A PRELIMINARY PROJECT REPORT ON TV COMMERCIAL DETECTOR

A PRELIMINARY PROJECT REPORT
ON
TV COMMERCIAL DETECTOR

SPONSORED BY
REPORO LTD.

BY
ARUNA MORALE (B8204207)
POOJA DESAI (B80204225)
NITA DHAPASE (B8204226)
NIKITA KODKANI (B8204256)


DEPARTMENT OF COMPUTER ENGINEERING
CUMMINS COLLEGE OF ENGINEERING FOR WOMEN
KARVE NAGAR, PUNE – 411052
[2012 – 2013]
CERTIFICATE

This is to certify that the following students

Aruna Morale
Pooja Desai
Nita Dhapase
Nikita Kodkani

Have completed the preliminary work for the project entitled

"TV COMMERCIAL DETECTOR"
Satisfactorily for the partial fulfillment of the requirements for the Bachelor's Degree in Computer Engineering of Pune University during academic year
2012-2013 [SEM1]



Mrs. Pranjali Deshpande Mr. Milind Naik
Internal Examiner External Examiner



Ms. S. S. Deshpande Dr. Mrs. Madhuri Khambete
Head Computer Department Principal,
Cummins College of Engineering Cummins College of Engineering




ACKNOWLEDGEMENT
We would like to thank our internal project guide Mrs. Pranjali Despande for her valuable guidance, suggestions and timely help in the completion of our Project Report on "Ad Detection on television".
We would like to thank our external guide Mr. Milind Naik, Reporo Ltd. for suggestions of project, helping us understand the statement thoroughly and continuous support and encouragement.
We would also like to thank Mr. S. Mengale and Mr. H. Khairnar for valuable help during the process of project allocation.
Last but not the least; we would like to extend a sincere gratitude to Ms. Shilpa Deshpande, Head of Department and all the staff members of the Computer Engineering department, our friends and family for their support, encouragement and timely guidance.



ABSTRACT
Every year companies spend sizable budget on marketing a large portion of which is spend on advertisement of their product bands on TV broadcasts. These advertisements usually consist of product logo and its information. These companies are keen to verify that their brand has the level of visibility they expect for such expenditure. Automatic detection of the commercial would be proved helpful to such companies.
The aim of the project is to detect the commercials in specific genre of program such as the serials and movies using feature based and recognition based methods. Our software will detect ad by presence of text, logo and amplitude variation. We are using the MATLAB 10.0 platform for processing input video and mapping with the stored commercial feature for detection of commercial.

INDEX

Chapter 1

INTRODUCTION


1.1
Problem Definition


1.2
Basic Introduction


1.3
Literature survey


1.3.1
Background for the project


1.3.2
Domain of study


1.3.3
Motivation for the project


1.3.4
Survey of existing


1.3.5
Limitation of existing system


1.3.6
Review of existing paper


1.3.7
Strengths and Weaknesses

Chapter 2

Software Requirement Specification(SRS)


2.1
Introduction


2.1.1
Purpose of this document


2.1.2
Scope of the development project


2.1.3
Definitions, Acronyms and Abbreviations


2.2
General description


2.2.1
User personas and Characteristics


2.2.2
Product Perspective


2.2.3
Overview of Functional Requirements


2.2.4
Overview of Data Requirements


2.2.5
Operating Environment


2.2.6
General Constraints, Assumptions, Dependencies, Guidelines


2.3
Specific Requirements


2.3.1
External Interface Requirements


2.3.2
Detail Description of Functional Requirements


2.3.3
Performance Requirements


2.3.4
Quality Attributes

Chapter 3

HIGH LEVEL DESIGN


3.1
3.2
3.3
3.4
Block Diagram
Use Case
Activity
Algorithms

Chapter 4
Technology

Appendix A

Appendix B
Appendix C

Appendix D
Mathematics Involved
SET Theory
Test Planning and Test Case Designing
Project Plan and Work progress
Individual Contribution etc.
Glossary


Chapter 1: Introduction
1.1Problem Definition
Our system detects the presence of the TV commercial based upon the various features of the Ad such as presence of text, matching of logo, and change in amplitude in audio. The features of ad will be predefined so that it will be easy to map the AD.
System should detect the presence of commercial TV Ad from recorded TV program using "Featured based" and "Recognition based" methods. Features of ad like text, logo is already stored used to map the logo and text frames
1.2 Basic Introduction
TV Commercial is a span of television program used for promoting product or service. Given a TV program, way of detecting the commercial advertisements in it is defined as commercial detection.
TV Commercial can be distinguished from the other program with the help of Ad features such as audio fingerprints, black frame, logo detection, shot segmentation, caption detection, scene break detection, aspect ratio etc. Methods used for commercial detection are feature based and recognition based [1]. In feature based method commercial is detected with the help of integral characteristics of TV commercials [1]. In Recognition based method commercial is identified by searching database containing known commercial [1]. Recognition based methods are found to be more efficient due to higher detection rate and are more reliable than feature based in detecting particular Ad .
1.3 Literature survey
1.3.1 Background for the project
There are a lot of practical applications for advertisement detection. Based on the end-user, there are three types of applications of advertisement detection. They are the TV viewer, advertiser and broadcast regulator.
TV viewer might want to record some programs for later viewing which are interesting. In such cases, it will be very irritating to see the advertisements in the recorded video. It also results in wastage of memory space since these advertisements will occupy useful space where programs can be stored. It is also tedious to sit over the entire video and remove the advertisements so these requirements make advertisement detection very useful problem for the benefit of these viewers.
The advertiser might want data about the timings of the airing of the advertisements of his brand. This will help in making an informed decision regarding the ratings of the advertisement, the influence of that advertisement and further ways to improve the advertisement. This will also help the advertising company in optimizing its revenue by deciding the timings in which the advertisement has to be aired. It will also help the advertiser in knowing which of its advertisements has the maximum influence among the TV viewers and develop publicity strategies accordingly.
The broadcast regulator wants data about their timings, their quality, their length and decency so as to have an eye on the advertisers whether they are following the various regulations regarding advertisement broadcasting. They will need a full time TV viewer dedicated for the sole purpose of watching the programs and pinpoint the advertisements to check whether they follow the regulations.
1.3.2 Domain of study/field of study
Image processing:
Image processing is a method to convert an image into digital form and perform some operations on it, in order to get an enhanced image or to extract some useful information from it. It is a type of signal dispensation in which input is image, like video frame or photograph and output may be image or characteristics associated with that image. The purpose of image processing is image retrieval i.e. seeks for the image of interest, measurement of pattern i.e. measures various objects in an image, and image recognition i.e. distinguishes the objects in an image. MATLAB Image Processing Toolbox provides a comprehensive set of reference-standard algorithms and graphical tools for image processing, analysis, visualization, and algorithm development. It can perform image enhancement, feature detection, noise reduction, image segmentation, geometric transformations and image registration.
Commercial study:
Characteristics of commercials

Commercials are almost always grouped into blocks, typically consisting of four to 10 commercials each. At the beginning and end of each commercial block and between each commercial in the block, several frames of monochrome black are displayed.
On many stations, the observation has been that the last two to three commercials of a block are commercials promoting upcoming shows.
Also, some countries (e.g. Germany) have laws requiring that every commercial block begin with a standard "commercial block introduction".
Many television stations also have a practice of displaying a network logo in the corner of the screen during regular programming and then removing this logo during commercial breaks.
Within a given television series, all episodes generally have commercial breaks scheduled at approximately the same time in the episode. Also, many commercials are repeated on a frequent basis, particularly for a given station.
The duration of individual commercials is short, almost always less than ninety seconds, and typically it is an integer multiple of fifteen seconds.
To capture viewers' attention in the small amount of time available to convey a message, commercials tend to be high in "action," typified by a high number of cuts between frames among other things. (The average "hard" cut rate in a sample of 200 commercials from German television was 20.9 cuts per minute, while the rate in the accompanying movie clips was only 3.7 cuts per minute.)
There are usually a large number of frames with text containing the product or company's name. Also, to leave the product in the viewer's mind, the last few seconds of many ads consist of "still" shots of the product or slogan.
The most noticeable characteristic, and the one most irritating to viewers, is the tendency of broadcasters to increase the v o l u m e level of the audio track during commercials.
Black frames at the beginning and end of commercials are accompanied by silence in the audio track.
Also, the dialogue on the audio track generally contains the product or company's name.
Finally, when closed captioning is available for a television show, it is generally discontinued during commercial breaks.
Detection schemes
There are two main categories of methods used to detect commercials:
Feature-based detection: relies on general characteristics of commercials to detect their presence. Any of the commercial characteristics mentioned earlier could be used to indicate the (possible) presence of a commercial.
Recognition-based detection: attempts to identify individual commercials in the broadcast as matching commercials it has already learned.

Study of Commercials on different channels:
Ads observed on COLORS channel (recorded on Friday 24/8/2012 from 7pm-8pm):-
Sr. No.
AD NAME
TEXT
LOGO
1
BRU
Yes
Yes
2
TUPPERWARE
Yes
Yes
3
MEDIKAR
Yes
No
4
DOCOMO
Yes
Yes
5
FORTUNE
Yes
Yes
6
MESHWAK
Yes
Yes
7
BRITANIA
No
Yes
8
FAIR & LOVELY
Yes
Yes
9
PEPSODENT
Yes
No
10
LAYS
Yes
Yes
11
CLINIC PLUS
Yes
No
12
DOMINOS
Yes
Yes
13
RED LABEL
Yes
Yes
14
BOURBON
No
Yes
15
LOREAL
Yes
Yes
16
KELOGGS
Yes
Yes
17
HONITUS
Yes
Yes
18
PURE IT
Yes
No
19
MAGGI
Yes
Yes
20
EVEREST
Yes
No
21
ALIVA
Yes
Yes
22
FORTUNE
No
No
23
IODEX
Yes
No
24
INCREDIBLE INDIA
Yes
Yes
25
BAJAJ ALMOND HAIR OIL
Yes
No
26
KAILASH TEL
Yes
Yes
27
ULTIMA
Yes
No
28
MAMY POKO PANTS
Yes
No
29
MAGGI MASALA
Yes
Yes
30
COLGATE SENSITIVE
Yes
Yes
31
EXO
Yes
Yes
32
RIN
Yes
No
33
PEARS
Yes
No
34
COMFORT
Yes
No
35
DETTOL
No
Yes
36
BENADRYL
Yes
No
37
NANODEAN
Yes
Yes

OBSERVATIONS:-
Every AD has product displayed at end.
Every serial has 2 intermediate commercial breaks (of about 5 minutes) and the ads in these breaks are unique.
In case of colors channel at the start of commercial break upcoming serial ads are shown followed by short commercial ads again followed by upcoming serial ads and then the actual commercial ad starts.
In case of color after 3 minutes, timer of 2 minutes gets displayed on top left side.
Ads observed on SAB TV channel (recorded on Friday 03/8/2012 from 8.30pm-9.45pm):-
Sr. No.
Ad Name
Text
Logo
1.
Diary milk
Yes
Yes
2.
Colgate
Yes
-
3.
Samsung galaxy S3
Yes
Yes
4.
Hipolin

Yes
5.
Godrej
Yes
Yes
6.
Badshah
Yes
Yes
7.
Titan
Yes
-
8.
Chaini
Yes
-
9.
Chocolate foundation
Yes
Yes
10.
Goodnight
Yes
Yes
11.
KFC
Yes
Yes
12.
Munch
Yes
-
13.
Mehndi
Yes
-
14.
Odomos
-
-
15.
Godrej no1 soAP
Yes
Yes
16.
Garnier
Yes
-
17.
Londonderry chocolate
Yes
-
18.
Asian Paints
Yes
-
19.
Maggi
Yes
Yes
20.
Mederma
Yes
-
21.
Coreal
Yes
Yes
22.
Kurkure
Yes
Yes
23.
Everyuth
Yes
-
24.
Horlicks
Yes
Yes
25.
BaadshahMilkmasala
Yes
Yes
26.
Nirma
Yes
-
27.
Iballslide
Yes
Yes
28.
Parle
Yes
Yes

Note:
The different standards of transmission are not compatible with each other. No American set can work in Europe and vice versa. Even in Europe, a set used in Germany cannot be used across the border in France. A receiver used in Indian subcontinent will not work in Myanmar and Japan.
1.3.3 Motivation for the project
The motivation of this project is from the following applications of ad detection-
System can be used by advertisers to know if their ads are published and also number of times ad is published in a day.
Removal of ad after detection is useful to end users who want to record TV serials without the interference of ads.
It is also useful to video database maintainers to decrease storage requirement, by eliminating ads from stored shows.
Ads which are not relevant for a particular age group (e.g. children) can also be replaced by other ads on detection.
1.3.4 Survey of existing system
There are systems for ad detection available which are developed for applications like removal, replacement or detection of ads. The systems use approaches like detecting black frames at the start and end of commercial block (used in US and Russia), audio silences. Such systems are used for detecting commercial block and not identifying individual ads.
There is a system designed for Spanish broadcast TV which detects ads based on the fact that TV logos are during commercials and shot detection i.e. video shots have short duration within commercials. But it does not happen in the Asian television.
On Japans TV system commercial segments are detected based on assumption that each commercial lasts for multiples of 15 seconds in length.
Existing systems use methods like rule-based methods, which use a set of features and rules to distinguish commercials from general programs (non-commercials), logo-based methods which identify commercials only by the existence of TV station logos.
Myth TV and MSU TV are the software which are already build to remove the commercial which are detected. Myth TV uses Blank frame, Logo and scene change methods for commercial detection. MSU TV uses black frame, shot detection and logo detection for detecting presence of commercial.

1.3.5 Limitations of existing system
Existing system work on assumption like disappearance of TV logos during commercial block, duration of commercial in multiples of 15 sec, presence of commercial bumper at start and end of commercial block.
1.3.6 Review of previous IEEE papers
1) Title: Detection of TV commercials
Description: This paper approach to commercial detection relies on two simple observations to label each video shot as a commercial or program shot. The first observation is based on the fact that TV logos are removed during commercials (at least in the Spanish broadcast). The second observation stems from the fact that video shots tend to have a shorter duration within commercials. These observations are modeled using HMM (Hidden Markov model) and the Viterbi algorithm is finally used to label each shot. They come with the new approach for logo detection that found out the area with stable contour which is applicable for animated, opaque, and transparent logos.

2) Title: Novel Real-time Commercial Detection Scheme
Description: This paper has proposed three methods of ad detection. First method is related with short silence sequence between commercials and other programs or between different commercials. Second method is related with shot change and scene change between commercials and other programs or between different commercials in which they have used short density algorithm. Third method is related with length of commercial.
3) Title: Billboard advertising Detection in Sport TV
Description: This paper has given technique for detection of commercial advertisement (Billboard advertisement) in sport channel. As advertisement billboard always appears between the sport ground and spectator stadium so to extract these ad they have used Hough transform and these ad are segmented by histogram based analysis which separate text from background. They have also used color to gray image transformation algorithm, edge detection algorithm (Sobel filter).
4) Title: A Novel Break Detection and Automatic Annotation of TV Program for content Based Retrieval
Description: This paper has used novel approach for automatic annotation and content based video retrieval by making use of the features extracted during the process of ad detection. They have used both audio and video approach for ad detection .Firstly they have detected commercial block by using audio features for this they have used Energy Peak rate(EPR)technique. After detection of commercial block they have detected each commercial by extracting different features (logo, text etc.) and all these features are store in database.
5) Title: TV Commercial Detection in News Program Videos
Description: This paper has given method to detect TV commercials in news programs. They have used segmentation Approach to detect ad .they have given labels to all possible commercial segment which are present in recorded video because of they are able to find exact boundary of commercial. This labeling consist of Cut in a minute (or shot change) is editing between two individual continuous camera shots, Strong cut in a minute.
6) Title: TV Advertisements Detection and Clustering based on Acoustic Information
Description: This paper has used approach of detection of commercial and it's clustering for this they have used acoustic information .Firstly they have observed that advertisement breaks are usually isolated from actual program by decrease in the audio signal occurring before and after each individual advertisement .From this observation they have used energy drops to detect ads (by using BIC algorithm).They have consider length of ads 10s,20s,30s to detect ad .After detection of ad they have used clustering(creation of database for commercial )while detection they have used this database to check whether that commercial is already present in database if not present then they have added database of new coming commercial .
7) Title: Real – time TV commercial monitoring based on Robust Visual Hashing
Description: This paper presents a method for TV commercial detection based on visual hash which is short length bit string used to identify broadcasted frames uniquely. They are calculated using commercial blocks DC value. Visual hashes of starting and ending frames of an individual commercial along with their duration are stored in database. First the start of commercial block is detected using commercial bumpers. Then broadcasted frames are compared with database to search for a known commercial using technique like Normalized Hamming Distance between their hashes. New ads are manually interpreted and stored in database. End of commercial block is detected using commercial bumpers. Drawback of this system is that if TV station changes the commercial bumper the system would fail.
8) Title: TV Commercial Detection Based on Shot Change and Text Extraction
Description: This paper is based on shot change for which histogram difference between adjacent frames is calculated. Shot change from histogram difference curve is detected using a threshold value. Cut, dissolve, fade in/out and wipe are the four shot transitions. Cut is an abrupt shot change whereas dissolve and fade in/out are gradual shot changes. In case of wipe, screen has a moving strip which converts old shot to a new shot. Thus a wipe has both portions of old and new shot present on frame. Shot change is determined using a sliding window. This paper is also based on text detection where first grayscale luminance value from an RGB image is computed and then horizontal and vertical maximum gradient difference is determined to find continuous text portions. Drawback of this system is that it relies on predetermined threshold value.
9) Title: Fast commercial detection based on audio retrieval
Description: This paper is based on audio signals to detect ads. Audio stream is segmented into units based on the energy levels. A unit has lower energy at the start and end and each unit contains one major peak. As commercials start with a short silence, matching of a known commercial in an audio stream is performed at the start of unit. Segmentation points in a unit is determined .Also segmentation points of known commercials are stored in database. Commercials are detected by identifying similarity between the segmentation points of units of audio stream and commercials from database. This system uses a unit stepping strategy.
10) Title: The detection of TV commercial based on multi feature fusion
Description: This paper describes about the detection the TV commercial by using shot segmentation, caption detection and audio feature extraction. TV commercial has high shot transitions such as fade in and fade out, abrupt transitions, high occurrence of flash, etc. It was also found that most of the TV serial had the caption at the beginning and at the end of the TV serial, which would appear mostly at the upper or bottom side, so by absence of that logo would identify commercial Ad. In the audio feature extraction, average short term energy was calculated and was compared with the threshold value, which detected the presence of the Ad
11) Title: TV Commercial Detection Using Constrained Viterbi Algorithm Based on Time Distribution
Description: This paper proposed the method of detection of commercial TV Ad by black frame, FMPI (Images Frames Marked with Product Information) and POIM (Program Oriented Informative Images). It was observed that Programs start mostly with POIM image. So it was basically used to mark the start and end of the program, which was further used to detect the block start of the TV Ad .FMPI contains the information of product or service. It was used to segment, recognize and retrieve commercials.
12) Title: Self-Optimized Spectral Correlation method for background music identification
Description: In order to make their TV commercials immediately identifiable to viewers and present a consistent theme, some companies put the same type of sounds into all their commercials. This paper proposes the method, to identify such background music from the speech.
13) Title: Multi-Modal Characteristics Analysis & Fusion for TV Commercial Detection
Description: In this paper, multi-modal (i.e. visual, audio and textual modalities) characteristics of the TV commercial are used to detect the Ad They have proposed the model, that when video is segmented, the frames which are obtained which are further simultaneously checked with the descriptors (visual change, audio modeling and text pattern detection) will give the final verdict about the conformation of Ad To do the final decision about the presence of ad they have designed the Tri-AdaBoost , which is heuristic system designed to boost the speed of the Ad detection.
14) Title: Segment Oriented Search (SOS) method for TV Repeats detection
Description: This paper focuses on detecting repeating commercials. The system proposed, didn't require any prior information such as logos. It first detects the scene breaks and use the hash based method to perform repeating video sequence matching. When the first scene matches it checks for the consecutive scene breaks. At the end, all such scene breaks are grouped and marked as one Ad. The experiments have shown that this method was faster than frame by frame comparison.
15) Title: Exploiting Visual-Audio-Textual Characteristics for Automatic TV commercial Block Detection and Segmentation
Description: This paper focuses on commercial block detection (CBD) and commercial block segmentation (CBS) by means of collaborative exploitation of visual-audio-textual characteristics embedded in commercials. Rather than utilizing exclusive visual-audio characteristics, an abundance of textual characteristics associated with commercials fully exploited. Additionally, Tri-AdaBoost, an interactive ensemble learning manner, is proposed to form a consolidated semantic fusion across visual, audio, and textual characteristics. In order to segment a detected commercial block into multiple individual commercials, additional informative descriptors including textual characteristics are introduced to boost the robustness in the detection of frame marked with product information (FMPI). Together with the characteristics of audio spectral variation pointer silent position, FMPI can provide a kind of complementary representation architecture to model the similarity of intra-commercial and the dissimilarity of inter-commercial. Experiments are conducted on a large video dataset from china central television (CCTV) channels.
1.3.7 Strengths and Weaknesses
Strength:
Our system can help detection of commercial from recorded TV program using audio text, logo simultaneously due to which system is more accurate.
Our system will also identify number of ads in a commercial block.
Weakness:
For detection, features of ads like logo needs to be stored, hence database is large.
Only ads whose logos are stored in database will be detected.
Our system will not detect ads at broadcast level.


Chapter 2: Software Requirement Specification
2.1 Introduction
2.1.1 Purpose of this Document
Every year companies spend sizable budget on marketing a large portion of which is spend on advertisement of their product bands on TV broadcasts. These advertisements usually consist of product logo and its information. These companies are keen to verify that their brand has the level of visibility they expect for such expenditure. Present document propose software of Automatic detection of the commercial which would be helpful to such companies.
2.1.2 Scope of development project
Television programs are of various types such as news, sports, serials, movies. The aim of the project is to detect the advertisements in specific genre of program such as the serials and movies using recognition based methods. Our software will detect ad by presence of text, logo and threshold energy (for audio detection), etc. These features can be used simultaneously for detecting ads giving higher accuracy.
Stored TV program, which was priory captured by TV tuner from television is used as input. Program is divided into frames and then text and logo is extracted from frame. Finally they are matched with stored logo and text to detect the individual commercial. MATLAB 10.0 provides rich set of libraries for image processing. Platform we are using is Windows XP.
2.1.2 Definitions, Acronyms and Abbreviations
1) Ad-Advertisement
2) TV-Television


2.2. General Description
2.2.1 Users personas and characteristics
1) User: Upload video containing Ad.
2) System: Detect ad from the video provided by user.
2.2.2 Product perspective
Our product is Software.
2.2.3 Overview of functional requirements
1 .User:
Upload video: upload video.
Start video: Start video provided by user.
Stop video: User can stop input video.
2. System:
Extraction of video (without audio): Extract video from given video.
Extract audio: Extracting audio from the given video.
Division in frame: Whole recorded video get divided into frames.
Logo processing: Match extracted logo with stored logo.
Text processing: Match extracted text with stored text.
Audio processing: Find change in amplitude.
Check Ad: If Ad is present then store data of ad in database else skip the part of frame.
2.2.4 Overview of data requirements
Our software uses TV to record video containing Commercial.


2.2.5 Operating Environment
Processor: Intel Pentium p4 and above.
Operating System: windows XP and above.
RAM: 256 MB.
2.2.6 General constraints, assumptions, dependencies, guidelines
1) Software will detect only those Ads whose features are with the system.
2) Programs are recorded from the television.
2.3. Specific Requirement
2.3.1 External interfaces and requirements
TV-tuner is used to record programs from television.
2.3.2 Detail Description of Functional Requirements
Function
Logo processing
Precondition
1) Logo should be extracted from frame.
2) Logo should be stored in database.
Steps
1)Match extracted logo to logo stored in database
2) If logo is matched then write to output file else skip frame.
Post condition
Ad is detected
Alternative Flow
Skip frame

2.3.3Performance Requirements
Software will detect Ad at faster rate due to simultaneous use of audio, text and logo.


2.3.4 Quality Attributes
1) Correctness: 90-95% ad will be detected from the video stream.
2) Usability: System can be used to detect commercial ad from the video.
3) Platform Independent: It can support all existing system
4) Reliability: The system is reliable in detecting ad present in video.


Chapter 3: High Level Design
Block Diagram

Commercial database:-Logos-commercial text-Anchor Message AI Engine
Commercial database:
-Logos
-commercial text
-Anchor
Message
LogoLogo processing
Logo
Logo processing
Frame processing
Frame processing


Text detectionSilence detectionMessage detectionAspect RatioTextExtract video and divide it into framesRecorded Samples
Text detection
Silence detection
Message detection
Aspect Ratio
Text


Extract video and divide it into frames
Recorded Samples
Anchor msg
Anchor msg
Output:-Adname-logo-textExtract audioSampling audio
Output:
-Adname
-logo
-text
Extract audio
Sampling audio

High amplitude
High amplitude
Audio processing
Audio processing



Is Ad
Is Ad




Skip frameNo
Skip frame



Block Diagram Description:
Input: Recorded TV program
Extract video and divide it into frames:
This block extract video from recorded program and divide whole video into frames.
Extract audio:
This block extracts audio from recorded program
Frame processing:
This block implements frame processing and extract text, logo from that frame and provide it as input to AI engine.
Sampling audio:
This block plot audio amplitude graph and provide value of amplitude to AI engine as input
AI Engine:
AI engine will think parallel like human on all features of commercial simultaneously for detection of Ad from the given input TV program. It consider input from frame processing and sampling audio block for parallel working .Frame processing provide text, logo, Anchor message, Aspect Ratio to AI engine and then AI Engine finds probability for existence of commercial from this whole data of frame .From this probability it decide whether it is commercial Ad or not with the help of logo processing, text detection and message detection block. And if it is not commercial frame then it sends to skip frame block.
Logo processing:
This block takes extracted logo as input from AI engine and maps it with commercial logo stored in commercial Database block and finds the probability and send to AI engine for its decision.


Text processing:
This block takes extracted text as input from AI engine and maps it with commercial text stored in commercial database block and fins the probability and send to AI engine for its decision.
Message detection:
This block takes extracted text as input from AI engine and maps it with commercial anchor message stored in commercial database block and fins the probability and send to AI engine for its decision.
Skip Frame:
This block skips frames which do not consist of commercial information.
Commercial Database:
This block consists of all stored commercial feature which is used by other block to detect the particular commercial.
Output:
This block stored the information of all the detected AD to file.



Use-Case Diagram
Extraction of audio
Extraction of audio

Extraction of frame
Extraction of frame
Text extraction
Text extraction
Detection by text
Detection by text
Logo extraction System
Logo extraction
Detection by logo
Detection by logo
Detection by audio amplitude
Detection by audio amplitude

Store details of detected AD
Store details of detected AD










Activity Diagram
Extraction of frames from videoExtracting logo from framesStore the details of AD in database
Extraction of frames from video
Extracting logo from frames
Store the details of AD in database



Skip Frames
Skip Frames


Check Extracted logo with database
Check Extracted logo with database





Yes


ALGORITHMS:
1) TEXT IDENTIFICATION:
The text detection stage detects the presence of text in a given image.
Algorithm:
1) Convert the image into binary picture that separates text object from background or non-text object. This method is known as preprocessing. Combination of filter is used to process original image into binary image.
2) After pre-processing, all connected pixels with same color index as a separate blob object are detected. This method is known as Blob extraction. It produces text blobs and also non text blobs.
3) Then classify a blob as a text blob or non-text blob. For that some features from text blob candidate is used for classification.
4)Then the text blob selection is used to select text blob and to remove noise or non-text blob, and text blob classification to classify blob into text word or sentence based on text blob position.
5) Finally the OCR (Optical Character Recognition) [11] is used to recognize the text from the blob. [12]

2) DETECTION USING PRESTORED LOGO OF COMMERCIAL:

Humans are capable to identify easily if an image is similar to another, in a way important to the viewer. This similarity evaluation could be based on shapes, colors, image composition, object proximity or a combination of all mentioned factors.
Computers are less efficient for this task, since are based on a linear logic. Using special methods and systems, it is possible to make the computers able to identify images like human beings. The basis of any software for image recognition it is the ability to realize adaptive matching, when we are comparing images. Like in human model of recognition, adaptive matching is giving the possibility to find out if similar, but not identical, images are essentially the same.
Applications realized for identification and image matching are offering the necessary solutions to handle the huge amount of images and video clips necessary to be processed.
Algorithm:
1) The image is modified in order to reduce defects that are unavoidably introduced by the image formation and acquisition stages (geometric distortion, de-focusing, noise) or enhances some desired property such as contrast between objects of interest and background.
2) Then the objects or parts of interest are roughly or accurately located using various techniques such as segmentation, edge detection or pattern matching.
3) When relevant locations are determined, processing concentrates on regions of interest and measurements are performed locally and the shape of objects can be quantified by appropriate geometric quantities.
4) The computed quantities can be used to assess the parts quality and detect defects by comparing them to the expected values for good parts.
3) DETECTION USING AUDIO SIGNALS:
Basic five features can be used for distinguishing the advertisements from program segments. The five audio features are audio break detection and audio types such as speech, music, silence and background sound.
Algorithm:
1) Extract the audio from audio-video.
2) Take the summation of the absolute value of all the audio samples corresponding to one video frame.
3) Determine the threshold value.
4) Then recognize the depression in audio volume. That would mark it as the ad.

Chapter 4: Technology
MATLAB:
MATLAB provides with variety of image processing functions like loading an image using right format, saving data as different data types, displaying image, converting image to different image formats, etc. For this, MATLAB needs to have Image processing toolbox installed with it. Image Processing Toolbox is a collection of functions that extend the capability of the MATLAB numeric computing environment. These functions and the expressiveness of the MATLAB language, make image-processing operations easy to write in compact and clear manner. It also provides a comprehensive set of reference-standard algorithms and graphical tools for image processing, image enhancement, feature detection, noise reduction, image segmentation. It supports a diverse set of image types. Graphical tools such as let us explore an image, examine a region of pixels, adjust the contrast, create contours or histograms, and manipulate regions of interest (ROIs). With toolbox algorithms one can restore degraded images, detect and measure features, analyze shapes and textures, and adjust color balance.
Analyzing Images
Image Processing Toolbox provides a comprehensive suite of reference-standard algorithms and graphical tools for image analysis tasks such as statistical analysis, feature extraction, and property measurement.
1) Edge-Detection Algorithms: Edge-detection is identifying object boundaries in an image.
2) Image Segmentation Algorithms: Image segmentation is determining region boundaries in an image.
3) Morphological Operators: Morphological operators enable to detect edges, enhance contrast, remove noise, and segment an image into regions and thin regions. Morphological functions in Image Processing Toolbox include Erosion and dilation, Reconstruction, Distance transform.


Advanced Image Analysis
Image Processing Toolbox also contains advanced image analysis functions to:
1) Measure the properties of a specified image region.
2) Detect lines and extract line segments from an image.
3) Measure properties, such as surface roughness or color variation.
Aspects of Image Processing
1) Image Enhancement: Processing an image i.e. sharpening image, highlighting edges, improving image contrast, or brightening an image, removing noise.
2) Image Restoration: Reversing the damage done to an image by a known cause. (Removing of blur caused by linear motion, removal of optical distortions)
3) Image Segmentation: Subdividing an image into constituent parts, or isolating certain aspects of an image i.e. finding lines, circles, or particular shapes in an image.
4) Image Thresholding: Converting an image into black and white image.
Types of Digital Images
1) Binary: Each pixel is just black or white with values 0 or 1.
2) Grayscale: Each pixel is a shade of gray, normally from 0 (black) to 255 (white).
3) True Color or RGB: Each pixel has a particular color which is described by the amount of red, green and blue in it. This means that every pixel corresponds to 3 values.
Types of processing

1) Block processing - An operation in which an image is processed in blocks rather than all at once. The blocks have the same size across the image. An operation is applied to one block at a time. Once processed, the blocks are re-assembled to form an output image.

2) Region-Based Processing - Region-based processing allows you to select a region of interest (ROI), and process only upon the selected area. A ROI is defined using a binary mask – The mask contains 1's for all pixels that are part of the region of interest and 0's everywhere else. A region of interest can be specified using one of the Image Processing functions or with any user defined binary mask. The options are:
Using roipoly allows specifying a polygonal region of interest.
Using roicolor allows specifying region of interest based on a color or intensity range.

VARIOUS FUNCTIONS IN MATLAB FOR IMAGE PROCESSING:
1) AVIREAD (filename)
AVIREAD Read AVI file.
MOV = AVIREAD (FILENAME) reads the AVI movie FILENAME into the MATLAB movie structure MOV. If FILENAME does not include an extension, then '.avi' will be used. MOV has two fields, "cdata" and "colormap". If the frames are truecolor images, MOV.cdata will be Height-by-Width-by-3 and MOV.colormap will be empty. If the frames are indexed images, then MOV.cdata field will be Height-by-Width and MOV.colormap will be M-by-3. On UNIX, FILENAME must be an uncompressed AVI file.
Example:
a=imread ('cameraman.tif ');
imshow (a);
2) WAVREAD (file)
WAVREAD Read is Microsoft WAVE (".wav") sound file.
Y=WAVREAD (FILE) reads a WAVE file specified by the string FILE, returning the sampled data in Y. The ".wav" extension is appended if no extension is given.
[Y, FS, NBITS]=WAVREAD (FILE) returns the sample rate (FS) in Hertz and the number of bits per sample (NBITS) used to encode the data in the file.
[...]=WAVREAD (FILE, N) returns only the first N samples from each channel in the file.
[...]=WAVREAD (FILE, [N1 N2]) returns only samples N1 through N2 from each channel in the file.
[Y ...]=WAVREAD (..., FMT) specifies the data type format of Y used to represent samples read from the file.
If FMT='double', Y contains double-precision normalized samples.
If FMT='native', Y contains samples in the native data type found in the file. Interpretation of FMT is case-insensitive, and partial matching is supported. If omitted, FMT='double'.
SIZ=WAVREAD (FILE, 'size') returns the size of the audio data contained in the file in place of the actual audio data, returning the 2-element vector SIZ= [samples channels].
[Y, FS, NBIT S, OPTS]=WAVREAD (...) returns a structure OPTS of additional information contained in the WAV file. The content of this structure differs from file to file. Typical structure fields include '.fmt' (audio format information) and '.info' (text which may describe title, author, etc.)
Output Scaling
The range of values in Y depends on the data format FMT specified. Some examples of output scaling based on typical bit-widths found in a WAV file are given below for both 'double' and 'native' formats.
FMT='native'
#Bits
MATLAB data type
Data range
8
Uint8 (unsigned integer)
0