Combining Template Matching and Background Subtraction Techniques to Detect Objects in Infrared Video Sequences

In this paper, an algorithm is introduced combining a template matching technique with background subtraction to detect moving targets on a road captured on video using a static camera. The designed algorithm based on combining the two techniques is to reduce the difficulties in background estimation and improve the processing time, etc. The proposed algorithm was tested in two different video sequences. The first video sequence was filmed with stabilized camera, while the second video was acquired with unstable camera. The results in both cases indicated that the background subtraction algorithm is successful in detecting moving objects and the template matching technique is suitable for object replacement from background frame to acquire the standard background frame.


Introduction
Background subtraction is an important technique in many computer vision applications such as video surveillance, people counting, human gesture recognition, moving object detection and tracking, traffic monitoring or in military application like imaging seekers.Many background subtraction methods have been proposed in publications such as running Gaussian average, temporal median filter, mixture of Gaussians, Kernel density estimation, etc.There are three representative approaches for background subtraction methods.First, they are pixel-based methods that extract foregrounds using each pixel independently.The second group of background subtraction techniques are the blockbased methods.The third class of background subtraction approaches are the texturebased methods [1][2][3][4].
In this paper, the presented method allows the user to choose the background frame at the start.Then, the algorithm automatically updates the background frame to eliminate environmental changes such as daytime, weather changes, etc.However, the background frame can contain objects of interest.So, a template matching technique needs to be employed to delete these objects from picture and set suitable intensity to pixels of removed objects.
There are many similarity criteria for template matching that are the sum of absolute differences (SAD), the locally scaled sum of absolute difference (LSAD), the zero -mean sum of absolute differences (ZSAD), the sum of squared differences (SSD), the locally scaled sum of squared differences (LSSD), the zero -mean sum of squared differences (ZSSD), the normalized cross correlation (NCC), the zero -mean normalized cross correlation (ZNCC), and the sum of hamming distances (SHD) [6].In this paper, similarity criteria of the normalized cross correlation were employed.And the normalized cross correlation coefficient used is 0.8 [6,7].
The algorithm employed the following formula to calculate the correlation coefficient c (u, v) [8,9]: ( ) y x f , is the pixel value at coordinates x, y of image, • ( ) y x g , is the pixel value at coordinates x, y of template image, • g is the mean of the template, • v u f , is the mean of ( ) y x f , in the region under the template.

Principle of the Algorithm
Fig. 1 outlines the principle of the algorithm.There are NumFram frames in the video sequence.The parameter step is an increment between the frames of video sequence.In this study, the step value is set as 1.The algorithm begins at frame i = a and finishes at frame i = b.In case of i = a, the unreal colour background frame B = Fa, and then the background frame B is converted to grey scale.
The parameter flag has the initial value equal to 0. If flag = 1, the background frame is changed, and if flag = 2, then the moving objects in background frame are removed.Parameters c1, c2 are calculated by the following formula.
where i is the frame number, when the background frame has chosen or has changed to suit the environmental conditions.The value of k is based on the experience that there are some moving objects in the background, so all objects will leave initial positions after k frames.The constant k is dependent on the velocity of the objects.In this paper, we used k = 60 equivalent to 2 s.At this time, it is easy to determine objects location and accurate shape.The acquired results help us to achieve more accurate results when Combining Template Matching and Background Subtraction Techniques to Detect Objects in Infrared Video Sequences 153 the template matching technique is employed.In the template matching block, the algorithm also removes moving objects on the background frame.We use the normalized cross correlation coefficient with the value equal to 0.8 in order to define clearly that new objects do not appear in the position of the moving objects in background frame.In a case that the cross correlation coefficient is less than 0.8, the algorithm will use the intensity of pixels in the current frame to replace the intensity of pixels in the background frame.The algorithm also converts each current colour frame Fi to greyscale frame, then it employs the background subtraction technique so as to find a matrix S. On the basis of matrix S, the mean value m can be calculated.It can be empirically predicted that the value m cannot exceed d.In this paper, d is equal to 14.In a case of exceeding the value of 14, the algorithm will automatically switch to the next frame Fi.
In other cases, the algorithm will compute the binary image BW1.The objects of interest should be then in the BW1 which is calculated by the following formula: where every element value of matrix S must be equal or greater than d1.In this paper, minimum pixel value d1 is 10.On BW1, we will find the number of objects (num) and the size of each object.If the BW1 does not contain any object (num = 0), the algorithm will automatically switch to the next frame.In other cases, the algorithm will check the size of the largest object in BW1, which is stored in variable size.If this value exceeds a limit e, the algorithm automatically selects the current frame as new background frame meaning B = Fi and the value of flag will be 1.In other cases, the algorithm will only select f objects and show them.In this paper, f is 20 and e is computed by the following equation: where S1 is a size of the matrix S, as multiplication of the width and height.

Experimental Results
In this paper, results of the designed algorithm are presented in two video sequences.The first video sequence has been acquired in stable air, while the second video is little blurred in initial frames of the video sequence.In both cases we obtained credible results and also the proposed algorithm proved its effectiveness when interference occurred.In each video sequence the following results are presented: the initial background frame Fa, the changed background frame Fi, the background frame at the time when the algorithm removed the moving objects containing Fi+k.95 frames in each video sequence were tested, where the initial frame a = 5 and b is 100, but only the results at 6 th , 46 th and 86 th frames are shown.The size of the frame in all video sequences is 120 × 160 pixels.

First Video Sequence
Fig. 2 shows a change of background frame during the video sequence.At the start (initial time), the algorithm choses the background frame B = F5.Then the algorithm continues to track two conditions to change the background frame, but these conditions were not fulfilled.Therefore, the algorithm employed the template matching technique at frame F65 and then it also removed moving objects to acquire a new background frame (on the right).Moving objects were omitted completely by the algorithm.
Combining Template Matching and Background Subtraction Techniques to Detect Objects in Infrared Video Sequences 155

Fig. 2 Background frame of 1 st video changed according to the time
The following table shows results for a few selected frames (see Tab. 1).

Binary
Tab. 1 indicates the result of the proposed algorithm for 3 frames.For all three frames the algorithm did not detect objects in the middle of frame.These objects belong to the objects of interest, but they did not change their position after 86 frames.If they changed their position at the next frame, the algorithm would successfully detect them and replace their intensity of pixels by more appropriate intensity such as background intensity.

Second Video Sequence
As in the previous case, Fig. 3 shows changes of background frame.At the start (initial time), the algorithm chose background frame B = F5.Then the algorithm continued to check two conditions to change the background frame.One of these conditions had happened and therefore, the background frame changed at the frame F8.From this point,

Initial background frame
Unchanged background frame Blurred background frame the algorithm employed the template matching technique at frame F68 and then the algorithm also removed the moving objects to acquire a new background frame (on the right).The background frame still contained heads of objects after being removed.Rectangle number 1 denotes the head of the first object after being removed.Rectangle number 2 denotes the head of the second moving object.Two objects were removed from the background.

Fig. 3 Background frame of the 2 nd video changed according to the time
The Tab. 2 shows the results for a few selected frames.

Tab. 2 Results of the algorithm in a few frames of 2 nd video
Type 6 th 46 th 86 th Colour

Grey scale
Binary Tab. 2 indicates the result of the proposed algorithm for three frames.For the same reason as in the previous case, the algorithm did not detect objects in the middle of the frame in all the three frames.In this video sequence, the noise was clearly visible in the 6 th frame.Because the new background frame still contained parts of the object, the result in frame 46 was unclear about the size of the object.In frame 86 the noise was visible, but still the algorithm was able to detect objects.

Initial background frame
Changed background frame Blurred background frame 1 Combining Template Matching and Background Subtraction Techniques to Detect Objects in Infrared Video Sequences 157 In this study, the focus was on moving object detection and objects that did not move were temporarily ignored.These objects stood still on the road for a few seconds.As soon as these objects started moving again, the proposed algorithm successfully found the accurate background frame and detected all the objects.

Conclusion
The following conclusions can be drawn from the results we have acquired.The proposed algorithm employs both techniques (background subtraction and template matching technique) to solve two main problems.The background subtraction technique solved the first problem with moving objects detection.However, there were some difficulties with predicting a standard background meaning where the background frame contained all objects that were not needed to be detected.This was due to the fact that the background subtraction is based on the principle of frame differences, i.e. difference of the current frame and the background frame, to acquire a foreground frame that would contain the objects of interest.If the background frame contained the objects of interest, then the effectiveness of background subtraction was reduced.
In order to solve the effectiveness drop mentioned above, the algorithm used the second technique-template matching technique to compare sub-images of objects on the background with sub-images on the current frame.The algorithm removed these objects from the background and replaced inappropriate intensities that were not similar to the intensities of the real background to get a more suitable background frame.
In this paper, the new algorithm was tested in two different video sequences.The first video sequence was filmed with stabilized camera, while the second video was acquired with unstable camera.The results in both cases indicated that the background subtraction algorithm is successful in detecting moving objects and the template matching technique is suitable for object replacement from background frame.However, there were some objects of interest that were not moving for a short period of time on the background.These objects were removed from the background after a period of time that depended on the time in which the object was motionless.After a certain time, a perfect background frame was acquired again.
Based on the above mentioned results, the following conclusions can be also drawn.Firstly, the proposed algorithm automatically accommodates the requirements such as timing change, environment change, the impact of noise or the change of lighting conditions.A user can choose the background at any time and the algorithm automatically adjusts the background to respond to the environmental conditions.Secondly, the algorithm can also solve difficulties with memory.The algorithm requires a memory with capacity of about the size of one frame.Thirdly, the algorithm does not require too much time for background frame creation.Therefore, this algorithm could be used for real time applications.
Finally, the proposed algorithm did not solve the problem of stand still objects detection in the background frame.This insufficiency will be covered in our future research.

Fig. 1
Fig. 1 Flowchart of the algorithm