Tracking

Tracking in computer vision refers to the process of locating and following objects or regions of interest in a sequence of images or video frames over time. The goal of tracking is to determine the movement of objects within a video stream, enabling applications such as object recognition, activity analysis, surveillance, and augmented reality.

There are many tracking approaches, including single-object tracking, multi-object tracking, and object tracking by detection. These methods use different techniques depending on the complexity of the scene, the number of objects tracked, and computational resources.

Description

In this example, we are going to track the movement of a car in a video. For this purpose, we will select the car we would like to track and compare between three different algorithms to understand its performance. The algorithms are:

1. MIL (Multiple Instance Learning): MIL is a tracking algorithm based on multiple instance learning. In the context of object tracking, it treats the tracking problem as a binary classification task, where the classifier needs to learn to distinguish between positive instances (e.g., the target object) and negative instances (i.e., other objects). MIL considers multiple instances of the target object and uses a learning algorithm to iteratively refine the tracking model based on these instances. MIL can be robust to variations in appearance, scale, and occlusions since it learns from multiple instances of the target object.

2. KCF (Kernelized Correlation Filters): KCF is a popular tracking algorithm that operates in the frequency domain. It correlates the target appearance with a learned filter in the Fourier domain to efficiently compute the target position in subsequent frames. By using the properties of correlation filters and employing kernel tricks, KCF can handle appearance changes, scale variations, and partial occlusions. It is computationally efficient, making it suitable for real-time applications. KCF has been widely used in various tracking scenarios due to its efficacy and speed.

3. CSRT (Channel and Spatial Reliability Tracking): CSRT is an extension of the CSR (Channel and Spatial Reliability) tracker, designed to improve robustness in challenging tracking scenarios. It utilizes both color (channel) and spatial information to track objects. CSRT maintains separate filters for each color channel to handle variations in appearance, illumination changes, and occlusions. It also incorporates a spatial reliability map to weigh the contributions of different spatial locations, focusing more on reliable regions for tracking. CSRT tends to be more robust compared to traditional correlation filter-based trackers, particularly in scenarios involving significant appearance changes, occlusions, and cluttered backgrounds.

The video provided is in the media folder. You can see an example of the first frame in the next image:

Requirements

(libraries to install)

pip install numpy
pip install cv2
pip install argparse

Execution

Inside the folder in the CMD, you can execute the program. You need to determine which tracking algorithm you would like to use by writing the following:

python 02_tracking.py -t {MIL,KCF,CSRT}

So in the case of wanting to try the MIL algorithm you would simply execute:

python 02_tracking.py -t MIL

Then the first frame will appear (you can change this manually in the code by changing the vairable init_frame), and you can select the ROI (Region of Interest) to follow, in this case a car using the your mouse over the image:

After pressing enter, the tracking will start. A blue mask with transparency will appear and you will be able to see the car’s trajectory in the form of a line:

These processed frames will be saved to a video called car_tracking_<tracker_type>.avi

Note: if you want the video in full resolution, comment the lines 52 and 93 where the cv2.resize is performed.