Tracking People using Multiple Cameras
Reliably tracking multiple people using ordinary cameras is challenging, mostly due to the severe occlusions that occur when many people are involved. We tackle this problem by using several cameras, observing the scene from different viewpoints.
To exploit the resulting images, we developed a people detection algorithm called POM and a Deep-Learning based improvement, that use a generative model and Bayesian reasoning to estimate people’s positions in each individual time frame.
Given these estimates we then rely on a global optimization method known as K-Shortest Path (KSP) to generate trajectories.
Combining these two algorithms let us track people reliably and in real-time in spite of very significant occlusions. This technology now forms the basis for a video-based commercial product that tracks team-sport players.
Our Deep-Occlusion Reasoning method, which combines discriminative and generative models, allows us to perform 3D tracking, even in very crowded scenes.
This video shows our recent tracking results using global appearance constraints – Basketball sequence.
This video shows our recent tracking results using global appearance constraints – Soccer sequence.
This video shows our recent tracking results using global appearance constraints.
- The C++ source code of our POM people detector is available under a GPL license on the Software POM page of our web site.
- The Theano-Python code of the Deep-Learning based version of POM is available on the Deep-Occlusion Reasoning Github .
- The C++ source code for the K-shortest path multiple object tracker used to generate the results shown on this page is available upon request for academic purposes. More information is available on the Software KSP page of our web site.
- The Multi-Tracked Paths (MTP) is another light weight implementation of the K shortest-paths algorithm for multi-target tracking, and is available under the GPL3 license.
- A Python wrapper for K-SP, is also available for download.
Some of the multi-camera video sequences that we acquired for this project are available for download on the Data part of our web site.
More annotations available from third-partie https://bitbucket.org/merayxu/multiview-object-tracking-dataset