Tracking People using Multiple Cameras
Embed of video is only possible from Mediaspace, SwitchTube, Vimeo or Youtube
Reliably tracking multiple people using ordinary cameras is challenging, mostly due to the severe occlusions that occur when many people are involved. We tackle this problem by using several cameras, observing the scene from different viewpoints.
To exploit the resulting images, we developed a people detection algorithm called POM and a Deep-Learning based improvement, that use a generative model and Bayesian reasoning to estimate people’s positions in each individual time frame.
Given these estimates we then rely on a global optimization method known as K-Shortest Path (KSP) to generate trajectories.
Combining these two algorithms let us track people reliably and in real-time in spite of very significant occlusions. This technology now forms the basis for a video-based commercial product that tracks team-sport players.
Results
Embed of video is only possible from Mediaspace, SwitchTube, Vimeo or Youtube
Our Deep-Occlusion Reasoning method, which combines discriminative and generative models, allows us to perform 3D tracking, even in very crowded scenes.
Embed of video is only possible from Mediaspace, SwitchTube, Vimeo or Youtube
This video shows our recent tracking results using global appearance constraints – Basketball sequence.
Embed of video is only possible from Mediaspace, SwitchTube, Vimeo or Youtube
This video shows our recent tracking results using global appearance constraints – Soccer sequence.
Embed of video is only possible from Mediaspace, SwitchTube, Vimeo or Youtube
This video shows our recent tracking results using global appearance constraints.
Technology Transfer
Embed of video is only possible from Mediaspace, SwitchTube, Vimeo or Youtube
This technology was initially commercialized by PlayfulVision, an EPFL spinoff, which as acquired by SecondSpectrum in 2016. It is now being used to track NBA players during basketball games.
Source Code
- The C++ source code of our POM people detector is available under a GPL license on the Software POM page of our web site.
- The Theano-Python code of the Deep-Learning based version of POM is available on the Deep-Occlusion Reasoning Github .
- The C++ source code for the K-shortest path multiple object tracker used to generate the results shown on this page is available upon request for academic purposes. More information is available on the Software KSP page of our web site.
- The Multi-Tracked Paths (MTP) is another light weight implementation of the K shortest-paths algorithm for multi-target tracking, and is available under the GPL3 license.
- A Python wrapper for K-SP, is also available for download.
Data Set
Some of the multi-camera video sequences that we acquired for this project are available for download on the Data part of our web site.
More annotations available from third-partie https://bitbucket.org/merayxu/multiview-object-tracking-dataset
Press
www.reuters.com
news.cnet.com
www.ibtimes.com
www.tecca.com
www.photonics.com
www.genbetadev.com
techline.hu
index.hu
www.tgdaily.com
creep.ru
tecmundo.com.br
www.gizmodo.es
www.fayerwayer.com
feber.se
computervisioncentral.com
www.eurekalert.org
www.physorg.com
sciencebusiness.technewslit.com
rovvysaip.posterous.com
news.medill.northwestern.edu
futureoftech.msnbc.msn.com
xage.ru
www.proho.gr
www.gizmag.com
science.compulenta.ru
References
Formation of electron traps in semiconducting polymers via a slow triple-encounter between trap precursor particles
Science And Technology Of Advanced Materials. 2024-12-31. Vol. 25, num. 1, p. 2312148. DOI : 10.1080/14686996.2024.2312148.Data-driven LPV Disturbance Rejection Control with IQC-based Stability Guarantees for Rate-bounded Scheduling Parameter Variations
2024-12-16. 63rd IEEE Conference on Decision and Control (CDC 2024)Milano Congressi (MiCo), Milan, Italy, December 16-19, 2024.Fine Arts in Transition: An Operative Research and Teaching Approach between Analogue and Digital
Lausanne: EPFL Press, 2024-09-30.Validation and Application of Hysteresis Loss Model for HTS Stacks and Conductors for Fusion Applications
Ieee Transactions On Applied Superconductivity. 2024-08-01. Vol. 34, num. 5, p. 4704105. DOI : 10.1109/TASC.2024.3369584.Performance Comparison of Cables With Partial Transposition
Ieee Transactions On Applied Superconductivity. 2024-08-01. Vol. 34, num. 5, p. 4803205. DOI : 10.1109/TASC.2024.3367804.Quench Detection and Temperature Measurement With Fiber Optic Sensors
Ieee Transactions On Applied Superconductivity. 2024-08-01. Vol. 34, num. 5, p. 9001005. DOI : 10.1109/TASC.2023.3346855.Design of a Beam Separator for FLASH Electron Therapy Facilities
Ieee Transactions On Applied Superconductivity. 2024-08-01. Vol. 34, num. 5, p. 4401005. DOI : 10.1109/TASC.2023.3341895.Flow and Snow in Complex Terrain
International Union of Geodesy and Geophysics (IUGG), Berlin, Germany, July 12-16, 2023.Data-Driven Control Synthesis using Koopman Operator: A Robust Approach
2024-07-12
MaskCLR: Attention-Guided Contrastive Learning for Robust Action Representation Learning
2024-06-17. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, Washington, USA, June 17-21, 2024.Multi-Commodity Network Flow for Tracking Multiple People
IEEE Transactions on Pattern Analysis and Machine Intelligence. 2014. Vol. 36, num. 8, p. 1614-1627. DOI : 10.1109/Tpami.2013.210.Formation of electron traps in semiconducting polymers via a slow triple-encounter between trap precursor particles
Science And Technology Of Advanced Materials. 2024-12-31. Vol. 25, num. 1, p. 2312148. DOI : 10.1080/14686996.2024.2312148.Data-driven LPV Disturbance Rejection Control with IQC-based Stability Guarantees for Rate-bounded Scheduling Parameter Variations
2024-12-16. 63rd IEEE Conference on Decision and Control (CDC 2024)Milano Congressi (MiCo), Milan, Italy, December 16-19, 2024.Fine Arts in Transition: An Operative Research and Teaching Approach between Analogue and Digital
Lausanne: EPFL Press, 2024-09-30.Validation and Application of Hysteresis Loss Model for HTS Stacks and Conductors for Fusion Applications
Ieee Transactions On Applied Superconductivity. 2024-08-01. Vol. 34, num. 5, p. 4704105. DOI : 10.1109/TASC.2024.3369584.Performance Comparison of Cables With Partial Transposition
Ieee Transactions On Applied Superconductivity. 2024-08-01. Vol. 34, num. 5, p. 4803205. DOI : 10.1109/TASC.2024.3367804.Quench Detection and Temperature Measurement With Fiber Optic Sensors
Ieee Transactions On Applied Superconductivity. 2024-08-01. Vol. 34, num. 5, p. 9001005. DOI : 10.1109/TASC.2023.3346855.Design of a Beam Separator for FLASH Electron Therapy Facilities
Ieee Transactions On Applied Superconductivity. 2024-08-01. Vol. 34, num. 5, p. 4401005. DOI : 10.1109/TASC.2023.3341895.Flow and Snow in Complex Terrain
International Union of Geodesy and Geophysics (IUGG), Berlin, Germany, July 12-16, 2023.Data-Driven Control Synthesis using Koopman Operator: A Robust Approach
2024-07-12
MaskCLR: Attention-Guided Contrastive Learning for Robust Action Representation Learning
2024-06-17. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, Washington, USA, June 17-21, 2024.Formation of electron traps in semiconducting polymers via a slow triple-encounter between trap precursor particles
Science And Technology Of Advanced Materials. 2024-12-31. Vol. 25, num. 1, p. 2312148. DOI : 10.1080/14686996.2024.2312148.Data-driven LPV Disturbance Rejection Control with IQC-based Stability Guarantees for Rate-bounded Scheduling Parameter Variations
2024-12-16. 63rd IEEE Conference on Decision and Control (CDC 2024)Milano Congressi (MiCo), Milan, Italy, December 16-19, 2024.Fine Arts in Transition: An Operative Research and Teaching Approach between Analogue and Digital
Lausanne: EPFL Press, 2024-09-30.Validation and Application of Hysteresis Loss Model for HTS Stacks and Conductors for Fusion Applications
Ieee Transactions On Applied Superconductivity. 2024-08-01. Vol. 34, num. 5, p. 4704105. DOI : 10.1109/TASC.2024.3369584.Performance Comparison of Cables With Partial Transposition
Ieee Transactions On Applied Superconductivity. 2024-08-01. Vol. 34, num. 5, p. 4803205. DOI : 10.1109/TASC.2024.3367804.Quench Detection and Temperature Measurement With Fiber Optic Sensors
Ieee Transactions On Applied Superconductivity. 2024-08-01. Vol. 34, num. 5, p. 9001005. DOI : 10.1109/TASC.2023.3346855.Design of a Beam Separator for FLASH Electron Therapy Facilities
Ieee Transactions On Applied Superconductivity. 2024-08-01. Vol. 34, num. 5, p. 4401005. DOI : 10.1109/TASC.2023.3341895.Flow and Snow in Complex Terrain
International Union of Geodesy and Geophysics (IUGG), Berlin, Germany, July 12-16, 2023.Data-Driven Control Synthesis using Koopman Operator: A Robust Approach
2024-07-12