Realistic Synthetic Data Generation

We propose a novel approach to synthesizing images that are effective for training object detectors. Starting from a small set of real images, our algorithm estimates the rendering parameters required to synthesize similar images given a coarse 3D model of the target object. These parameters can then be reused to generate an unlimited number of training images of the object of interest in arbitrary 3D poses, which can then be used to increase classification performances.

A key insight of our approach is that the synthetically generated images should be similar to real images, not in terms of image quality, but rather in terms of features used during the detector training. We show in the context of drone, plane, and car detection that using such synthetically generated images yields significantly better performances than simply perturbing real images or even synthesizing images in such way that they look very realistic, as is often done when only limited amounts of training data are available.

Results

We have evaluated the quality of the synthetic images, generated by our method in application to three detection task, which include:

  • UAV detection
  • Aircraft detection
  • Car detection (tested on PASCAL VOC dataset)

We used the following models:

UAVs Aircrafts Cars

Optimisation of rendering parameters

Our system depends on the number of parameters which are defining the influence of different post-processing effects that we apply to the synthetic image. These parameters are usually hard to chose by hand that’s why we use and automatic system, for optimisation.

UAV example Aircraft example

We use simulated annealing approach for selection of the optimal values of parameters.

Performance on the UAV dataset

The dector is trained using both real and synthetic images, which allows it to robustly detect UAVs in various indoor and outdoor environments.

UAV flying indoors UAV flying outdoors

Detection results on the PASCAL VOC Car Dataset

 Missed Detections

References

A. Rozantsev; V. Lepetit; P. Fua : On Rendering Synthetic Images for Training an Object Detector; Computer Vision and Image Understanding. 2015. DOI : 10.1016/j.cviu.2014.12.006.
A. Rozantsev; V. Lepetit; P. Fua : On Rendering Synthetic Images for Training an Object Detector. 2014.