Data Augmentation Example with TensorFlow Lite (in iDrone)

 

Training Data Augmentation (in Object Detection API)

Training Data Augmentation refers to the generation of new training data samples from existing samples by augmenting or modifying them.   There have been numerous proposals in literature on how to do this. With regards to SSD MobileNet Architecture in Object Detection API a few techniques are possible and described.    The first is called ssd_random_crop and comes from the original SSD paper [1] which attempts to help alleviate the difficulty of trying to detect objects at different scales.   Consider Figure 1 which show people at different distances and hence scales.   While your training data might primarily consist of the people at the middle scale you could generate samples from them and greater and lesser scales.    The problem of detecting objects at different scales is hardest when the object is relatively small (say less than 10% of width/height of image).   As discussed in [1] a technique referred to as “random cropping” involves changing the scale of the training image and also cropping.   Figure 2 shows some results of performing different kinds of data augmentation including and shows the idea of scaling.    Cropping is illustrated in Figure 3 which in essence crops different parts of the object for training.

Figure 1:  People exist at different scales



Figure 2:  Different kinds of data augmentation performed.   Blue - scaling.  Green - Horizontal flip.

Figure 3: Illustration of cropping used in data augmentation

In[2] a discussion of the dangers of random cropping is given and a solution of selectively applying cropping is given.  This is just one of the many derivatives of the basic data augmentation schemes that have been discussed in literature.

In the Object Detection API the use of random cropping for data augmentation in SSD MobileNet architecture is stipulated in the pipeline.conf file as shown in Table 1. 

data_augmentation_options {     ssd_random_crop {THIS is about scaling the data and then cropping to help creating more scaled versions of the object --I think we should keep the original SSD paper shows an increase in mAP of 2-3%      }   }

Table 1:  Random Cropping specified for use in SSD architecture, part of training_config: in pipeline.config

Another form of data augmentation commonly used is flipping as illustrated in Figure 2.   Unfortunately, this is a dangerous kind of data augmentation for our task of user direction identification as shown in Figure 3.  Currently, we are only using the ssd_random_crop data augmentation to help with scale.


Figure 3: Data Augmentation of Horizontal Flipping not allowed for User Heading Detection (a) original image, here you see labeled appropriately as West heading user.  (b) if perform a horizontal flip for data augmentation the user label of West is no longer appropriate and instead East would be the true label.

data_augmentation_options {     random_horizontal_flip {  BAD for us to do --as we have different classes (E to W) with a flip....GET RID OF     }   }

REFERENCES

[1] https://arxiv.org/pdf/1512.02325.pdf 

[2] Domain-Specific Data Augmentation for On-Road Object Detection Based on a Deep Neural Network Hui Eun Kim1 , Youngwan Lee2 , Hakil Kim1 , Xuenan Cui1 * 2,  2017 IEEE Intelligent Vehicles Symposium (IV) June 11-14, 2017, Redondo Beach, CA, USA