Exercise: Learn OBJECT DETECTION Training BY Example :PART 2

 

**** Watch this video (please excuse the non polished video as it was a late night recording) *********

 

 

You will get the notebook and data for the covidID_Mask github repository as follows: There are actually two ML sub-projects and yousing on the maskDetect part of covidID_Mask.

• maskDetect of the covidID_Mask:https://github.com/grewe/covidID_mask/tree/master/maskDetect- download and seutp as YOUR OWN colabs. This notebook has some additional nice cells on evaluation +which mayl be helpful for your Project 3. You will need to download data and upload it to your own Google Drive to host it.
  • this contains a colab using Google Drive to mount data (data also in github, you will have to put on your drive), 3 class problem ussing efficientDet D0 - face_no_mask, face_with_mask, face_with_mask_incorrec. t NOTE: As this is hosted on github you will need to clone and then copy to your google drive and setup your own colab to run it.
    
    
  • NEED TO DOWNLOAD AND UPLOAD TO YOUR DRIVE (Object Detection API Code): (My copy on my drive) "models" directory which is the Object Detection github repository clone at the time covidID_mask project was created.
    
    
  • DATA CONVERTED TO TFRECORD FORMAT FROM PASCAL VOC: This colab takes as input the data found at which is Pascal VOC formatted images and annotations and label_map.pbtext and then reads in the files and randomly selects the bulk for training and some for validation/testing. It then creates what are called TFRecord files which is a single file format of the data that can make training faster. It uses the TFRecord file as input in training.
    
    
  • SPECIAL NOTE on label_map.pbtxt and TFRecord records must have matching ids
    
    
    
• SPECIAL NOTE 1: one of the more challenging parts of this exercise has been the conversion of the detection model which is an EfficientDet D0 model. First read documentation and see if supported and try it. OTHERWISE:
  • The MaskDetectTflite notebook can be loaded in google colab and it did do conversion to TFlite efficientDet D0 model at the time of creation of this notebook. However, versions changed and we had to do experiementation to get it to work with a particular TF version. This is something that I except you will need to edit and fix if there are any problems. SO, this is one part of the Exercise that will involve you doing investigation, debugging and coding.
  • some old issues tracking on EfficientDet conversion potential support and possible solutions: issue1, issue 2
    
    
• SPECIAL NOTE 2: Another part of that will involve coding is the modification of the baseMLComputerVisionANdroid app to support the EfficientDet D0 model rather than the SSD Mobile detector it uses. You can find some code in the covidID mobile app github repository here that has the class to support EfficientDet D0 tflite models.
  
  
  
  
  
  

NEW (2022) ALTERNATIVE TO ABOVE - Using TensorFlow Lite Model Maker : It works by loading the tflite-model-maker python library (new 2022). so rather than using the colabs in the covidID_mask github, you could use the data from it along with the instructions on how to train a TFLite using this new Tensorflow Lite Model Maker.

• • MODELS SUPPORTED: It supports EfficientDet Lite modles for retraining ( I believe this only supports EfficientDet model architecture). Read the Obect Detection with Tensroflw Lite Model Maker example that includes a colab(local copy). The python API for the tflight-model-maker python library api is also documented on the tensorflow.org size
  
  
  • EXAMPLE: Here is an EXAMPLE on how to do this on custom data. It includes a step by step Codelab.
  • DATA: However at the bottom of a related colab that shows how to use the tflite-model-maker library for Object Detection you can see that they discuss that you can instead of what they do in the example(which is a CSV format) instead load PASCAL VOC formatted data which LabelImg can create . It ALSO shows how to freeze certain layers and other parameters that can be tuned for training. (NOTE: the covidID_mask dataset is both in the form of PASCAL VOC and TFRecord (these are too large to host on github and are instead on Google Drive).. Here is an example from covidID_mask in PASCAL VOC. However, you may have to edit the paths, etc in the xml files if the obejct_detector.DataLoaders uses the path specified in the xml file rather than the image_dir in the from_packal_voc() method call.
    
    

    load your data with a different data format

    The Model Maker library also supports the object_detector.DataLoader.from_pascal_voc method to load data with PASCAL VOC format. makesense.ai and LabelImg are the tools that can annotate the image and save annotations as XML files in PASCAL VOC data format:

    object_detector.DataLoader.from_pascal_voc(image_dir, annotations_dir, label_map={1: "person", 2: "notperson"})

  • FINE TUNING MODELS POSSIBLE IN ADDITION TO RETRAINING: It ALSO shows how to freeze certain layers and other parameters that can be tuned for training. (NOTE: the covidID_mask dataset is both in the form of PASCAL VOC and TFRecord (these are too large to host on github and are instead on Google Drive)..
    
    
  • WHY / WHY NOT THIS ALTERNATIVE? What is nice about this option is that the ModelMaker supports EfficientDet for both training and deployment to TFlite with proven example so this would eliminate potenial problems discussed in Special Note 2. You will need to alter the colab so that the data from covidID_mask repository is loaded in its PASCAL VOC format for training/validation/testing (see previous bullet point) and depending on its use of path from the xml annotations correct those too. Also, as discussed in the next point, it seems to have built in support ofor EfficinetDet models for andoroid deployment but, there is only a Kotlin ready android app example.
    
    
  • ANDROID USE: The Example includes pointer to a github that includes code for an android application using the created TFLite model--but in Kotlin NOT Jav. It uses the ObjectDetector api in Tensorflow. Here is the MainActivity class from the app that loads the tflite model into an instance of the class ObjectDetecor. The basics in Java would look like (and see for java) : However, until tried it is possible the support in Java is different than Kotlin.

• // Initialization from the modelFile like "modelfile.tflie" ObjectDetectorOptions options =     ObjectDetectorOptions.builder()         .setBaseOptions(BaseOptions.builder().useGpu().build())         .setMaxResults(1)         .build(); ObjectDetector objectDetector =     ObjectDetector.createFromFileAndOptions(         context, modelFile, options); // Run inference List<Detection> results = objectDetector.detect(image);

  //ANOTHER EXAMPLE ObjectDetectorOptions options =               ObjectDetector.ObjectDetectorOptions.builder().setMaxResults(5).setScoreThreshold(0.3f).build(); ObjectDetector objectDetector = ObjectDetector.createFromFileAndOptions(this, "salad.tflite", options );

 

check out the new colab I made that uses the mask data and Tensorflow Lite Model Maker It uses the data you will need to grab, unzip and put in your drive found here You can see it training below.   I will continue to work on it after it finishes with 5 epochs to see that it runs the evaluation and testing cells in the colab.    If you choose this alternative I am fine with you using the Kotlin Android sample they provide that uses the ObjectDetector class... again see the Exercise statement info.   If you choose this alternative I am fine with you using the Kotlin Android sample Tensorflow provides (see below) that uses the ObjectDetector class... again see the Exercise statement info.   with training for only a 5 epoch (too small) here are the results: