Reading MV- Chapter 1

• Course Overview,    Introduction lecture , [Names,Applications, Illusions ]
• Unix Tips
• Group Collaboration (work is individual unless specifically denoted as group work)
  
• For Fun:
  • what is Computer Vision? and a video on applications

Work: start Project 1 - research tutorial

2 Creating Images

• Image formation lecture 1 [pinhole cameras and lenses] and video , Image formation lecture 2 [sampling and quantization] and video
• [Sampling and Quantization,Bits and Bytes, Lenses]

3 Our Visual System + Color

• Human Abilities lecture     video    [The Human Visual System ]
• Illumination/ Light. CVA- 2.6,Color Space lecture [basics ]

Fun:driverless car talk

4 Image/Video Formats

Reading - OPTIONAL Video: Chapter 2 of "Practical Image Processing in C" by C. Lindley, OPTIONAL Image Formats: p.g. 185-187,214-240 of "Practical Image Processing in C" by C. Lindley

• image formats      [image header example ]
• video standards
• display types

Begining Android
Exercise: Exercise A1 (see below)

5 Simple Image Operations

Reading - MV- Chapter 2

• Point-based Processing Lecture and video    [Point Operators]
• Histogram
• Binary Images

6 Software

• Arrays and Indexing (or see specific data structures/classes used in any package your plan to employ)

LAB 1: Learn more about OpenCV using these Google Colabs. INDIVIDAUL WORK start of class Sep 1

• REQUIRED - must show when called upon in class your Colab up and running:
  • THROUGH STEP 3: Google Colab (not mine): Python + OpenCV calls ( and you can also download the corresponding python notebook)
• OPTIONAL -another example
  • Google Colab (not mine): Simple example on Python and a few OpenCV calls

7 Probability+

Reading -CV-chapter 7, probability (local copy)

• probability basics and video, distribution basics and video, [more basics]
• more yet: MSE and crosscorrelation

8 Histograms, Equalizaton

Reading - MV- Chapter 4 (thresholding)

• Lecture on Histograms and Equalization and video     [Histograms,   Histogram Equalization]

Exercise 2: Learn more about OpenCV using Google Colabs. INDIVIDAUL WORK (NOT TURNED IN)

• REQUIRED - must show when called upon in class your Colab up and running:
  • continued from before...NOW show STEP 4: Google Colab (not mine): Python + OpenCV calls ( and you can also download the corresponding python notebook)

Project 1 - proposal presentation -week 3

More Begining Android

OpenCV & Android

READ: IP -chapter 1 (only creating our application)
Exercise: A2

9 Geometric Relationships:
Scaling/Sizing, Rotation, Mirror Images

Reading -CV-section 6.1, analytical (local copy)

• simple geometric operations      [basics ]
• math-tranformations: linear algebra lecture, part1, part2

10 Area Processing: edges and filters

Reading -MV- Chapter 3,5, OPTIONAL CVA-section 9.7 (edge), OPTIONAL - CV-chapter 8, linearfilters (local copy),CV-chapter 9, edgedetection (local copy)

• Area Filters lecture     [Neighborhoods,    Area Processing]

Exercise 3: Learn more about OpenCV using Google Colabs. INDIVIDAUL WORK

• REQUIRED - must show when called upon in class your Colab up and running: (not recorded)
  • continued from before...go over STEP 5&6: Google Colab (not mine): Python + OpenCV calls ( and you can also download the corresponding python notebook)
  • Add a new cell for a STEP 7 that performs Sobel Edge Detection and displays the output image
  • Add another new cell for a STEP 8 that performs Laplacian of Gaussian Edge Detection (LoG) and displays the output image
  • DUE -before start of class Sept 12 turn in to Canvas->Assignments->Exercise 3
    • the link to your colab AND a screen shot of STEP 7 & 8 running (show both the code AND the displayed output image)

11 Recognition : Deep Learning based techniques

• Training: Training Data (how much and what), What is Validation Data,     Tensorflow Training Options
  
  
• Tensorflow: tensorflow.org :watch Video  &Video2
  • Beginning Classification
    • Keras Getting Started training w/Keras+Tensorflow & do Google CoLab (w/ image classification of Fashion MNIST dataset built in tf.keras.datasets.fashion_mnist )
    • :training tutorial 2 w/image classification via download of image in tgz files,
    • Optional training tutorial 3
    • Optional Getting Started (numeric data not images):
      
      
  • Beginning Object Detection
    • TF2 Object Detection Released Blog
    • TF2: Object Detection Training (from command line)
    • Simple Colab tutorial for running in python
      
      
  • LSTM - video example
  • Mobile: mobile, setup on android, how to create your own android tensorflow model app
    • TFLearn: watch Video
      
      
• Google Cloud Vision API & Google Cloud Machine Learning (related to tensorflow), (optional: TF- chapters 3-5, 8)
  • FIRST: try the python based Google Cloud Vision API example (it will give you some familiiarity with the steps---even though you are doing it from an Android client not a phtyon client run in Google Cloud)
    
  
  
  

Deep Learning: CNN(classification) & part 2(object detection), & Learning/Training your network & RNN/LSTM & Attention to Transformers

• deep learning & Intro Video
• CNN overview
• Classification Output Logits to Softmax to Percentages
• Coco Metrics - Poor Performance
• Object Detection Post Processing
• Metrics: recognitiion rate/percision/recall 
• video understanding the math behind backpropogation for Fully Connected NN layers
• blog on how backpropogation in convolutional layers works. AND video 1 AND video 2
• Batch Size, Learning Rate & Algorithms
• Retraining on Colab
• ML Frameworks
  • video discussing frameworks and featuring deployment to hardware frameworks
  • compare 2022 Tensorflow & Pytorch: article (high end)
• Tensorflow & Checkpoints/Save-Load Models(tf w/keras)
• Media Pipe in Tensorflow Training Example
• Average Percision, mAP and More
  
• Recommendations for MultiModal Data
• Regression (not classification) - continuous variables for output (e.g. predict age of a person or their weight from an image)
  • Colab on MNIST dataset as a regression problem (output a number not a label).-note uses a secondary framework. (local notebook version)
  • Regression: Steering wheel angle for self driving car using Images as input
    • OLD (2016) Predicting Sterring Angle to keep on road (self-driving cars domain) from Images (regression example)
    • Another blog on steering wheel angle prediciton
    • Yet another blog on steering wheel angle prediction
    • Yet another example steerig wheel angle prediciton ---use Resnet + dense connected layers
    • FINALLY some REAL images example Colab +Tensorflow -does NOT use keras and is a simple network (and a local copy)

Deep Learning Lab Experiences:

1) Classification Colab Overview video Part 1 AND video Part 2

2) Detection Colab and Overview video -- NOTE this is using Google's AI Edge ModelMaker API which focuses on the creation of mobile ready model (can be converted to TFLite/LiteRT)

• [Optional, Old] TF2 Object Detection Training and their LONGish blog ---this is not using Google AI Edge
• [Again not using Google AI Edge] TF Object Detection Retraining of ResNet

3) Regression Colab

• [Optional] Beginning deep learning with tensorflow colab (NN + CNN) [ PART1-regression from features w/NN, PART2- classification w/CNN] (and local jupyter notebook version -w/ errors fixed) The code above in the "part 1 Boston Housing Price Prediction" AND "part2" has "errors" because the Tensorflow API changed since its creation. The first error is regarding the movement of tf.train.AdamOptimzer to tf.optimzers.Adam and the solution (search on error to find) shows how you have to modify the code for the newer versions of tensorflow). So, if you find code or get code samples you may be doing this kind of discovery and change. The 2nd error has to do with the keys to the hist[] output in the same part1 --lets see if you can fix that. (hint: print out entire array and see the keys). There will be other places in the colab it tries to access elements with wrong key name.
  

4) Multi Input Multi Output Regression Colab and Overview video

5) Tensorflow implementation of ViT (not training just network coding)

6) Vision Transformer Retraining Colab uses MoViNet for retraining for video activity recognition

7) Inference/Prediction in Python Colab and corresponding documentation AND read blog on inference from saved model and inference from checkpoint

8) Explainable AI - Understanding what your network and example : class activation mapping (visualization of areas of image that contribute more to the network decision/activation)

Google Cloud Vision API - coding: VIDEO part 1, VIDEO part 2 -[OLD] Project 3 with Google Cloud Vision OR Tensorflow

Tensorflow 2 w/ Keras

Tensorflow more on Object Detection API

ABOUT documents - notes to read

Useful Machine Learning Stuff/Tools

• Kaggle (datasets, competitions, resources -own colab like hosting) -Example (past) competition: Google Visual Relationships (bounding box +) Competition
• LabelImg (open source tool for labeling images in PASCAL VOC XML format) , VOTT , from microsoft). CVAT (from Intel) & CVAT Online , labelme (online tool from MIT) and local download labelme tool
  • About use of polygon rather than bounding boxes --> segmentation learning tasks
  • There are other commercial tools (e.g. from Roboflow) , supervise.ly (ai assisted)
  • CVAT allows for you to not only create labels but, to set attributes for them read blog on how to use and take a look at this
• Netron (visualizer ...tensorboard has some)
• Anaconda --easy-to-install package manager, environment manager, and Python distribution with a collection of 1,500+ open source packages with free community support. Anaconda is platform-agnostic, so you can use it whether you are on Windows, macOS, or Linux
• Python install link
• Jupyter Notebook
• JupyterLab is a web-based dev environment for Jupyter notebooks+ and you can optionally install this if you wish    and Video on JupyterLab w/Github
• JupyterLab debugging extension --how you can debug your notebooks.****NEW*****
• SERVICES for labeling (not free): Google labeling service, RoboFlow Pro assistive tool, more --search on your own
• Learning Python resources
  • Python @ w3schools,
  • one of many interactive online consoles (to practice your learning python) ,
  • go to safari index at library.csueastbay.edu and find a book like Learning Python by Mark Lutz

• Training Data Example Labeling
• Python for Training and Jupyter Notebook setup
• Data Augmentation Example (when you have "sparse"data)

Class Exercises - (all of the following exercises are done in your groups, only turn in one person in the group, MUST put everyone's name in the text submission

• (individual part 1/ group part 2) Exercise: Classification - Learn Training BY Example - (50 points - points may change to be determined)
  • PART 1- DUE BEFORE class Sept 24 (INDIVIDUAL WORK)
  • PART-2 DUE 11pm Sept 28
    
    
    
• (group) Exercise: Learn OBJECT DETECTION Training BY Example - (50 points - points may change to be determined) exercise: work on this in your project 3 groups
  
  
  • PART1 DUE BEFORE class Oct. 3.(30 points)
    
    
  • PART 2 - DUE BEFORE class Oct. 10 (30 points)- Turn in 2 Videos.
    
    
• (group) Deploying a model (SavedModel format) to Tensorflow Serving (OPTIONAL) - Learn how to use Tensorflow Serving and deploy the CLASSIFICATION model you created in PART1 (you may need to recreate it and create code in the codelab to save the model to a SavedModel form --see this modified version of the classification colab that does exporting to a saved model- look for the STEP X: Save the model cells ) to a web endpoint, Use the following Colab to deploy and exercise the deployment to create predictions via a Web REST call This Colab runs the endopint in the Colab VM and it will disappear when the Colab shuts off --what do you have to do to deploy permanently (discuss this). Show screenshots of it running.
  • Alternative - Learn how to train and deploy via REST call on AutoML. Show screenshots of it running + deployed
    
    
• (group) Retraining a ViT + exploration(vision transformer)
  • PART 1 Due 11pm class Oct. 19 (30 points)
  • PART 2 Due BEFORE class Oct 24 (30 points)
    
    
• (group- OPTIONAL) Exercise Deeper
  • Exercise: Learn by Example: LSTM for Video Activity Recognition: (turn in partial - results of training + tensorboard + do predictions from python) Turn in ideally deployment in Android App OR MINIMALLY find blogs/githubs/resources that show how to run live an LSTM model (with CNN feature extraction for each frame) using LSTM --- +30 extra points if get Android working
    
    
  • Training on CLOUD Exercise: Re deploy training for one of tdhe previous exercises (your same small group picks based on your Project 3 needs --either CNN or RNN based) on Google Cloud by launching a Virtual Machine using ML (50 points - points may change to be determined) You will need to read and follow official directions
    • we will split into 2 groups:
      • Part 1:(Training) Google Cloud Training
      • Part 2:(Run Time Deployment) Android a model (from exercise above) deployed to Cloud and receiving the result from an Android call (after incomming image processed by your canned CNN). I believe you will have to deploy using a Saved Model format (not h5) https://www.tensorflow.org/guide/saved_model. You will "deploy" using Tensorflow Serving .QUESTION (is this video a solution to getting TFlite to convert our LSTM--one group work on this----only for limited ops or for flow-control like RNNs?? OR look at For models with LSTM/RNNs, you can also try the experimental API OpHint to convert). ROADMAP saying LSTM will come 2019
        • Regarding deployment --- you can see me "starting this" for the lstm-- or look at the possibly easier console based method I started --- either way follow official instructions). Turn into Canvas->Assignments->Ex:Cloud . Create documentation helping other students to reproduce what you did as well as showing it working. Note it seems the AI Platform only supports 1.14 Tensorflow not 2.0 at this moment? ALSO -read here about Google Cloud class API from Java/Android
          
        • Android call to Deployed Google AI Platform Model (under construction)
  • If using Pytorch over Tensorflow (only allowed if functionality not present) - PYTORCH TRAINING COLAB WITH document AND Conversion to Mobile and Applicaiton with Pytorch Mobile detector model.
    

OpenCV & Android

READ: IP -chapter 1 and chapter 2 , Recommended : on Safari index on library.csueastbay.edu read chapter 2 of "OpenCV Android Programming By Example", by A. Mohammad - covers histograms and histogram equalization

Exercise: A3

Discuss Project 2

Project 2 - Computer Vision

12 Recognition (non deep-learning) : numerous techniques

Reading- Model-Based: MV- chapter 12, Statistical Classifiers: MV- chapter 13, 14, Neural Networks: MV - chapter 13, With OPENCV: IP-Chapter 3,4

• Intro,Recognition Algorithms - a sampling
  • Nearest Neighbor   intro video and     
    •  openCV implementation,     github implementation of a KNN for color identification and   colab using skilearn (GO HERE for data) --see KNeighborsClassifier
      
      
  • Statistical Classification Gaussian Mixture Models with Notebook;     simple color Image example, and ; see end for example
    
  • Neural Networks lecture and video understanding the math behind backpropogation
  • Support Vector Machines Video: AND The idea and Simple Math Explanation and Colab using Sklearn and hypothetical geometeric features for Maglinent/Benign cancer and Colab SVM explained & at bottom example with features=principal component analysis on aligned cropped small face images and Python code with HOG features + SVM
  • Bag of Words (based on Text retrieval ideas)
    • paper on topic
  • BoostingColor
  • Optional: Hidden Markov Models
    
    
• Applications lecture
• intro, more (web)
  • graph matching
  • interpretation tree search
    • demo
  • scene labeling
  • neural networks

Optional

• Do the colab on training a NN (local notebook copy)---simply show me that you have run the last cell of the colab (save a screenshot).
  • actually this is a CNN --but, we are going to ignore the Convolution part of it.
  • Note this code uses Cafe which is a framework for Machine Learning and the definition of the neural network used is in the file downloaded from web and you can see involves a 4 layer fully connected set of neurons at the end.
• colab with pre-built Haar Classifiers for facial features using OpenCV (local jupyter notebook version) AND to understand more abou Haar Cascading Classifier and Training your own Cascading Classifiers in OpenCV tutorial
  
  
• colab with GMM: Gaussian Mixture Models (local notebook version) - first part is done with feature vectors (imagine you have processed your images somehow into 2D feature vector) AND 2nd part does modeling of images of numerals -> performs a PCA (principle component analysis) to reduce dimension of input so dont do on ALL pixels values THIS serves as a "kind" of feature extraction -> GMM -> use the GMM to generate new images of numerals (instead of classification)
  • NOTE: this colab uses sklearn which is a simple machine learning framework for Python (has Nearest Neighbor, GMM, SVM, KMeans clustering and more)

Week 11

13 Image Noise and intro to the frequency domain

Reading - CV-chapter 8, chapter 10, nonlinearfilts (local copy), m(recommend/ not required Fourier Transform: CVA:section 8.6-8.7 )

• frequency lecture
• intro
• fourier
• sample code
• Noise lecture [kinds of noise and reductions ]

14 Binary Image processing

Reading -MV: chapter 8,9

• binary images and morphology lecture
• binary image: geometric feature lecture
• binary image processing, blob detection, labeling and more
  • geometric features

15 Features and Texture

Reading -Texture:MV-chapter 7 , Features and hough: MV- Chapter 10,11 , SIFT - local features (scale and rotation invariant) READ IP- Chapter 3 , OpenCV is ORB and matching with ORB , paper comparing different Local features for recognition in different viewing conditions(image transformations) (PAMI 2005) , OpenCV and SIFT explained. (in python but, same idea)

Class Lecture, Materials

• texture lecture
• feature detection
• hough transform

Exercise 4: More OpenCV using these Google Colabs. GROUP WORK (Not graded but, demoed)

• continue from YOUR exercise 3 and add a new cell 9 that performs your choice of a feature detector (corner points, hough lines, etc) --discuss and show it.

Features: OpenCV & Android -features

Reading: IP - Chapter 2, Chapter 3: SIFT & more , Chapter 4 - Cascade filters(HOG), Chapter Chapter 7 :OCR with KNN, OCR with Support Vector Machines (SVM)

Recommended : on Safari index on library.csueastbay.edu read chapter 3 & 6 of "OpenCV Android Programming By Example", by A. Mohammad - covers edges, shapes with Hough transform,corner detection, cascade classifier ( using Haar-like features and adaptive boosting)
FYI: there is an OCRTesseract class in OpenCV that does OCR relatively well that I recommend using if OCR is not a main topic you want to alter do research on. This chapter's reading is more about how to do KNN and SVM

16 Vision as Backend Process (cloud)

• OpenCV deployed with Docker onto Google Cloud (IaaS)
• OpenCV on AWS Lambda with Python deploy script (Paas)
• OpenCV on AWS Lambda with NodeJS (PaaS)
• Google Vision Library in cloud (specific tasks like ocr, object recognition, face recognition, label detection, not as image processing rich as OpenCV)
• Google Cloud Machine Learning

• LAB (done in class potentially finish at home): Show the VM deployed and run EITHER YOUR PROJECT 3 colab OR SOME PAST Tensorflow training based Colab for a few hours and show it results of training --email instructor by 11pm on 11/18 EACH person. NOTE: only one person in your group should be training the Project 3 Colab --so decide this amongst your group. Others will choose a past exercise colab to use.

  >>> turn in to Canvas ->Assignments -> Colab Running on Google Cloud VM

  1) screen shot of training after say 1 hour running with the GCE VM (Google Compute Engine VM), (show me the training cell results)

  2) screenshot of colab where hover mouse over connection pulldown showing you are connected to a GCE VM

  3) screen shot showing the VM in the Google Compute Engine VM conole window.

16 Segmentation and Fitting

Reading - MV- Chapter 12 , Fitting: CV-chapter 17, fitting (local copy) (chapter 18 for those who are interested...not covered), IP: p.g. 42-44 (previously assigned - contours). See also OpenCV and contours

• segmentation lecture Video     [segmentation notes]

17 Data Structures in Vision

• structures

18 Motion

Reading - Motion: MV- Chapter 20, Tracking-MV - Chapter 22 (surveillance motion & tracking application), optional advanced: CVA-section 10.2-10.3 optical flow

• Motion Part 1 lecture & Part 2 lecture
• Super naive approach ---image differencing --a first step
• Video - naive approach -- using OpenCV to do image difference, dtect contours --> towards motion detection
• video on using RANSAC fitting parameters to a model (any model, a transformation, motion esitmation, etc)
• paper on using RANSAC for camera parameter motion estimation
• The New OpenCV RANSAC algorithm - a blog
• SLAM - Simultaneous Localization and Mapping
• general &differential

19 3D Imaging

Reading- MV- chapter 16 ,

OPTIONAL: CV--chapter 12, (local copy), chapter 13 stereo (local copy), CV- chapter 24 range (local copy), CVA-section 11.3, Depth from motion with OpenCV (non android version)

• stereo vision
• sensing
• RANSAC for image point matching

20 Image Databases

Reading -Digital Libraries: CV-Chapter 25 , diglib (local copy )

• basic concepts

21 Applications in work - face detection & surveillance & towards self-driving cars - case studies from book

Reading - MV - Chapter 21, 22, 23

• general

22 Compression (go over on own)

• introduction lecture
• basics

Other

PyTorch Mobile from past students

Android

A.0 Android Misc.

Android *****NOTE: Some of the links will take you to Mobile Prog. OR Software Engineering website*****

• Android Developer Fundamentals
• Android Developer Website
• Android Studio
  • Create a project (e.g Empty Views Activity - for simply "empty" interface) AND run your project
  • Testing and JUnit creation
•  Firestore with Android
• Android SQLlite Database: as Video

Android Studio:

• Android Studio - Download

  Android Studio - User Guide

  Android Studio +Git + GitHub AND Android Studio and .gitignoreJUnit testing in AndroidStudio And AndroidStudio Generate->Test tool demonstrated (works same for Kotlin/Java)

   

Android

A.1 Android

Android *****NOTE: Some of the links will take you to Mobile Prog. website*****

Learning Labs/Experiences  
LAB:A.1 -1 : [Extensive option] Build Your First (Kotlin) Android App

                  [Shortcut option] Create a project (e.g Empty Views Activity - for simply "empty" interface) AND run your project AND Testing and JUnit creation

• Example Kotlin/Java APP

LabA.1-2: OpenCV + Android and video [Uses OpenCV for image capture] (on emmulator you may need to have cameras set to emulator not webcam --or use physical device as suggested)

Experience: A.1-3: Follow Media Pipe + Android Video AND Android+MediaPipe For Classification example (Github hosted code , Note: MediaPipe is part of Google AI Edge) and to create an Android Classification App.

• AND android studio + Media Pipe setup
  
  

General Overview (Java only Based):

• Beginning Android and How to create an Android Project - asPowerPoint alone,    as MP4 Video OR as YouTube
  
  
• LECTURE: START Programming: (HOW to Create a Project in Android Studio) -as Powerpoint, , as MP4 Video OR as YouTube

***Demo - overview of using Android Studio: MP4 Video OR as YouTube (search for official documation, more up to date demos)

Emulator and AVD Manager and Running on a Real Device  

• Android Emulator, AVD Manager and how to creat an AVD
  • AVD Sample Configs (give me your favorite to add)
  • can take a long time to launch emulator ---snapshots?
• How to Debug/Run on a Physical (real) Device
  • Install APK on Physical Device video demo (from previous student ) YouTube

SDK Manager

• Android SDK manager
  
  

Loading Existing APK, Pushing/Pulling Files

• Loading Existing Android Project (what you need and how to do it)
• Loading Existing APK (application)
• Pulling/Pusing File from/to Device in Emulator

Exercises:

• Exercise A1:(not recorded) Install on your laptop the AndroidStudio IDE and Android SDKs, Create a Hello World code. app. Demonstrate working in BOTH an emulator AND on an Android Physical device DUE -before start of class Sept 5 turn in to Canvas->Assignments->Exercise A1 a screen shot of it running on the emulator(15 points)AND a second screen shot of it running on the physical device (10 points). I do not want the code
  • TIP: Leverage the Android Get Started CodeLab
    

Useful Apps / Tools (including important SCREEN RECORDER)

A.2 Android the Interface (Activity, Layout and Views) - older

LECTURE: more ideas and review of Android Activity - as Powerpoint alone, as MP4, as YouTube

LECTURE: Android Interface - as Power point, as MP4 , as YouTube

A.3 Android and event handling and intents - older

LECTURE: Android more on Event Handling - as Power point, as MP4 , as YouTube

A.4.1 Android and Android-based Image Capture with Tensorflow MediaPipe ML processing

EXPERIENCES - Go through tutorial and run code given to you ---try to understand it -- SEE Experience: A.1-3 (above)

1. Image Classification w/ MediaPipe (using model.tflite EfficientDet model provided in repository)
2. Object Detection w/ MediaPipe (using model.tflite EfficientDet model provided in repository)
3. For Fun ONLY - Image Generation (text to image) w/ provided model

A.4.2 Android and Android-based Image Capture with Tensorflow ML processing (Older - now use MediaPipe)

Alternative image capture--direct from Android SDK

• LECTURE: Android using the Camera lecture using Android API, as MP4, as YouTube
• LECTURE: Going over the baseMLAndroidAppComptuerVision repository code example (YouTube video overview of code) that uses Tensorflow ML for both Classification and ObjectDetection.
  
  
• Camera Issues and Fake Image
• Camera api 2
• READ latest on developer
  • Basics Overview
  • Building Your Own Camera App
  • Newer android.hardware.camera2 (only added API 21) and tutorial on camera2 api
• Web pages:
  • Example Using Camera (android 4.0 or greater)
  • Example Using Camera -built into App
  • Example Using Camera 2 api (api 21 or greater)

A.4.3 Android and OpenCV centric frame capture featuring OpenCV calls (older, see example A.1)

OpenCV API for version 4.0.1 (if you are using a different version search for api online)

3rd party OpenCV videos: (some desktop not android examples) -- showing the power of OpenCV

• object tracking
• hand recognition
• face recognition
• face detection
• image processing and another image processing
• SIFT operator
• OCR
• Sodoku Solver
• Object Recognition
• with a "robot"
  
• for the blind: color detector reading signs money reading
  crowd sourcing + "be my eyes" everything
• many more ---search "OpenCV Android XXX"

Non-free OpenCV extesions/modules

Some of the OpenCV modules are no longer free in later versions of OpenCV

• Incorporating NonFree Module (opencv3* has non-open/free modules , some free for research)
• word document showing steps for OpenCV3.*
• NOTE: ALTERNATIVE for SIFT/SURF (not free) that comes part of standard OpenCV is ORB and matching with ORB

Camera Capture in Android using OpenCV

VIDEOS:

1) Discussion of OpenCV w/ Android setup and code MP4 YouTube
2) Demo OpenCVAndroid proj. Create from Scratch:MP4 YouTube
3) Demo - overview of using Android Studio: MP4 YouTube

Coding Basics with OpenCV camera capture

• Create Project OpenCV Android & Code Explained (using OpenCV Native Code)

• How to Setup Android Project for Debugging that includes OpenCV Native Code

• Code Example Extended (adding a spinner and image processing) EXPLAINED

OTHER

• Tips

• Writing your Own pixel manipulation algorithms and Speed issues

• Special Note on running on Device
  

• Setup for using Tesseract (OCR) with OpenCV
  
  
• Samples Downloaded with OpenCV:

• optional: consider running Sample – image-manipulations – this example demonstrates how OpenCV can be used as an image processing and manipulation library.

Exercises - even though these are not graded they must be done and demoed by due date to succeed in your project work.

• Exercise A2(not recorded): First, l ook but do NOT dothe following java basedcodelab. Next, look at the vision quickstart sample from Official MLKit GitHub which is taken from the Google Vision API Sample Application page. Next, use this to modify your previous exercise or find online an alternative App that is a Kotlin/Java blend that performs Object Detection using a predefined supplied mode. Consider using the sample you downloaded when you were doing Experience: A.1-3 (Here is an older Java only version --do not use -- base Android App w/ML ). Use your application for part 2 of Exercise: Learn OBJECT DETECTION. listed above. .
  • NOTE: the sample Java code above works for an SSD Mobilenet model. Remember you are looking for a Kotlin/Java hybrid (why- to make the Android part more "modern"). You will need to make modifications on the code to accomodate other models. See the following and an example that has code to support an EfficientDet-D0 model (again Java based).
    
    
    
• Exercise A3 (required, not recorded): THis exercise is to explore the use of Google Vision API
  
  • GOOGLE CLOUD VISION SERVICE- Add to your aExercise A2 pplication the ability to call the Google Cloud Vision labeling service using HTTP request (as shown in lecture and you will have to use Google Cloud Vision documentation), https://cloud.google.com/vision/overview/docs/get-started

A.5 EXTENDED Android and Git

create new github repository and commit from Android Studio

REST OF MATERIAL WILL NOT BE COVERED BUT HERE FOR YOUR INTEREST

Wavelets

Reading

Class Lecture, Materials

• basics

Speech

Class Lecture, Materials

• speech interfaces

Fuzzy Image Processing

Reading

• CVA-Chapter 16

Camera Calibration

Reading

• CV-section 6.3-6.4, analytical (local copy) ,
• CVA- Chapter 6

K.1 Kinect

Kinect

• Kinect for Windows Sensor Overview
• Microsoft Kinect Getting Started (for version 1.8) ( this will reference how to dowload SDK, runtime tools and setup your sensor)
• Visual Studio and Microsoft Visual Studio Online for Code Versioning, Collaboration and Cloud Hosting
• Kinect Developer Site (and SW)
• Developer site (note we will use kinect for windows and 1.8 version of sdk and runtime)
• Tutorials
• Code Samples
• Version 1.8 general info
• Exercises:
  • Ex K1: Download Kinect Studio and Play --- recorder showing color, depth
  • Ex K2: Download Code currently the C# sample called "Skeleton Basics -WPF C#" , see http://www.microsoft.com/en-us/kinectforwindows/develop/sample-code.aspx or if link has changed search ON MICROSOFT page for it. Turn in screen shots of running
    
    • this will perform skeleton tracking and display it. Alter the code to write the Skeleton Tracking ID (you will need to read about Microsoft.Kinect.Skeleton class ---see http://msdn.microsoft.com/en-us/library/dn791989.aspx)