CPS Research

You will need to do research on a number of topics including

• System design (suggestion using Kinect) and calibration (finding the floor)
• Data Gathering from System
• Human Detection and Tracking
  • Using Kinect: Real-Time Human Pose Recognition in Parts from Single Depth Images. Shotton, et al, CVPR 2011.
• Detection of Fallen Person
• Response

 

 

A few Research and System Questions to answer

• how do you calibrate the RGB to Depth data that comes out of the Kinect sensor
• where is person or persons in scene
• will you track the person(s) (motion detction) OR just look for non (or little) moving person(s) (this somewhat implies the stop of a detected motion)
• how will you detect a person is on the floor
• how will you determine where the floor is (suggestion: system configuration/calibration)
• how will you determin if a person needs help on the floor
• can the person turn on and off the detection system (what if they want to lay on floor and watch TV or do exercises).

 

 

Kinect Links ( YOU MUST FIND YOUR OWN....these are to start ---ones I quickly found)

Other Computer Vision Courses experimenting with Kinect

• http://www.cs.cornell.edu/courses/CS7670/2011fa/lectures/zhaoyin_kinect.pdf
• http://golancourses.net/2011spring/projects/project-3-interaction/
• Student Project fog simulation,etc. (uses middleware OpenNI too with Kinect)
• Kinect Programming for Computer vision course

 

Getting Data from Kinect --- color, depth maps

• RGB-D Mapping: Using depth cameras for dense 3D modeling of indoor environments. Henry, Krainin, Herbst, Ren, Fox. ISER 2010.
• How Kinect gets its 3D measurements (Kinect range operation = Operation range: 0.8m~3.5m)
• Book chapter covering how to capture using OpenNI middleware a picture using Java and Kinect and storing in a data structure so can be processed by JavaCV (Java version of OpenCV)

(Online) Calibration Toolkit for Kinect - relating the Color to Depth images

• http://www.ee.oulu.fi/~dherrera/kinect/
• Article showing the registration

 

Kinect Developer sites

• OFFICIAL product site
• MS SDK developer site
• OPENNI developer site
• http://developkinect.com
• resources from wiki
• http://www.kinecthacks.com/,
• http://kinect.dashhacks.com/
• http://www.kinecthacks.nl/,
• http://kinect-hacks.net/
• http://hackaday.com/category/kinect-hacks/
• http://www.engadget.com/tag/kinect,hack
• Search also on YouTube

Sample Computer Vision Kinect Systems

• Blob Detection using Kinect (video)
• Posting of Numerous Systems
• Students doing a Computer Vision Course
• L. Schwarz, D. Mateus, N. Navab: Manifold learning for ToF-based human body tracking
  and activity recognition. British Machine Vision Conference (BMVC), 2010

 

 

PART of the beginning of this Project is to find research or ideas on the following topics

Floor Detection

• Using 2D camera input only: this is much harder then if you have 3D depth. Will you try to create a depth map from multiple images or how? How will you get these (2 cameras)? Will you instead ignore 3D and use color or texture to classify if something is a floor.
• Using 3D data input: this is easier, you can recover the 3D equation of the plane (assume single level floor) that is the floor by having user specify 3 points in the image corresponding to the floor that you have XYZ information on. Find research on how to get parametric equation of a plane from 3 points.
  
  
  

Person Detection and Tracking or Any moving object

• Will you track anything moving and then if "large enough" and it comes to rest investigate only then if it is a person? Or will you find the people in the scene and track them.
• Instructor IDEA: use Skeleton tracker to find joints....big question ( is microsoft's skeleton tracker so much better you need to program with it or not)????

 

 

Person on Floor Detection

• How if you see a person (or a moving object) do you know it is LAYING on the floor?
• Instructor IDEA: use Skeleton tracker if all joints (or some subset) are on floor (in a parallel and close plane to plane of floor) then person is on the floor.....big question ( is microsoft's skeleton tracker so much better you need to program with it or not)????