Project 1 Research Tutorial

individual work

points: 100 points (separately 20 points proposal). Willtake off up to 25 points for no or bad peer reviews

proposal due:11pm Sept 10 (note: topics/featured articles are first-come first-serve) will hold presentations and discussions in class(20 points)

tutorial due: start of class Sept 27

review of peer's tutorials due: 11pm, ONE WEEK AFTER THE LAST PRESENTER IN CLASS (take off up to 25 points)

**** You will be creating your tutorial using github pages built off of a github repository -- see this quickstart

Your are going to create an interactive Web-based tutorial that you will post for review by your classmates. This project has 3 phases:

Proposal (must be approved by instructor),
Project delivered
Peer Reviews

Proposal Submission Guidelines

(20 points) Proposal: Adequately given proposal posted on time on discussion board.

Select a topic from the list presented below (these topics will help with our Project 3). Topics are given out on a first come first serve basis. The list of topics is repeated on a canvas discussions board topic "Project 1 Proposals". You MUST look there and not select some topic that has already been claimed via another student's posting. The proposal must contain the following information (please number them as seen here) AND IMPORTANTLY the subject title of your posting should be the title of your topic e.g. "Computer Vision for Automatic Driving".

Title of topic (e.g. "Mobile: Previous work on Sign reading")
Author (your name)
1 paragraph detailing what you are going to talk about (this requires that you have done some research /reading on the subject).
table of contents - break down the subject into sub-topics (each could be a major link on your website). For example, you might have for "Mobile: Previous work on Sign reading"
1. Introduction and Problem Definition
  - what is the problem of mobile sign reading- a person using a mobile camera app is walking around (inside/outside) and the application using still images or video detects signs (what is a sign) and reads the writing (or sybmols ---think bathroom) and conveys this information to the user.
2. Sensor Possibilities
  - describe different sensors that people have used to solve this problem in the past --- i.e. still images from mobile devices or video?
3. Finding and Reading Signs
  -here describe (maybe choose 1 or a few) different techniques people have used to find find signs in an image and then how they might be read
  -give details on image processing algorithms used to accomplish this
4. Success and Failures
  -describe (the 1 or few you are comparing) system(s) success and where / why it (they) failed
5. The Challenges
  -I (meaning you) will give readers some sense of what the greatest challenges are (i.e. speed of user movement in enviornment and quickness of Computer Vision app response, sensor problems with missallignment, noise from enviornment, etc).
  -I (meaning you) will also give the results of an interview with a professor/student doing research in this field (contact one of the authors via email)
6. The Future and Conclusions
  -I (meaning you again) will let readers know about the research labs (CMU, etc) doing research in this area and what they are currently working on (you should send and email and contact professor in charge of lab to ask about it).
At least 2 sources you have read (preferrably online) on the subject and used to create the paragraph.

On-line Tutorial Requirements:

You are going to read the references you have found on your chosen topic and use this knowledge to create a tutorial that reviews this topic at least in terms of the papers you have read. In this way it is much like what we call a "review paper" which gives the reader knowledge about the state of the art - current trends in a topic. Now I don't expect you to be an expert the way an author of a "review paper" is. Rather you are going to teach your reader what you have learned about the topic and at least a few solutions/implementations (from your references).

As discussed above you are creating a online research tutorial/"website" using github pages. Some of the requirements for a good online research tutorial include:

(5 points) Format& Ease of Use: You must implement the tutorial as a web tutorial. That means HTML/CSS with supporting media.
(5 points) Mixed-Media: You must have a minimum of 5 pictures or diagrams illustrating the concept you are discussing. As appropriate include video or audio also. If you borrow images, you must reference in the legend where you retrieved them from. You must add voice to each of your individual web pages
(5 points) Organization: The tutorial should be organized. It SHOULD not look like a long paper simply poped-up on the web. You should have a nice intro page with an easy navigation scheme present throughout the site's pages.
(10 points) Originality: Your writing should be original. You can NOT paraphrase large blocks of text. If you need to paraphrase smaller items, you need to refer to the article in which the text comes from and surround the text with quotes. You should keep to a bare minimum the number of times you paraphrase others words.
(10 points) Length: Someone should be able to complete the tutorial within 30 minutes, and it should minimally take 15 minutes unassisted.
(5 points) Grammar: good grammar
(30 points) Level of Detail: This is a graduate level course and as such, I expect that you will delve into your topic at a sufficient level of detail for the audience to be able to understand how things are implemented or created and with possible reference to other articles (that you quote) be able to recreate or build the system(s) described. Includes Detail -overall (summary of concepts, results given--how well works) , Detail- tech algorithms/system details, Detail -applications discussed. Detail-limitations of system discussed. For example
- If you are discussing a topic that involves the development of software or algorithms, you should present a minimum of one algorithm or piece of code on the topic. You should discuss how the algorithm works and if there are alternatives compare them to others, summarizing with what you think (and others may think) are superior algorithms (and why). Also, give the user an idea when you might apply one technique over another.
- If you are discussing a topic that is hardware-based, you should discuss the various hardware options, the components of the hardware system(s), how the hardware works, the conditions under which the hardware system(s) should be used, the output format of the hardware. Discuss the limitations of the hardware and assumptions made about the working environment. Contrast and compare different hardware system(s) that may accomplish the same or similar (or different if appropriate) goals.
(5 points) Interactivity: You are to incorporate some kind of interactive activity - either a test/ quiz (you don't have to grade it) or some kind of special video of a demonstration (you create) or a slide show (you create) ,etc. If there is an existing implementation you can use that in a demo.
(5 points) Annotated Bibliography: You are required to have links in your on-line (or powerpoint) tutorial to an annotated bibliography. You must have a minimum of 5 different references (please note different pages on the same web-site do NOT count). Throughout your on-line tutorial, each time you wish to quote a reference, it should be represented by a number that is hyper-text linked to the annotated bibliography. In this bibliography page, you will have listed in order of appearance (hence the numbers) your references each followed by an annotation of the contents of the reference. Specifically, you must include not only the title, author, date, source, but also, a sysnopsis, and a reliability rating. For example, for the second reference you would have:

[2] "Java is Great", Lynne Grewe, PC Weekly, May, 1996., p.g. 21-33.

synopsis: this is a journal article that discusses the benefits Java brings to the programming world and contrasts it with other high-level languages.

reliablity: L. Grewe is an independent researcher and professor. PC Weekly is an popular computer industry magazine that publishes technical articles, mainly to a non-research but, technical audience.

(20 points) Presentation:
- You need to create a small power point to help you create a YouTube video that is a presentation using BOTH this powerpoint AND your github pages Tutorial.
- Your voice should be hear clearly as well as any text in the YouTube video should be easily readable.
- .You can use Zoom to record yourself giving the presentation (it is not necessary to show yourself in the video, but, audio is required).
- Please make your YouTube video Unlisted mode where anyone who knows the URL can access it but, does not have to be invited to it...(however, it can't be searched in youtube).

Grading Criteria:

Your grade will be determined through peer review. Here is the formula: average of points determined by your peers.. The instructor reserves the right to remove peer reviews and to review projects instead.

REVIEW - online with shared google doc (link will be made active later) MUST be invited to share... will setup in class. If you are absent you are responsbile for contacting me to get added appropriately) and it is your responsibility to email me before AT LEAST 1 WEEK BEFORE THE due date with your horizon email address to get the invitation. I prefer to do this in person during office hours for security validation reasons.

How to use Google Document FOR EACH STUDENT YOU WANT TO REVIEW

Go to Google Doc, say your name is Lynne Grewe and you are going to review the student "Patel" (last name) work.

Go to the "Patel" tab, and find your row with your last name on it...in this example we go to the Grewe row. NOTE: every person has a different colored row --this is to help avoid you editing the wrong row. So, in this example,

Student Abe, will review Patel by going to Patel tab and ONLY editing the row with the name Abe on it

Student Grewe, will review Patel by going to Patel tab and ONLY editing the row with the name Grewe on it

Student Grant, will review Patel by going to Patel tab and ONLY editing the row with the name Grant on it

Student Zan, will review Patel by going to Patel tab and ONLY editing the row iwht hte name Zan on it

************** IF YOU DO NOT UNDERSTAND THIS WE NEED TO MEET ***************

NEVER fill out anything in the RED boxes or in other peoples rows.

Deliverables (turn in times are start of class on the due date)

1)PROPOSAL[start of class Sept 6]]

turn in pdf to Canvas->Discussions->Project 1-Proposal

this IS FIRST COME FIRST SERVE....meaning if a prior (by time/date) posting has taken the topic you may NOT re-use that topic.

This means you should post your topic quickly and then go back and update your posting to include the full proposal. You don't want to have spent a lot of time researching a topic to find someone else has already taken it.

2)PROJECT [start of class Sept 27] turn in the URL to your github pages web tutorial AND the YouTube video of you presenting the tutorial. UPLOAD URL to YouTube video AND URL to the github pages Web tutorial in a SINGLE posting to Canvas->Discussions->Project 1-Final Tutorial

3)PEER REVIEWS 11pm, ONE WEEK AFTER THE LAST PRESENTER IN CLASS]: REVIEW OTHERS by looking at their YouTube Presentation & Web Tutorial botth posted on each student's canvas discussion board "Project 1 - Final Tutorial". Make your reviews for each person on their tab in the google document AND save your reviews locally in case we have a problem.

You will have approximately a week to go through ALL your fellow classmates tutorials and give a review. This review will be available for ALL to see and we will use the online shared google doc you were invited to share. If you did not get an invitation it is YOUR RESPONSIBILITY to email the instructor to ask for an invitation.
Note: if there are 15 students in class you will do 14 evaluation postings (1 for every student except yourself).
IMPORTANT: you must save a local copy on your machine of all the reviews you make of others --why? Sometimes people make mistakes and edit the wrong field in the document and we have to fix this.
IMPORTANT: if you do not evaluate adequately you will loose points on this.
we are using the shared drive document (I will only accept requests from a campus email). WATCH VIDEO that discusses how to use the shared google sheet for review. SAVE to a local file everytime you make edits to the document in case you need to recover information (remember this is shared and multi-user revision is not not possible per user changes in Google documents at this time).

Possible Topics:

IMPORTANT: You can NOT choose a topic that is taken already by someone else --who ever post's first on discussion board their proposal gets that topic.

IMPORTANT: choose a topic ideally related to what app you might develop for our last Project 3.

IMPORTANT: I DO NOT WANT ANYONE FOR PROJECT 3 and 1 to do simple OCR ---that is done and apps out there ---you can apply it to very specific tasks like sign reading etc.

going to propose reserarch related to Computer Vision that in some way will ASSIST One of the following groups:

LV = a blind or low-vision (not completely blind) person
S = senior citizen
P= someone with reduce physical mobility
A = an animal
C= children
AH = athletes or Health

Mobile: mHealth and Vision: using computer vision in mobile health (mHealth) applicaitions. Example stroke patient monitoring (facial changes, gait qualification, etc).
Mobile: Situation Awareness. Wearing a mobile device using computer vision to report to user what is in the environment around them (
Mobile/Rover: User following - Mounting a mobile device on a rover have it follow a person

But we are looking at reserch & how to make cheap/our own - https://www.theverge.com/circuitbreaker/2018/3/6/17086450/segway-rideable-robot-sidekick-loomo-indiegogo

Mobile/Rover Situation Awareness: Much like number 1 but, this time mounted on a rover rather than a person.

But we are looking at research & how to make cheap/our own -https://www.theverge.com/circuitbreaker/2018/3/6/17086450/segway-rideable-robot-sidekick-loomo-indiegogo

Mobile: Previous work on Cross Walk Guidance/ Navigation
Mobile: Previous work on Door Finding [LV, S, C, A?)
Mobile: Teaching Children to do X

Mobile: Previous work on use of 2D vision for obstacle detection on mobile phones (here you monitor the environment and tell user audibly when an obstacle is in their path) [LV, S, C, A]
MObile: detecting obstacles for a wheel chair???

Mobile: Computer Vision detecting falling: keeping senior citizens at home longer, but, using mobile devices to monitor using computer vision a seniors activites
https://www.ualberta.ca/science/science-news/2018/october/using-ai-to-protect-the-elderly
Mobile: Computer Vision and seniors: remembering where they left items (monitoring):
Mobile: Previous work on use of 3D vision for obstacle detection on mobile phones (will need to purchase special dual lens recording for 3D data generation and detect obstacles in person way when say walking and warn them audibly)
**** this would require purchasing hardware*****
Mobile: Previous work on use of 3D vision - NON obstacle detection applications (can this be applied to atheltes to watch them work or ????)
**** this would require purchasing hardware*****
caution: you would need to think of how 3D would benefit a low vision person ---for Project 3 you couldn't just have a 3D display of the 3D input data
Mobile: Previous work on Label Reading -- cans, packages, price tags
Mobile: Computer Vision to monitor activity : example monitoring senior activity at home and loggin it or using to determine if they need help, same for children or animals. http://alan.vision/publications/luo2017computer.pdf
Mobile: Computer Vision and Atheletes:
Mobile: Computer Vision and Animals --detecting animals on side of road for avoidance of hitting
Mobile: Computer Vision and Animals/Children -- detecting animal cruelty photos on internet (saving animals/children from abuse) --problem: data
Mobile: Computer Vision and Animals - detecting glacoma (or ??) from portrait images --problem: data?
Mobile: Computer Vision and Animals: detecting & counting wild animals - monitoring of wild animals.
Mobile: Vision & Farm Animals: monitoring/tracking animals - http://www.diva-portal.org/smash/record.jsf?pid=diva2%3A1269787&dswid=-4469
Mobile: VIsion & Pets: detecting the emotion of your animals --problem: data that is real/meaningful
Mobile: Vision & Pets: training/correction monitoring of bad behavior or restriction of animal entering area:

Mobile/Drone: Vision & Pets: can we make a automatic dog walking drone

Mobile: Vision& Pet & Sensors: for what??
Mobile: Vision & Stray Animals: detecting lost animals?? data?? https://www.researchgate.net/publication/319252971_CrowdPet_Deep_learning_applied_to_the_detection_of_dogs_in_the_wild
Mobile; Computer vision and Athletes - http://www.vap.aau.dk/cvsports/
http://vision.cse.psu.edu/publications/publications.shtml [AH]

Mobile: Previos work indoor navigation -- using vision --where the goal is to go from current point to point B.
Mobile: Previos work on Fashion Matching (help blind match clothes)
Mobile: Previous work on people detection (tell blind person number of people in scene)
Mobile: Previous work on scene understanding - not people detection(by this I mean more specifically describe contents of scene ---this is hard anything can be in a scene you could focus on
indoor furniture like chairs, tables, couches)
Mobile: Previous work on interpertreting facial expressions/emotion (can assist blind in "reading" emotions of others OR help people/children with certain mental disabilities)
Mobile: Previoius work on smile detection --very similar to pevious facial expressions/emotion but, focus only on smile
Mobile: Previouis work on use of headsets for augmented reality (visual enhancement etc but, with headset so 3D)**** this would require purchasing headset*****
caution: you would need to think of how 3D would benefit a low vision person ---for Project 3 you couldn't just have a 3D display of data---there has to be a purpuse
Mobile: Previous work on Electronic Sign Reading (like reading airport signs for take off /landing)
Mobile: how can use computer vision to recognize food on plate (for assisting blind, see Purdue U. research on plate monitoring for diet tracking as a hint)
Mobile: Previous work on detecting vehicles in scene
Mobile: Using Mobile vision to assit blind with small vehicle (bike, scooter) navigation (look to some of the mobile vehicle vision based tracking)
Mobile: Previous work on Sign reading (bathrooms, room signs)
Mobile: Picking out ripe fruit or rejected bad fruit using 2D color vision. (idea here would be for project 3 to select say a minimum of 4 fruit that you would assist user with this process).
Mobile: Combining Vision and Haptic (touch) -
caution: would require some significant arduino type work --hardware kit ---
like here suit that buzes in direction on shirt where computer vision senses some dramatic change
like replacing peripheral vision
Mobile: Face Recognition - blind can know who is approaching and phone can tell them who it is
Mobile: Recognizing Agressive actions - (like security applications, hard problem on phone)
Mobile: Tell who is lying - would require face detection followed by a lye detector (or done at same time)
Mobile: Reading body language - using vision to tell what the person is "feeling" based on their body posture (not facial expression specifically but, entire body)
Mobile: Assistance with Makeup Application for low vision
caution - is this even a topic???? how could you direct the person to apply makeup even?
AH: Detection of diseases
AH: Gait Imballance Detection
AH: Situation Awarness around Communal Diseases
AH: Illness Identification via Unusual Behavior/Movements
AH: MOnitoring of good sports "form"
Mobile: Magnification App with ZOOM Lens (this would require you purchase a lens and caution ---you would need to control not just display video do something interesting with it) --- this is tricky and not so much a research topic but, going towards an app idea ---and maybe too trivial you could not just display the magnified image --but, how could it be controlled/ used for a low vision person.

=======THE FOLLOWING ARE GENERIC AND NOT APPLICATION ORIENTED=========
Mobile: Previous work on creating apps for 3D mobile Google platform (not really blind focused but, could be???) **** this would require purchasing hardware*****
Mobile: Computer Vision Frameworks specifically for mobile platform (i.e. https://developer.qualcomm.com/mobile-development/add-advanced-features/computer-vision-fastcv, OpenCV on android/ios --how does it perform)
Mobile: Project Tango and how to use and applications (not really focused on blind and needs special hardware but, if someone wants to explore)
Mobile AND WEB Crowd Sourcing: letting say blind take pictures and other people tell what is in them and annotating and delivering in some kind of augemented speech reality --- don't know if this exits.
Related but not a topic ---here they are teaching a guidedog to make telephone emergency call

IMPORTANT: If a topic you is presented in our class materials your treatement of it must go significiantly beyond what we will be discussing