Project 2 - A ERN Web App(SPA) featuring SocialNetwork X Photos , Google SaaS (Vision + Gemini APIs), Google PaaS w/"Database"

355 points

Points: 20 proposal, 335 (see rubric): ApplicationCompletion/function, Documentation+, Analytics

 

GROUP Work

 

Due dates -

• Proposal (GROUP)- App Purpose + interface due start of class March13 (you will need to research what Google Vision API + Gemini API can do)

• Application and Documentation, Presentations& Reports (GROUP) Due April 29@ 9:30am, April 29 or May 1 will present

***PEER Grading ---(INDIVIDUAL) Details to be discussed in class. DUE May 6, start of class (note: I will take off up to 40 points from your INDIVIDUAL score if you do not evaluate groups appropriately/fairly) ******* General Information on Peer Reviews

 

---

You will be working to create a SPA(single page application) Web application using NodeJS/Express/ReactJS that will feature the non-trivial use Google SaaS Computer Vision API +Gemini API on SocialNetwork X User Photos and Storage of results to the a Google Cloud "Database"(Datastore, Firestore) and deployed on a Google PaaS platform (Google Cloud Run or GAE). This Web application should be launched from your Project 1 static site. Note some approved SocialNetwork X sites inlcude Pintrest and Google Photos. Unfortuantely, Facebook is not possible as it requires a verified business account (the others required only business pages you can upload something to an AWS site to create the account). Remember you are to integrate the use of Google Gemni into your SPA.

 

IMPORTANT: If you want to do something else than Pintrest or Google Photos for the SocialNetwork X you must get approval from the intstructor first. The resulting application should be something that I can use and I do not need to be the owner/developer of the application. Make sure you do not need more than a business website (which you will need to create on AWS) to register to develop the applicaiton an have users.

 

 

 

GOAL/OBJECTIVES: Design a SPA web application that uses Images from SocialNetwork X(i.e. Pintrest, Instagram, Google Photos) and the Google Vision API & A foundational model from Google Gemini (how to in Webstorm-- CONSIDER adding user authentication to futher protect your API key) in a "unique" NON trivial way. It should be more than some kind of simple image processing application (w/ Google Vision API) and should be a non-trivial use of Gemini model. Be creative.

 

• CLOUD: Use your choice of Google Cloud Run or GAE as a Platform as a Service cloud to deploy your Web Application.

• FRAMEWORK: Use ERN (Express, React, NodeJS) +Cloud Database code to implement your Web Application as a Single Page Application.

• IMAGES CONTENT You will use JavaScript to perform Authentication to your SocialNetwork X site to allow a user to Login into their account to get their images (and any other social data you wish to use). This is something you will need to learn on your own.

• GOOGLE VISION API: You are required to learn about (on your own) the Google Vision API that is accessible via HTTPS request and use it to process images towards the "goal/objectives" of your application.

• GEMINI API: You are required to learn about (on your own) Google Gemini API that is accessible via HTTPS requests and use it in your application.

 

 

The Concept - System Stages AND the PROPOSAL PHASE

 

PROPOSAL PHASE: your group will need to submit a proposal to Canvas->DiscussionBoard->Project2 Proposals. That shows a interface mockup that discusses how the results generated using BOTH Google Cloud Vision + Gemni and stored/retrieved from the "Database" for a user are used in the interface of your application (the purpose of the application)? What is the output?

 

"GENERAL STEPS" (**** LOG all calls to APIs & insert Analytics****)

(STEP1+2) APP -> Retrieve User Photos [UP] from their Social Network X account.

(STEP 3)

    3a) From Retrieved User Photos [UP] keep only the ones that do not have their URLs listed as already processed resultings in entries inside "Google Database" --> [UP']

    3b) for remaining User Photos [UP'] cycle through each Image i and --> process using Google Vision API calls + Gemini API Calls--> output XXX --> store in some datastructure like Hash[Image i SocialNetwork X URL] = {user name, photo url, output XXXX]

(STEP 4)Process with Google Gemini [user name, photo url + outputXXX] --> Storage Record ZZZ + app output

(STEP 5) store Hash to "Google Database" [user name, photo url + outputXXX] --> Storage Record YYY+ZZZ

(STEP 6) Retrive ALL/MOST/??? Storage Records YYY for user--> PROPOSED App PURPOSE should determine what results you present.

 

Below shows you non-coding results of Steps 1,2 and 3 which process retrieve a user's SocialNetwork Xphotos using the SocialNetwork X's API on and the Google SaaS Computer Vision API+Gemni processing results on one of these photos. STEP 1: ask using SocialNework X's API for access to user's photos ---for example it MIGHT LOOK SOMETHING LIKE THE FOLLOWING where the user of the app ("me" id) this would be the request and the results Request: see the photos field being requested me?fields=id,name,photos,friends,gender,email      Result: see the list of photos with ids for them(note have cut out real id information) -- you get a JSON list with the information including photos as an array.   STEP 2: Then for each returned Photo in the JSON array above grap the ID (suppose it is 101XXXXX707)and make a request like 101XXXXX707?fields=link     and you will get the results: (in this example I am using a hypothetical SocialNetwork X--the way to make REST requests to your SocialNetwork X's API may be different. { "link": "https://www.SocialNetwork X.com/photo.php?fbid=1015XX3XXXX7707&set=a.101547382939824899ype=3", "id": "101XXXXX707" } You can use the URL to retrieve the image ---here is one of my images   STEP 3: lets pass this image to the Google Cloud Computer Vision API and see the results (okay this is not from code but, done manually) --> WHAT CAN YOU DO WITH THIS INFORMATION???     STEP 4: what will you do with Gemini?

 

A word about Google Gemini API VERSUS Vertex AI API

 

 

Analytics Implementation & Comparison

 

 

 

 

• Create a GitHub remote repository that hosts both your code and has a wiki to document creation, delivery, testing of your application.

• Required: use GitHub Issues for your grou work on this project. You must have a unique issue for each component / code file of your applicaiton

• SEE Rubric for tentative points

 

 

 

   Create shared Github repository (should be open for me and others to look at given URL) that contains BOTH the code and Documentation. Regarding the Wiki documentation you must have the following PAGES (keep these titles)

 

 

 

 

Page names in github wiki	Contents you should have
1) IIntro	Name your Application URL of deployed application (make a link)
2) Description	Screen shot showing history of commits 5 screenshots taken over time showing the changes in your Issue board as work was done on the project. List each major code file and describe its function.
3) Demonstration of Application working	SHOW A SEPERATE SCREEN SHOT showing results for each user in your team (minimum 2 people must be demonstrated) SHOW A SCREEN SHOT after adding a new image to the SocialNetwork X account and the resulting change in output when running the Application Screenshots of the data stored in "Google Database" Screenshots showing how the application uses the Google Vision API and what calls are made and example of returned results Screenshots showing how the application uses the Google Gemini API and what calls are made and example of returned results
4) "Google Database"	Part 1: You should discuss the setup of the chosen "Googel Database" (firestore or datastore) and structure of stored data. Part 2: show how data is added, updated and removed while using your app. Specify Querries done from the app.
5) Code Styling and Comments	Consisten CamelCase styling All code must be FULLY commented. This is a graduate class.... I expect comments everywehere throughout the code.
6) Code	THIS just refers use to URL of code in the github repository. This must include ALL code, HTML, images, media and any other supporting files. EVERYTHING!
7) Presentaton (pdf) + YouTube Video (must be unlisted not private)	USE This Slide Deck to create a slide deck for a presentation about your project. . ALSO convert the slide deck to a PDF and turn in this PDF (called Presentation.pdf). Record your team giving the presentation and turn in BOTH the mp4 (Presentation.mp4) and YouTube URL to this video (upload as unlisted)
8) Analytics+Logging	Post the PDF described in the Analytics+Loggin Report for this project on this wiki page in your github repostiory.

 

 

 

 

1. PROPOSAL (20 points) : Create github for project and create in the wiki a page called "proposal" that you document the purpose of your app and give a mockup of the interface(s) showing how the user would use the app and any results displayed. Turn in github URL to Canvas->Discussion Board->Project 2 Proposal
2. COMPLETION: Code and Documentation to be present on github. ALL documentation including Analytics report to on github AND Deliver github URL to Canvas->Projects->Project 2
   You must fully test (with IE, Safari and Chrome)!! NOTE: see above for details of what is needed in wiki pages.
3. DEMONSTRATION: Give demonstration of system working for instructor -- as requested. YOU MUST HAVE A YOUTUBE DEMONSTRATION OR WILL GET A 0 and it MUST BE COMPLETE
4. PEER EVALUATON (INDIVIDUAL) : YOU WILL BE GRADED ON DOING THIS WELL. This can account for up to 40 points taken off of your individual grade.