Project 2 - A ERN Web App(SPA) featuring SocialNetwork X Photos & Audio gathering, Google SaaS (Gemini APIs), Google PaaS w/"Database"

355 points

Points: 20 proposal, 335 (see rubric): ApplicationCompletion/function, Documentation+, Analytics

 

GROUP Work

UNDER CONSTRUCTION

Due dates -

• Proposal (GROUP)- App Purpose + interface due 11pm March 10 (you will need to research what Gemini API can do)

• Application and Documentation, Presentations& Reports (GROUP) Due April 30 @ 9:30am, May 5 + 7 will present

***PEER Grading ---(INDIVIDUAL) Details to be discussed in class. DUE May 8 @ 4pm (note: I will take off up to 40 points from your INDIVIDUAL score if you do not evaluate groups appropriately/fairly) ******* General Information on Peer Reviews

 

---

You will be working to create a SPA(single page application) Web application using NodeJS/Express/ReactJS that will feature the non-trivial use Google SaaS Gemini API on SocialNetwork X User Photos & Audio (recorded or collected from whatever source is appropriate for you app) and Storage of results to the a Google Cloud "Database"(Datastore, Firestore) and deployed on a Google PaaS platform (Google Cloud Run or GAE).

This Web SPA should be launched from your Project 1 static site. Note some approved SocialNetwork X sites inlcude Pintrest and Google Photos. Unfortuantely, Facebook is not possible as it requires a verified business account (the others required only business pages you can upload something to an AWS site to create the account). Part of the project is researching and meeting needs to access user's SocialNetwork X content.

Remember you are to integrate the use of Google Gemni into your SPA to process BOTH the Photos & Audio. Note the processing may be separate for separate purposes in your proposed application or it may be related. However, the use of photos & audio MUST make sense.

 

IMPORTANT:Regarding SocialNetwork X content, the use in your application should be something that I can use and I do not need to be the owner/developer of the application to access (ofcourse user authentication will be necessary). Make sure you do not need more than a business website (which you will need to create on AWS) to register to develop the applicaiton an have users.

 

 

 

GOAL/OBJECTIVES: Design a SPA web application that uses Images from SocialNetwork X(i.e. Pintrest, Instagram, Google Photos) and you also need to get audio (or record it). Both the photos and audio are processed using a foundational model from Google Gemini (how to in Webstorm-- CONSIDER adding user authentication to futher protect your API key) in a "unique" NON trivial way. Be creative. Audio and Photo processing may be highly correlate or have separate purposes in the application but, both must make sense and be non-trivial.

 

• CLOUD: Use your choice of Google Cloud Run or GAE as a Platform as a Service cloud to deploy your Web Application.

• FRAMEWORK: Use ERN (Express, React, NodeJS) +Cloud Database code to implement your Web Application as a Single Page Application.

• IMAGES CONTENT You will use JavaScript to perform Authentication to your SocialNetwork X site to allow a user to Login into their account to get their images (and any other social data you wish to use). This is something you will need to learn on your own.

• AUDIO CONTENT: This may be captured by the application or retrieved from a SocialNetwork. However, a set of fixed pre-stored audio files does not meet the requirement of this project.

• GEMINI API: You are required to learn about (on your own) Google Gemini API that is accessible via HTTPS requests and use it in your application. You are to use a Gemini model to process photos and to process audio.

 

 

The Concept - System Stages AND the PROPOSAL PHASE

 

PROPOSAL PHASE: your group will need to submit a proposal to Canvas->DiscussionBoard->Project2 Proposals. That shows a interface mockup (check out figma.com) that discusses how the results generated using BOTH Photos & Audio with Gemni and stored/retrieved from the "Database" for a user are used in the interface of your application (the purpose of the application)? What is the output?

ORDER OF CONTENT:

• App purpose/description

  MockUP Interfaces

  System Diagram

  Flow of Control Details (steps in processing w/ technical details - what you are doing/using)

   

 

"Flow of Control Details" (**** LOG all calls to APIs & insert Analytics****) WITH DIAGRAM (check out LucidCharts.com)

(STEP1+2) APP -> Retrieve User Photos [UP] from their Social Network X account.

(STEP 3) PHOTO processing

    3a) From Retrieved User Photos [UP] keep only the ones that do not have their URLs listed as already processed resultings in entries inside "Google Database" -->UserPhotosNew[]

    3b) for UserPhotosNew[] cycle through each image i and --> process using Google Gemini API Calls--> output XXX --> store in some datastructure like Hash[Image i SocialNetwork X URL] = {user name, photo url, output XXXX} & store in "Google Database"

 

(STEP 4) AUDIO processing

    4a) Gather Audio (either recording or social network, etc)     

    4b) check if audio is new --> for a new audio file if App recorded/created Audio store media in Google Cloud ->audio url AND process with Gemini --> output YYY --> store in some datastructure like Hash[Audio i Storage or SocialMedia URL] = {user name, audio url, output YYY} & store in "Google Database"

 

 

(STEP 5)Possibly Process with Google Gemini the outputXXX & output YYY from the photo(s) and audio(s) --> output ZZZ---> store in "Google Database"

(STEP 7) Use or Retreive Storage Records as appropriate for user--> PROPOSED App PURPOSE should determine what results you present.

 

A word about Google Gemini API VERSUS Vertex AI API

 

 

Analytics Implementation & Comparison

 

 

 

 

• Create a GitHub remote repository that hosts both your code and has a wiki to document creation, delivery, testing of your application.

• Required: use GitHub Issues for your grou work on this project. You must have a unique issue for each component / code file of your applicaiton

• SEE Rubric for tentative points

 

 

 

   Create shared Github repository (should be open for me and others to look at given URL) that contains BOTH the code and Documentation. Regarding the Wiki documentation you must have the following PAGES (keep these titles)

 

 

 

 

Page names in github wiki	Contents you should have
1) IIntro	Name your Application URL of deployed application (make a link)
2) Description	Screen shot showing history of commits 5 screenshots taken over time showing the changes in your Issue board as work was done on the project. List each major code file and describe its function.
3) Demonstration of Application working	SHOW A SEPERATE SCREEN SHOT showing results for each user in your team (minimum 2 people must be demonstrated) SHOW A SCREEN SHOT after adding a new image to the SocialNetwork X account and the resulting change in output when running the Application SHOW A SCREEN SHOT demonstrating how audio is gathered and the resulting chage in output when runing the Applicatio. Screenshots of the data stored in "Google Database" Screenshots showing how the application uses the Google Gemini API for both Audio & Photos and what calls are made and example of returned results
4) "Google Database"	Part 1: You should discuss the setup of the chosen "Googel Database" (firestore or datastore) and structure of stored data. Part 2: show how data is added, updated and removed while using your app. Specify Querries done from the app.
5) Code Styling and Comments	Consisten CamelCase styling All code must be FULLY commented. This is a graduate class.... I expect comments everywehere throughout the code.
6) Code	THIS just refers use to URL of code in the github repository. This must include ALL code, HTML, images, media and any other supporting files. EVERYTHING!
7) Presentaton (pdf) + YouTube Video (must be unlisted not private)	USE This Slide Deck to create a slide deck for a presentation about your project. . ALSO convert the slide deck to a PDF and turn in this PDF (called Presentation.pdf). Record your team giving the presentation and turn in BOTH the mp4 (Presentation.mp4) and YouTube URL to this video (upload as unlisted)
8) Analytics+Logging	Post the PDF described in the Analytics+Loggin Report for this project on this wiki page in your github repostiory.

 

 

 

 

1. PROPOSAL (20 points) : Create github for project and create a PDF of your proposal and link it on a wiki a page called "proposal. See above for proposal requirements that include the purpose of your app and mockups of the interface(s) showing how the user would use the app and any results displayed. Turn in github URL to Canvas->Discussion Board->Project 2 Proposal
2. COMPLETION: Code and Documentation to be present on github. ALL documentation including Analytics report to on github AND Deliver github URL AND DEPLOYED APPLICATION URL to Canvas->Projects->Project 2
   You must fully test (with IE, Safari and Chrome)!! NOTE: see above for details of what is needed in wiki pages.
3. DEMONSTRATION: Give demonstration of system working for instructor -- as requested. YOU MUST HAVE A YOUTUBE DEMONSTRATION OR WILL GET A 0 and it MUST BE COMPLETE
4. PEER EVALUATON (INDIVIDUAL) : YOU WILL BE GRADED ON DOING THIS WELL. This can account for up to 40 points taken off of your individual grade.