Project 2 - A ERN Web App(SPA) featuring SocialNetwork X Photos & Audio gathering, Google SaaS (Gemini APIs + Google ADK), Google PaaS w/"Database"

355 points

Points: 20 proposal, 335 (see rubric): ApplicationCompletion/function, Documentation+, Analytics

 

GROUP Work

UNDER CONSTRUCTION

Due dates -

• Proposal (GROUP)- App Purpose + interface due 11pm Oct 13 (you will need to research what Gemini API can do)

• Application and Documentation, Presentations& Reports (GROUP) Due Nov. 30 @ 11pm, Dec. 1 + 3 will present work to class

***PEER Grading ---(INDIVIDUAL) Details to be discussed in class. DUE December 4 @ 4pm (note: I will take off up to 40 points from your INDIVIDUAL score if you do not evaluate groups appropriately/fairly) ******* General Information on Peer Reviews

 

---

You will extend the ReactJS Single Page Application (App) developed in Project 1 into a complete ERN (Express, React, Node.js) web application. Rather than creating a new frontend, you will reuse and enhance the React App from Project 1 by integrating a Node/Express backend, Google Cloud services, Google Gemini APIs, Google ADK and a Google Cloud database. The "App" is a SPA(single page application) Web application using NodeJS/Express/ReactJS that will feature the non-trivial use Google SaaS Gemini API on SocialNetwork X User Photos & Audio (recorded or collected from whatever source is appropriate for you app) and also somehow creates a Google ADK Agent (possibly to further transform the Gemini results). Also required is storage of results to the a Google Cloud "Database"(Datastore, Firestore) and deployed on a Google PaaS platform (Google Cloud Run or GAE).

This Web SPA should be launched from your Project 1 website and should reuse the React App created in Project 1 as its frontend. Note some approved SocialNetwork X sites inlcude Pintrest and Google Photos. Unfortuantely, Facebook is not possible as it requires a verified business account (the others required only business pages you can upload something to an AWS site to create the account). Part of the project is researching and meeting needs to access user's SocialNetwork X content.

Remember you are to integrate the use of Google Gemni into your SPA to process BOTH the Photos & Audio. Note the processing may be separate for separate purposes in your proposed application or it may be related. However, the use of photos & audio MUST make sense. Also, you must create Agent(s) and use them in your SPA. Read a hypothetical example.

 

IMPORTANT:Regarding SocialNetwork X content, the use in your application should be something that I can use and I do not need to be the owner/developer of the application to access (ofcourse user authentication will be necessary). Make sure you do not need more than a business website (which you will need to create on AWS) to register to develop the applicaiton an have users.

 

 

 

GOAL/OBJECTIVES: Extend the ReactJS SPA developed in Project 1 into a complete ERN web application that uses Images from SocialNetwork X(i.e. Pintrest, Instagram, Google Photos) and you also need to get audio (or record it). Both the photos and audio are processed using a foundational model from Google Gemini (how to in Webstorm-- CONSIDER adding user authentication to futher protect your API key) in a "unique" NON trivial way. Be creative. Audio and Photo processing may be highly correlate or have separate purposes in the application but, both must make sense and be non-trivial.

 

• CLOUD: Use your choice of Google Cloud Run or GAE as a Platform as a Service cloud to deploy your Web Application.

• FRAMEWORK: (The "App") Use the React App developed in Project 1 together with Express, Node.js, and a Google Cloud database to implement a complete ERN Single Page Application.

• IMAGES CONTENT You will use JavaScript to perform Authentication to your SocialNetwork X site to allow a user to Login into their account to get their images (and any other social data you wish to use). This is something you will need to learn on your own.

• AUDIO CONTENT: This may be captured by the application or retrieved from a SocialNetwork. However, a set of fixed pre-stored audio files does not meet the requirement of this project.

• GEMINI API: You are required to learn about (on your own) Google Gemini API that is accessible via HTTPS requests and use it in your application. You are to use a Gemini model to process photos and to process audio.

  Google ADK: You are required to implement your proposed Agent Architecture and the Agent(s) should have meaningful use in the SPA.

   

 

 

The Concept - System Stages AND the PROPOSAL PHASE

 

PROPOSAL PHASE: your group will need to submit a proposal to Canvas->DiscussionBoard->Project2 Proposals. That shows a interface mockup (check out figma.com) that discusses how the results generated using BOTH Photos & Audio with Gemni and stored/retrieved from the "Database" for a user are used in the interface of your application (the purpose of the application)? What is the output?

During the proposal phase, your group should explain how the React App developed in Project 1 will evolve into the completed application.

The proposal should clearly identify:

• Which React components from Project 1 will be reused.
• Which new React components will be added.
• How the interface will evolve.
• How the backend integrates with the existing frontend.

 

ORDER OF CONTENT:

• App purpose/description - include a discussion of :

  • purpose of app
    
    authentication process for user

    how the Gemini API is used to process Audio + Video

    how Google ADK is used to create an Agent Architecture (and what this architecture is) to perform meaningful use in your SPA
    
    what information is stored in Firestore

• MockUP Interfaces

  System Diagram (including your Agent Architecture)

  Flow of Control Details (steps in processing w/ technical details - what you are doing/using)

   

 

A word about Google Gemini API VERSUS Vertex AI API

 

 

Analytics Implementation & Comparison

 

 

 

 

• Create a GitHub remote repository that hosts both your code and has a wiki to document creation, delivery, testing of your application. The React frontend from Project 1 should be used as the starting point for this project. You may reorganize and refactor the code as necessary, but the frontend should be extended rather than recreated.

• Required: use GitHub Issues for your group work on this project. You must have a unique issue for each component / code file of your applicaiton

  SEE Rubric for tentative points

  SEE Code structure below

   

 

 

 

   Create shared Github repository (should be open for me and others to look at given URL) that contains BOTH the code and Documentation. Regarding the Wiki documentation you must have the following PAGES (keep these titles)

 

 

 

 

Page names in github wiki	Contents you should have
1) IIntro	Name your Application URL of deployed application (make a link)
2) Description	Screen shot showing history of commits 5 screenshots taken over time showing the changes in your Issue board as work was done on the project. List each major code file and describe its function.
3) Demonstration of Application working	SHOW A SEPERATE SCREEN SHOT showing results for each user in your team (minimum 2 people must be demonstrated) SHOW A SCREEN SHOT after adding a new image to the SocialNetwork X account and the resulting change in output when running the Application SHOW A SCREEN SHOT demonstrating how audio is gathered and the resulting chage in output when runing the Applicatio. Screenshots of the data stored in "Google Database" Screenshots showing how the application uses the Google Gemini API for both Audio & Photos and what calls are made and example of returned results. Screenshots showing ADK use and Deployment of Agent(s)
4) "Google Database"	Part 1: You should discuss the setup of the chosen "Googel Database" (firestore or datastore) and structure of stored data. Part 2: show how data is added, updated and removed while using your app. Specify Querries done from the app.
5) Code Styling and Comments	Consisten CamelCase styling All code must be FULLY commented. This is a graduate class.... I expect comments everywehere throughout the code.
6) Code	THIS just refers use to URL of code in the GitHub repository. This must include ALL code, HTML, images, media and any other supporting files. EVERYTHING! Clearly separate the frontend and backend code. For example with the following (or similar) file structure: Project2/ │ ├── frontend/ React SPA │ ├── public/ │ ├── src/ │ │ ├── components/ │ │ ├── pages/ │ │ ├── hooks/ │ │ ├── services/ │ │ └── App.jsx │ ├── package.json │ └── ... │ ├── backend/ Express / Node │ ├── routes/ │ ├── controllers/ │ ├── middleware/ │ ├── services/ │ ├── models/ │ ├── app.js │ ├── package.json │ └── ... │ ├── README.md └── documentation/
7) Presentaton (pdf) + YouTube Video (must be unlisted not private)	USE This Slide Deck to create a slide deck for a presentation about your project. . ALSO convert the slide deck to a PDF and turn in this PDF (called Presentation.pdf). Record your team giving the presentation and turn in BOTH the mp4 (Presentation.mp4) and YouTube URL to this video (upload as unlisted)
8) Analytics+Logging	Post the PDF described in the Analytics+Loggin Report for this project on this wiki page in your github repostiory.

 

 

 

 

1. PROPOSAL (20 points) : Create github for project and create a PDF of your proposal and link it on a wiki a page called "proposal. See above for proposal requirements that include the purpose of your app and mockups of the interface(s) showing how the user would use the app and any results displayed. Turn in github URL to Canvas->Discussion Board->Project 2 Proposal
2. COMPLETION: Code and Documentation to be present on github. ALL documentation including Analytics report to on github AND Deliver github URL AND DEPLOYED APPLICATION URL to Canvas->Projects->Project 2
   You must fully test (with IE, Safari and Chrome)!! NOTE: see above for details of what is needed in wiki pages.
3. DEMONSTRATION: Give demonstration of system working for instructor -- as requested. YOU MUST HAVE A YOUTUBE DEMONSTRATION OR WILL GET A 0 and it MUST BE COMPLETE
4. PEER EVALUATON (INDIVIDUAL) : YOU WILL BE GRADED ON DOING THIS WELL. This can account for up to 40 points taken off of your individual grade.