Google App Engine -- Logging

• DEFAULT LOGGING DONE BY GAE: App Engine logs all incoming requests for your application, including application requests, static file requests, and requests for invalid URLs (so you can determine whether there is a bad link somewhere).

  • logs the date and time

  • IP address of the client

  • URL requested (including the path and parameters)

  • the domain name requested,

  • the browser's identification string (the "user agent"),

  • the referring URL if the user followed a link,

  • HTTP status code in the response returned by the app or by the frontend.

  • TIMMING:

    • amount of time it took to handle each request

    • amount of "CPU time" that was spent handling the request with size of the response

      • The CPU time measurement is particularly important to watch because requests that consistently consume a large amount of CPU may be throttled, such that the CPU use is spread over more clock time. This can effect your $$ charges
        

Logging from your code

• occurrence of notable events and data using a logging API.

• Logging a message associates a line of text with the request that emitted it, including all of the data logged for the request. Each message has a log level indicating the severity of the message to make it easier to find important messages during analysis.
  

App Engine "Levels" of Logging

• debug

• info

• warning

• error

• critical.

How to View your Logs

To access the Logs Viewer:

1. Open your project in the Developers Console.

2. Select Logging
   
   
   
   

3. If your app runs in a Managed VM and writes custom logs, select the desired log name.

• Here are the results

• You can also download your log data for offline analysis and recordkeeping.
  
  

GAE: Logging in NodeJS Express

In Google App Engine, logging is an essential part of monitoring and troubleshooting your applications. Google Cloud provides integrated logging via Google Cloud Logging (formerly Stackdriver Logging), which captures log entries from your app and makes them available for analysis in the Google Cloud Console.

For a Node.js application using the Express framework, logs are automatically captured by Google App Engine. However, you can enhance your logging with structured logging, log levels, and error tracking.

Key Concepts in Google Cloud Logging:

1. Structured Logging: Logs that are captured as structured data (usually in JSON format) allow for more sophisticated queries and easier analysis in the Cloud Logging dashboard.

2. Log Levels: Different log levels such as DEBUG, INFO, WARNING, ERROR, and CRITICAL help distinguish the severity of log messages.

3. Error Reporting: Google Cloud Logging integrates with Google Cloud Error Reporting, which can notify you about any critical errors in your app.

STEP 1: Install @google-cloud/logging-winston

The Google Cloud Logging client library for Node.js integrates well with Winston, a popular logging library for Node.js. You’ll use this to capture structured logs and send them to Google Cloud.

Install the necessary packages: npm install express winston @google-cloud/logging-winston

STEP 2: Configure Winston with Google Cloud Logging ..an example

Create a simple Express app that logs different events using Winston with Google Cloud Logging.

// Create the logger instance with transports
const logger = winston.createLogger({
   level: 'info',
   transports: [
     // Log to the console
     new winston.transports.Console(),
     // Log to Google Cloud Logging
     loggingWinston,
    ],
});

// Home route with log
app.get('/', (req, res) => {
   logger.info('Home route was accessed.');
   res.send('Welcome to the home page!');
});

// A route that logs a warning
app.get('/warn', (req, res) => {
  logger.warn('This is a warning log');
  res.send('Warning logged');
});

// A route that logs an error
app.get('/error', (req, res) => {
   logger.error('This is an error log');
   res.status(500).send('Error logged');
});

// A route that logs structured data (like a user object)
app.get('/user', (req, res) => {
   const user = { id: 123, name: 'John Doe', email: 'john@example.com' };
   logger.info('User data:', user);
   res.send('User info logged');
 });

const PORT = process.env.PORT || 8080;
app.listen(PORT, () => {
   logger.info(`App listening on port ${PORT}`);
});
               

Explanation of Key Components:

• Winston Logger: winston.createLogger() creates a logger that logs messages to both the console and Google Cloud Logging.

• Logging Levels: Logs are categorized by level (e.g., info, warn, error), which allows you to filter logs in Google Cloud Logging.

• Structured Logging: In the /user route, structured data (the user object) is logged, which can be queried in Cloud Logging.

Customizing Log Levels - example NodeJS Express

You can set different log levels depending on the environment or purpose. For instance, during development, you might want more DEBUG level logs, while in production, you may only want INFO and above.

const logger = winston.createLogger({
  level: process.env.NODE_ENV === 'production' ? 'info' : 'debug',
  transports: [
    winston.transports.Console(),
    loggingWinston,
  ],
 });
            

Error Reporting

Google Cloud integrates logs with Google Cloud Error Reporting, automatically capturing uncaught exceptions and errors that happen in your application.

Example: Logging an Error and Reporting to Cloud Error Reporting

app.get('/crash', (req, res) => {
  try {
    throw new Error('Something went wrong!');
  } catch (err) {
    logger.error('Caught an error:', err);
    res.status(500).send('Error reported!');
  }
 });
            

Summary:

• Logging Basics: Use Winston and the @google-cloud/logging-winston package to log structured data in your Node.js app.

• Integration with App Engine: Logs are automatically captured in Google Cloud Logging when your app runs on App Engine.

• Log Levels: Use different log levels like info, warn, and error to categorize logs.

• Error Reporting: Google Cloud automatically reports uncaught exceptions and critical errors.

Resources:

• Google Cloud Logging for Node.js: Google Cloud Logging Node.js Docs (https://cloud.google.com/logging/docs/setup/nodejs)

• Google Cloud Error Reporting: Google Cloud Error Reporting Docs (https://cloud.google.com/error-reporting)