Write and query log entries with the gcloud CLI

Write and query log entries with the gcloud CLI

This document introduces you to some of the capabilities of Cloud Logging and shows you how to do the following:

• Write log entries by using the Google Cloud CLI.
• List log entries by using the gcloud CLI.
• List log entries by using the Logging API.
• View and query log entries by using the Logs Explorer.

Before you begin

You must have a Google Cloud project with billing enabled to complete this quickstart. If you don't have a Google Cloud project, or if you don't have billing enabled for your Google Cloud project, then do the following:

1. Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.
2. Install the Google Cloud CLI.

3. To initialize the gcloud CLI, run the following command:

   gcloud init

   Note: If you installed the gcloud CLI previously, make sure you have the latest version by running gcloud components update.

4. Create or select a Google Cloud project.

   Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

   • Create a Google Cloud project:

     gcloud projects create PROJECT_ID

     Replace PROJECT_ID with a name for the Google Cloud project you are creating.

   • Select the Google Cloud project that you created:

     gcloud config set project PROJECT_ID

     Replace PROJECT_ID with your Google Cloud project name.

5. Make sure that billing is enabled for your Google Cloud project.

6. Install the Google Cloud CLI.

7. To initialize the gcloud CLI, run the following command:

   gcloud init

   Note: If you installed the gcloud CLI previously, make sure you have the latest version by running gcloud components update.

8. Create or select a Google Cloud project.

   Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

   • Create a Google Cloud project:

     gcloud projects create PROJECT_ID

     Replace PROJECT_ID with a name for the Google Cloud project you are creating.

   • Select the Google Cloud project that you created:

     gcloud config set project PROJECT_ID

     Replace PROJECT_ID with your Google Cloud project name.

9. Make sure that billing is enabled for your Google Cloud project.

Write log entries by using the gcloud CLI

Logging supports log entries with structured and unstructured data. Structured data consists of a JSON data structure; for example, {"weather": "partly cloudy"}. Unstructured data is a string of characters; for example, "A simple entry".

In the next steps, you use the gcloud CLI to write a log entry with unstructured data and a log entry with structured data. The gcloud CLI provide a command-line interface to the Cloud Logging API.

1. Write a log entry with unstructured data to the log my-test-log, run the gcloud logging write command:

   gcloud logging write my-test-log "A simple entry."
   

   After the command completes, you see the message: Created log entry.

2. Write a log entry with structured data to the log my-test-log:

   gcloud logging write --payload-type=json my-test-log '{ "message": "My second entry", "weather": "partly cloudy"}'
   

   When you write a log entry with structured data, you must include --payload-type=json. If you omit this field, then Logging interprets the payload as unstructured data.

If the log my-test-log doesn't exist, then Logging creates the log when the log entry is received.

List log entries by using the gcloud CLI

You can retrieve log entries from Logging and display them by using the gcloud CLI. For example, to retrieve and display the log entries with a resource type of global, run the following command:

gcloud logging read "resource.type=global"

The command returns a result similar to the following:

---
insertId: jpj9zjf73t1mn
jsonPayload:
  message: My second entry
  weather: partly cloudy
logName: projects/myloggingproject/logs/my-test-log
receiveTimestamp: '2018-11-01T18:39:31.114507977Z'
resource:
  labels:
    project_id: myloggingproject
  type: global
timestamp: '2018-11-01T18:39:31.114507977Z'
---
insertId: vd4m1if7h7u1a
logName: projects/myloggingproject/logs/my-test-log
receiveTimestamp: '2018-11-01T18:39:19.718100792Z'
resource:
  labels:
    project_id: myloggingproject
  type: global
textPayload: A simple entry
timestamp: '2018-11-01T18:39:19.718100792Z'

For information about reading logs, see the gcloud logging read reference documentation.

List log entries by using APIs Explorer

To run Logging API methods without writing any code, see Using the APIs Explorer. To read a list of log entries from Logging, do the following:

1. Go to the API reference page for the entries.list API method:

   Go to entries.list API page

2. Configure and run the API command:

   1. Replace PROJECT_ID in the following text:

      "resourceNames": [
      "projects/PROJECT_ID"
      ],
      "filter": "resource.type=global",
      "orderBy": "timestamp desc"
      

   2. Copy the updated text from the previous step, and paste it into the Request body field of APIs Explorer.

   3. Click Execute.

   The method returns a response similar to the following:

   {
     "entries": [
       {
         "textPayload": "A simple entry",
         "insertId": "vd4m1if7h7u1a",
         "resource": {
           "type": "global",
           "labels": {
             "project_id": "myloggingproject"
           }
         },
         "timestamp": "2018-11-01T18:39:19.718100792Z",
         "logName": "projects/myloggingproject/logs/my-test-log",
         "receiveTimestamp": "2018-11-01T18:39:19.718100792Z"
       },
       {
         "insertId": "jpj9zjf73t1mn",
         "jsonPayload": {
           "message": "My second entry",
           "weather": "partly cloudy"
         },
         "resource": {
           "type": "global",
           "labels": {
             "project_id": "myloggingproject"
           }
         },
         "timestamp": "2018-11-01T18:39:31.114507977Z",
         "logName": "projects/myloggingproject/logs/my-test-log",
         "receiveTimestamp": "2018-11-01T18:39:31.114507977Z"
       }
     ]
   }
   

View log entries in the Logs Explorer

To view log entries in the Google Cloud console, you can use the Logs Explorer. Most Google Cloud projects store a large number of logs; you can select certain log entries by writing a query.

To view the log entries that you wrote using the Logs Explorer, do the following:

1. In the Google Cloud console, go to the Logs Explorer page:

   Go to Logs Explorer

   If you use the search bar to find this page, then select the result whose subheading is Logging.

   Ensure your Google Cloud project is selected in the Google Cloud navigation bar. If necessary, use the Google Cloud project drop-down list to select your Google Cloud project.

2. In the Resource menu, select Global.

   If you don't see the Global menu option or if you don't see your log entries, then wait a few minutes and refresh the page. It can take a few minutes for Logging to receive log entries.

3. To view the details of a log entry, click its Menu.

   The first log entry has its data stored in textPayload. The second log entry contains structured data that is stored in jsonPayload. The structured payload contains the keys message and weather.

   Note: When message is a key for a structured log entry, Logging uses it as the summary of the log entry when the entry isn't expanded.

For information about the data format of log entries, see the LogEntry type.

Query log entries in the Logs Explorer

You can query log entries by using the query editor and, with structured logs, by the key and value. For example, to display all log entries that contain the text simple, do the following:

1. In the Google Cloud console, go to the Logs Explorer page:

   Go to Logs Explorer

   If you use the search bar to find this page, then select the result whose subheading is Logging.

2. In the Resource menu, select Global.

3. In the query editor, enter the string simple in quotation marks. The logs display shows only the log entry A simple entry.

4. After you have viewed your log, remove the query string you added and click Run query. Both log entries reappear in the display.

To display all log entries with structured data that have a key of weather where the value field contains partly, do the following:

1. The query editor contains the line resource.type="global". Enter the following command:

   jsonPayload.weather:partly
   

2. Click Run query. The result is the single log entry My second entry.

Logs Explorer also offers saved, suggested, and recent queries. For more information about queries, see Build queries in the Logs Explorer.

For sample queries, see Sample queries using the Logs Explorer.

Troubleshooting

• Typographical errors and unknown field names result in the gcloud CLI commands completing with invalid argument messages. For example, if you forget the period in resource.type, then it results in the error:

   ERROR: (gcloud.logging.read) INVALID_ARGUMENT: Field not found: 'resourcetype'.
  

• When Cloud Logging hasn't been granted the necessary access permissions, the gcloud CLI commands complete with permission denied messages. For example, if a Compute Engine VM instance is configured with the default API settings, then the list command completes with a permission denied error:

   ERROR: (gcloud.logging.read) PERMISSION_DENIED: Request had insufficient authentication scopes.
  

  To fix this condition, modify your Compute Engine VM instance permissions to grant Cloud Logging read permission by doing the following:

  1. Go to the VM instance details page for your VM instance. Click Stop. This action might take a minute or two to complete.
  2. To modify the configuration, click Edit.
  3. Search for the header Cloud API access scopes, and click Details to display the settings for each API. Change the entry from Cloud Logging API to Full. Click Save.
  4. To restart your VM instance, click Start. After a few moments, your VM is ready to use.

• When APIs Explorer can't complete your command, or requires additional authorization, it displays a message or error code:

  • 200 response code and no entries: If the message nextPageToken is displayed, then it indicates that APIs Explorer didn't have time to complete the search. Add a pageToken to your request, set the value to be the same as that given with the key nextPageToken, and then retry the command.
  • 400 response code: The query value is invalid. For example, if you misspell global as gloobal, then the message is Unsupported resource type: gloobal.
  • 404 response code: The project ID is invalid. Check the spelling of your project identifier.
  • You might be asked to sign into your Google Cloud project and permit APIs Explorer to access your account.

Clean up

To avoid incurring charges to your Google Cloud account for the resources used on this page, delete the Google Cloud project with the resources.

1. (Optional) To delete the log entries you created, run the following gcloud command:

   gcloud logging logs delete my-test-log
   

   If you don't delete your log entries, then they expire and are removed. For retention information, see Quotas & limits.

What's next

• For details on the Logging command-line interface, read the reference pages for the gcloud logging command group.
• For documentation on the Logging API, see Cloud Logging API.
• For details on the Logs Explorer, see Using the Logs Explorer.

• To learn how to collect log entries from your VM instances in Logging, see Google Cloud Observability agents.