Send Security Command Center data to IBM QRadar

This page explains how to automatically send Security Command Center findings, assets, audit logs, and security sources to IBM QRadar. It also describes how to manage the exported data. QRadar is a security information and event management (SIEM) platform that ingests security data from one or more sources and lets security teams manage responses to incidents and perform real-time analytics.

In this guide, you ensure that required Security Command Center and Google Cloud services are properly configured and enable QRadar to access findings, audit logs, and assets in your Security Command Center environment.

Before you begin

This guide assumes you are using QRadar (v7.4.1 Fix Pack 2 or later). To get started with QRadar, see Sign up for QRadar.

Configure authentication and authorization

Before connecting to QRadar, you need to create an Identity and Access Management (IAM) service account in each Google Cloud organization that you want to connect and grant the account both the organization-level and project-level IAM roles that the Google SCC App for QRadar needs.

Create a service account and grant IAM roles

The following steps use the Google Cloud console. For other methods, see the links at the end of this section.

Complete these steps for each Google Cloud organization that you want to import Security Command Center data from.

In the same project in which you create your Pub/Sub topics, use the Service Accounts page in the Google Cloud console to create a service account. For instructions, see Creating and managing service accounts.
Grant the service account the following role:
- Pub/Sub Editor (roles/pubsub.editor)
Copy the name of the service account that you just created.
Use the project selector in the Google Cloud console to switch to the organization level.
Open the IAM page for the organization:

Go to IAM
On the IAM page, click Grant access. The grant access panel opens.
In the Grant access panel, complete the following steps:
1. In the Add principals section in the New principals field, paste the name of the service account.
2. In the Assign roles section, use the Role field to grant the following IAM roles to the service account:
3. Security Center Admin Editor (roles/securitycenter.adminEditor)
4. Security Center Notification Configurations Editor (roles/securitycenter.notificationConfigEditor)
5. Organization Viewer (roles/resourcemanager.organizationViewer)
6. Cloud Asset Viewer (roles/cloudasset.viewer)
7. Click Save. The service account appears on the Permissions tab of the IAM page under View by principals.
  
  By inheritance, the service account also becomes a principal in all child projects of the organization. The roles that are applicable at the project level are listed as inherited roles.

For more information about creating service accounts and granting roles, see the following topics:

Provide the credentials to QRadar

Depending on where you are hosting QRadar, how you provide the IAM credentials to QRadar differs.

If you are hosting the QRadar deployment in Google Cloud, consider the following:
- The service account that you created and the organization-level roles that you granted to it are available automatically by inheritance from the parent organization. If you are using multiple Google Cloud organizations, add this service account to the other organizations and grant it the IAM roles that are described in steps 5 to 7 of Create a service account and grant IAM roles.
- If you deploy QRadar in a service perimeter, create the ingress and egress rules. For instructions, see Granting perimeter access in VPC Service Controls.
If you are hosting QRadar in your on-premises environment or on IBM Cloud, create a service account key for each Google Cloud organization. You will need the service account keys in JSON format to complete this guide.

Note: Service account keys are a security risk if not managed correctly. You are responsible for the security of the private key and for other operations described by Best practices for managing service account keys. If you are prevented from creating a service account key, service account key creation might be disabled for your organization. For more information, see Managing secure-by-default organization resources.
If you are hosting QRadar in Microsoft Azure or Amazon Web Services, configure workload identity federation and download the credentials configuration files. If you are using multiple Google Cloud organizations, add this service account to the other organizations and grant it the IAM roles that are described in steps 5 to 7 of Create a service account and grant IAM roles.

Configure notifications

Complete these steps for each Google Cloud organization that you want to import Security Command Center data from.

Set up finding notifications as follows:
1. Enable the Security Command Center API.
2. Create a filter to export desired findings and assets.
3. Create three Pub/Sub topics: one each for findings, audit logs, and assets. The NotificationConfig must use the Pub/Sub topic that you create for findings.
Create a sink for the audit logs, as described in Collate and route organization-level logs to supported destinations. The sink must use the Pub/Sub topic that you created for audit logs. For example:
```
gcloud logging sinks create SINK_NAME SINK_DESTINATION \
  --include-children \
  --organization=ORGANIZATION_ID \
  --log-filter=FILTER
```
Replace the following:
- SINK_NAME with the name for the audit log sink.
- SINK_DESTINATION with pubsub.googleapis.com/projects/PROJECT_ID/topics/TOPIC_ID
- ORGANIZATION_ID with your organization's ID.
- FILTER with logName:activity, logName:data_access, logName:system_event, or logName:policy.
Grant the Pub/Sub Publisher (roles/pubsub.publisher) role to the sink's service account.
Enable the Cloud Asset API for your project.
Create feeds for your assets. You must create two feeds in the same Pub/Sub topic, one for your resources and another for your Identity and Access Management (IAM) policies.
- The Pub/Sub topic for assets must be different than the one used for findings.
- For the feed for your resources, use the following filter: content-type=resource.
- For the IAM policies feed, you must use the following filter: content-type=iam-policy --asset-types="cloudresourcemanager.googleapis.com/Project".

You will need your organization IDs and Pub/Sub subscription names to configure QRadar.

Install Google SCC App for QRadar - QRadar v7.4.1FP2+

In this section, you install the Google SCC App for QRadar - QRadar v7.4.1FP2+ (v3.0.0). The app, which is maintained by Security Command Center, automates the process of scheduling Security Command Center API calls, and regularly retrieves Security Command Center data for use in QRadar.

App installation requires access to the QRadar console machine through a web interface.

To complete the installation, do the following:

Download the Google SCC App for QRadar from IBM App Exchange.
Log into your QRadar console at https://QRadar_Console_IP.
In the console menu, click Admin, and then select Extension Management.
To select the download zip file, click Add. Follow the prompts as the install is prepared.
Select Start a default instance for each app.
Click Install. After installation completes successfully, you see a list of application components.
Click the Admin tab, and then click Deploy changes.
Clear the browser's cache and refresh the browser window.
Navigate to Extension Management. You should see Google SCC App For QRadar with a status of Installed.

Configure the Google SCC app

In this section, you configure the Google SCC App. To complete the configuration, do the following:

Navigate to the Admin tab in QRadar.
Click Google SCC App Settings.
Click Add Google SCC Organization.
Enter the following variables as needed:
- Service Account JSON: the JSON file that includes the service account key
  
  If you are hosting the QRadar deployment in Google Cloud, this field is not available. Ensure that you provide the service account that is linked to the VM with the IAM permissions for each Google Cloud organization. For more information, see Provide the credentials to QRadar.
- Credential Configuration: the credential configuration file that you downloaded when you set up workload identity federation
- Organization ID: the ID for your organization
- Findings Subscription Name: Pub/Sub subscription name for your finding notifications
- Assets Subscription Name: Pub/Sub subscription name for your assets feed
- Enable Audit Logs Collection: select to send audit logs to your QRadar instance
  - Audit Logs Subscription Name: Pub/Sub subscription name for your audit logs sink
- Interval: the number of seconds between Pub/Sub calls during real-time data collection
- QRadar Authorization Token: the token for your QRadar instance. To retrieve a token, do the following:
  1. Navigate to the Admin tab in QRadar.
  2. Under User Management, click Authorized Service.
  3. Copy your authorization token with Admin as a user role and Admin as Security Profile. If you don't have a token, create one by clicking Add Authorized Service.
  4. Click Deploy changes, and then refresh the browser window.
To enter optional proxy configuration details, click the Enable/Disable Proxy toggle, and then enter your proxy settings:
- IP/Hostname: the IP address or hostname of your proxy server (don't include the HTTP/HTTPS prefix)
- Port: the port of your proxy server
- Username: the username used for the authentication proxy
- Password: the password used for the authentication proxy
Click Save.
Repeat these steps for each Google Cloud organization that you want to integrate.

The app configuration is stored and your organizations are added to the app configuration page. The following sections explain how to view and manage Security Command Center data in the service.

Upgrade the Google SCC app

In this section, you upgrade an existing Google SCC App for QRadar to the latest version.

To complete the upgrade, do the following:

Download the latest version of the Google SCC App from the IBM App Exchange.
Log into your QRadar console at https://QRadar_Console_IP.
In the console menu, click Admin, and then select Extension Management.
To select the download zip file, click Add. Follow the prompts as the upgrade is prepared.
Select Replace Existing Items and Start a default instance for each app.
Click Install. After the upgrade process completes successfully, you see a list of application components.
Click the Admin tab, and then click Deploy changes.
Clear the browser's cache and refresh the browser window.
Navigate to Extension Management. You should see Google SCC App For QRadar with a status of Installed.
Remove application logs from users who access the application from QRadar using SSH:
1. Download the latest version of the Reference Data Management app from the IBM App Exchange.
2. Log into your QRadar console at https://QRadar_Console_IP.
3. In the console menu, click Admin, and then select Extension Management.
4. To select the download ZIP file, click Add. Follow the prompts to install the application.
5. In the console, navigate to the Reference Data Management dashboard.
6. Click Reference Map.
7. Select asset_owners and click Clear Data.

View the exported data in QRadar

This section describes relevant functionality available in QRadar, including searching for findings, audit logs, and assets, viewing IAM policies, and viewing custom dashboards.

Search for data

To search Security Command Center data in QRadar, you use the Log Activity panel. You can see ingested findings, assets, audit logs, and security sources and apply SQL-style filters to refine the data.

View IAM policy data

To view IAM policy data for your assets, do the following:

Download and install the Reference Data Management application from the IBM App Exchange Portal.
Click on the Reference Data Management dashboard in QRadar.
In the navigation panel, click Reference Map.
Select asset_owners. The dashboard is populated with your IAM policy data.

Custom dashboards

You can use custom dashboards in QRadar to visualize and analyze your findings, assets, and security sources.

Overview

The Overview dashboard displays the total number of findings, threats, and vulnerabilties in your Google Cloud organizations. Findings are compiled from Security Command Center's built-in services, such as Security Health Analytics, Web Security Scanner, Event Threat Detection, and Container Threat Detection and any integrated services you enable.

You can filter data to update visualizations, specify Google Cloud organization, and fetch new data on-demand.

Assets

The Assets tab displays a table of your Google Cloud assets. Table data includes asset name, asset type, resource owners, last update time, and links to Security Command Center's Assets page in the Google Cloud console.

You can search and filter asset data by organization, time range, and asset type, and drill down to findings for specific assets.

Sources

The Sources tab displays a table of your security sources, including source name, source display name, and description. By clicking a source name, you can view findings for that source.

Findings

The Findings tab displays a table of your organization's findings. You can search the table and filter the list by time range, category, severity, security source, asset, and project name.

Table columns includes finding name, category, asset name, security source name, security marks, severity, project name, event time, event time, finding class, and update status. If you click a finding name, you are redirected to Security Command Center's Findings page in the Google Cloud console and shown details for the selected finding.

In the Update Status column, you can update the state of a finding. To indicate that you are actively reviewing a finding, click Mark as ACTIVE. If you are not actively reviewing a finding, click Mark as INACTIVE.

Audit logs

The Audit logs dashboard displays a series of charts and tables that show audit log information. The audit logs that are included in the dashboard are the administrator activity, data access, system events, and policy denied audit logs. The table includes the time, log name, severity, service name, resource name, and resource type.

Check application logs

List all installed applications and their App-ID values:

/opt/qradar/support/recon ps

The output is similar to the following. Take note of the App-ID of the Google SCC app.

App-ID  Name                                    Managed Host ID Workload ID             Service Name    AB       Container Name          CDEGH          Port          IJKL
1101    QRadar Log Source Management            53              apps                    qapp-1101       ++           qapp-1101           +++++          5000          ++++
1104    QRadar Assistant                        53              apps                    qapp-1104       ++           qapp-1104           +++++          5000          ++++
1105    QRadar Use Case Manager                 53              apps                    qapp-1105       ++           qapp-1105           +++++          5000          ++++
1163    IBM QRadar Pre-Validation App Service   53              apps                    qapp-1163       ++           qapp-1163           +++++          5000          ++++
1164    IBM QRadar Pre-Validation App UI        53              apps                    qapp-1164       ++           qapp-1164           +++++          5000          ++++
1170    Google SCC                              53              apps                    qapp-1170       ++           qapp-1170           +++++          5000          ++++

Connect to the Google SCC app container:
```
/opt/qradar/support/recon connect APP_ID
```
Replace APP_ID with the App-ID of the Google SCC app.
Go to the log directory:
```
cd /opt/app-root/store/log
```
List all files in the directory:
```
ls
```
View the contents of a file:
```
cat FILENAME
```
Replace FILENAME with the name of the file.

Uninstall Google SCC App

To uninstall the Google SCC App, do the following:

Go to the Admin tab.
Select Extension Management.
Select Google SCC App For QRadar - QRadar v7.4.1FP2+.
Click Uninstall.

If you uninstall the application, custom event properties, reference maps, dashboards, and log sources provided by the Google SCC App are removed.

Known issues

This section lists known issues with the Google SCC App and QRadar dashboards.

v1.0.0

In the Overview dashboard, the panel Findings By Severity Over Time displays a technical error for data greater than 250,000 findings and the flask process, which populates the dashboards, is restarted in the backend. To avoid this issue, select a smaller time range for the dashboard.

This issue is resolved in v2.0.0.
Deleted assets might appear on the Assets dashboard because of unexpected behavior from the GROUP BY AQL function.

v2.0.0

Deleted assets might appear on the Assets dashboard because of unexpected behavior from the GROUP BY AQL function.
The Findings dashboard might not display the latest finding data after you update the Google SCC app because of unexpected behavior from the GROUP BY AQL function.

v3.0.0

The dashboard might not display the latest events when multiple events are available with the same unique key because of unexpected behavior from the GROUP BY AQL function.
For data already ingested using v2, the Organization ID filter isn't applicable. You can view the data by selecting the All value in the Organization ID filter.

Troubleshoot

This section describes solutions for some common problems.

Google SCC events are shown as Google SCC messages

Problem: Security Command Center events will show up as Security Command Center messages rather than getting identified as the right QRadar category. Messages are seen in the Log Activity tab in QRadar when a user searches for an event from a Google Cloud log source.

This issue is occurs when a required field is not present in a raw log event, or if the event payload size is more than the default 4,096 bytes, which can cause events to be truncated.

Solution: If payloads are truncated, perform the following steps to increase the maximum payload size:

Navigate to Admin tab and select System settings.
Under Switch to, click Advanced.
In the settings list, do the following:
1. Select Max TCP Syslog Payload Length and increase its value; the recommended value is 32,000.
2. Select Max UDP Syslog Payload Length and increase its value; the recommended value is 32,000.
Click Deploy changes and use the Full Deploy option.

Google SCC events listed as unknown events

Problem: Security Command Center events are listed as Unknown. This issue occurs when the event ID and category from the payload are not mapped in QRadar.

Solution: Perform the following steps to fix this issue:

Navigate to Log Activity, and then click Add Filter.
Select Parameter, and then select Log Source Type (Indexed).
Select Operator, and then select Equals.
Select Log Source Type, and then select Google SCC.
In the Views filter drop-down menu, seelect Last 7 days.
If events are displayed as Unknown, perform the following steps:
1. Right click on the event, and select View in DSM editor.
2. Under Log Activity Preview, check the values of Event ID and Event Category.
3. If the values are unknown, contact Cloud Support.

App configuration fails with error messages

If you get an app configuration error, follow these steps to fix the issue.

Error	Description	Solution
"Please enter valid Service Account JSON."	This error occurs if a properly formatted JSON is provided but authentication fails when attempting to save the configuration.	Enter a valid JSON with the correct account credentials.
"Service Account JSON should be JSON string."	This error occurs if an improperly formatted JSON is provided or the file is in a format other than JSON.	Enter a valid JSON file.
"Please enter valid Organization ID".	This error occurs when an incorrect or incomplete organization ID is entered.	Verify your organization ID and re-enter it.
"Please enter valid Project ID or Findings Subscription ID."	This error occurs when an incorrect or invalid project ID or subscription ID is entered.	Verify your project ID and organization ID, and re-enter them.
"Please enter valid Assets Subscription ID."	This error occurs when an incorrect or invalid asset subscription ID is entered.	Verify your asset subscription ID, and re-enter it.
"Error while validating authorization token."	This error when an incorrect or invalid QRadar Authorization Token is provided.	Verify your QRadar Authorization Token, and re-enter it. It must have Admin as the user role and security profile. The token must also not be expired.

Error while initiating socket connection with QRadar

Problem: An error message, "Error while initiating socket connection with IBM QRadar" is observed in data collection log files. This issue might be observed in the QRadar v2 app framework (< v7.4.2 P2).

Solution: Perform the following steps to fix this issue:

Review support note regarding QRadar deploy changes.
Upgrade QRadar.

Interface issues

Problem: A dashboard panel or configuration page shows errors or unintended behavior.

Solution: Perform the following steps to fix this issue:

Clear the browser cache and reload the webpage.
Reduce the time range of the filter. QRadar queries might expire if the number of responses is too large.
If the issue is not resolved, contact Cloud Support.

Dashboard panels fail to load and flask process gets killed

Problem: The flask process times out and some dashboard panels fail to load.

Solution: Perform the following steps to fix this issue:

Clear the browser cache and reload the webpage.
Reduce the time range of the filter. QRadar queries might expire if the number of responses is too large.
If the issue is not resolved, please contact Cloud Support.

All other performance issues

If your issue is not resolved by following instructions in this guide, do the following:

Navigate to the Admin tab, and then click on System and License Management.
Select the host on which Google SCC App For QRadar - QRadar v7.4.1FP2+ is installed.
Click Action, and then select Collect Log Files.
In the dialog, click on Advance Options.
Select the checkboxes next to Include Debug Logs, Application Extension Logs, and Setup Logs (Current Version).
Select two days as data input, and then click on Collect Log Files.
Select Click here to download files.

Log files will be downloaded in a zip file. Contact Cloud Support and share the log files.

What's next

Learn more about setting up finding notifications in Security Command Center.
Read about filtering finding notifications in Security Command Center.