Logs are essential for maintaining your air-gapped deployments' health, security, and operational efficiency. They provide valuable insights into the following aspects of an environment:
- Application behavior: Identify errors, performance issues, and unusual activity within your applications.
- System activity: Monitor the performance and health of underlying infrastructure components.
- Security auditing: Track user actions, access control events, and potential security threats.
- Troubleshooting: Diagnose and resolve issues by analyzing historical log data.
Therefore, Google Distributed Cloud (GDC) air-gapped provides a logging platform for collecting and analyzing your logs. GDC captures the following two types of logs:
- Audit logs: Record user and administrative activities on privileged operations and achieve auditing and compliance requirements on GDC.
- Operational logs: Record conditions, changes, and actions as you manage ongoing operations in applications and services on GDC. These logs help developers and operators test and debug applications.
This page provides an overview of logging features in GDC environments and explains the key concepts and components involved in managing logs generated by your applications and the platform itself.
Key components
GDC deployments use a logging platform to collect and store logs from various sources. This platform includes the following components:
- Fluent Bit: A lightweight log processor and forwarder deployed on each cluster node. Fluent Bit collects logs from applications, system components, and Kubernetes itself.
- Loki: An open-source log aggregation system that stores and queries logs from your applications and infrastructure.
- Log sinks: Log routers that export log entries to other destinations, such as a local storage system or a security information and event management (SIEM) tool. These log sinks give you flexibility in managing and analyzing your log data.
- Logging pipeline: A managed service that stores log data and implements the logic for log collection. In air-gapped environments, the logging pipeline runs locally within your data center.
- Grafana: An analytics platform that lets you create insightful dashboards and explore your logging data. You can query logs stored in the logging platform using the Grafana user interface.
Considerations for your deployments
Consider the following aspects when using the logging platform to collect logs:
- Storage capacity: Plan for sufficient storage capacity to accommodate the volume of log data your applications and system components generate.
- Log retention policies: Define log retention times based on your compliance and operational needs during the creation of your organization. For more information, see Log retention.
Benefits
You obtain the following benefits when using the logging platform in GDC:
- Automation: Automate log-related tasks, such as exporting logs to a central repository or generating reports.
- Customization: Build custom logging solutions tailored to your needs and integrate them with existing tools and workflows.
- Efficiency: Programmatically manage large volumes of log data and perform complex queries.
- Flexibility: Access log data from various sources and integrate your systems with analysis and monitoring tools.
By using the logging platform in your air-gapped environment, you can manage and analyze log data, even in an isolated environment.
Log retention
Retention policies define how long metrics and logs are stored. These policies are crucial for meeting compliance requirements and supporting operational analysis. GDC uses retention times to configure lifecycle and retention policies for resource configurations.
If specific retention times are not provided during the organization's creation
or are set to 0 in the Organization custom resource during creation time,
the system applies the following default values for logs:
- Infrastructure audit logs: 2000 days
- Platform audit logs: 400 days
- Operational logs: 90 days
Further resources
Consult the comprehensive list of audit logs and operational logs for detailed descriptions of all collected logs from GDC components. These resources provide valuable context and facilitate advanced logging strategies.