Managed Service for Apache Kafka limitations

• Managed Service for Apache Kafka requires each cluster to have equal resources in each of the three zones. Single-zone or two-zone Managed Service for Apache Kafka clusters are not supported.

• You cannot choose the zones when you create the cluster.

• You cannot configure the volume of local storage on a cluster.

• Managed Service for Apache Kafka runs in the KRaft mode. Zookeeper mode is not supported.

• When using VPC Service Controls with Managed Service for Apache Kafka, you cannot restrict the Cloud DNS API within the perimeter. Restricting the Cloud DNS API prevents Managed Service for Apache Kafka clusters from being created because it blocks the creation and renewal of TLS certificates. To exclude the Cloud DNS API, run the following command:

  gcloud access-context-manager perimeters update PERIMETER_ID \
    --remove-restricted-services="dns.googleapis.com" \
    --project=PROJECT

  Replace the following:

  • PERIMETER_ID: the ID of the access perimeter that you want to update.
  • PROJECT: the project ID of your Google Cloud project.
  Caution: Excluding the DNS API from the VPC Service Controls perimeter creates a potential security risk. This configuration lets you create public DNS records that are readable outside your protected perimeter.

• Managed Service for Apache Kafka does not offer a schema registry or a Kafka Connect API. It can be used with open source and third-party services implementations of these APIs.

• JMX APIs for metrics are not supported.

• While you can change broker configurations with read-only update mode at any time, these changes take effect only when the brokers restart. Restarts happen periodically as part of Google's maintenance and upgrade processes, but there's no set schedule or a way to manually trigger them. As a result, you can't control when these changes take effect. Examples of read-only configurations include auto.create.topics.enable and background.threads. Updates to configurations with the cluster-wide update mode, such as message.max.bytes, don't require restarts and take effect immediately.

• Some broker configs managed by the service cannot be updated. This includes broker.id and storage-related settings, such as remote.log.storage.system.enable.

Apache Kafka® is a registered trademark of The Apache Software Foundation or its affiliates in the United States and/or other countries.