Access resources in another project

Cloud Composer 1 | Cloud Composer 2 | Cloud Composer 3

This page describes how to access resources that are located in a different Google Cloud project than your Cloud Composer environment.

We recommend to access resources in other Google Cloud projects in the following way:

  1. In your DAGs, use the default connections that are preconfigured in your environment.

    For example, the google_cloud_default connection is used by many Google Cloud operators and is automatically configured when you create an environment.

  2. Grant extra IAM permissions and roles to the service account of your environment, so that it can access resources in a different project.

Determine the service account of your environment

To determine the service account of your environment:

Console

  1. In Google Cloud console, go to the Environments page.

    Go to Environments

  2. In the list of environments, click the name of your environment. The Environment details page opens.

  3. Go to the Environment configuration tab.

  4. The service account of your environment is listed in the Service account field.

    The value is an email address, such as service-account-name@example-project.iam.gserviceaccount.com.

gcloud 

gcloud composer environments describe ENVIRONMENT_NAME \
    --location LOCATION \
    --format="get(config.nodeConfig.serviceAccount)"

The value is an email address, such as service-account-name@example-project.iam.gserviceaccount.com.

Grant IAM roles and permissions to access resources in another project

The service account of your environment requires permissions to access resources in another project. These roles and permissions can be different based on the resource that you want to access.

Access a specific resource

We recommend to grant roles and permissions for specific resources, such as a single Cloud Storage bucket located in a different project. In this approach, you use resource-based access with conditional role bindings.

To access a specific resource:

  1. Follow the Configure resource-based access guide.
  2. When granting roles and permissions, specify the service account of your environment as a principal.

Access a resource type

As an alternative, you can grant roles and permissions based on the resource type, such as all Cloud Storage buckets located in a different project.

To access a resource type:

  1. Follow the Manage access to other resources guide.
  2. When granting roles and permissions, specify the service account of your environment as a principal.

After you grant the required permissions and roles, you can access resources in a different project with the same default Airflow connections that you use to access resources in the project where your environment is located.

What's next