2.0.0 Migration Guide

The 2.0 release of the google-cloud-bigquery-datatransfer client is a significant upgrade based on a next-gen code generator, and includes substantial interface changes. Existing code written for earlier versions of this library will likely require updates to use this version. This document describes the changes that have been made, and what you need to do to update your usage.

If you experience issues or have questions, please file an issue.

Supported Python Versions

WARNING: Breaking change

The 2.0.0 release requires Python 3.6+.

Import Path

WARNING: Breaking change

The library was moved into google.cloud.bigquery namespace. Existing imports need to be updated.

Before:

from google.cloud import bigquery_datatransfer
from google.cloud import bigquery_datatransfer_v1

After:

from google.cloud.bigquery import datatransfer
from google.cloud.bigquery import datatransfer_v1

Method Calls

WARNING: Breaking change

Methods that send requests to the backend expect request objects. We provide a script that will convert most common use cases.

  • Install the library
python3 -m pip install google-cloud-bigquery-datatransfer
  • The script fixup_datatransfer_v1_keywords.py is shipped with the library. It expects an input directory (with the code to convert) and an empty destination directory.
$ scripts/fixup_datatransfer_v1_keywords.py --input-directory .samples/ --output-directory samples/

Before:

from google.cloud import bigquery_datatransfer

client = bigquery_datatransfer.DataTransferServiceClient()

parent_project = "..."
transfer_config = {...}
authorization_code = "..."

response = client.create_transfer_config(
    parent_project, transfer_config, authorization_code=authorization_code
)

After:

from google.cloud.bigquery import datatransfer

client = datatransfer.DataTransferServiceClient()

parent_project = "..."
transfer_config = {...}
authorization_code = "..."

response = client.create_transfer_config(
    request={
        "parent": parent_project,
        "transfer_config": transfer_config,
        "authorization_code": authorization_code,
    }
)

More Details

In google-cloud-bigquery-datatransfer<2.0.0, parameters required by the API were positional parameters and optional parameters were keyword parameters.

Before:

def create_transfer_config(
    self,
    parent,
    transfer_config,
    authorization_code=None,
    version_info=None,
    service_account_name=None,
    retry=google.api_core.gapic_v1.method.DEFAULT,
    timeout=google.api_core.gapic_v1.method.DEFAULT,
    metadata=None,
):

In the 2.0.0 release, methods that interact with the backend have a single positional parameter request. Method docstrings indicate whether a parameter is required or optional.

Some methods have additional keyword only parameters. The available parameters depend on the google.api.method_signature annotation specified by the API producer.

After:

def create_transfer_config(
    self,
    request: datatransfer.CreateTransferConfigRequest = None,
    *,
    parent: str = None,
    transfer_config: transfer.TransferConfig = None,
    retry: retries.Retry = gapic_v1.method.DEFAULT,
    timeout: float = None,
    metadata: Sequence[Tuple[str, str]] = (),
) -> transfer.TransferConfig:

NOTE: The request parameter and flattened keyword parameters for the API are mutually exclusive. Passing both will result in an error.

Both of these calls are valid:

response = client.create_transfer_config(
    request={
        "parent": project_path,
        "transfer_config": {"foo": "bar"},
    }
)

response = client.create_transfer_config(
    parent=project_path,
    transfer_config={"foo": "bar"},
)

This call is invalid because it mixes request with a keyword argument transfer_config. Executing this code will result in an error:

response = client.create_transfer_config(
    request={"parent": project_path},
    transfer_config= {"foo": "bar"},
)

NOTE: The request parameter of some methods can also contain a more rich set of options that are otherwise not available as explicit keyword only parameters, thus these must be passed through request.

Removed Utility Methods

WARNING: Breaking change

Most utility methods such as project_path() have been removed. The paths must now be constructed manually:

project_path = f"project/{PROJECT_ID}"

The only two that remained are transfer_config_path() and parse_transfer_config_path().

Removed client_config Parameter

The client cannot be constructed with client_config argument anymore, this deprecated argument has been removed. If you want to customize retry and timeout settings for a particular method, you need to do it upon method invocation by passing the custom timeout and retry arguments, respectively.