Using the Python client library

---

This tutorial describes how to use the Google API Client Library for Python to call the AI Platform Training REST APIs in your Python applications. The code snippets and examples in the rest of this documentation use this Python client library.

You will be creating a model in your Google Cloud project in this tutorial. That is a simple task that's easily contained to a small example.

Objectives

This is a basic tutorial designed to familiarize you with this Python client library. When you're finished you should be able to:

• Get a Python representation of the AI Platform Training services.
• Use that representation to create a model in your project, which should help you understand how to call the other model and job management APIs.

Note: The code samples in this tutorial use Python 2.7. You can choose either Python 2.7 or Python 3 for use with AI Platform Training. See how to specify a Python version for training.

Costs

You will not be charged for the operations in this tutorial. Refer to the pricing page for more information.

Before you begin

Set up your GCP project

1. Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.

2. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

   Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

   Go to project selector

3. Make sure that billing is enabled for your Google Cloud project.

4. Enable the AI Platform Training & Prediction and Compute Engine APIs.

   Enable the APIs

5. Install the Google Cloud CLI.

6. To initialize the gcloud CLI, run the following command:

   gcloud init

7. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

   Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

   Go to project selector

8. Make sure that billing is enabled for your Google Cloud project.

9. Enable the AI Platform Training & Prediction and Compute Engine APIs.

   Enable the APIs

10. Install the Google Cloud CLI.

11. To initialize the gcloud CLI, run the following command:

    gcloud init

Set up authentication

To set up authentication, you need to create a service account key and set an environment variable for the file path to the service account key.

1. Create a service account:

   1. In the Google Cloud console, go to the Create service account page.

      Go to Create service account

   2. In the Service account name field, enter a name.
   3. Optional: In the Service account description field, enter a description.
   4. Click Create.
   5. Click the Select a role field. Under All roles, select AI Platform > AI Platform Admin.
   6. Click Add another role.

   7. Click the Select a role field. Under All roles, select Storage > Storage Object Admin.

      Note: The roles you select allow your service account to access resources. You can view and change these roles later by using the Google Cloud console. If you are developing a production app, you might need to specify roles with fewer permissions than AI Platform Admin and Storage Object Admin. For more information, see access control for AI Platform Training.

   8. Click Done to create the service account.

      Do not close your browser window. You will use it in the next step.

2. Create a service account key for authentication:

   1. In the Google Cloud console, click the email address for the service account that you created.
   2. Click Keys.
   3. Click Add key, then Create new key.
   4. Click Create. A JSON key file is downloaded to your computer.
   5. Click Close.

3. Set the environment variable GOOGLE_APPLICATION_CREDENTIALS to the file path of the JSON file that contains your service account key. This variable only applies to your current shell session, so if you open a new session, set the variable again.

   Example: Linux or macOS

   Replace [PATH] with the file path of the JSON file that contains your service account key.

   export GOOGLE_APPLICATION_CREDENTIALS="[PATH]"

   For example:

   export GOOGLE_APPLICATION_CREDENTIALS="/home/user/Downloads/service-account-file.json"

   Example: Windows

   Replace [PATH] with the file path of the JSON file that contains your service account key, and [FILE_NAME] with the filename.

   With PowerShell:

   $env:GOOGLE_APPLICATION_CREDENTIALS="[PATH]"

   For example:

   $env:GOOGLE_APPLICATION_CREDENTIALS="C:\Users\username\Downloads\[FILE_NAME].json"

   With command prompt:

   set GOOGLE_APPLICATION_CREDENTIALS=[PATH]

Set up a Python development environment

Choose one of the options below to set up your environment locally on macOS or in a remote environment on Cloud Shell.

For macOS users, we recommend that you set up your environment using the MACOS tab below. Cloud Shell, shown on the CLOUD SHELL tab, is available on macOS, Linux, and Windows. Cloud Shell provides a quick way to try AI Platform Training, but isn't suitable for ongoing development work.

macOS

1. Check Python installation
   Confirm that you have Python installed and, if necessary, install it.

   python -V

2. Check pip installation
   pip is Python's package manager, included with current versions of Python. Check if you already have pip installed by running pip --version. If not, see how to install pip.

   You can upgrade pip using the following command:

   pip install -U pip

   See the pip documentation for more details.

3. Install virtualenv
virtualenv is a tool to create isolated Python environments. Check if you already have virtualenv installed by running virtualenv --version. If not, install virtualenv:

   pip install --user --upgrade virtualenv

   To create an isolated development environment for this guide, create a new virtual environment in virtualenv. For example, the following command activates an environment named aip-env:

   virtualenv aip-env
   source aip-env/bin/activate

4. For the purposes of this tutorial, run the rest of the commands within your virtual environment.

   See more information about using virtualenv. To exit virtualenv, run deactivate.

Cloud Shell

1. Open the Google Cloud console.

   Google Cloud console

2. Click the Activate Google Cloud Shell button at the top of the console window.

   

   A Cloud Shell session opens inside a new frame at the bottom of the console and displays a command-line prompt. It can take a few seconds for the shell session to be initialized.

   

   Your Cloud Shell session is ready to use.

3. Configure the gcloud command-line tool to use your selected project.

   gcloud config set project [selected-project-id]

   where [selected-project-id] is your project ID. (Omit the enclosing brackets.)

Install the Google API Client Library for Python

Install the Google API Client Library for Python.

This is a basic tutorial designed to familiarize you with this Python client library. When you're finished you should be able to:

• Get a Python representation of the AI Platform Training services.
• Use that representation to create a model in your project, which should help you understand how to call the other model and job management APIs.

Note: The code samples in this tutorial use Python 2.7. You can choose either Python 2.7 or Python 3 for use with AI Platform Training. See how to specify a Python version for training.

You will not be charged for the operations in this tutorial. Refer to the pricing page for more information.

Importing the required modules

When you want to use the Google API Client Library for Python to call the AI Platform Training REST APIs in your code, you must import its package and the OAuth2 package. For this tutorial (and for most standard uses of AI Platform Training) you only need to import specific modules:

• GoogleCredentials from OAuth2
• discovery from Google API Client Library for Python.
• errors from Google API Client Library for Python

Refer to the documentation for those packages to learn about the other available modules.

Create a new Python file using your favorite editor, and add these lines:

from oauth2client.client import GoogleCredentials
from googleapiclient import discovery
from googleapiclient import errors

Building a Python representation of the API

Get your Python representation of the REST API. The method you call is build because the API client library uses service discovery to dynamically set up connections to the services as they exist when you make the call. Call your object that encapsulates the services ml:

ml = discovery.build('ml','v1')

Configuring your parameters and request body

To make a call to a service, you must create the parameters and request body that will be passed to the REST API. You pass parameters as regular Python parameters to the method that represents the call. The body is a JSON resource just as you would use if calling the API with an HTTP request directly.

Take a look at the REST API for creating a model in a new browser tab, projects.models.create:

• Notice the path parameter parent, which is the part of the URI of the request that identifies the project. If you were making the HTTP POST request directly, you would use the following URI:

  https://ml.googleapis.com/v1/projects/YOUR_PROJECT_ID/models

  When using the API client library, the variable part of the URI is represented as a string-typed parameter to the API call. You'll set it to 'projects/<var>YOUR_PROJECT_ID</var>'. Store your project in a variable to make API calls cleaner:

  project_id = 'projects/{}'.format('YOUR_PROJECT_ID')

• The body of the request is a JSON resource representing the model information. You can see in the model resource definition that it has two values for input: name and (optionally) description. You can pass a Python dictionary in the place of JSON and the API client library will perform the necessary conversion.

  Create your Python dictionary:

  request_dict = {'name': 'your-model-name',
                 'description': 'This is a machine learning model entry.'}

Creating your request

Making calls to APIs with the Python client library has two steps: first you create a request, then you make the call using that request.

Create the request

Use the built client objects that you created earlier (if you followed the code snippet exactly, it's called ml) as the root of the API hierarchy and specify the API you want to use. Each collection in the API path behaves like a function that returns a list of the collections and methods within it. For example, the root of all the AI Platform Training APIs is projects, so your call begins with ml.projects().

Use this code to form your request:

request = ml.projects().models().create(parent=project_id, body=request_dict)

Send the request

The request that you constructed in the last step exposes an execute method that you call to send the request to the service:

response = request.execute()

It's common for developers to combine this step with the last one:

response = ml.projects().models().create(parent=project_id,
                                         body=request_dict).execute()

Handle simple errors

A lot of things can go wrong when you make API calls over the Internet. It's a good idea to handle common errors. The simplest way to deal with errors is to put your request in a try block and catch likely errors. Most of the errors you're likely to get from the service are HTTP errors, which are encapsulated in the HttpError class. To catch these errors, you'll use the errors module from the googleapiclient package.

Wrap your request.execute() call in a try block. Also put a print statement in the block, so that you will try to print the response only if the call succeeds:

try:
    response = request.execute()
    print(response)

Add a catch block to handle HTTP errors. You can use HttpError._get_reason() to get the reason text fields from the response:

except errors.HttpError, err:
    # Something went wrong, print out some information.
    print('There was an error creating the model. Check the details:')
    print(err._get_reason())

Of course, a simple print statement might not be the right approach for your application.

Putting it all together

Here is the complete example:

from googleapiclient import discovery
from googleapiclient import errors

# Store your full project ID in a variable in the format the API needs.
project_id = 'projects/{}'.format('YOUR_PROJECT_ID')

# Build a representation of the Cloud ML API.
ml = discovery.build('ml', 'v1')

# Create a dictionary with the fields from the request body.
request_dict = {'name': 'your_model_name',
               'description': 'your_model_description'}

# Create a request to call projects.models.create.
request = ml.projects().models().create(
              parent=project_id, body=request_dict)

# Make the call.
try:
    response = request.execute()
    print(response)
except errors.HttpError, err:
    # Something went wrong, print out some information.
    print('There was an error creating the model. Check the details:')
    print(err._get_reason())

Generalizing to other methods

You can use the procedure you learned here to make any of the other REST API calls. Some of the APIs require much more complicated JSON resources than creating a model does, but the principles are the same:

1. Import googleapiclient.discovery and googleapiclient.errors.

2. Use the discovery module to build a Python representation of the API.

3. Use the API representation as a series of nested objects to get to the API you want and create a request. For example,

   request = ml.projects().models().versions().delete(
       name='projects/myproject/models/mymodel/versions/myversion')

4. Call request.execute() to send the request, handling exceptions in an appropriate way for your application.

5. When there is a response body, you can treat it like a Python dictionary to get at the JSON objects specified in the API reference. Note that many of the objects in responses have fields that are present only in some circumstances. You should always check to avoid key errors:

   response = request.execute()
   
   some_value = response.get('some_key') or 'default_value'

What's next

• Explore reference architectures, diagrams, and best practices about Google Cloud. Take a look at our Cloud Architecture Center.
• Familiarize yourself with the AI Platform Training API.
• Learn more about managing models in the model and job management overview.