Configuring the order of build steps

You can specify the order in which your build steps are executed. By default, build steps run sequentially, but you can configure them to run concurrently.

This page explains how to configure the order of the build steps.

Build step order and dependencies

Use the waitFor field in a build step to specify which steps must run before the build step is run. If no values are provided for waitFor, the build step waits for all prior build steps in the build request to complete successfully before running.

To run a build step immediately at build time, use - in the waitFor field.

The order of the build steps in the steps field relates to the order in which the steps are executed. Steps will run serially or concurrently based on the dependencies defined in their waitFor fields.

A step is dependent on every id in its waitFor and will not launch until each dependency has completed successfully.

Steps without the optional waitFor field (or with an empty waitFor) will wait for all prior steps to complete successfully before executing. Therefore, if no step contains an id in its waitFor field, then all steps will execute serially in the order they are defined.

Steps can depend on the start of the build by having waitFor contain only -. By declaring that a step depends only on -, the step runs immediately when the build starts. The first step defined depends implicitly on start.

The following snippet shows a build config with two steps that runs serially:

YAML 

steps:
- name: foo
- name: bar

JSON 

{
    "steps": [{
        "name": "foo"
    },
    {
        "name": "bar"
    }
    ]
}

The following snippet shows two concurrent steps that both depend on start; the third step waits for the first two to complete successfully before launching. This concurrent build runs steps A and B at the start of the build. The third step will wait implicitly until both previous steps are finished before starting. This example could be simplified by omitting the id fields, which are not referenced in a subsequent waitFor.

YAML 

steps:
- name: foo
  id: A
- name: bar
  id: B
  waitFor: ['-']
- name: baz

JSON 

{
    "steps": [
    {
        "name": "foo",
        "id": "A"
    },
    {
        "name": "bar",
        "id": "B",
        "waitFor": ["-"]
    },
    {
        "name": "baz"
    }
    ]
}

The following snippet shows concurrent steps that depend on a previous step. Step A runs immediately when the build starts. Steps B and C run concurrently after A has completed successfully. Note that the id and waitFor fields in step B, and the id field in step C, could be omitted without changing the order of execution.

YAML 

steps:
- name: foo
  id: A
- name: bar
  id: B
  waitFor:
  - A
- name: baz
  id: C
  waitFor:
  - A

JSON 

{
    "steps": [
    {
        "name": "foo",
        "id": "A"
    },
    {
        "name": "bar",
        "id": "B",
        "waitFor": [
            "A"
        ]
    },
    {
        "name": "baz",
        "id": "C",
        "waitFor": [
            "A"
        ]
    }
    ]
}

Examples

The example below calls the gsutil and wget steps concurrently. After these steps have completed, the ubuntu step is called.

YAML

steps:
# Download the binary and the data in parallel.
- name: 'gcr.io/cloud-builders/wget'
  args: ['https://example.com/binary']
- name: 'gcr.io/cloud-builders/gsutil'
  args: ['cp', 'gs://$PROJECT_ID-data/rawdata.tgz', '.']
  waitFor: ['-']  # The '-' indicates that this step begins immediately.

# Run the binary on the data, once both are downloaded.
- name: 'ubuntu'
  args: ['./binary', 'rawdata.tgz']

JSON

{
  "steps": [
    {
      "name": "gcr.io/cloud-builders/wget",
      "args": [
        "https://example.com/binary"
      ]
    },
    {
      "name": "gcr.io/cloud-builders/gsutil",
      "args": [
        "cp",
        "gs://$PROJECT_ID-data/rawdata.tgz",
        "."
      ],
      "waitFor": [
        "-"
      ]
    },
    {
      "name": "ubuntu",
      "args": [
        "./binary",
        "rawdata.tgz"
      ]
    }
  ]
}

The example below uses the id field to identify certain build steps. The values from id are used in waitFor to define build step order:

  • First, fetch-resources step uses gsutil to copy the local resources from Cloud Storage. Concurrently, go generates, tests, and installs the source code.

  • Then, the docker build step builds the image after all other steps are complete.

YAML 

steps:
- name: 'gcr.io/cloud-builders/go'
  args: ['generate']
- name: 'gcr.io/cloud-builders/go'
  args: ['test', './...']
- name: 'gcr.io/cloud-builders/go'
  args: ['install', 'mytarget']
  id: 'go-install'

- name: 'gcr.io/cloud-builders/gsutil'
  args: ['cp', '-r', './somefiles', 'gs://my-resource-bucket/somefiles']
  waitFor: ['-']  # The '-' indicates that this step begins immediately.
  id: 'fetch-resources'

- name: 'gcr.io/cloud-builders/docker'
  args: ['build', '-t', 'gcr.io/$PROJECT_ID/mytarget', '.']
  waitFor: ['go-install', 'fetch-resources']

images: ['gcr.io/$PROJECT_ID/mytarget']

JSON 

{
  "steps": [
    {
      "name": "gcr.io/cloud-builders/go",
      "args": [
        "generate"
      ]
    },
    {
      "name": "gcr.io/cloud-builders/go",
      "args": [
        "test",
        "./..."
      ]
    },
    {
      "name": "gcr.io/cloud-builders/go",
      "args": [
        "install",
        "mytarget"
      ],
      "id": "go-install"
    },
    {
      "name": "gcr.io/cloud-builders/gsutil",
      "args": [
        "cp",
        "-r",
        "./somefiles",
        "gs://my-resource-bucket/somefiles"
      ],
      "waitFor": [
        "-"
      ],
      "id": "fetch-resources"
    },
    {
      "name": "gcr.io/cloud-builders/docker",
      "args": [
        "build",
        "-t",
        "gcr.io/$PROJECT_ID/mytarget",
        "."
      ],
      "waitFor": [
        "go-install",
        "fetch-resources"
      ]
    }
  ],
  "images": [
    "gcr.io/$PROJECT_ID/mytarget"
  ]
}

What's next