Authenticating with OpenID Connect (OIDC)

Alpha

This product or feature is subject to the "Pre-GA Offerings Terms" in the General Service Terms section of the Service Specific Terms. Pre-GA products and features are available "as is" and might have limited support. For more information, see the launch stage descriptions.

Caution: Be sure to plan your installation ahead of time. Your clusters' configurations, as declared in the GKE On-Prem configuration file, become immutable after you create the clusters. You can't change most aspects of your clusters after they are created, including networking and authentication. The only configurable exceptions are:

• Resizing user clusters
• Creating additional user clusters
• Upgrading clusters
• Creating and managing node pools

This page shows how to configure GKE On-Prem to use an OpenID provider for authentication to user clusters. To learn how to use OIDC with AD FS, see Authenticating with OIDC and AD FS.

For an overview of the GKE On-Prem authentication flow, see Authentication.

Overview

GKE On-Prem supports OpenID Connect (OIDC) as one of the authentication mechanisms for interacting with a user cluster's Kubernetes API server. With OIDC, you can manage access to Kubernetes clusters by using the standard procedures in your organization for creating, enabling, and disabling employee accounts.

There are two ways an employee can use the OIDC authentication flow:

• An employee can use kubectl to initiate an OIDC flow. To make this flow automatic, GKE On-Prem provides the Kubectl Plugin for OIDC, a kubectl plugin.

• An employee can use Google Cloud console to initiate an OIDC authentication flow.

In this exercise, you configure both options: kubectl and Google Cloud console.

Before you begin

This topic assumes you are familiar with OAuth 2.0 and OpenID Connect. This topic assumes you are familiar with OpenID scopes and claims.

Choosing an OpenID provider

This section is for administrators.

You can use any OpenID provider of your choice. For a list of certified providers, see OpenID Certification.

One possibility is to use Active Directory Federated Services (AD FS) as your OpenID provider, and use an on-premises instance of Active Directory as your employee database. For more information, see Authenticating with OIDC and AD FS.

Downloading the Kubectl Plugin for OIDC

This section is for administrators and employees who want to use the Kubectl Plugin for OIDC.

Download the plugin and set access permissions:

Linux

gcloud storage cp gs://gke-on-prem-release/oidc-plugin/v1.1alpha/linux_amd64/kubectl-oidc .
chmod +x kubectl-oidc

Windows

gcloud storage cp gs://gke-on-prem-release/oidc-plugin/v1.1alpha/windows_amd64/kubectl-oidc .

macOS

gcloud storage cp gs://gke-on-prem-release/oidc-plugin/v1.1alpha/darwin_amd64/kubectl-oidc .
chmod +x kubectl-oidc

Installing the plugin

Install the plugin by moving the executable file to any location on your PATH. The executable file must be named kubectl-oidc. To learn more, see Installing kubectl plugins.

Creating a redirect URL for the Kubectl Plugin for OIDC

This section is for administrators.

As part of establishing a relationship with your OpenID provider, you must specify a redirect URL that the provider can use to return ID tokens to Kubectl Plugin for OIDC. The Kubectl Plugin for OIDC runs on each employee's local machine and listens on a port of your choice. Choose a port number greater than 1024 that is suitable for this purpose. Then, the redirect URL is:

http://localhost:[PORT]/callback

where [PORT] is your port number.

When you configure your OpenID provider, specify http://localhost:[PORT]/callback as one of your redirect URLs. How you do this depends on your provider.

Configuring a redirect URL for Google Cloud console

This section is for administrators.

In addition to having a redirect URL for kubectl, you need a redirect URL for Google Cloud console. The redirect URL for Google Cloud console is:

https://console.cloud.google.com/kubernetes/oidc

When you configure your OIDC provider, specify https://console.cloud.google.com/kubernetes/oidc as one of your redirect URLs. How you do this depends on your provider.

Registering your clients applications with the OpenID provider

This section is for administrators.

Before your employees can use the Kubectl Plugin for OIDC or Google Cloud console with your OpenID provider, you need to register those two clients with the OpenID provider. Registration includes these steps:

• Learn the provider's issuer URI. This is where the Kubectl Plugin for OIDC or Google Cloud console sends authentication requests.

• Give the provider the redirect URL for the Kubectl Plugin for OIDC.

• Give the provider the redirect URL for Google Cloud console. This is https://console.cloud.google.com/kubernetes/oidc.

• Establish a single client ID. This is the ID that the provider uses to identify both the Kubectl Plugin for OIDC and Google Cloud console.

• Establish a single client secret. The Kubectl Plugin for OIDC and Google Cloud console both use this secret to authenticate to the OpenID provider.

• Establish a custom scope that the Kubectl Plugin for OIDC or Google Cloud console can use to request the user's security groups.

• Establish a custom claim name that the provider will use to return the user's security groups.

How you perform these steps depend on your OpenID provider. To learn how to perform the registration steps with AD FS, see Authenticating with OIDC and AD FS.

Populating the oidc specification in GKE On-Prem configuration file

This section is for employees who want to create a cluster that is configured to use OIDC.

Before you create a user cluster, you generate a GKE On-Prem configuration file using gkectl create-config. The configuration includes the following oidc specification. You populate oidc with values specific to your provider:

oidc:
  issuerurl:
  clientid:
  clientsecret:
  username:
  usernameprefix:
  group:
  groupprefix:
  scopes:
  extraparams:
  usehttpproxy:
  capath:

• issuerurl: Required. URL of your OpenID provider, such as https://example.com/adfs. Client applications, like the Kubectl Plugin for OIDC and Google Cloud console, send authorization requests to this URL. The Kubernetes API server uses this URL to discover public keys for verifying tokens. Must use HTTPS.
• clientid: Required. ID for the client application that makes authentication requests to the OpenID provider. Both the Kubectl Plugin for OIDC and Google Cloud console use this ID.
• clientsecret: Optional. Secret for the client application. Both the Kubectl Plugin for OIDC and Google Cloud console use this secret.
• username: Optional. JWT claim to use as the username. The default is sub, which is expected to be a unique identifier of the end user. You can choose other claims, such as email or name, depending on the OpenID provider. However, claims other than email are prefixed with the issuer URL to prevent naming clashes with other plugins.
• usernameprefix: Optional. Prefix prepended to username claims to prevent clashes with existing names. If you do not provide this field, and username is a value other than email, the prefix defaults to issuerurl#. You can use the value - to disable all prefixing.
• group: Optional. JWT claim that the provider will use to return your security groups.
• groupprefix: Optional. Prefix prepended to group claims to prevent clashes with existing names. For example, given a group foobar and a prefix gid-, gid-foobar.
• scopes: Optional. Additional scopes to send to the OpenID provider as a comma-delimited list.
• extraparams: Optional. Additional key-value parameters to send to the OpenID provider as a comma-delimited list.
  • For a list of authentication parameters, see Authentication URI parameters
  • For a list of Microsoft Azure's authentication parameters, see Send the sign-in request.
  • If you are authorizing a group, pass in resource=token-groups-claim.
  • If your authorization server prompts for consent, pass in prompt=consent.
• usehttpproxy: Optional. Specifies whether to deploy a reverse proxy in the cluster to allow Connect Agent access to the on-premises OIDC provider for authenticating users. Value must be a string: "true" or "false".
• capath: Optional. Path to the certificate for the certificate authority (CA) that issued your identity provider's web certificate. This value might not be necessary. For example, if your identity provider's certificate was issued by a well-known public CA, then you would not need to provide a value here.

Example: Authorizing users and groups

Many providers encode user-identifying properties, such as email and user IDs, in a token. However, these properties have implicit risks for authentication policies:

• User IDs can make policies difficult to read and audit.
• Emails can create both an availability risk (if a user changes their primary email) and potentially a security risk (if an email can be re-assigned).

Therefore, it's a best practice to use group policies, as a group ID can be both persistent and easier to audit.

Suppose your provider creates identity tokens that include the following fields:

{
  'iss': 'https://server.example.com'
  'sub': 'u98523-4509823'
  'groupList: ['[e-mail]', '[e-mail]']
  ...
}

Given this token format, you'd populate your configuration file's oidc specification like so:

issueruri: 'https://server.example.com'
username: 'sub'
usernameprefix: 'uid-'
group: 'groupList'
groupprefix: 'gid-'
extraparams: 'resource=token-groups-claim'
...

After you've created the user cluster, you could then use Kubernetes role-based access control (RBAC) to grant privileged access to the authenticated users. For example, you could create a ClusterRole that grants its users read-only access to the cluster's Secrets, and create a ClusterRoleBinding resource to bind the role to the authenticated group:

ClusterRole

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: secret-reader
rules:
- apiGroups: [""]
  # The resource type for which access is granted
  resources: ["secrets"]
  # The permissions granted by the ClusterRole
  verbs: ["get", "watch", "list"]

ClusterRoleBinding

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: read-secrets-admins
subjects:
  # Allows anyone in the "us-east1-cluster-admins" group to
  # read Secrets in any namespace within this cluster.
- kind: Group
  name: gid-us-east1-cluster-admins # Name is case sensitive
  apiGroup: rbac.authorization.k8s.io
  # Allows this specific user to read Secrets in any
  # namespace within this cluster
- kind: User
  name: uid-u98523-4509823
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: ClusterRole
  name: secret-reader
  apiGroup: rbac.authorization.k8s.io

Saving the certificate of the CA of the Kubernetes API server

This section is for employees who have created a user cluster and now want to use the Kubectl Plugin for OIDC.

Your user cluster has a Kubernetes API server. And your user cluster's kubeconfig file stores the certificate of the CA that issued a certificate to the Kubernetes API server. The CA's certificate is the base-64 encoded value of the certificate-authority-data field. You need to decode this value and store it in a local file, like server-ca-cert:

cat [USER_CLUSTER_KUBECONFIG]  | grep certificate-authority-data | awk '{ print $2}' | base64 --decode > server-ca-cert

Generating the client authentication configuration file

This section is for employees who have created a user cluster and now want to use the Kubectl Plugin for OIDC.

To generate a client authentication configuration file, enter the following command:

Linux

kubectl oidc client-config \
--issuer-uri [ISSUER_URI] \
--redirect-uri [REDIRECT_URL] \
--client-id [CLIENT_ID] \
--client-secret [CLIENT_SECRET] \
--scopes "[CUSTOM_SCOPES]" \
--cluster-name [USER_CLUSTER_NAME] \
--server [CLUSTER_URL] \
--server-ca-file [SERVER_CA_CERT] \
--issuer-ca-file [PROVIDER_CA_CERT] \
--extra-params [KEY]=[VALUE], ... # e.g. --extra-params "resource=token-groups-claim"
> client-config.yaml

where:

• [ISSUER_URI] is your issuer URI.
• [REDIRECT_URL] is the redirect URL for the Kubectl Plugin for OIDC
• [CLIENT_ID] is the client ID for the Kubectl Plugin for OIDC.
• [CLIENT_SECRET] is the client secret for the Kubectl Plugin for OIDC.
• [USER_CLUSTER_NAME] is your user cluster's name.
• [CLUSTER_URL] is the URL of the user cluster's Kubernetes API server.
• [SERVER_CA_FILE] is the path to the certificate of the CA that issued a certificate to the Kubernetes API server. This is the certificate file that you created in the previous section.
• [PROVIDER_CA_CERT] is the path to the certificate of the CA that signed the OpenID provider's certificate. This is the same as the value of oidc:cacert in your cluster configuration file.
• [CUSTOM_SCOPES] is the comma-separated list of your custom scopes for security groups. This is the same as the value of oidc:scopes in your cluster configuration file.
• --extra-params [KEY]=[VALUE], ... is a list of comma-delimited key-value pairs to be included in authorization requests to the OpenID provider.
  • For a list of authentication parameters, see Authentication URI parameters
  • If you are authorizing a group, pass in resource=token-groups-claim.
  • If your authorization server prompts for consent, pass in prompt=consent.

PowerShell

kubectl oidc client-config `
--issuer-uri [ISSUER_URI] `
--redirect-uri [REDIRECT_URL] `
--client-id [CLIENT_ID] `
--client-secret [CLIENT_SECRET] `
--scopes "[CUSTOM_SCOPES]" `
--cluster-name [USER_CLUSTER_NAME] `
--server [CLUSTER_URL] `
--server-ca-file [SERVER_CA_CERT] `
--issuer-ca-file [PROVIDER_CA_CERT] `
--extra-params [KEY]=[VALUE]
> client-config.yaml

where:

• [ISSUER_URI] is your issuer URI.
• [REDIRECT_URL] is the redirect URL for the Kubectl Plugin for OIDC
• [CLIENT_ID] is the client ID for the Kubectl Plugin for OIDC.
• [CLIENT_SECRET] is the client secret for the Kubectl Plugin for OIDC.
• [USER_CLUSTER_NAME] is your user cluster's name.
• [CLUSTER_URL] is the URL of the user cluster's Kubernetes API server.
• [SERVER_CA_FILE] is the path to the certificate of the CA that issued a certificate to the Kubernetes API server. This is the certificate file that you created in the previous section.
• [PROVIDER_CA_CERT] is the path to the certificate of the CA that signed the OpenID provider's certificate. This is the same as the value of oidc:cacert in your cluster configuration file.
• [CUSTOM_SCOPES] is the comma-separated list of your custom scopes for security groups. This is the same as the value of oidc:scopes in your cluster configuration file.
• --extra-params [KEY]=[VALUE], ... is a list of comma-delimited key-value pairs to be included in authorization requests to the OpenID provider.
  • For a list of Google's authentication parameters, see Authentication URI parameters
  • For a list of Microsoft Azure's authentication parameters, see Send the sign-in request.
  • If you are authorizing a group, pass in resource=token-groups-claim.
  • If your authorization server prompts for consent, pass in prompt=consent.

This command produces a client authentication configuration file called client-config.yaml. Do not manually edit this file.

Authenticating against a user cluster using the Kubectl Plugin for OIDC

This section is for employees who have created a user cluster and now want to use the Kubectl Plugin for OIDC.

1. Initialize the plugin using the client-config.yaml file:

   kubectl oidc login --clientconfig-file=client-config.yaml --user [NAME] \
       --kubeconfig [KUBECONFIG_OUTPUT_PATH]

   where:

   • [NAME] is your username.
   • [KUBECONFIG_OUTPUT_PATH] is the path to the kubeconfig file where the Kubectl Plugin for OIDC will store credentials.

   kubectl oidc login launches a browser where you can enter your credentials.

   The kubeconfig file provided now contains an ID token that kubectl can use to authenticate to the Kubernetes API server on the user cluster.

   Note: Windows users might have to run the command as kubectl-oidc.exelogin instead of kubectl oidc login.

2. Verify that the authentication was successful by running any kubectlcommand. For example:

   kubectl get nodes --kubeconfig [KUBECONFIG_OUTPUT_PATH]

Using OIDC with Google Cloud console

This section is for employees who have created a user cluster and now want to use Google Cloud console to authenticate against the cluster.

1. Verify that your cluster is configured for OIDC.

2. Verify that your cluster has been registered with Google Cloud, either automatically during cluster creation or manually.

3. Visit the Kubernetes clusters page in Google Cloud console.

   Visit the Kubernetes clusters page

4. In the list of clusters, locate your GKE On-Prem cluster, and click Login.

5. Select Authenticate with the Identity Provider configured for the cluster, and click LOGIN.

   You will be redirected to your identity provider, where you might need to log in or consent to Google Cloud console accessing your account. Then you will be redirected back to the Kubernetes clusters page in Google Cloud console.

Troubleshooting OIDC in GKE On-Prem

Invalid configuration

If Google Cloud console cannot read the OIDC configuration from your cluster, the LOGIN button will be disabled.

Invalid provider configuration

If your identity provider configuration is invalid, you will see an error screen from your identity provider after you click LOGIN. Follow the provider-specific instructions to correctly configure the provider or your cluster.

Invalid permissions

If you complete the authentication flow, but still don't see the details of the cluster, make sure you granted the correct RBAC permissions to the account that you used with OIDC. Note that this might a different account from the one you use to access Google Cloud console.

Error: missing 'RefreshToken' field in 'OAuth2Token' in credentials struct

You might get this error if the authorization server prompts for consent, but the required authentication parameter wasn't provided. Provide the prompt=consent parameter to GKE On-Prem configuration file's oidc: extraparams field, and regenerate the client authentication file with the --extra-params prompt=consent flag.