Set up an Envoy sidecar service mesh on GKE

This page describes how to set up an Envoy sidecar service mesh on GKE.

Prerequisites

As a starting point, this guide assumes that you have already:

Set up the Service

  1. Create a sample HTTP service:

    kubectl apply -f - <<EOF
    kind: Namespace
    apiVersion: v1
    metadata:
      name: sidecar-example
    ---
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: whereami
      namespace: sidecar-example
    spec:
      replicas: 2
      selector:
        matchLabels:
          app: whereami
      template:
        metadata:
          labels:
            app: whereami
        spec:
          containers:
          - name: whereami
            image: gcr.io/google-samples/whereami:v1.2.20
            ports:
            - containerPort: 8080
    ---
    apiVersion: v1
    kind: Service
    metadata:
      name: whereami
      namespace: sidecar-example
    spec:
      selector:
        app: whereami
      ports:
      - port: 8080
        targetPort: 8080
    EOF
    
  2. Create a baseline HTTPRoute for the service:

    apiVersion: gateway.networking.k8s.io/v1beta1
    kind: HTTPRoute
    metadata:
      name: whereami-route
      namespace: sidecar-example
    spec:
      parentRefs:
      - name: whereami
        kind: Service
        group: ""
      rules:
      - backendRefs:
        - name: whereami
          port: 8080
    EOF
    

    Alternatively, the following manifest describes a sample gRPC Service:

    apiVersion: v1
    kind: Service
    metadata:
      name: sample-service
      namespace: sample-ns
      annotations:
        networking.gke.io/app-protocols: '{"50051": "HTTP2"}' # 50051 is backendref.port
    spec:
      ports:
      - port: 50051
        targetPort: 50051
    

Set up the Client

  1. Run the following command to enable sidecar injection in the sidecar-example namespace:

    kubectl label namespace sidecar-example csm-injection=enabled
    
  2. Create a client:

    kubectl apply -f - <<EOF
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      labels:
        run: client
      name: client
      namespace: sidecar-example
    spec:
      replicas: 1
      selector:
        matchLabels:
          run: client
      template:
        metadata:
          labels:
            run: client
        spec:
          containers:
          - name: client
            image: curlimages/curl
            command:
            - sh
            - -c
            - while true; do sleep 1; done
    EOF
    
  3. Verify that the client Pod has an Envoy sidecar container automatically injected:

    kubectl get pods -n sidecar-example -l run=client
    

    The output is similar to:

    NAME                    READY   STATUS    RESTARTS   AGE
    client-xxxx             2/2     Running   0          20s
    

    Wait for the client to be ready and have the Status Running before continuing.

  4. Verify Envoy-sidecar service mesh setup. The following command sends a request to the whereami service from the client

    CLIENT_POD=$(kubectl get pod -n sidecar-example -l run=client -o=jsonpath='{.items[0].metadata.name}')
    
    # The VIP where the following request will be sent. Because all requests
    # from the client container are redirected to the Envoy proxy sidecar, you
    # can use any IP address, including 10.0.0.2, 192.168.0.1, and others.
    VIP='10.0.0.1'
    
    TEST_CMD="curl -v -H 'host: whereami.sidecar-example.svc.cluster.local' $VIP"
    
    kubectl exec -it $CLIENT_POD -n sidecar-example -c client -- /bin/sh -c "$TEST_CMD"
    

    The output is similar to:

    < Trying 10.0.0.1:80...
    < Connected to 10.0.0.1 (10.0.0.1) port 80 (#0)
    < GET / HTTP/1.1
    < Host: whereami
    < User-Agent: curl/7.82.0-DEV
    < Accept: */*
    <
    < Mark bundle as not supporting multiuse
    < HTTP/1.1 200 OK
    < content-type: application/json
    < content-length: 318
    < access-control-allow-origin: *
    < server: envoy
    < date: Tue, 12 Apr 2022 22:30:13 GMT
    <
    {
      "cluster_name": "${CLUSTER_NAME}",
      "location": "${LOCATION}",
      "host_header": "whereami",
      ...
    }