Overview
This document shows how to deploy a privileged daemonset in each node of Google Distributed Cloud to modify kubelet parameters to disable read-only ports.
Prerequisite
Make sure your Google Distributed Cloud is healthy before running the following patch script. This solution can be used to patch both admin clusters and user clusters. And it should work for all Google Distributed Cloud versions.
Save the following Daemonset YAML to your local file (e.g. patch.yaml)
apiVersion: apps/v1 kind: DaemonSet metadata: name: onprem-node-patcher namespace: kube-system spec: selector: matchLabels: name: onprem-node-patcher updateStrategy: type: RollingUpdate template: metadata: labels: name: onprem-node-patcher spec: tolerations: - operator: Exists volumes: - name: host hostPath: path: / hostPID: true initContainers: - name: read-only-patcher image: "ubuntu" env: - name: KUBELET_READONLY_PORT value: "0" # Number of 1G hugepages. Update the value as desired. command: - /bin/bash - -c - | set -xeuo pipefail configfile="/host/var/lib/kubelet/config.yaml" kubeletservice="/host/etc/systemd/system/kubelet.service" # $1: The read-only port for the kubelet to serve on with no # authentication/authorization (set to 0 to disable) function set-readonly-port-in-config() { [[ "$#" -eq 1 ]] || return local readonlyport; readonlyport="$1" local actual; actual="$(grep readOnlyPort "${configfile}")" if [[ "${actual}" == "" ]]; then echo "readOnlyPort: ${readonlyport}" >> "${configfile}" else sed -E -i 's/readOnlyPort: [0-9]+/readOnlyPort: 0/g' ${configfile} fi echo "Successfully append readOnlyPort: ${readonlyport} to ${configfile}" } sed -E -i 's/--read-only-port=[0-9]+/--read-only-port='"${KUBELET_READONLY_PORT}"'/g' ${kubeletservice} [[ -f ${configfile} ]] && set-readonly-port-in-config "${KUBELET_READONLY_PORT}" echo "Restarting kubelet..." chroot /host nsenter -a -t1 -- systemctl daemon-reload chroot /host nsenter -a -t1 -- systemctl restart kubelet.service echo "Success!" volumeMounts: - name: host mountPath: /host resources: requests: memory: 5Mi cpu: 5m securityContext: privileged: true containers: - image: gcr.io/google-containers/pause:3.2 name: pause # Ensures that the pods will only run on the nodes having the correct # label. nodeSelector: "kubernetes.io/os": "linux"
Patch on the admin cluster
kubectl apply -f patch.yaml \ --kubeconfig ADMIN_CLUSTER_KUBECONFIG
Patch on the user cluster
kubectl apply -f patch.yaml \ --kubeconfig USER_CLUSTER_KUBECONFIG
Restore
To re-enable the read-only port, manually edit the environment variable
KUBELET_READONLY_PORT
in the Daemonset YAML.Once you save the changes, the daemonset will be re-run to modify the kubelet accordingly.
Caveats
This patch has the same lifecycle as your installed 3P apps. You can run it anytime as a day 2 operation. But it might not persist after you re-create the cluster. To make this change persistent, deploy this daemonset as a step in the Google Distributed Cloud post-initialization action.
After running once, the Kubelet config file should be modified and reloaded. You can safely run
kubectl delete -f patch.yaml
to clean up daemonset resources.Anthos running on Windows does not currently support this patch.
The following metrics are lost for 1.13 and older cluster versions: