Debugging node issues

This page explains how to debug node issues using a suite of preinstalled debugging tools.

Overview

Each GKE on-prem cluster you create is composed of several nodes. Each GKE on-prem node includes a distribution of CoreOS' toolbox, a shell script that unpacks and runs a debugging container, debug-toolbox. debug-toolbox is a container image that includes several useful debugging tools.

If you encounter issues with a specific node, you can attempt debugging by connecting to the affected node, run the toolbox script to unpack and run the debug-toolbox container, and run the tools included in the container.

Tools included in debug-toolbox container

The debug-toolbox container runs a Debian base image that includes the following packages:

• bash
• curl
• dnsutils
• hping3
• iperf3
• lsof
• netcat
• mtr
• procps
• strace
• tcpdump
• traceroute
• util-linux

Since these tools are included in the container, they don't require an internet connection. If you want to install additional debugging tools, you use apt-get, which does require an internet connection.

Using toolbox

1. SSH into the cluster node.

2. Run the toolbox command:

   sudo toolbox

   This command starts a debug-toolbox container.

3. While inside the container, run one of the tools. For example, tcpdump.

4. When you're finished, exit the container and close the SSH connection to the node.