Unit Testing with the Kubernetes Client Library

How do you unit test code that makes Kubernetes API calls?

Using the Kubernetes client library can help you mock out a cluster to test your code against.

As one of the first consumers of the kubernetes/client-go library when building kubernetes/minikube, I built elaborate mocks for services, pods, and deployments to unit test my code against. Now, there's a much simpler way to do the same thing with significantly fewer lines of code.

I'm going to be showing how to test a simple function that lists all the container images running in a cluster. You'll need a Kubernetes cluster, I suggest GKE or Docker for Desktop.

Setup

Clone the example repository https://github.com/r2d4/k8s-unit-test-example if you want to run the commands and follow along interactively.

main.go

package main

import (
	"github.com/pkg/errors"
	meta_v1 "k8s.io/apimachinery/pkg/apis/meta/v1"
	"k8s.io/client-go/kubernetes/typed/core/v1"
)

// ListImages returns a list of container images running in the provided namespace
func ListImages(client v1.CoreV1Interface, namespace string) ([]string, error) {
	pl, err := client.Pods(namespace).List(meta_v1.ListOptions{})
	if err != nil {
		return nil, errors.Wrap(err, "getting pods")
	}

	var images []string
	for _, p := range pl.Items {
		for _, c := range p.Spec.Containers {
			images = append(images, c.Image)
		}
	}

	return images, nil
}

Writing the Tests

Let's start with a definition of our test cases, and some skeleton code for running the tests.

func TestListImages(t *testing.T) {
	var tests = []struct {
		description string
		namespace   string
		expected    []string
		objs        []runtime.Object
	}{
		{"no pods", "", nil, nil},
	}

	// Actual testing code goes here...
}

What's Happening

This style of writing tests is called "table driven tests" and in Go, this is the prefered style. The actual test code iterates over the table entries and performs the necessary tests. Test code is written once and used for each case. Some interesting things to note:

• Anonymous struct to hold the test case definition. They allow us to define test cases concisely.
• The Runtime Object Slice objs will hold all the runtime objects that want our mock API server to hold. We'll be populating it with some pods, but you can use any Kubernetes object here.
• The trivial test case. No pods on the server shouldn't return any images.

Test Loop

Let's fill out the actual test code that will run for every test case.

	for _, test := range tests {
		t.Run(test.description, func(t *testing.T) {
			client := fake.NewSimpleClientset(test.objs...)
			actual, err := ListImages(client.CoreV1(), test.namespace)
			if err != nil {
				t.Errorf("Unexpected error: %s", err)
				return
			}
			if diff := cmp.Diff(actual, test.expected); diff != "" {
				t.Errorf("%T differ (-got, +want): %s", test.expected, diff)
				return
			}
		})
	}

Some interesting things to note:

• t.Run executes a subtest. Why use subtests?
  • You can run specific test cases using the -run flag to go test
  • You can do setup and tear-down
  • And subtests are the entrypoint to running test cases in parallel (not done here)
• Actual and expected results are diffed with cmp.Diff. Diff returns a human-readable report of the differences between two values. It returns an empty string if and only if Equal returns true for the same input values and options.

fake.NewSimpleClientset returns a clientset that will respond with the provided objects.
It's backed by a very simple object tracker that processes creates, updates and deletions as-is,
without applying any validations and/or defaults.

Test Cases

Let's create a pod helper function that will help provide some pods for us to test against. Since we are concerned about namespace and image, lets create a helper that creates new pods based on those parameters.

func pod(namespace, image string) *v1.Pod {
	return &v1.Pod{ObjectMeta: meta_v1.ObjectMeta{Namespace: namespace}, Spec: v1.PodSpec{Containers: []v1.Container{{Image: image}}}}
}

Let's write three unit tests. The first will just make sure that we grab all images if we use the special namespace value "" to list pods in all namespaces.

{"all namespaces", "", []string{"a", "b"}, []runtime.Object{pod("correct-namespace", "a"), pod("wrong-namespace", "b")}}

The second case will make sure that we filter correctly by namespace, ignoring the pod in wrong-namespace

{"filter namespace", "correct-namespace", []string{"a"}, []runtime.Object{pod("correct-namespace", "a"), pod("wrong-namespace", "b")}}

The third case will make sure that we don't return anything if there are no pods in the desired namespace.

{"wrong namespace", "correct-namespace", nil, []runtime.Object{pod("wrong-namespace", "b")}}

Putting it all together.

func TestListImages(t *testing.T) {
	var tests = []struct {
		description string
		namespace   string
		expected    []string
		objs        []runtime.Object
	}{
		{"no pods", "", nil, nil},
		{"all namespaces", "", []string{"a", "b"}, []runtime.Object{pod("correct-namespace", "a"), pod("wrong-namespace", "b")}},
		{"filter namespace", "correct-namespace", []string{"a"}, []runtime.Object{pod("correct-namespace", "a"), pod("wrong-namespace", "b")}},
		{"wrong namespace", "correct-namespace", nil, []runtime.Object{pod("wrong-namespace", "b")}},
	}

	for _, test := range tests {
		t.Run(test.description, func(t *testing.T) {
			client := fake.NewSimpleClientset(test.objs...)
			actual, err := ListImages(client.CoreV1(), test.namespace)
			if err != nil {
				t.Errorf("Unexpected error: %s", err)
				return
			}
			if diff := cmp.Diff(actual, test.expected); diff != "" {
				t.Errorf("%T differ (-got, +want): %s", test.expected, diff)
				return
			}
		})
	}
}