Fast Kubernetes Development with File Sync and Smart Rebuilds

Oct 11, 2018

What if I told you that you didn't have to rebuild your docker images every time you made a change?

I'm happy to share with you a feature I added in the last release of skaffold that instantly syncs files to your running containers without any changes to your deployments or extra dependencies.

You can get it today with skaffold 0.16.0


All you need for this tutorial is a running Kubernetes cluster and kubectl.

I'm going to be:

  1. Creating a Flask python app
  2. Dockerizing it
  3. Kuberneterizing it
  4. Watching my changes instantly get reflected in the cluster

If you'd prefer to just clone the repository, you can get these 4 files at https://github.com/r2d4/skaffold-sync-example

Creating the Flask App

app.py

from flask import Flask
app = Flask(__name__)

@app.route('/')
def hello_world():
    return 'Hello, World from Flask!'

Dockerizing it

Dockerfile

FROM python:3.7-slim

RUN pip install Flask==1.0
COPY *.py .

ENV FLASK_DEBUG=1
ENV FLASK_APP=app.py
CMD ["python", "-m", "flask", "run"]

The two environment variables tell flask to print stack traces and reload on file changes.

  • I used python-slim to work with a smaller image
  • With more than one requirement, you'll want to create a separate requirements.txt file and COPY that in. We're only using flask so I kept it simple here.
  • Did you know? That before Docker 1.10 ENV and other commands used to create layers. Now, only RUN, COPY, and ADD do. So go ahead and add those cheap commands to the end of your Dockerfile.

Kuberneterizing it

k8s-pod.yaml

apiVersion: v1
kind: Pod
metadata:
  name: python
spec:
  containers:
  - name: python
    image: gcr.io/k8s-skaffold/python-reload
    ports:
    - containerPort: 5000

If Kubernetes were carbon based life, the Pod would be the atom. If you're not using minikube or Docker for Desktop, you're going to need to change that image name to something you can push to.

The Magic

skaffold.yaml

apiVersion: skaffold/v1alpha4
kind: Config
build:
  artifacts:
  - image: gcr.io/k8s-skaffold/python-reload
    sync:
      '*.py': .
deploy:
  kubectl:
    manifests:
    - k8s/**

This is the last YAML file I'm going to make you copy, I swear.

The magic here is the "sync" field, that tells skaffold to sync any python file to the container when it changes.

Make sure the image name matches the image name you used above if you changed it.

  • Did you know? Skaffold supports building a few types of "artifacts" other than Dockerfiles. Anything that produces a Docker image.
  • I used a glob pattern in the deploy part of the config, and when new Kubernetes manifests are added, skaffold will be smart enough to redeploy.
  • Skaffold can also detect any changes in the skaffold.yaml itself and reload

Development

Run

skaffold dev

You should see some output ending with

$ skaffold dev
...
Port Forwarding python 5000 -> 5000
[python]  * Serving Flask app "app.py" (lazy loading)
[python]  * Environment: production
[python]    WARNING: Do not use the development server in a production environment.
[python]    Use a production WSGI server instead.
[python]  * Debug mode: on
[python]  * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
[python]  * Restarting with stat
[python]  * Debugger is active!
[python]  * Debugger PIN: 289-130-309

Follow the link to http://127.0.0.1:5000/. But, isn't my application running on Kubernetes, in Docker, possibly in a VM on my computer or in the cloud?

Yep. Skaffold is smart enough to port-forward any ports in your deployments to your laptop. You don't have to worry about exposing your development environments to the internet just to ping a URL. The connection is secure between you and your cluster.

Go ahead and make some changes to your Flask app. Whatever you want: change the message, add more routes, add more files python files, delete some files.

Now check the output on your skaffold dev terminal.

....
Synced files for gcr.io/k8s-skaffold/python-reload:dirty-2db9f3d...
Copied: map[app.py:app.py]
Deleted: map[]
Watching for changes...
[python]  * Detected change in '/app.py', reloading
[python]  * Restarting with stat
[python]  * Debugger is active!
[python]  * Debugger PIN: 289-130-309
...

If you visit http://127.0.0.1:5000/, you'll see the changes that you made to your image, nearly instantly.

$ kubectl get pods
NAME      READY     STATUS    RESTARTS   AGE
python    1/1       Running   0          6m

If you want to see a Node JS example in action, I've added this example and another in the official skaffold repository.