Update: minikube 0.29.0 has been released and includes my merged PR so you can enable embedded certificates with minikube config set embed-certs true once and then just symlink your .kube/config file from your WSL home directory to the same file in your Windows home directory.

---

 

I recently blogged about how I work with minikube from the Windows Subsystem for Linux (WSL), describing some of the friction points and workarounds.

At the time I recommended using the Windows version of kubectl to avoid needing to translate the Windows file paths found in the .kube/config file to be usable with the Linux version of kubectl. Also, I hadn’t at that time encountered any use cases where using the Linux kubectl inside WSL would work better than the Windows version.

The first scenario most people would likely encounter with different or breaking behaviour would be passing absolute file paths, e.g. kubectl apply -f /home/jason/my.yaml would usually fail to locate the file with the Windows version. This is worked around most often by using paths relative to the working directory.

Another scenario where the Linux version of kubectl is preferred is the TTY support when running kubectl exec --tty mypod. This was the reason I personally decided to get the Linux version of kubectl working with minikube in my WSL environment.

My first approach was to copy the .kube/config file that is created in my Windows user profile directory during minikube start, modify the three certificate paths to be WSL-compatible paths, and save the result in my WSL home directory.

Later I realised (from the files generated by kubeadm init on production Kubernetes clusters) that the certificate entries in the config file don’t need to be paths, but can have the certificate content embedded as base64 blobs. Naturally I wrote a bash script that I could run from WSL to perform these steps for me each time my minikube IP address or certificates changed (which to be fair, isn’t often). The script will use the translated paths approach by default but if executed with the --embed parameter it will use the embedded certificates alternative.

After using this solution for a while I began wondering why minikube didn’t just generate a .kube/config file with embedded certificates so WSL support could be solved with a simple symlink instead of copying and rewriting the file each time it changed.

So I dived into the minikube source, and raised a pull request, and as at the time of writing this post, the PR has been merged into master and is just awaiting an official release. Once the new version of minikube is published (or if you’re keen to build it from source yourself), you will be able to execute minikube config set embed-certs true once and then minikube will always generate a .kube/config file with the certificates embedded as base64 blobs.

Then you can symlink your WSL ~/.kube/config file to your Windows %USERPROFILE%/.kube/config file and use either version of kubectl with no ongoing management.

PS: hat tip to Nuno do Carmo who also found a solution to embedding the certificates in the .kube/config file by using a pair of kubectl config ... --embed-certs commands. See the “Bonus 3: Do the same with Minikube” section of his extensive blog on WSLinux+K8S: The Interop Way”.

I develop services that run on Kubernetes. During development minikube provides an convenient way to run a local Kubernetes “cluster” regardless of whether you use Windows, OS X, or a Linux distribution as your host OS.

Day-to-day I use minikube on Windows 10 and I prefer to use the Windows Subsystem for Linux (WSL) bash shell to have a scripting environment consistent with my colleagues, some of whom do not use Windows, and consistent with the CI system.

The Linux binary of minikube isn’t very useful in WSL since it doesn’t support the Hyper-V driver and the Virtualbox driver cannot deal with the path differences it sees within WSL compared to those reported by VboxManage.exe.

However, when running the Windows minikube.exe binary, many of the commands (e.g. start, stop, ip, dashboard) just work without any special configuration. Furthermore, creating a symlink so minikube can be executed on the PATH without the .exe extension easily improves the default experience. Beyond these initial commands though, some extra effort is required.

SSH can be a little flakey with minikube ssh so I find it better to create an alias to use WSL’s ssh client:

ssh -a -i "$(wslpath -u "$(minikube ssh-key)")" -l docker "$(minikube ip)"

You may get an error from this SSH command that the permissions of the identity file are too open. This is fixed in two steps. Firstly ensure you have added metadata to the automount options in your /etc/wsl.conf and restart your WSL session.

Secondly, change WSL’s view of the permissions of the key file:

chmod 0600 "$(wslpath -u "$(minikube ssh-key)")"

The minikube docker-env command doesn’t recognise the WSL environment and outputs the PowerShell environment commands instead. This can be worked around by passing the --shell bash arguments but the DOCKER_CERT_PATH environment variable value won’t work with the docker Linux binary as-is and the Windows binary needs the WSLENV environment variable set appropriately. These extra steps are enough to justify a helper script which I’ve published as a GitHub Gist. With this script dot-sourced, both the Windows and Linux binaries for the Docker client will work with Minikube’s Docker daemon.

Lastly, use the Windows binary for kubectl. The paths for Kubernetes certificates in the .kube/config file make it difficult to use the kubectl Linux binary and so far I haven’t found a problem with the Windows version.