Some time ago I decided to setup seperate homelab network in my house, as my previous homelab server is more like a production homeserver which I don’t want to disrupt. It is not much, but I wanted to build something small, energy-efficient and from the parts I already had.

But back to the topic that made me curious - I kinda needed some persistent storage for pods in lab. Then I remembered that the Mikrotik (RB951G-2HnD) router i’m using have a USB port and I saw an option in winbox to create SMB share. I think you can image where it is going.

Recently, I was trying to upgrade my toy K3s cluster using System Upgrade Plans operator. Applied example manifest, control-plane upgraded without issues, but pod for upgrading worker node was constantly restarting with error in logs exec /bin/upgrade.sh: exec format error. I would kinda expect that issue on ARM system (although we come a long way in the last couple of years!), but my nodes are running on x86 CPU, so what is happening?

Currently, I have small, two-node sandbox k3s cluster on my Proxmox homeserver. It is great for testing new things in Kubernetes ecosystem, but currently i’m missing one thing that is pretty awesome on “real” cloud environments with managed Kubernetes services - additional persistent volumes. Of course, I can use local-path to get persistent storage in my cluster, but it would be cool to attach separate virtual disks per pod. That’s when i found proxmox-csi-driver.

Issue

$ minikube tunnel
✅  Tunnel successfully started
📌  NOTE: Please do not close this terminal as this process must stay alive for the tunnel to be accessible ...
🏃  Starting tunnel for service balanced.
[email protected]'s password:

Solution

You need to add minikube’s ssh key to your ssh agent

ssh-add C:\Users\user\.minikube\machines\minikube\id_rsa

Issues with ssh-add

Cannot use ssh-add, agent not running

Make sure ssh-agent is running

Get-Service ssh-agent | Set-Service -StartupType Automatic
Start-Service ssh-agent

Permissions for ‘C:\Users\user.minikube\machines\minikube\id_rsa’ are too open.

In case you get “Permissions for ‘C:\Users\user\.minikube\machines\minikube\id_rsa’ are too open.” you might need to copy id_rsa to desktop and change permissions https://superuser.com/a/1296046