Time has come to migrate away from Docker Desktop on my Mac work laptop. Rancher Desktop is one of the usual suggestions, but since it uses a tool called colima underneath, and I really never used the Docker Desktop GUI, might as well go full-on command line. Let’s get started!
What is colima?
colima is a nice little project which facilitates installing a container runtime on macOS (and Linux). On macOS Docker is never fully native, the CLI tool connects to a Docker daemon running on a tiny Linux VM.
colima uses lima under the hood to accomplish this. It automates the creation of a small Alpine Linux based VM to run Docker on.
Getting everything installed
Assuming you already uninstalled Docker Desktop, let’s go ahead and install colima and the Docker CLI tools:
brew install colima docker
Since I will need them later, let’s also install Docker Compose and Buildx:
brew install docker-buildx docker-compose
mkdir -p ~/.docker/cli-plugins
ln -sfn /opt/homebrew/opt/docker-buildx/bin/docker-buildx ~/.docker/cli-plugins/docker-buildx
ln -sfn /opt/homebrew/opt/docker-compose/bin/docker-compose ~/.docker/cli-plugins/docker-compose
Once we have everything installed we can get colima started and test if Docker is running:
docker run --rm hello-world
You should get something like this:
Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
70f5ac315c5a: Pull complete
Status: Downloaded newer image for hello-world:latest
Hello from Docker!
This message shows that your installation appears to be working correctly.
To generate this message, Docker took the following steps:
1. The Docker client contacted the Docker daemon.
2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
3. The Docker daemon created a new container from that image which runs the
executable that produces the output you are currently reading.
4. The Docker daemon streamed that output to the Docker client, which sent it
to your terminal.
To try something more ambitious, you can run an Ubuntu container with:
$ docker run -it ubuntu bash
Share images, automate workflows, and more with a free Docker ID:
For more examples and ideas, visit:
Assigning more resources to the colima VM
The default VM colima creates has 2 vCPUs, 2GBs of RAM and 60GBs of disk storage, let’s customize that:
colima start --cpu 4 --memory 8 --disk 60
That’s it, now we have colima running with 4 vCPUs and 8GBs of RAM, nice!
Problem 1: permission errors when running chown
Here I feel like I need to do a quick aside. Docker is not part of my everyday development workflow when working on Jitsi, but I do maintain our Docker setup. Our Docker setup is reasonably involved, and we do a couple of non-mainstream things here and there, such as fiddling with permissions in files / folders inside mounted volumes.
The first problem I found was issue #83, I was getting permission errors when starting my containers. Thankfully, the answer was right there in the comments, we need to change the mount type to
9p (the default is
sshfs) and add an override file.
First make sure you stop and delete the colima VM:
Add this content into a file in `
- location: "/Users/<username>"
- location: "~"
- location: /tmp/colima
Start colima again, with the
9p mount type (remember to still set the previous CPU and RAM options!):
colima start --cpu 4 --memory 8 --disk 60 --mount-type 9p
No more permission errors, nice!
Problem 2: UDP forwarding to the host does not work
UDP support in the Docker ecosystem has always felt a bit like an afterthought and this time it took me a bit to figure it out.
With colima all the (TCP) ports used by your containers are forwarded to the host machine, so you can access them comfortably in
localhost. That doesn’t work for UDP though. The reason is that lima uses SSH to forward the ports, which doesn’t support UDP. Here is the lima issue.
In order to get Jitsi Meet working on macOS we need to configure the
JVB_ADVERTISED_IPS environment variable, which must point to the public (or LAN since it’s for testing) IP of the host machine running the JVB container. A way around this is to tell colima to assign an IP address to the VM, and then use that:
colima start --cpu 4 --memory 8 --disk 60 --mount-type 9p --network-address
We can now check with
INFO colima is running using QEMU
INFO arch: aarch64
INFO runtime: docker
INFO mountType: 9p
INFO address: 192.168.106.2
INFO socket: unix:///Users/saghul/.colima/default/docker.sock
The IP address we can use is
192.168.106.2 and won’t change, so that’s handy. Another potential solution would be to configure bridging and have the VM get a “real IP”, but honestly this Just Works (TM) so I didn’t bother.
Problem 3: docker login
When attempting to use
docker login I stumbled upon this error:
Error saving credentials: error storing credentials
This looked very similar to issue #167, likely caused by the fact I had Docker Desktop installed before.
~/.docker/config.json and remove the
"credsStore" line, that will do!
Problem 4: Buildx woes
Buildx is the new way to build Docker images using BuildKit. It allows one to build multi-architecture images easily. Since we release amd64 and arm64 images for Jitsi Meet, Buildx is a tool I’d like to use, but ran into colima issue #792, all amd64 images were producing an error at build time.
The problem seems to be some cgroups v2 shenanigans with the default Alpime image colima uses, but thankfully there is a workaround, creating a custom Buildx builder with the
docker buildx create --name mybuilder --driver-opt 'image=moby/buildkit:rootless' --bootstrap --use
That did it! Now we can build multi-arch images with no problem.
While it wasn’t a walk in the park, replacing Docker Desktop with colima forced me to look under the hood and understand how a number of the moving parts work. Overall I’m happy I made the switch!