Docker osx development: fix slow calls to mounted filesystem

If you develop in Docker you probably, you probably mount a volume containing your source code from your host’s filesystem. If your host’s is running on osx (or windows), you might encounter slowness issues.

I myself stumbled upon this problem as I was running php tests, while a colleague had it with some python. I have had a bit of trouble figuring out why and how to fix it. Is it language related? Is it OS related? … I wrote this article to help you quickly understand the cause and tackle the problem.

The cause: the file system calls

Your source code is mounted from your osx host’s filesytem to your Docker (Linux) filesystem. For them to communicate together, every call to your mounted volume uses the osxfs shared file system solution. As described in this post, a call will undergo the following process, back and forth:

  • turned into a FUSE message in the Linux VFS
  • proxied over a virtio socket by transfused
  • forwarded onto a UNIX domain socket by HyperKit
  • deserialized, dispatched and executed as a macOS system call by osxfs

Now, some languages, such as php or python, expect system calls to be instantaneous. Thus, they will retry until they get an answer. These thousands of parasite calls will result in an overhead causing latency. Although Docker has made some effort to reduce this problem, it is still insufficient, but cool opensource projects such as docker-sync solve the problem.

A workaround with docker-sync

Docker-sync is a tool that allows a two way and asynchronous synchronization. It will drastically improve the performances and make the comparable to running docker on a linux host. If you want to know more about the process, go check the documentation here.

Here is a guide to quickly solve your issue:

 

Install docker-sync

Install docker sync with rubby on any OS with the command: `gem install docker-sync`.

 

Configure your project to use docker-sync

Let’s say you have a Dockerfile and a docker-compose.yaml. A minimal docker-compose.yaml would look something like that

Create a docker-sync.yaml file and past in this configuration. More options are available, you can check them here:

Create the volume that will keep the src defined above by running docker volume create api-sync

Update your docker compose to refer to this volume

Run the command docker-sync start --config=<path-to-docker-sync.yaml>

You should define sync for every volume you mount from your host. For instance, if your db stores persistent data...

Note that if you change your docker-sync.yaml, run docker-sync clean and docker-sync start  to refresh your volumes. Run docker-sync stop to stop docker-sync

 

Use docker-sync

You are now ready to use your app docker-compose up -d.

 

I hope it will solve your problem quickly. Note that this solution is also available for windows. Please go support docker-sync on Github. Also, check our other articles on docker and DevOps tools.

Emmanuel Lilette

Emmanuel Lilette

Emmanuel est Cloud Engineer chez Padok

What do you think? Leave your comments here !