Writing An Optimized Dockerfile

Introduction

Optimization is a pretty good idea in general.  When it comes to Docker images, the way your Dockerfile is structured can have a significant impact on your CI runtime and local environment build time.  By making use of the optimization tools docker provides to us,  we will be reducing the final image size by ~90% and reducing the image rebuild time by ~95%.

Bold claims, let’s get started!

The Available Techniques

Since this article is pretty long, here is a list of the strategies/techniques that are covered, each of them linking to the appropriate heading on this page.

Multi-Stage Dockerfile

You’re likely familiar with a Dockerfile looking something like this:

FROM node:13.3.0
WORKDIR /home/node/app
ADD package.json package-lock.json ./
RUN npm run build
RUN npm ci
EXPOSE 4000

ENTRYPOINT ["sh", "-c", "./start.sh"]

It gets the job done, which is usually good enough for a first iteration.  However, when it comes to docker images, there is often a significant contrast between an image being functional, and an image is properly designed.  Let’s take a look at what a multi-stage image might look like:

FROM node:13.3.0 as builder
RUN mkdir -p /home/node/app/build
WORKDIR /home/node/app
COPY ./package.json ./package-lock.json .
COPY ./src ./src
RUN npm ci
RUN npm run build

FROM alpine
RUN apk add --update --no-cache nodejs-current-npm && mkdir /home/app
WORKDIR /home/app
COPY --from=builder /home/node/app/start.sh /home/node/app/package.json ./
COPY --from=builder /home/node/app/build ./build
COPY --from=builder /home/node/app/migrations ./migrations
COPY --from=builder /home/node/app/node_modules ./node_modules
EXPOSE 4000

ENTRYPOINT ["sh", "-c", "./start.sh"]

A few things to point out, the first being multiple FROM commands. When we use more than one FROM command in a Dockerfile, docker will create intermediate images that are excluded from the final artifact. This is handy for when you need to compile your code before executing it.

In the example above, I use the node image to install my dependencies and compile my source code and then use the alpine image to execute it. Doing this results in a significantly smaller final artifact due to the alpine image being 5MB in size while the node:13.3.0 image is 934MB.

A final thing to point out is the use of the –from flag on the COPY commands, the value specified here needs to map to an image in a previous stage of the Dockerfile; the behavior of this flag is to change the source volume of the COPY.

Optimal Syntax (Reducing Layers)

Every time you use the RUN, COPY, or ADD command in a docker file, it will generate a new layer.  I personally don’t know exactly what layers consist of, but every layer you create will increase the size of your image so you want to try and combine/remove as many of them as possible.  There are a couple of strategies to reduce your layers, here are a few that I’ve found to be most effective.

Combining RUN commands

Combining your RUN commands is a super-easy way to cut down on layers, take the following commands:

RUN apk update
RUN apk upgrade
RUN apk add bash

These can easily be combined into a single command, resulting in a single cache layer instead of 3.

RUN apk update && apk add --upgrade --no-cache bash

Notice the use of –no-cache, this flag will prevent APK from caching the packages, which helps with keeping the image small.

Combining COPY commands

Similar to RUN commands, COPY commands can in some cases be combined into a one-liner, saving even more layers.

Unoptimized:

COPY /home/node/app/start.bash .
COPY /home/node/app/knexfile.js .
COPY /home/node/app/package.json .
COPY /home/node/app/build ./build

Optimized:

COPY ./start.bash ./package.json ./package-lock.json  . 
COPY ./build ./build

Notice how copying the build directory is still its own layer, this is because it is a directory rather than an individual file.

Order Layers By Likelihood of Change

Before docker builds an image, it decides what layers need to be rebuilt, and what layers can be pulled from the cache.  As soon as a layer is marked as needing to be rebuilt, all subsequent layers will also be rebuilt.  This has the potential to ruin a lot of the work we’ve done to optimize the image so far, so let’s make sure we have things in the optimal order.

First off, we need to know how docker decides if a layer needs to be rebuilt.  There are likely multiple factors it takes into account, but the most straightforward conditions are:

  1. Have the contents of a COPY/ADD command changed?
  2. Has a previous layer of this image been rebuilt?

Given those facts, let’s take a look at an example.

FROM node:13.3.0
WORKDIR /home/node
COPY . .
RUN npm ci && npm run build

There are a few things wrong here, the first being the COPY command.  The resulting layer from this command will almost never be cached because if anything is changed in the project (that is not listed in .dockerignore), docker will need to rebuild the layer.  An optimal setup would look more like this:

FROM node:13.3.0
WORKDIR /home/node
COPY package.json package-lock.json ./
RUN npm ci
COPY webpack.config.js .babelrc ./
COPY src ./src
RUN npm run build

This will result in more layers, yes, but it’s better to optimize for the cache first, layer count second; the cache will have a greater overall impact.  The largest improvement this setup results in would be better dependency caching since docker will only rebuild the RUN npm ci layer if the contents of package/package-lock.json changes.

Multi-Stage Dockerfile with Dependency Cache

Separating our build process into an intermediate image is a great first step into multi-stage dockerfiles.  Let’s now take things a step further and separate our dependency installation process into yet another intermediate image.  Continuing the node project example, the following commands would install our dependencies:

FROM node:13.3.0 as cache
WORKDIR /tmp
ADD package.json package-lock.json ./
RUN npm ci

From there, we would just need to pull the dependencies out of the cache for runtime.  Combined with the above, that would look something like:

FROM node:13.3.0 as cache
WORKDIR /tmp
ADD package.json package-lock.json ./
RUN npm ci

FROM node:13.3.0 as builder
# ...compile our source into /home/app/build

FROM alpine
RUN apk add --update --no-cache nodejs-current-npm && mkdir /home/app
WORKDIR /home/app
COPY --from=cache /tmp/node_modules ./node_modules
COPY --from=builder /home/app/build ./build
EXPOSE 4000
ENTRYPOINT ["node", "./build/bundle.js"]

Make use of .dockerignore

In my experience, the .dockerignore file is wrongfully one of the most overlooked features of Docker.  So, if you aren’t aware, a .dockerignore is a file in the root of your project that contains a list of files and patterns.

The purpose of a .dockerignore file is to tell docker what to ignore when sending your source code to the daemon.  If you have a COPY . . anywhere in your Dockerfile and you’re not yet using a dockerignore file, you’ll see an immediate performance increase once you add one.

I’d recommend ignoring node_modules and .git for sure, as they can get quite large.  Past that, it’s a matter of going through your project and looking for files you don’t need at build/runtime, it won’t end up being much, but every little bit adds up so adding things like .eslintrc, other ignore files, CI files, test screenshots, etc is a good idea.

For specifics on the capabilities of this file, I encourage you to check out the official docker documentation for this topic.

Audit Production Image Dependencies

By this point, we’ve used all of the builtin docker image optimization techniques (at least the ones I’m aware of).  The only thing left is to manually go through our Dockerfile and see if there is anything we can get rid of.

It’s tough to give step-by-step instructions for this, so I will instead share an example of refactoring my Dockerfile to remove a dependency.  Take the following example, where we are executing a bash script as our entrypoint:

FROM alpine
RUN apk add --update --no-cache bash nodejs-current-npm
# ...
ENTRYPOINT ["bash", "-c", "./start.bash"]

This is a pretty standard setup, but it can be optimized.  You may have guessed this already, but since the alpine image ships with shell, we can use the sh executable instead of bash.  (As long as the bash script can be easily converted to a shell script, which it could in my case.)  First, make sure that you change #!/bin/bash to #!/bin/sh in your bash(soon to be shell) script.  Then, simply remove the bash dependency and use the sh executable instead of bash:

FROM alpine
RUN apk add --update --no-cache nodejs-current-npm
# ...
ENTRYPOINT ["sh", "-c", "./start.sh"]

Putting It All Together (My Ideal Dockerfile) (NodeJS)

So now that we’ve learned all these new optimization methods, let’s put them to use and see what implementing them into a real-world, production dockerfile looks like.  This example is again a NodeJS example, simply because that is the language I work the most with; these optimization methods are specific to docker, not NodeJS, and will work with any language.  Starting our dockerfile off with a cache stage, responsible for installing and caching NPM packages:

FROM node:13.3.0 as cache
WORKDIR /tmp
ADD package.json package-lock.json ./
RUN npm ci

The next stage is responsible for compiling the source JavaScript and then pruning the NPM packages so we don’t ship dev dependencies to production.

FROM node:13.3.0 as builder
RUN mkdir -p /home/node/app/build
WORKDIR /home/node/app
ARG NODE_ENV=development
ENV NODE_ENV $NODE_ENV
COPY --from=cache /tmp/node_modules ./node_modules
COPY . .
RUN npx webpack && npm prune

The ARG NODE_ENV=development and the following ENV commands allow this dockerfile to accept a NODE_ENV build argument, build args can be passed to a dockerfile by using the –build-arg flag, see the official docker documentation for more info.

Everything has now been compiled, the only remaining step is to copy the artifact into an alpine image and run it.  This stage is the most important because the resulting image is the one that will actually be pushed to production.

FROM alpine
RUN apk add --update --no-cache nodejs-current-npm && mkdir /home/app
WORKDIR /home/app
COPY --from=builder /home/node/app/node_modules ./node_modules
COPY --from=builder /home/node/app/start.sh /home/node/app/knexfile.js /home/node/app/package.json ./
COPY --from=builder /home/node/app/migrations ./migrations
COPY --from=builder /home/node/app/build ./build
EXPOSE 4000

ENTRYPOINT ["node", "./build/dist.js"]

And that’s it!

Conclusion

Thanks for reading, I hope you learned something new that you can take and apply to your Dockerfiles.  If you have any questions or comments, my Twitter is @DevZacx so feel free to reach out on there, or in the comments section of this article.

Will The Layer Caching Work In My Pipeline?

Short answer: no, but it can.

Longer Answer: The layer caching optimizations are what yield the greatest benefit, but since it is a cache-based optimization, we will need to figure out a way to preserve the cache between consecutive pipeline runs so we can realize the full benefit of layer caching.

This can be a complex task that can be different depending on what CI provider is in use, my next article will cover how to build, test, and deploy this Dockerfile.  So stay tuned for that.

Further Reading

After all this optimization research, I was put into an optimization-obsessed mindset and found myself embarking on a quest to write the perfect dockerfile.  Though that is a noble quest, it’s not really practical after a certain point because the law of diminishing returns starts to kick in and the performance gains start to become trivial.

I’ve ended this article at what I feel is the point where the performance improvements have peaked and are starting to decline.  If you’d like to take things even further, here are a few resources to get you started in the right direction:

Dockerfile Best Practices

Creating a Custom Base Image

Setting up automated builds on Docker hub

Leave a Reply