Creating a Docker image

Overview

Docker simplifies COMPAS deployment by containerizing all dependencies and build tools, eliminating manual environment configuration across different operating systems and platforms. This enables reliable deployment on cloud platforms and consistent development environments.

For information on how to use Docker to run COMPAS, see the COMPAS and Docker page.

GitHub CI/CD

Docker images of all versions of COMPAS (dev branch) are available at:

Building of these images is performed automatically by the GitHub CI/CD process.

Whenever a push to TeamCOMPAS/dev occurs, a continuous deployment process automatically builds[1] a new image and deploys it to both dockerHub and the GitHub Container Registry with a tag[2] that corresponds to the value of VERSION_STRING in changelog.h (see Change log for detailed information regarding changelog.h).

At time of writing, GitHub Actions[3] facilitates the above process. While this is convenient (because it's free and well supported) it is somewhat slow - the COMPAS Docker image is available 5 - 10 minutes after pushing/merging. A future improvement may be to create a runner[4] locally with a high core count that can be used to compile COMPAS quickly.

The Github Actions configuration is in /.github/workflows/dockerhub-ci.yml.

See the Atlassian CI/CD documentation for detailed information regarding the GitHub CI/CD process.

Using GitHub Container Images

The GitHub Container Registry makes COMPAS images available without requiring a dockerHub account. You can use the GitHub image in the same way as a dockerHub image:

# Pull the latest development image
docker pull ghcr.io/teamcompas/compas:latest

# Pull a specific version
docker pull ghcr.io/teamcompas/compas:v02.43.01

# Run a container
docker run -it ghcr.io/teamcompas/compas:latest /bin/bash

For more information, see the GitHub Container Registry.

Dockerfile

The Dockerfile[5] defines how the docker image is constructed.

Images are created as a combination of layers. During the build process, each layer is cached and only updated on subsequent builds if that layer would change.

The Dockerfile for COMPAS is made up of 8 layers:

FROM ubuntu:18.04[6]
Use Ubuntu 18.04 as a base (provided by Docker Hub)

WORKDIR /app/COMPAS[7]
Effectively cd /app/COMPAS within the container.

RUN apt-get update && apt-get install -y ...[8]
Install the required dependencies.

  • -y so there's no prompt to install any of the packages.

  • update and install are in the same layer because now if there are any updates, it will force all of the dependencies to be re-installed

RUN pip3 install numpy[8]
Install numpy.

COPY src/ src[9]
Copy ./src/ directory from the local machine to ./src in the container (remembering that WORKDIR changes the cwd).

RUN mkdir obj bin logs[8]
Create the directories required by COMPAS.

ENV COMPAS_ROOT_DIR /app/COMPAS[10]
Set the required environment variable(s).

RUN cd src && make -f Makefile.docker -j $(nproc)[8]
Change to the src directory; make COMPAS using a specific makefile (see below) and as many cores as possible.

A Dockerfile usually ends with a CMD directive that specifies what command should run when the container is started[11]. The COMPAS Dockerfile doesn't have a CMD directive because some users will want to run the executable directly and some will want to use runSubmit.py.

Makefile.docker

A separate makefile is required for Docker in this scenario for two reasons:

  1. To separate compiled files from source files

  2. To prevent the usage of the gcc -march=native compiler option

-march=native is a very useful optimisation for users who compile and run COMPAS on the same machine, however it causes fatal errors when running COMPAS on a machine for which it was not compiled. The -march=native compiler option selects the CPU for which code should be generated by determining the processor type of the compiling machine. Using -march=native enables all instruction subsets supported by the compiling machine, thus producing an executable file that will perform better than it would if the full native instruction set was not available, but some of those instructions may not be available for use on machines of different architectures - hence the possible fatal run-time errors if the code is run on machines with different architectures). See gcc x86-Options for detailed information regarding -march.

The Docker makefile provided (Makefile.docker) functions exactly like the local makefile provided (Makefile) in all other respects. See Building COMPAS locally for a detailed description of the local makefile functionality.

Footnotes