Creating a Docker image¶

GitHub CI/CD¶

Docker images of all versions of COMPAS (dev branch) are available at the TeamCOMPAS dockerHub page. Building of these images is performed automatically by the GitHub CI/CD process.

Whenever a push to TeamCOMPAS/dev occurs, a continuous deployment process automatically builds1 a new image and deploys it to dockerHub with a tag2 that corresponds to the value of VERSION_STRING in changelog.h (see Change log for detailed information regarding changelog.h).

At time of writing, GitHub Actions3 facilitates the above process. While this is convenient (because it's free and well supported) it is somewhat slow - the COMPAS Docker image is available 5 - 10 minutes after pushing/merging. A future improvement may be to create a runner4 locally with a high core count that can be used to compile COMPAS quickly.

The Github Actions configuration is in /.github/workflows/dockerhub-ci.yml.

See the Atlassian CI/CD documentation for detailed information regarding the GitHub CI/CD process.

Dockerfile¶

The Dockerfile5 defines how the docker image is constructed.

Images are created as a combination of layers. During the build process, each layer is cached and only updated on subsequent builds if that layer would change.

The Dockerfile for COMPAS is made up of 8 layers:

FROM ubuntu:18.046
Use Ubuntu 18.04 as a base (provided by Docker Hub)

WORKDIR /app/COMPAS7
Effectively cd /app/COMPAS within the container.

RUN apt-get update && apt-get install -y ...8
Install the required dependencies.

-y so there's no prompt to install any of the packages.

update and install are in the same layer because now if there are any updates, it will force all of the dependencies to be re-installed

RUN pip3 install numpy8
Install numpy.

COPY src/ src9
Copy ./src/ directory from the local machine to ./src in the container (remembering that WORKDIR changes the cwd).

RUN mkdir obj bin logs8
Create the directories required by COMPAS.

ENV COMPAS_ROOT_DIR /app/COMPAS10
Set the required environment variable(s).

RUN cd src && make -f Makefile.docker -j $(nproc)8
Change to the src directory; make COMPAS using a specific makefile (see below) and as many cores as possible.

A Dockerfile usually ends with a CMD directive that specifies what command should run when the container is started11. The COMPAS Dockerfile doesn't have a CMD directive because some users will want to run the executable directly and some will want to use runSubmit.py.

Makefile.docker¶

A separate makefile is required for Docker in this scenario for two reasons:

To separate compiled files from source files

To prevent the usage of the gcc -march=native compiler option

-march=native is a very useful optimisation for users who compile and run COMPAS on the same machine, however it causes fatal errors when running COMPAS on a machine for which it was not compiled. The -march=native compiler option selects the CPU for which code should be generated by determining the processor type of the compiling machine. Using -march=native enables all instruction subsets supported by the compiling machine, thus producing an executable file that will perform better than it would if the full native instruction set was not available, but some of those instructions may not be available for use on machines of different architectures - hence the possible fatal run-time errors if the code is run on machines with different architectures). See gcc x86-Options for detailed information regarding -march.

The Docker makefile provided (Makefile.docker) functions exactly like the local makefile provided (Makefile) in all other respects. See Building COMPAS locally for a detailed description of the local makefile functionality.

Footnotes