Skip the build queue and save money by reducing (if not eliminating) dedicated build machines by Bringing Your Own Device (BYOD) as CI/CD build machine to run pipelines using Docker.
I have been utilising GitLab to build and host my blog since mid of August 2017. They offer 2000 hours of free build time per month. Occasionally, I found that my build was queued up to slightly over 10 minutes. Build time also fluctuates between a low of 1 minute and a high of 2.5 minutes. Understandably, these are shared runners with varying machine specifications. Usually, the build runs as soon as code gets pushed into remote branch.
Last weekend, I read that it is possible to register my own build runner into GitLab so I decided to give it a try.
What is Docker?
Docker is the world’s leading software container platform. Developers use Docker to eliminate “works on my machine” problems when collaborating on code with co-workers. Operators use Docker to run and manage apps side-by-side in isolated containers to get better compute density. Enterprises use Docker to build agile software delivery pipelines to ship new features faster, more securely and with confidence for both Linux, Windows Server, and Linux-on-mainframe apps.
Source: https://www.docker.com/what-docker
With Docker, you can run builds on a container. Trusted containers can be found on Docker Store. In my case, I use NodeJS image as it comes with npm. This gives me access to a huge library of Node modules useful for running my builds.
Follow the Docker installation instructions. Specifically, I installed Docker Community Edition.
GitLab Runner can be installed onto a variety of operating systems including Raspberry Pi Raspbian Jessie.
See a list of arm32v7 Docker images. Notice that Node 8.6.0 Alpine image is not available, with the issue still open on GitHub. As an alternative, I switched to the default Node image:
image: node:8.6.0
I have tried the Slim variant but it does not have git which I need for pushing the build output to GitHub. If you do not need or you are not sure, give the Slim variant a try:
image: node:8.6.0-slim
Go to your GitLab repository. Then click on Settings and navigate to CI/CD.
Next, Expand the Runners settings. You should be seeing Specific Runners and Shared Runners. Disable the latter.
Under Specific Runners, you should be seeing:
“Use the following registration token during setup: <your token>”.
You are going to need this token in the next step.
Follow GitLab’s instructions on Registering Runners.
In my case, I literally followed all instructions written there except tags association and disallow running untagged jobs. I tagged my runner with docker,linux following the pattern of tag names defined on shared runners.
You need to add tags to your GitLab CI YAML file.
pages:
tags:
- docker
Push your code to remote branch and watch the pipeline spring to life.
The first few lines of the output should be:
Running with gitlab-runner 10.0.0 (2055cfdc)
on <my machine name> (<runner ID>)
Using Docker executor with image node:8.6.0-alpine ...
Using docker image sha256:60c6962e4e7c3e4390c8af1b833e3a9ed80aedcb2d933f7c04f77ccfed49ec62 for predefined container...
Pulling docker image node:8.6.0-alpine ...
Using docker image node:8.6.0-alpine ID=sha256:b7e15c83cdaf2bf47a6db862f342d8d176477e54cc271b0803e0f1c7273cd5a7 for build container...
The final few lines should be:
Uploading artifacts...
public: found <number> matching files
Uploading artifacts to coordinator... ok
Note: On first pull of a Docker image that you have not pulled down before. Build time is expected to be longer as the image has to be downloaded and cached locally. Subsequent pulls are taken from local cache.
If you no longer have any use of one or more Docker images, you could save disk space by removing them.
To see a list of Docker images in your local cache:
sudo docker images
To delete a specific Docker image from your local cache:
sudo docker rmi <IMAGE ID>
I setup Docker on my Pi to use a USB stick as its data-root. Builds for my blog were taking almost 10 minutes to complete. That is if it completes successfully. Over 90% of the time, I encountered the following error:
ERROR: Job failed (system failure): Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
I suspected that the I/O performance and latency of USB stick on Pi 2 was not playing nice with GitLab Runner so I shifted the Docker data-root directory back to its default directory on my micro SD card and builds now take less than half the time to complete and most importantly without errors!