GitLab CI push output to GitHub

Utilise GitLab CI runner to build and publish static website to GitLab and GitHub pages. By having mirrored copies, it is possible to switch DNS records to point to either for high availability reasons.

Motivation

As with most free service providers, GitLab Pages and GitHub Pages do not offer SLA. One can argue that they have historically been averaging a very respectable uptime but past performance if used as an indication should be taken with a grain of salt. That being said, it is of course in their interest to ensure high uptime and performance for their reputation.

As a user of such service, I need high availability. One way to do this is by having mirrored copies so that I could switch DNS settings at any time. Given that I use CloudFlare with orange cloud, traffic gets diverted to new destination almost instantly.

I will be sharing how to setup GitLab CI to build and publish to GitLab as well as GitHub. You can take it further by configuring 3rd party DNS traffic manager to monitor and automatically failover. This is beyond scope of this post.

Pre-requisites

Basic knowledge on how to use Git. Repositories already exist in both GitLab and GitHub. For the latter, you can start with an empty repository.

Please read my earlier post Hugo Static Site Generator with CI Deployment using GitLab. The bits below is an addition over this.

The solution

Below is my new GitLab CI YAML script:

image: node:6.11.2-alpine
before_script:
  - apk update && apk add openssl ca-certificates git
  - npm install
  - PATH=$(npm bin):$PATH
  - hugo version
pages:
  script:
  - npm run build
  - git clone --depth 1 https://<username>:$GITHUB_ACCESS_TOKEN@github.com/<username>/<username>.github.io.git
  - mkdir <username>.github.io.new
  - cp -a <username>.github.io/.git <username>.github.io.new/.git
  - cp -a public/* <username>.github.io.new
  - cd <username>.github.io.new
  - git config user.email "<GitHub email address>"
  - git config --global user.name "<Your name>"
  - git add -A
  - git diff-index --quiet HEAD || git commit -m "$CI_SERVER_NAME $CI_PIPELINE_ID"
  - git push
  artifacts:
    paths:
    - public
  only:
  - master

The key additions are:

  1. apk add git - Installation of Git since the Docker image used does not have it.
  2. git clone --depth 1 - Clones only the latest revision of repository from GitHub. The clone is downloaded to <username>.github.io directory.
  3. $GITHUB_ACCESS_TOKEN is a GitLab secret variable which stores the GitHub personal access token. When creating the token at GitHub, name the token GitLab or whatever you prefer then assign it with public_repo access.
    Then head over to GitLab Settings > Pipelines and define a secret variable named GITHUB_ACCESS_TOKEN and paste the GitHub personal access token into the Value field.
  4. mkdir <username>.github.io.new - A temporary folder; to be published from.
  5. cp -a <username>.github.io/.git <username>.github.io.new/.git - Copy the .git folder to the temporary directory. Not interested in the other stuff in the cloned directory.
  6. cp -a public/* <username>.github.io.new - Hugo static site generator outputs to public directory by default. Copy the output directory to the temporary directory.
  7. cd <username>.github.io.new - Change the current directory to the temporary directory as subsequent commands are against this.
  8. git config user.email "<GitHub email address>" - Put your GitHub email address here.
  9. git config --global user.name "<Your name>" - Put your name here.
  10. git add -A - Stage all (new, modified, and deleted) changes.
  11. git diff-index --quiet HEAD || git commit -m "$CI_SERVER_NAME $CI_PIPELINE_ID" - Commits changes. There is a chance for Git to detect no changes to public directory. i.e. when updating .gitlab-ci.yml which sits outside the public directory. When committing, set commit message to GitLab server name and pipeline ID.
  12. git push - Pushes commited changes to GitHub.

To test this out, simply Git push to GitLab and wait for the CI runner to do its thing.

Troubleshooting

GitHub CNAME file

In order for GitHub to serve content on your custom domain, a file named CNAME must be created.