Auto DevOps with GitLab CI and Docker-Compose


Sean BradleyOverview of the Finished GitLab CI Process

You are here because you want to set up an auto deployment pipeline for your amazing full stack project.

In this article I will show you how to do this yourself.

The project I will use is a prebuilt NodeJS API, which when run, is exposed behind a Nginx proxy, and containerised using docker-compose. The project exists in GitHub under several branches Dev, UAT, Staging and Production. You can create and manage your branches any way you wish.

The NodeJS API Application as seen from .gitLab-ci.yml

The code is hosted at https://github.com/Sean-Bradley/Seans-TypeScript-NodeJS-CRUD-REST-API-Boilerplate

The finished process will be,

  1. You commit and push your changes to a GitHub branch
  2. GitLab CI will detect the latest changes on GitHub and mirror the changes to your GitLab repository.
  3. GitLab CI will then trigger the GitLab Runner to start the pipeline for the particular branch, which will pull the latest code, build and run the docker-compose process.
  4. If no errors occurred during build and deployment, your latest commit will now be live.

The NodeJS API process is started within a docker-compose process and exposed behind another Nginx proxy process, with a static folder where you can place static CSS, JavaScript and HTML. The Nginx proxy forwards all /api/ calls to the NodeJS API, and everything else to the static folder. So you can choose the front end framework of your choice.

Docker-compose is a great tool in this setup since the internal application, your chosen components and internal architecture are treated agnostically by GitLab runner. You internal application does not have to be NodeJS and Nginx in order for docker-compose and GitLab CI to work together to create a CI/CD workflow for your application. This method works equally well for Python, Java, C#, Angular, React, MongoDB, MySQL, Redis or whatever stack you desire. I have done all of them in the past and there are sure to be more. Basically, whatever you can start within Docker-Compose, can be deployed using the method in this video and only requires one person, yourself.

Since I use GitHub for my projects version control, I setup GitLab to mirror the existing GitHub repository.

In this examples, I’ve purchased an Ubuntu server from Digital Ocean. I could have used many of the other cloud providers, for example AWS, AZURE, GCP, Hetzner, LeaseWeb, Linode or millions of them. But I chose Digital Ocean since it is very easy to use, the cheapest out of all of them, it’s very quick to set up, and has been 100% reliable from my experience.

I chose the $5 Ubuntu Droplet from Digital Ocean

After I get my Ubuntu server details in my email, I then connect to it, log in, then install GitLab runner and register it.

The GitLab runner, upon detecting changes to the chosen branches, will pull the latest commit and rebuild on the servers where I’ve installed the GitLab runners.

The commands used on my Ubuntu server where,

Add GitLab’s official repository:

$ curl -L https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.deb.sh | sudo bash

Install the latest version of GitLab runner:

$ sudo apt-get install gitlab-runner

I also have another article that shows how to install GitLab Runner on your Ubuntu or Centos.

Register the runner. During registering, it will require a coordinator URL and CI token. You can find these in your GitLab repository → Settings → CI/CD → Runners page. Disable ‘Shared Runners’ while you are there. Take note of the details in the ‘Specific Runners’ section

$ sudo gitlab-runner register [follow prompts]

Open the nano editor on your Ubuntu server

$ sudo nano /etc/sudoers

add

gitlab-runner ALL=(ALL) NOPASSWD: ALL

to the bottom of the file, then save

Then install docker from docker.io. This last install step is optional, but I found it prevents many issues I’ve had when trying to run docker-compose from the GitLab runners shell executor. So I manually install it now.

$ sudo apt install docker.io

In the video, I set up only the UAT deployment branch. This was very quick, and from this you can understand how you would set up the other branches you’d require for your CI/CD. You can view the files on GitHub and see how I set up the .gitlab-ci.yml file to watch for changes in each particular branch.

Create your .gitlab-ci.yml file

In my sample .gitlab-ci.yml above, the important lines are at line 5, where I indicate the steps, being test and deploy, I then create nodes for deploy and name them step-develop, step-uat, step-deploy-staging and step-deploy-production. Each of these sub steps indicates which stage(s) it relates to. Eg, on line 10, step-develop is part of stage test, and on line 21, step-uat is part of stage deploy.

Another important factor of this .gitlab-ci.yml is the only property on lines 14, 25, 41, 57. This is the GitLab branch it is listening to. So any push to the staging branch for example, will trigger a new step-deploy-staging pipeline and all the commands from lines 45 -50 will be executed on the server that I setup specifically for Staging

When setting up the GitLab runners on your specific servers, you are asked for a tag. On my UAT specific server, I entered uat. The gitlab runner on my UAT server now only runs pipelines with the uat tag. See line 27 in the above .gitlab-ci.yml

Now, with everything setup, any commits and pushes to the UAT branch, will auto deploy and build on my dedicated UAT server. This same setup would be built for my dedicated Staging and Production servers. The limits of the details of your setup are up to you.

Watch the 11 minute video tutorial to see it done

Hire me, to build the proto type for your next amazing application. https://www.linkedin.com/in/seanbradleymcdba/

Leave a Comment

Your email address will not be published. Required fields are marked *