The Introduction

This is the second part of a multi-part tutorial covering a simple(ish) setup of a continuous integration/deployment pipeline using Drone.io. Since Part 1, I’ve added a GitHub project outlining a simple Django application that you can use as a reference.

In Part 2, we will be adding a publish step to our application’s drone.yml in order to push our image to Docker Hub. Having our image in Docker Hub allows us to easily pull our app’s image onto our staging/production server to allow easy automated deployment. After the publish step we will add a deploy step to the drone.yml that will be responsible for SSHing into an EC2 instance (or wherever your app lives) to pull our newly pushed image and update our app.

Let’s get started!

Step 1: Publish our app’s image to Docker Hub after a successful pull request.

If you recall, our drone.yml should successfully be running our application’s test suite on push and pull events and reporting back to GitHub if our test suite failed or succeeded. After a successful build, we would like to push our app’s Docker image to Docker Hub.

Sign up/Log in to Docker Hub

Having an account on Docker Hub is free and it will allow us to easily update our application once it is on our EC2 instance.

After you have set up an account, keep note of your username and password. We will need to inject those as Drone secrets in order for Drone to push our app’s image. Read more about pushing images to Docker Hub.

In your .drone.yml, add a “publish” section so that it looks like this:

pipeline:
  build:
    image: python:3.5.2
    environment:
      - DATABASE_URL=postgres://postgres@localhost
    commands:
      - sleep 5
      - pip3 install -r requirements.txt # make sure gunicorn is installed
      - cd projectDir
      - python ./manage.py test
      - cd ..
    when:
      branch: [ master, develop ]
      event: [push, pull_request ] # trigger step on push and pull events
  publish:
    image: plugins/docker
    username: ${DOCKER_USERNAME}  # we will inject your dockerhub username using drone secrets. 
    password: ${DOCKER_PASSWORD} # we will inject your dockerhub password using drone secrets.
    email: octocat@catmail.com
    repo: octocat/repoName # refer to dockerhub documentation for repo naming conventions
    tag: latest
    file: Dockerfile
    environment:
      - DOCKER_LAUNCH_DEBUG=true #( usefull for debugging but not necessary )
    when:
      branch: [ master ]
      event: [ push ] # step only triggers on push events
services:
  database:
    image: postgres
    environment:
      - DATABASE_URL=postgres://postgres@localhost

We are telling Drone to use the plugins/docker image (read more about this particular plugin or plugins in general). In order for the plugin to login to your Docker Hub account so it can push your image it will need your username and password passed in as Drone secrets.

On your local machine (assuming the drone CLI is setup correctly in Part 1 ), run:

$ drone secret add --image=plugins/docker octocat/repoName DOCKER_USERNAME yourDockerUsername
​$ drone secret add --image=plugins/docker octocat/repoName DOCKER_PASSWORD yourDockerPassword 

To make sure your secrets have been set correctly, you can run:

$ drone secret ls octocat/repoName 

and you should see something like:

DOCKER_USERNAME 
Events: push, tag, deployment
SkipVerify: false
Conceal: false

DOCKER_PASSWORD 
Events: push, tag, deployment
SkipVerify: false
Conceal: false

Now that we are using secrets, you must sign your project by running the following command inside your projects folder (so that the Drone CLI will create a .drone.yml.sig file next to your .drone.yml file: 

$ drone sign octocat/repoName 

Now, every time you change your .drone.yml locally, you will have to run the sign command to generate a new sig file. If you don’t, then the next Drone build will not have access to your secrets. Drone is pretty good at displaying a warning in the console of the build if this is the case.

Go ahead and open another pull-request. After the tests pass and you’ve merged your new code in, you should see Drone run another build but this time executing the new publish step we outlined in the drone.yml. If problems occur you should at least see some helpful output thanks to the DOCKER_LAUNCH_DEBUG=true line in your .drone.yml. Drone secrets and plugins can be fickle. I’ve found that even if you do not explicitly declare environment variables in your .drone.yml all secrets you’ve set for that plugin and repo are passed into the resulting Docker container pertaining to that build step. So if you were to remove the following two lines from your .drone.yml file: 

username: ${DOCKER_USERNAME}  # we will inject your dockerhub username using drone secrets. 
password: ${DOCKER_PASSWORD} # we will inject your dockerhub password using drone secrets.

Your Docker username and password would still be passed into the publish step’s Docker container, resulting in a successful Docker Hub login. This can lead to some odd behavior with certain plugins (this phenomenon might be specifically plugin-dependant so I encourage you to read all plugin documentation).

If the build passes, go ahead and visit your Docker Hub account. You should see a newly pushed image as a result! We’re almost there!

Step 2: Create a service to start your app’s Docker container

If you already have a service to run your Django application inside a Docker container, you can skip this step.

Here you have two choices. Either spin up a new EC2 container and install Docker (which will you cost you a small amount of money with two running free-tier instances) or you can execute the below steps on the same EC2 instance that you put your Drone server/agent on (though I have not tested this and the exact steps might deviate). SSH into the instance you want you app to be hosted on.

Create an Init Script for your Django Application

This example uses systemd, but there are some alternatives such as upstart. Create a service script as follows:

$ vim /etc/systemd/system/projectName.service 

[Unit]
Description=yourApp Container
Requires=docker.service
After=docker.service

[Service]
Restart=always
ExecStart=/usr/bin/docker run --name=containerName -p 8000:8000 octocat/repoName bash -c "gunicorn smashDB.wsgi -b 0.0.0.0:8000"
ExecStop=/usr/bin/docker stop -t 2 repoName
ExecStopPost=/usr/bin/docker rm -f repoName

[Install]
WantedBy=default.target

Here we are assuming Gunicorn is installed in your app. The first -p 8000:8000 is telling all traffic on port 8000 that reaches our EC2 container to be forwarded to the created Docker container on port 8000. This allows traffic from 0.0.0.0 to be translated so Django can process them. We’ve set the Restart flag to always so that if the container goes down for whatever reason, a new one will take it’s place. The ExecStop and ExecStopPost command here just tell systemd to clean up after itself.

To start using the service, reload systemd and start the service:

systemctl daemon-reload
systemctl start projectName.service

If you want the service execute whenever your EC2 instance starts, run:

systemctl enable docker-redis_server.service

Now if you run:

sudo docker ps

you should see a Docker container running your Django app.

Create a deploy.sh file

Now that our container is running, we can create a sh file that will be responsible for stopping our service, stopping and removing our app’s Docker container, pulling our app’s latest image from Docker Hub, and finally restarting our service. Create a deploy.sh file (I created mine in ~/deploy/) . Your deploy.sh file should look something like this:

#!/bin/bash
echo "Updating staging Server"

echo "stopping projectName.service"
sudo systemctl stop projectName.service

# remove all outdated images and containers
echo "removing outdated/dangling images and containers"
sudo docker rm $(sudo docker ps -aq)
sudo docker rmi $(sudo docker images --filter dangling=true --quiet)

# pull new image for projectName
echo "pulling new image for myProject"
sudo docker pull octocat/repoName

# restart service which will use the newly pulled image
echo "restarting projectName service"
sudo systemctl start projectName.service

# App is updated!
echo "projectName successfuly updated!"

Make the file executable

sudo chmod +x deploy.sh

and execute manually just to make sure it is working:

sh deploy.sh

You should see the echo statements and if you run:

docker ps

you should see that your app was just newly created. With your service and deploy.sh we can now add a publish step to our drone.yml!

Step 3: Add a deploy step to your drone.yml

In your drone.yml, add a ssh-deploy step so that it looks like this:

pipeline:
  build:
    image: python:3.5.2
    environ ment:
      - DATABASE_URL=postgres://postgres@localhost
    commands:
      - sleep 5
      - pip3 install -r requirements.txt # make sure gunicorn is installed
      - cd projectDir
      - python ./manage.py test
      - cd ..
    when:
      branch: [ master, develop ]
      event: [push, pull_request ] # trigger step on push and pull events
  publish:
    image: plugins/docker
    username: $DOCKER_USERNAME  # we will inject your dockerhub username using drone secrets. 
    password: $DOCKER_PASSWORD # we will inject your dockerhub password using drone secrets.
    email: octocat@catmail.com
    repo: octocat/repoName
    tag: latest
    file: Dockerfile
    environment:
      - DOCKER_LAUNCH_DEBUG=true #( usefull for debugging but not necessary )
    when:
      branch: [ master ]
      event: [ push ] # step only triggers on push events
  ssh-deploy:
    image: appleboy/drone-ssh
    pull: true  # always pull the latest version of the `drone-ssh` plugin
    host: ${HOST} # passed in as a drone secret
    user: ${USER} # passed in as a drone secret
    key: ${SSH_KEY} # passed in as a drone secret
    port: 22
    pull: true
    command_timeout: 180
    script:
      - cd /home/ubuntu/deploy  # or whereever you put your `deploy.sh`
      - sh deploy.sh
    when:
      event: [push, tag, deployment]
services:
  database:
    image: postgres
    environment:
      - DATABASE_URL=postgres://postgres@localhost

The $HOST, $USER and $SSH_KEY variables will be passed in as Drone secrets. Read up on the appleboy/drone-ssh plugin.

Set your secrets:

drone secret add --image=applyboy/drone-ssh octocat/repoName HOST yourEC2InstancePublicDNS
drone secret add --image=applyboy/drone-ssh octocat/repoName USER ubuntu
drone secret add --image=applyboy/drone-ssh  octocat/repoName SSH_KEY @/path/to/pem/key.pem

Remember to resign your drone.yml

drone sign ocotcat/repoName

Open up one last pull request. When your build passes merge in your latest code. Upon merge you should see the deploy step trigger (along with all of the output from your deploy.sh file) after the publish step in the Drone console. Congratulations! We now have a very simple CI/CD pipeline.

In Part 3 I will outline setting up Nginx for your app and add a slight tweak to the deploy.sh to accomodate the containerized nature of your app with Nginx.