The Introduction

This is the first part of a multi-part tutorial covering a simple(ish) setup of a continuous integration/deployment pipeline using Drone.io. In Part 1 we will set up some simple automated testing for a Django, Docker/Docker-Compose application.

Who is tutorial this for?

This tutorial is for anyone who is still somewhat new to web development and is looking for an excuse to set up a relatively simple continuous integration/delivery pipeline. I’m going to assume a basic understanding of Amazon Web Services , Docker , Gunicorn , Nginx and some familirity with Django .

The basic outline of the pipeline is as follows:

1) After opening a pull request on your Github repo, a webhook will queue up a drone build for a drone agent to pick up, build your specified environment outlined in your projects .drone.yml , run any tests your project has before finally reporting back its status to Github.

2) After tests have passed and the new code has been merged in, the same webhook will queue up another build on the drone server. This time, after building and testing your application, drone will create a Docker image of your application and push this image to DockerHub .

3) After the image is pushed, Drone will SSH into your EC2 container (where your application is hosted), pull your newly updated image, and stop, update and restart your web application.

Step 1: Setting up a Drone 0.5 server

My initial setup for my Drone server was based off of this tutorial and except for the fact that it uses Drone:0.4 the steps that outline setting up an EC2 instance and registering a github application, etc, are mostly still correct and I’m including it just as a reference (I don’t recommend using Drone:0.4 since Drone:0.5 is far more flexible).

With that said, let’s begin!

Spin up an EC2 instance and SSH into it.

I usually spin up basic ubuntu 16.04 instances with free tier/basic settings. After the instance is spun up you will need to reference it’s public DNS when registering a github application.

Register a github application

• Application name Drone 0.5 (or whatever)
• Homepage URL http://yourNewEC2PublicDNS.com
• Application description Drone 0.5 (or whatever)
• Authorization callback URL http://yourNeWEC2PublicDNS.com/authorize

You will use the generated Client Id and Client Secret in your dronerc file in the step after next.

Install Docker

The installation instructions for installing docker are relatively straight forward. If not, there are plenty of other good tutorials out there.

Install Drone

After docker is installed it’s time to set up Drone. There are three entities you must install for you Drone to work correctly:

1) The drone Server is responsible for organizing your repos and queuing up what are essentially build tasks for drone agents to start working on.

The linked installation guide I’ve found to be perfectly adequate except I used a dronerc file to pass in my environment variables to the drone server, so go ahead and make a dronerc file in /etc/drone/ . Here I am assuming you are integrating drone with a Github account but it should be fairly easy to use Bitbucket (for example) instead. Your dronerc should look something like this:

DRONE_DEBUG=true
DRONE_GITHUB=true
DRONE_SECRET=yourDroneSecret
DRONE_GITHUB_CLIENT=yourGitHubClientIdFromStep2
DRONE_GITHUB_SECRET=yourGitHubSecretFromStep2
DRONE_OPEN=true
DRONE_ADMIN=yourGitHubUsername

You can read about each of these environment variables inside the drone documentation. If you don’t explicitly specify, Drone assumes you are using SqlLite3 as a backend. If you don’t have it installed on your EC2 instance, do it now:

$ sudo apt-get install sqlite3 libsqlite3-dev

You also must create a directory for drone’s sql database to live. You will be mounting this directory to your drone docker container when you run drone as a docker container.

$ mkdir /var/lib/drone

After you’ve done that you can create your drone instance. Here is my docker run command which is very similar to the one provided in the official drone documentation :

$ sudo docker run -d --env-file /etc/drone/dronerc -v /var/lib/drone/:/var/lib/drone -p 80:8000 --restart=always --name=drone drone/drone:0.5

Notice that we are mounting the directory /var/lib/drone/ and specifying /etc/drone/dronerc as our environement file.

If the contianer is successfully created you should see some positive output (no errors) when running:

$ sudo docker logs drone .

2) The drone agent is responsbile for actually running your builds and executing tasks outlined in your drone.yml . It is not present in Drone:0.4 and it isn’t terribly clear from the documentation that you need to install the agent if you are coming from Drone:0.4.

You can install the agent as outlined in the drone documentation. My Docker run command looked like:

$ sudo docker run -d -e DRONE_SERVER=ws://yourEC2PublicDNS.com/ws/broker -e DRONE_DEBUG=true -e DRONE_SECRET=yourDroneSecret -v /var/run/docker.sock:/var/run/docker.sock --restart=always --name=agent drone/drone:0.5 agent

3) The drone CLI is responsbile for easily interacting with your drone server. Just to be super clear, you should install the CLI locally and not on your EC2 instance like you did the server and the agent. You will need the CLI to conveniently pass secrets (essentially hidden varaibles) to your execution steps in your .drone.yml later when we SSH into our container and also to push our app’s image to DockerHub. Your can verify you’ve set up the CLI correctly by running:

$ drone info

If set up correctly this will show output like:

User: octocat
Email: octocat@catmail.com

4) With everything installed you can go to your instance’s public DNS http://yourNewEC2PublicDNS.com and activate your repo from the drone console that should now be present (if you don’t see it then you have installed the The drone Server incorrectly). You should be able to click to open up the menu in the top right of the console, exposing an “Account” page. this should show a list of repos associated with your github account. After activating, Drone will create a webhook for your application in Github (go to your Repo in Github, click setings in the top right and then webhooks to verify).

Step 2: Add a .drone.yml to Your Django App and Test Your Drone Server

Now that your drone server is set up, your django application will need a .drone.yml . Here is a representation of a basic Django app project structure:

- projectDir
    - projectSrcCodeDir
    - .drone.yml
    - requirements.txt
    - Dockerfile
    - docker-compose.yml

To set up the first part of our pipeline we will need our .drone.yml to look something like this:

pipeline:
  build:
    image: python:3.5.2
    environment:
        # I use dj_database_url to specify my DB settings
      - DATABASE_URL=postgres://postgres@localhost 
    commands:
      - sleep 5 # (probably not necessary)
      - pip3 install -r requirements.txt
      - cd projectDir
      - python ./manage.py test
      - cd ..
    when:
      branch: [ master ]
      event: [push, pull_request] # trigger step on push and pull events
services:
  database:
    image: postgres
    environment:
      - DATABASE_URL=postgres://postgres@localhost

Feel free to reference the documentation but essentially what we have done is tell drone to use the python:3.5.2 image to build our application, install our requirements in your requirements.txt file and then run our applications tests.

Now, when you open a pull request to your repo and go to your drone server at http://yourNewEC2PublicDNS.com you should see a build being triggered and ran in the drone console. Under the branches section in your Github Repo’s settings you should be able to “protect” your main branch, so that if your drone tests fail you are unable to merge in the new pull request.

If your tests pass then congratualtions! You’ve now set up automated testing which is the first big step in our pipline. In Part 2 I will outline adding a publish step to the .drone.yml which will push your app’s image to DockerHub. After that we will set up a deployment step which will SSH into your EC2 instance, pull your newly updated image, and stop, update and restart your web application.