Auto Jenkins Setup

I recently talked about how to automate Jenkins using job-dsl. We are going a bit further here.

Jenkins is usually the frontend for projects to check their build status, but for some, Jenkins is the product that we develop, or where we drop plugins etc. In that case, we don’t want to do this into production.

In our case we work with Open Nebula as a VM provider, so it is fairly easy to create a quick VM to test something and then just remove it. It is not as common as having your own Jenkins instance, but since I found that I was doing this quite commonly, it was time to automate.

In this post I present an automatic way to set up a quick Jenkins host. So I just have to type a command and come back later.

Different Options

Configuring a host with some specific set-up is something known as Configuration Management. We live in a world of Virtual Machines, and we are creating environments every day. To configure this environments we use specific software.

This is an itch that a lot of people tried to scratch, that is why we have a big list of configuration management software. The big contenders in this area used to be Chef and Puppet. Both written in Ruby. But two other interesting options are Ansible and Salt, both written in Python.

Puppet, Chef and Salt are designed to work in a client/server mode. They can run in standalone, but on the other hand Ansible is designed to run over ssh, with no additional client installation.

For this reason and for being Python I decided to try a bit of Ansible, and see how it works.

Ansible Quick Start

Ansible works with playbooks. In each playbook (text file) you define modules to be applied in order in any kind of machine.

The machines are defined in the file /etc/ansible/hosts. This file (called inventory) contains some basic configuration for each host, and groups of hosts. That is, if I want to configure 20 machines the same way I can define them in the same group.

mail.example.com

[webservers]    #this is a group
foo.example.com #this is a host
bar.example.com

[dbservers]
one.example.com
two.example.com
three.example.com

Once a playbook is written, ansible can apply it by simply running
ansible-playbook my-playbook.file

What are we going to set up?

Since this Jenkins is a “copy” of an existing Jenkins, I want it to be as similar as possible. We are going to set up our hosts with the following list:

• Linux Host. (I worked with a CentOS)
• Tomcat
• Jenkins
• Specific version of Jenkins
• Set of plugins
• Predefined templates
• job-dsl

Base Ansible playbook

The playbooks are written in a data format called YAML. It is human readable, and easy to follow. (But bitchy with the formatting).

All the playbooks start with a quick setup:

---
- hosts: Jenkins-deploy
  vars:
    http_port: 80
    max_clients: 200
    Jenkins_version: 1.549
    Jenkins_plugins: plugins-100214.tar.gz
    Jenkins_templates: templates-100214.tar.gz
    Jenkins_jobs: jobs-100214.tar.gz
    ssh_keys: keys.tar.gz
    juser: Tomcat
    jgroup: Tomcat
  remote_user: root
  tasks:
(continues)

hosts defines which hosts from the ansible inventory to target (or which group).
vars sets up the different variables used in all the playbook, we may split that into a different file later on.
remote_user is an option to tell ansible which user to run in the remote machine.
tasks is signaling that the following will be tasks to perform.

Packages

To set up the basic Jenkins, we will package different configurations from an already existing Jenkins. This mostly means: ssh keys, basic templates, the basic job dsl and plugins. The setup expected by my playbook is the following directory structure:
/root
|
|- playbook
|- files/
|
|- confs/
| |- Tomcat.conf
|- keys/
| |- keys.tar.gz
|- jobs/
| |- jobs-100214.tar.gz
|- templates/
| |- templates-100214.tar.gz
|- plugins/
|- plugins-100214.tar.gz


We separe different packages in different directories so we can easily keep various versions. It will be a matter of defining the variables when deploying.

keys.tar.gz

This is plainly all the ssh keys needed by Jenkins to connect to git. Since I will be installing the Git plugin and the credentials plugin too.

jobs-100214.tar.gz

To transfer some existing Jobs to this instance, simply pack their directory under JENKINS_HOME to this tar.gz.

This should be plain, no extra layers of directories. All the jobs will be deployed in the new Jenkins.

templates-100214.tar.gz

Templates and Jobs are the same, but with nits. In my case we create jobs that never run but that are used as templates to create other jobs. For this reason I also add this package.

plugins-100214.tar.gz

This tarball will contain all the .hpi and .jpi (hudson/Jenkins plugin) files to deploy into Jenkins.

In case of copying from another Jenkins, just copy those files from JENKINS_HOME/plugins. If it’s a new installation, download the hpi files directly and create this compressed file.

Ansible install Tomcat/Jenkins

Ansible has a lot of modules and it is well documented. Our first task is to ensure that Tomcat and Jenkins are installed in the host.

To install Tomcat we can use a very simple yum task. For other package managers you can use the corresponding task.

  - name: ensure Tomcat is at the latest version
    yum: pkg=Tomcat state=latest

Since we are going to configure Tomcat with our own file, and start deploying Jenkins, we’ll stop the service:

  - name: ensure Tomcat is stopped
    service: name=Tomcat state=stopped

Now we can download Jenkins. This can be done directly in the host, but in my case, the hosts have limited access to Internet via proxy. I will download it locally and push the file to the VM:

  - name: download Jenkins
    local_action:
        get_url
            url=http://mirrors.Jenkins-ci.org/war/{{Jenkins_version}}/Jenkins.war
            dest=/tmp/Jenkins.war
            mode=0644

  - name: Upload Jenkins to host
    copy:
        src=/tmp/Jenkins.war
        dest=/usr/share/Tomcat/webapps/Jenkins.war
        mode=0644

To do so I used the get_url module and the copy_module. Notice that the get_url is preceeded by local_action. That tells ansible to run this module locally instead of remotely.

I did not talk about how to use variables in Ansible. Ansible uses jinja2, but more to the point, to use any defined variable we just need to write the variable name between brackets, just as I used {{Jenkins_version}}

This is mostly it. Now we have Tomcat installed and Jenkins in the webapps directory. The last steps are to push our own Tomcat configuration and restart the service.

My Tomcat configuration is a default configuration but with an extra option:
CATALINA_OPTS="-DJenkins_HOME=/var/lib/Jenkins/ -Xmx512m"
This option is used for Jenkins to know where all the files should reside. We have to make sure that this directory exists, and that can be done with another task:

  - name: Create the /var/lib/Jenkins directory
    file:
        path=/var/lib/Jenkins/
        owner={{juser}}
        group={{jgroup}}
        mode=0777
        state=directory

Finally we’ll start Tomcat and wait for Jenkins to set up:

  - name: start Tomcat to pickup and install Jenkins
    service: name=Tomcat state=started

  - wait_for: path=/var/lib/Jenkins/plugins/

The wait_for module will stop the execution until a condition is met. In this case, we are waiting for Tomcat to deploy Jenkins. When that happens the plugin directory will be created.

And with this we have Tomcat + Jenkins installed on our VM.

Send the packages

All the packages are sent in the same way. Using a local_execution followed by a call to the unarchive module. That module will upload and unpack a compressed file. In some cases an extra step is needed to set the correct permissions and ownerships. Since it is always the same I will show only the plugin case:

  - name: Get path of plugins tarball
    local_action:
        shell find `pwd` -name {{Jenkins_plugins}}
    register: plugins_tarball

  - name: Upload Jenkins plugins
    unarchive:
      src={{plugins_tarball.stdout}}
      dest=/var/lib/Jenkins/plugins/
      owner={{juser}}
      group={{jgroup}}
      mode=744

  - name: Force a change of owner for all plugins
    shell: chown -R {{juser}}:{{jgroup}} /var/lib/Jenkins/plugins/*

That is.

• Find the correct file in the directory.
• Use unpack to upload the tarball and unpack.
• An extra task to set the correct ownership to the files.

It is a little bit more of a hassle to have everything packed in a tar.gz file. But with this way I can assure faster data transmissions (only a single compressed file) and the Ansible playbook is clear.

The full Set

The full playbook can be downloaded from here It contains the main playbook, and a setup.yml file to configure which tar.gz files to use.

Note that I do not include any tar.gz as example, simply the folder structure and the yaml files.

What next?

This is a basic Jenkins set up, able to run job-dsl to populate everything. That does not mean that those jobs will work if Jenkins is not properly configured. I did not configure any of the plugins at all, and believe me, they will need configuration.

I would need to add more configurations on a known state and push them to the Jenkins VM.

Conclusions

That was a piece of cake. Ansible combined with the fact that Jenkins is mostly configured via files makes everything so easy. For other software packages I would have had to configure something in SQL or who knows what.

Ansible gets big quite fast, and the looping ability is not exactly the best. My first attempt was to send the plugin files one by one (reading them from the local plugins folder). That worked as expected, but was too slow for my taste. I left the code in the final file in case you want that functionality.

References

Jenkins CI ⇒GO
Ansible ⇒GO
Jenkins Deploy Playbook ⇒GO