Shell Scripts vs Ansible: Fight!

Note: In this article, I talk mostly about Shell Scripts as the enemy. However, in practice, there are much worse offenders like not using any kind of script or not even having a checklist or any documentation whatsoever.

Surprise

After writing my book on configuration management (CM) tools, I sent out review copies to about 20 people, including some very well-known developers. What really surprised me was the response from some of these more famous devs. They basically said, "This is really cool, but I probably won't read it since my manual-install/shell-script workflow is fine for now."

I was a little shocked, but once I thought about it for a few minutes, I realized that their choice was perfectly sane and rational given what they knew about CM tools.

Example:

Aaron Patterson (a top Ruby on Rails contributor) demonstrated this problem perfectly:

He's a top programmer in the community and even he struggles with this.

Complexity & Cost

For them, using a CM tool meant weeks of effort learning complex concepts, struggling with a complex installation process, and maintaining that complex system over time. They were somewhat aware of the benefits, but the costs of using a CM tool just seemed too high to make it worth the effort.

Michael Booth captures the sentiment well in his Hacker News comment:

As a veteran user of Puppet I'm a firm believer in using a tool like Puppet, but Puppet-and-Chef are overdesigned for small jobs (and, arguably, for most other jobs as well) so actually recommending them to a beginner has always felt like this:

A: "I just set up a cloud instance by running some shell commands by hand."

B: "You shouldn't do that, because of X and Y and Z. You should learn Puppet or Chef."

A: "Wait... did you just tell me to go spend thirty hours banging my head against solid objects, in exchange for nebulous benefits that I can't even perceive yet?"

B: "Why, yes, I believe I did!"

Fortunately, there are new CM tools on the scene that are keenly focused on simplicity and a smooth user experience. One of the simplest and most powerful is Ansible.

Why it Matters

Most engineers would laugh at a company that had zero tests and didn't version control or even backup their code. But that's exactly what many companies do with their systems.

And these businesses often don't realize the astronomical risks and costs of this neglect.

Your server systems are the underlying foundation of your application. They are basically the "app" that your app code runs on. As such, your systems should be version controlled and tested. There are many other benefits of course like speed, scalability, lower costs, etc, but I won't go into them all here.

At a recent client job, the former system administrator set up all of their servers manually and left ZERO documentation. I had to spend a few weeks divining what he did and try to capture everything into a CM tool (Chef in this case). Not one week after I finished doing that, one of their key servers inexplicably failed. Fortunately, I had done the work needed and they were able to bring up a replacement within minutes. Had that work not been done, the company would have been severly crippled and might have even died due to the days of downtime that would have been inevitable.

That's just a tiny example. The cost savings for a business are huge when the team uses a CM tool. When it takes a couple minutes to bring up a new server instead of hours/days/weeks, that's a huge savings (assuming your engineers cost greater than $0/hour).

Fight!

Ansible has brought the learning curve down so far that it can actually be EASIER to use Ansible than to do a manual install or a shell script.

The high cost of using a CM tool is now gone and you also get all the benefits of configuration management and remote execution.

Hopefully this will convince folks who are still on the fence to start using a CM tool like Ansible for more awesome systems.

Scenario

The great guys over at Phusion recently released an APT repository for their Passenger web/application server.

I've been looking for a good real-world example in order to compare Manual Installs vs Shell Scripts vs Ansible and this scenario seemed ideal.

Demonstration

I'm going to briefly show you setting up Phusion Passenger with a Manual Install, then create a Shell Script, then show how to do it in Ansible. The server I'm using is Ubuntu 13.04 (Raring Ringtail) with the software-properties-common package installed for some supporting utilities.

Manual Install

root@server:~# # Install the PGP key
root@server:~# gpg --keyserver keyserver.ubuntu.com --recv-keys 561F9B9CAC40B2F7
root@server:~# gpg --armor --export 561F9B9CAC40B2F7 | sudo apt-key add -

root@server:~# # Install https support for apt
root@server:~# apt-get install apt-transport-https

root@server:~# # Add the passenger apt repository
root@server:~# vi /etc/apt/sources.list.d/passenger.list
root@server:~# chown root: /etc/apt/sources.list.d/passenger.list
root@server:~# chmod 600 /etc/apt/sources.list.d/passenger.list

root@server:~# # Update the apt cache so we can use the new repo
root@server:~# apt-get update

root@server:~# # Install nginx
root@server:~# apt-get install nginx-full passenger

root@server:~# # Set up passenger in the nginx configuration
root@server:~# vi /etc/nginx/nginx.conf

root@server:~# # Start nginx
root@server:~# service nginx restart

Shell Script

# Install the PGP key
gpg --keyserver keyserver.ubuntu.com --recv-keys 561F9B9CAC40B2F7
gpg --armor --export 561F9B9CAC40B2F7 | apt-key add -

# Install https support for apt
apt-get install apt-transport-https -y

# Add the passenger apt repository
echo "deb https://oss-binaries.phusionpassenger.com/apt/passenger raring main" > /etc/apt/sources.list.d/passenger.list
chown root: /etc/apt/sources.list.d/passenger.list
chmod 600 /etc/apt/sources.list.d/passenger.list

# Update the apt cache so we can use the new repo
apt-get update

# Install nginx
apt-get install nginx-full passenger -y

# Set up passenger in the nginx configuration
sed -i "s/# passenger_root/passenger_root/" /etc/nginx/nginx.conf
sed -i "s/# passenger_ruby/passenger_ruby/" /etc/nginx/nginx.conf

# Start nginx
service nginx restart

Assuming that's in passenger.sh, then run:

root@server:~# bash passenger.sh

Ansible

---
- hosts: all
  tasks:

  - name: Ensure the PGP key is installed
    apt_key: >
      state=present
      id=AC40B2F7
      url="http://keyserver.ubuntu.com/pks/lookup?op=get&fingerprint=on&search=0x561F9B9CAC40B2F7"

  - name: Ensure https support for apt is installed
    apt: >
      state=present
      pkg=apt-transport-https

  - name: Ensure the passenger apt repository is added
    apt_repository: >
      state=present
      repo='deb https://oss-binaries.phusionpassenger.com/apt/passenger raring main'

  - name: Ensure nginx is installed
    apt: >
      state=present
      pkg=nginx-full

  - name: Ensure passenger is installed
    apt: >
      state=present
      pkg=passenger
      update_cache=yes

  - name: Ensure the nginx configuration file is set
    copy: >
      src=/app/config/nginx.conf
      dest=/etc/nginx/nginx.conf

  - name: Ensure nginx is running
    service: >
      name=nginx
      state=started

Assuming that's in passenger.yml, then run:

root@server:~# ansible-playbook passenger.yml

Some Differences

You may notice that I copied over the nginx config file for Ansible rather than just modifying the file directly on the server like I did for the manual install and shell script. Initially I looked into editing the file with Ansible's lineinfile module, but it just felt so wrong. That's another nice thing about Ansible, it encourages better practices. So, I decided to save the nginx config file locally so I could track it in version control and copy it over instead.

You'll also notice that I start the comments for Ansible with "Ensure". That is because Ansible will not only do the initial installation, but can also be run again and again to 'ensure' that the system is in the desired state. Ansible can perform the function of not only setting up your systems, but also testing them for correctness.

By contrast, a shell script can be very dangerous to run more than once unless you're extremely careful and know exactly what every line does. For example, what will those gpg commands do if I run them again? I have no idea. I could invest time to see if they are safe to run multiple times, or I could just use Ansible.

Note: The Ansible script can be improved. I kept it simple so it was easy to see how it correlated to the shell script. However, there are several things I could have done to make it better.

I could have added enabled=yes to the nginx service. That would have set the service to start automatically if the server is rebooted.

I could have combined the installation of the nginx-full and passenger packages by using with_items.

I could have set a handler to notify nginx to restart whenever nginx.confis updated.

The Winner

• Which method is most likely to end up in source control?
• Which method can be run multiple times safely with confidence?
• Which method can easily be run against multiple servers?
• Which method actually verifies (tests) your server for correctness?
• Which method can target certain servers easily (web, db, etc)?
• Which method supports easily templating your configuration files?
• Which method will grow to easily support your whole stack?

I didn't cover some of these topics, but the answer to all of these is: Ansible.

Not only can you start using it right away for trivial scripts like this, but it also gives you a ton of power that you can use later for free.

Getting Started

To keep this article brief, I'm skipping over a lot of important things. However, here are some helpful tips to get started...

If you're on OSX and have Homebrew installed, then you can install Ansible like this:

> brew install ansible

Note: Ansible is written in Python, but if you're not a Python programmer - don't worry! You never have to touch Python unless you really really want to. The Ansible scripts that you work with are written in the very simple YAML format. Not using Ansible because it's written in Python is like not using Linux because it's written in C.

To install Ansible on other systems, see the official Installation docs.

For setting up connectivity to your servers, see my post-install instructions.

Note: Unlike other CM tools, Ansible doesn't need to have a client agent installed on your individual servers, so that's one less step you need to worry about.

Migrating

If you already have shell scripts for setting up your systems, you can easily start switching over by calling them via Ansible:

- script: /some/local/script.sh --some-arguments 1234

Then you can start extracting the shell script commands into Ansible and then remove the shell script altogether. That will give you a nice iterative path to migrating.

You can similarly do this with individual shell commands:

- shell: apt-get install nginx -y

Then you can go back and convert them to Ansible. That previous command would turn into:

- apt: pkg=nginx state=present

Use Ansible's modules documentation to help you through the migration process.

Note: For both the script and shell modules, you can use the creates parameter in order to tell the script or shell command not to run if a certain file already exists on the server. That way you can set Ansible up to ensure that the script or shell command isn't run more than once if it shouldn't be.

Excelsior!

Hopefully this will convince you (or some friend you'll send this to), to move over to a more awesome way of managing your systems. They are the foundation of your app and should be treated with love too.

If you're now interested in Ansible, then be sure to sign up for our new Ansible Weekly newsletter (warning to cow afficianodos: there's a potentially disturbing image of a Borg Cow on that page).