Infrastructure as Cake

Testing Your Configuration Management in the Kitchen, with Sprinkles and Love

By Jamie Tanna, https://jamietanna.co.uk

Overview

  • A Crash Course in Configuration Management
  • Writing Cookbooks with Chef
  • Chef Utensils
  • Questions

About Me

Capital One UK

Bash

Ansible

A Crash Course in Configuration Management

Let’s Guage Experience Levels

"Configuration Management"

Used Chef?

Used another CM tool?

Deploy an application/website/provision instances regularly?

Used Ruby?

What is it?

Creating a repeatable, controlled fashion to make changes to a server

A way to maintain parity between environments

“the objective is to have everything that can possibly change at any point stored in a controlled manner”

Continuous Delivery, Jez Humble & David Farley

You’re building quality-driven software

You’re deploying said software

Shouldn’t you be deploying with quality-driven config?

“Why Can’t I Just Use a Shell Script?”

Using a “real language”

  • Standard library
  • Package ecosystem
  • Edge cases / language subtleties
  • Auditability / grokability

Tooling

Testable

Versionable artefacts

DSL can be exposed

Templating languages

What happens if I don’t have Configuration Management?

Story time

Server troubles

How would you respond to this?

What Configuration Management is Not

Silver bullet

“Glorified Bash scripts”

Writing Cookbooks with Chef

What is Chef?

Ruby-based Configuration Management tool

DSL provided to make a lot of common tasks easier

A whole host of community cookbooks hosted on https://supermarket.chef.io/: - mysql, mariadb, postgresql - apache, nginx - java - node.JS - jenkins - openssh

Deployment

  • Generally run in a Client-Server model
  • Can be run server-less

Three main components:

  • Client
  • Server
  • Supermarket

Apache 2.0 licensed

Cookbook File Structure

So what does a cookbook file structure look like?

new-cookbook
├── attributes
│   └── default.rb               # what are the default configuration values?
├── Berksfile                    # dependency management
├── chefignore
├── LICENSE
├── metadata.rb                  # what is the version, name, license, dependencies?
├── README.md
├── recipes
│   └── default.rb               # how do I do stuff?
├── spec
│   ├── spec_helper.rb           # ChefSpec helper
│   └── unit
│       └── recipes
│           └── default_spec.rb  # unit tests for the default recipe
└── test
    └── smoke
        └── default
            └── default_test.rb  # integration/smoke test for the default suite

Cooking Some Welcome Muffins

user 'create user' do
	username 'jamie'
	manage_home true
end

file 'creates the muffins.txt file' do
	path '/home/jamie/muffins.txt'
	content 'Welcome to the team!'
	mode '0600'
	owner 'jamie'
end

Adding a Special Touch

Attributes are a great way to make cookbooks configurable:

# attributes/default.rb
node.default['user'] = 'jamie'

# recipes/welcome.rb
user "create user #{node['user']}" do
	username node['user']
	manage_home true
end

file 'creates the muffins.txt file' do
	path "/home/#{node['user']}/muffins.txt"
	content "Welcome #{node['user']}, it's nice to meet you!"
	mode '0600'
	owner node['user']
end

Hey, I Thought Testing Was Important?

ChefSpec

Quickest path to testing a cookbook

Matchers for Chef resources

expect(chef_run).to_not install_package('wibble')
expect(chef_run).to run_execute('ls -al')
expect(chef_run).to render_file('/etc/os-release')
  .with_content('Hello!')

Mock and test out different paths + scenarios

describe 'user-cookbook::default' do
	context 'When all attributes are default, on an Ubuntu 16.04' do
		let(:chef_run) do
			runner = ChefSpec::ServerRunner.new(platform: 'ubuntu', version: '16.04')
			runner.converge(described_recipe)
		end

		it 'converges successfully' do
			expect { chef_run }.to_not raise_error
		end

		it 'creates the jamie user' do
			expect(chef_run).to create_user('create user jamie')
				.with(username: 'jamie')
				.with(manage_home: true)
		end

		it 'creates the muffins.txt file' do
			expect(chef_run).to create_file('creates the muffins.txt file')
				.with(content: 'Welcome jamie, it\'s nice to meet you!')
				.with(mode: '0600')
				.with(owner: 'jamie')
		end
	end
end

describe 'user-cookbook::default' do
	# ...
	context 'When the user attribute is set' do
		let(:chef_run) do
			runner = ChefSpec::ServerRunner.new(platform: 'ubuntu', version: '16.04') do |node|
				node.automatic['user'] = 'test'
			end
			runner.converge(described_recipe)
		end

		it 'converges successfully' do
			expect { chef_run }.to_not raise_error
		end

		it 'creates the jamie user' do
			expect(chef_run).to create_user('create user test')
				.with(username: 'test')
				.with(manage_home: true)
		end

		it 'creates the muffins.txt file' do
			expect(chef_run).to create_file('creates the muffins.txt file')
				.with(content: 'Welcome test, it\'s nice to meet you!')
				.with(mode: '0600')
				.with(owner: 'test')
		end
	end
end

Using context blocks for testing different OS configurations

describe 'user-cookbook::package' do
	context 'On Ubuntu 16.04' do
		let(:chef_run) do
			runner = ChefSpec::ServerRunner.new(platform: 'ubuntu', version: '16.04')
			runner.converge(described_recipe)
		end

		it 'installs the apache2 package' do
			expect(chef_run).to install_package('Install Apache')
				.with(package_name: 'apache2')
		end
	end

	context 'On RHEL 7.2' do
		let(:chef_run) do
			runner = ChefSpec::ServerRunner.new(platform: 'redhat', version: '7.2')
			runner.converge(described_recipe)
		end

		it 'installs the httpd package' do
			expect(chef_run).to install_package('Install Apache')
				.with(package_name: 'httpd')
		end
	end
end

package 'Install Apache' do
	case node['platform']
	when 'redhat'
		package_name 'httpd'
	when 'ubuntu'
		package_name 'apache2'
	end
end

Using context blocks for testing for error states

describe 'cookbook-spectat::install' do
	context 'When site_type is valid' do
		let(:chef_run) do
			runner = ChefSpec::SoloRunner.new(platform: 'debian', version: '9.1') do |node|
				node.automatic['site_type'] = 'static'
			end
			runner.converge(described_recipe)
		end

		it 'installs nginx' do
			expect(chef_run).to install_package('Install Nginx')
				.with(package_name: 'nginx')
		end
	end

	context 'When site_type is not set' do
		let(:runner) do
			ChefSpec::SoloRunner.new(platform: 'debian', version: '9.1')
		end

		it 'throws an error' do
			expect { runner.converge(described_recipe) }.to\
				raise_error('Attribute site_type needs to be set to `static`')
		end
	end
end

if node['site_type'].nil? || 'static' != node['site_type']
	raise 'Attribute site_type needs to be set to `static`'
end

package 'Install Nginx' do
	package_name 'nginx'
end

Chef Utensils

Linting

Cookstyle

Ruby-specific linting

FoodCritic

FC004: Use a service resource to start and stop services:

# Don't do this
execute 'start-tomcat' do
	command '/etc/init.d/tomcat6 start'
	action :run
end

# Do this instead
service 'tomcat' do
	action :start
end

FC041: Execute resource used to run curl or wget command

# Don't do this
execute "cd /tmp && wget 'http://example.org/'" do
	action :run
end

# Do this instead
remote_file '/tmp/testfile' do
	source 'http://www.example.org/'
end

Integration Testing - Test Kitchen

Unit tests are for assumptions

Does that package exist in the cache?

Unit tests can’t possibly know!

Different ways to spin up the nodes:

  • kitchen-docker
  • kitchen-vagrant
  • kitchen-ec2
  • kitchen-digitalocean
---
# .kitchen.yml
driver:
	name: vagrant

provisioner:
	name: chef_zero

verifier:
  name: inspec

platforms:
  - name: debian
    driver_config:
      image: debian:jessie

suites:
  - name: hello
    run_list:
      - recipe[user-cookbook::default]
    attributes:
      user: 'everybody'
      hello: true

Stages of a kitchen test:

kitchen create:

kitchen converge:

kitchen verify:

kitchen destroy:

Integration Testing - InSpec

describe package('nginx') do
	it { should be_installed }
end

describe port('443') do
	it { should be_listening }
	its('processes') { should include 'nginx' }
end

Compliance and audit tool

Confirm nodes are in correct state

Agent- and dependency-less

Not just for Chef

Review

  • A Crash Course in Configuration Management
  • Writing Cookbooks with Chef
  • Chef Utensils
  • Questions

Learning Resources

Learn Chef Rally - https://learn.chef.io/

Chef Certification - https://training.chef.io/certification

Food Fight Podcast - http://foodfightshow.org/

My personal blog - https://www.jvt.me/posts/tags/chef/

Intro to TDD Chef: How I Write Cookbooks (Coming Soon!) - https://gitlab.com/jamietanna/jvt.me/issues/184

Contact

@jamietanna on Twitter

https://jamietanna.co.uk

Slides Tech Stack

@jamietanna