Deploying a multi-node application to Vagrant using chef-provisioning

This post is for people who are getting started with chef-provisioning and want to use it to deploy to Vagrant. It will take you through creating a couple of machines and deploying a simple application to them.

You may also be interested in my other post, Deploying a multi-node application to AWS using chef-provisioning.

For an overview of chef-provisioning ( (formerly known as chef-metal), take a look at this Chef-Provisioning: Infrastructure as Code blog post. And see the Chef provisioning docs for more details.

Getting setup with chef-provisioning

Chef-provisioning is included in the latest ChefDK (0.3.6 at time of writing). Make sure you have this version or later installed by typing:

chef --version

If not, you can download or upgrade it here.

Create a new Chef repository to explore chef-provisioning:


cd ~
chef generate repo chefprov


We are going to use chef-client in local mode to run our provisioning recipes, so we want to set up a .chef directory that will be used specifically for this repo.

cd ~/chefprov
mkdir .chef
cd .chef


In the .chef directory, create a knife.rb file containing the following:

log_level                :info
current_dir = File.dirname(__FILE__)
node_name                "provisioner"
client_key               "#{current_dir}/dummy.pem"
validation_client_name   "validator"

Our workstation is going to behave like a chef-client talking to the local-mode server on our workstation, so it needs a node name and a key. The key can be any well-formed key as the local-mode server will not validate it. For example:

ssh-keygen -f ~/chefprov/.chef/dummy.pem

Check the setup is working by performing an empty chef-client run:


cd ~/chefprov
chef-client -z


This will perform a local mode chef-client run with no recipes, using the built-in chef-zero server running on port 8889. You should see output similar to:

Starting Chef Client, version 11.18.0
[2015-01-31T16:16:43-06:00] INFO: *** Chef 11.18.0 ***
[2015-01-31T16:16:43-06:00] INFO: Chef-client pid: 14113
[2015-01-31T16:16:44-06:00] INFO: Run List is []
[2015-01-31T16:16:44-06:00] INFO: Run List expands to []
[2015-01-31T16:16:44-06:00] INFO: Starting Chef Run for provisioner
[2015-01-31T16:16:44-06:00] INFO: Running start handlers
[2015-01-31T16:16:44-06:00] INFO: Start handlers complete.
[2015-01-31T16:16:44-06:00] INFO: HTTP Request Returned 404 Not Found : Object not found: /reports/nodes/provisioner/runs
[2015-01-31T16:16:44-06:00] WARN: Node provisioner has an empty run list.
Converging 0 resources
[2015-01-31T16:16:44-06:00] INFO: Chef Run complete in 0.032696323 seconds
Running handlers:
[2015-01-31T16:16:44-06:00] INFO: Running report handlers
Running handlers complete
[2015-01-31T16:16:44-06:00] INFO: Report handlers complete
Chef Client finished, 0/0 resources updated in 1.117898047 seconds

If you’re curious, take a look at the ‘nodes/provisioner.json’ file. This is where the local-mode server stores its node data. You can also run commands like:

knife node show provisioner -z

This command will query the local-mode server and show summary details that it has about your provisioner node (i.e. your workstation).

Deploy the Application using Vagrant

Get the application cookbooks

The basic application we will install can be found in the ‘test-repo’ for the ‘knife-topo’ plugin on Github. Download the latest release of the knife-topo repository and unzip it. We will use ‘berks vendor’ to assemble the cookbooks we need to deploy this application.

cd knife-topo-0.0.11/test-repo
berks vendor
cp -R berks-cookbooks/* ~/chefprov/cookbooks

Line 2 uses the Berksfile to assemble all of the necessary cookbooks into the ‘berks-cookbooks’ directory. Line 3 copies them into our ‘chefprov’ repo, where the local-mode server will look for them when it runs the chef-provisioning recipes.

Create recipes to provision the machines

Create a recipe to setup the Vagrant environment.

vagrant-setup.rb

require 'chef/provisioning/vagrant_driver'
with_driver 'vagrant'

vagrant_box 'ubuntu64' do
  url 'http://files.vagrantup.com/precise64.box'
end

with_machine_options :vagrant_options => {
  'vm.box' => 'ubuntu64'
}

Line 2 specifies to use the Vagrant driver, which is included in ChefDK.

Lines 4 to 6 create a local Vagrant box called ‘ubuntu64’ using the standard Ubuntu 12.04 box. Lines 8 to 10 tell chef-provisioning to use that box when creating machines.

Use the following recipe to provision the machines:

topo.rb

require 'chef/provisioning'

machine 'db' do
  run_list ['apt','testapp::db']
end

machine 'appserver' do
  run_list ['apt','testapp::appserver']
end

and then run the chef-client to do the provisioning:
chef-client -z vagrant_setup.rb topo.rb

This will create these two machines using Vagrant, bootstrap them and run the specified recipes, installing nodejs on ‘appserver’ and mongodb on ‘db’.

The Vagrant machines by default are stored in “.chef/vms”. You can see their status by going to this directory and running normal vagrant commands, e.g.:


cd ~/chefprov/.chef/vms
vagrant status

You can also use the ‘vagrant global-status’ command to see the status of any VM on your workstation.

Working around SSH issue

If you are trying this with ChefDK 0.3.6 on Ubuntu, you may encounter the following error:

         ================================================================================
         Chef encountered an error attempting to load the node data for "db"
         ================================================================================

         Unexpected Error:
         -----------------
         NoMethodError: undefined method `gsub' for nil:NilClass

This is a known issue with chef-provisioning providing a bad URL for the local-mode server. If you can upgrade to chefDK 0.4.0, this problem has been fixed (but be aware that chefDK 0.4 embeds Chef 12 and not Chef 11).

A workaround for chefDK 0.3.6 is to create the following Gemfile in your chefprov directory:

source 'https://rubygems.org'

gem 'chef-dk'
gem 'chef-provisioning'
gem 'chef-provisioning-vagrant'
gem 'net-ssh', '=2.9.1'

and then run chef-client using:

bundle exec chef-client -z vagrant_setup.rb topo.rb

This will run the chef-client using a previous version of ‘net-ssh’, which avoids the problem.

You will likely need to use ‘bundle exec’ in front of all of the chef-client runs described in this post.

UPDATE: If the above command fails with:

 
         Unexpected Error:
         -----------------
         ChefZero::ServerNotFound: No socketless chef-zero server on given port 8889

then add the following to the setup recipe:

with_chef_server "http://localhost:8889"

This problem exists with chef-dk 0.6.0 to 0.6.2.

Deploy the application

Create the following recipe to deploy the application.

vagrant_deploy.rb

require 'chef/provisioning'

myconfig = <<-EOM
  config.vm.network 'forwarded_port', guest: 3001, host: 3031
EOM

machine 'appserver' do
 add_machine_options :vagrant_config => myconfig
 run_list ['testapp::deploy']
 attribute ['testapp', 'user'], 'vagrant'
 attribute ['testapp', 'path'], '/home/vagrant'
 attribute ['testapp', 'db_location'], lazy { search(:node, "name:db").first['ipaddress'] }
end

ruby_block "print out public IP" do
 block do
   Chef::Log.info("Application can be accessed at http://localhost:3031")
 end
end

Lines 3 to 5 and 8 setup port forwarding for our application. You can see how this gets converted into a Vagrantfile by looking at what is generated in ‘~/.chef/vms/appserver.vm’.

Lines 6 to 8 setup attributes used to customize the test application. We use ‘lazy’ to ensure the IP address lookup is not done until the ‘db’ server has been created in the converge phase of the chef-client run.

Lines 11-15 print out a message so you know how to access the application.

To deploy the application, run the chef-client with the setup and deploy recipes:
chef-client -z vagrant_setup.rb vagrant_deploy.rb

When you navigate to the URL, you should see a message from the application:

 Congratulations! You have installed a test application using the knife topo plugin.

 Here are some commands you can run to look at what the plugin did:

    knife node list
    knife node show dbserver01
    knife node show appserver01
    knife node show appserver01 -a normal
    knife data bag show topologies test1
    cat cookbooks/testsys_test1/attributes/softwareversion.rb

Go to the knife-topo plugin on Github

Ignore the example commands as we did not use the knife-topo plugin.

Destroy the machines

The recipe to destroy the machines is:

destroy.rb

require 'chef/provisioning'
machine 'db' do
  :destroy
end

machine 'appserver' do
  :destroy
end

Run this using:

chef-client -z destroy.rb

If you need to clean up manually, use ‘vagrant global-status’ to get the IDs of the machines, and then use ‘vagrant destroy ‘ to destroy them. If you do this, you will also want to remove the contents of the ‘chefprov/nodes’ and ‘chefprov/clients’ directory so the local-mode server does not think they still exist.