Spend less time waiting, more time Cheffing

Vagrant is wonderful, but I hate waiting for my virtual machines to come up. Here’s some things I have done to reduce that wait:

• Use local boxes
• Use vagrant-cachier to cache the omnibus installer as well as packages used during the chef-client run
• Run vbguest plugin only when needed
• Package a custom box

Use Vagrant’s locally managed boxes

When you are starting up a machine based on a completely new box, the most painful wait is usually for the box to download. Make sure you don’t have to wait more than once: use vagrant box add to add it into vagrant’s locally managed boxes. For example, to add Ubuntu 12.04:

vagrant box add precise64 http://files.vagrantup.com/precise64.box

This will give you a box that you can now refer to as “precise64” in your Vagrantfiles. You can use whatever name you want for the first parameter (‘precise64’) in the example. The second parameter is the URL to obtain the box, see this list of base boxes.

In this example, your Vagrantfile will contain something like line 1 in the following:

  config.vm.box ="precise64";
  config.vm.box_url = "http://files.vagrantup.com/precise64.box";

Line 2 is entirely optional, and tells vagrant where to get the box if it is not found in the local cache. It’s useful for when you reuse your Vagrant file on another machine, share it with someone else, or when you forget where you got the box from!

Use vagrant-cachier with vagrant-omnibus

After storing the box locally, my next longest waiting time was for the omnibus installer to download the Chef image. I looked into how to do a knife bootstrap from a local image, but that involved replacing the entire chef bootstrap script. Matt Stratton instead pointed me at vagrant-cachier, a plugin that provides a shared package cache. You can also use it to cache other packages like apt and gems that you are installing on the virtual machine. What it does is configure the package manager to use a package cache that is a shared folder between host and guest. This cache is used by the vagrant-omnibus plugin to bootstrap chef onto the virtual machine. Make sure you have recent versions of both plugins. Here’s how to install them:

vagrant plugin install vagrant-omnibus
vagrant plugin install vagrant-cachier

It seems that vagrant-omnibus is quite specific about when it uses the cache. Here’s what worked for me.

if Vagrant.hasPlugin?("vagrant-cachier")
  config.cache.auto_detect = true
  config.cache.scope = :machine
  config.omnibus.cache_packages = true
  config.omnibus.chef_version = "11.16.0"
end

Line 1 makes sure that you dont get errors if the cachier plugin isn’t installed.

Line 2 enables caching for all types of packages. Beware of this – if you’re actually trying to test downloading from various package repositories, this setting may not work for you. I tried enabling only Chef with config.cache.auto_detect = false and config.cache.enable :chef but it seems like this doesn’t work for the omnibus installer, only for things like cookbooks that would be placed in ‘/var/chef/cache’ during a chef-client run.

Line 3 restricts package sharing to a specific machine. Although it’s tempting to use the :box setting and share across all machines using the same box, it’s dangerous (if you ever run more than one machine at once you may well get locking problems – the package managers will all treat the cache as if its a local filesystem). Further, the omnibus plugin appears to require a scope of machine before it will use the cache.

Line 4 is needed to tell omnibus that you really do want to use the cache.

Line 5 is optional and sets the specific Chef version you want to use.

With this configuration, the packages once downloaded are stored on the host machine in ‘.vagrant/machines/dbserver/cache’ (where ‘dbserver’ is the machine name in the Vagrantfile). You may need to go to this folder from time to time, to check on the cache size or to clear it out. They are shared with the guest in ‘/tmp/vagrant-cache’.

Run vbguest plugin only when needed

Plugins can make a difference to startup time. Case in point: early on I had some problems with Vagrant shared folders caused by different Guest Addition version on the guest versus Virtualbox, and so I started using the vagrant-vbguest plugin to resolve that problem. It’s great, but it takes a little while to apply the kernel updates when creating a new machine. If you are really shaving time off getting a new machine running, you may reconsider updating guest additions automatically.

Turn auto-update off for vbguest plugin

Much of the time, the mismatch between guest additions and Vagrant is not a showstopper, so you may want to start by just reporting the mismatch, i.e. turning auto_update off for the vbguest plugin (if you have it).

  config.vbguest.auto_update = false

If you decide to update guest additions, change the property in the Vagrantfile and vagrant up, or simply run:

  vagrant vbguest --do install

You are advised to reboot the virtual machine afterwards, e.g. using vagrant reload.

Remove vbguest plugin

Use vagrant plugin list to see which plugins you have, and sudo vagrant plugin remove vagrant-vbguest to remove the vbguest plugin if you have it.

Package a custom box

When I have a stable setup, I sometimes package my own box with the right version of Guest Additions and Chef. To do this, get a machine setup with the Chef and Virtual Additions that you want (plus anything else). Halt the machine, then package it as a box using something like:

vagrant package dbserver --output myprecise64.box
vagrant add myprecise64 ./myprecise64.box

Where ‘dbserver’ is the name of the machine in the Vagrantfile from which to create the box, and ‘myprecise64.box’ is the filename to output the box to. Now replace “precise64” with “myprecise64” in your Vagrantfile, and your machines will (at least for now) have the right version of Chef and Guest Additions.