Spend less time waiting, more time Cheffing

Vagrant is wonderful, but I hate waiting for my virtual machines to come up. Here’s some things I have done to reduce that wait:

Use Vagrant’s locally managed boxes

When you are starting up a machine based on a completely new box, the most painful wait is usually for the box to download. Make sure you don’t have to wait more than once: use vagrant box add to add it into vagrant’s locally managed boxes. For example, to add Ubuntu 12.04:

vagrant box add precise64 http://files.vagrantup.com/precise64.box

This will give you a box that you can now refer to as “precise64” in your Vagrantfiles. You can use whatever name you want for the first parameter (‘precise64’) in the example. The second parameter is the URL to obtain the box, see this list of base boxes.

In this example, your Vagrantfile will contain something like line 1 in the following:

  config.vm.box ="precise64";
  config.vm.box_url = "http://files.vagrantup.com/precise64.box";

Line 2 is entirely optional, and tells vagrant where to get the box if it is not found in the local cache. It’s useful for when you reuse your Vagrant file on another machine, share it with someone else, or when you forget where you got the box from!

Use vagrant-cachier with vagrant-omnibus

After storing the box locally, my next longest waiting time was for the omnibus installer to download the Chef image. I looked into how to do a knife bootstrap from a local image, but that involved replacing the entire chef bootstrap script. Matt Stratton instead pointed me at vagrant-cachier, a plugin that provides a shared package cache. You can also use it to cache other packages like apt and gems that you are installing on the virtual machine. What it does is configure the package manager to use a package cache that is a shared folder between host and guest. This cache is used by the vagrant-omnibus plugin to bootstrap chef onto the virtual machine. Make sure you have recent versions of both plugins. Here’s how to install them:

vagrant plugin install vagrant-omnibus
vagrant plugin install vagrant-cachier

It seems that vagrant-omnibus is quite specific about when it uses the cache. Here’s what worked for me.

if Vagrant.hasPlugin?("vagrant-cachier")
  config.cache.auto_detect = true
  config.cache.scope = :machine
  config.omnibus.cache_packages = true
  config.omnibus.chef_version = "11.16.0"
end

Line 1 makes sure that you dont get errors if the cachier plugin isn’t installed.

Line 2 enables caching for all types of packages. Beware of this – if you’re actually trying to test downloading from various package repositories, this setting may not work for you. I tried enabling only Chef with config.cache.auto_detect = false and config.cache.enable :chef but it seems like this doesn’t work for the omnibus installer, only for things like cookbooks that would be placed in ‘/var/chef/cache’ during a chef-client run.

Line 3 restricts package sharing to a specific machine. Although it’s tempting to use the :box setting and share across all machines using the same box, it’s dangerous (if you ever run more than one machine at once you may well get locking problems – the package managers will all treat the cache as if its a local filesystem). Further, the omnibus plugin appears to require a scope of machine before it will use the cache.

Line 4 is needed to tell omnibus that you really do want to use the cache.

Line 5 is optional and sets the specific Chef version you want to use.

With this configuration, the packages once downloaded are stored on the host machine in ‘.vagrant/machines/dbserver/cache’ (where ‘dbserver’ is the machine name in the Vagrantfile). You may need to go to this folder from time to time, to check on the cache size or to clear it out. They are shared with the guest in ‘/tmp/vagrant-cache’.

Run vbguest plugin only when needed

Plugins can make a difference to startup time. Case in point: early on I had some problems with Vagrant shared folders caused by different Guest Addition version on the guest versus Virtualbox, and so I started using the vagrant-vbguest plugin to resolve that problem. It’s great, but it takes a little while to apply the kernel updates when creating a new machine. If you are really shaving time off getting a new machine running, you may reconsider updating guest additions automatically.

Turn auto-update off for vbguest plugin

Much of the time, the mismatch between guest additions and Vagrant is not a showstopper, so you may want to start by just reporting the mismatch, i.e. turning auto_update off for the vbguest plugin (if you have it).

  config.vbguest.auto_update = false

If you decide to update guest additions, change the property in the Vagrantfile and vagrant up, or simply run:

  vagrant vbguest --do install

You are advised to reboot the virtual machine afterwards, e.g. using vagrant reload.

Remove vbguest plugin

Use vagrant plugin list to see which plugins you have, and sudo vagrant plugin remove vagrant-vbguest to remove the vbguest plugin if you have it.

Package a custom box

When I have a stable setup, I sometimes package my own box with the right version of Guest Additions and Chef. To do this, get a machine setup with the Chef and Virtual Additions that you want (plus anything else). Halt the machine, then package it as a box using something like:

vagrant package dbserver --output myprecise64.box
vagrant add myprecise64 ./myprecise64.box

Where ‘dbserver’ is the name of the machine in the Vagrantfile from which to create the box, and ‘myprecise64.box’ is the filename to output the box to. Now replace “precise64” with “myprecise64” in your Vagrantfile, and your machines will (at least for now) have the right version of Chef and Guest Additions.

Advertisements

Confirm before navigating away from unsaved edits, using javascript and Backbone

Here’s an approach to putting up a confirmation dialog whenever the user navigates away from a view containing unsaved edits, so that the user doesn’t unintentionally lose those edits.

A working example is available on github.

Use Case

My use case is as follows. Our application is a single page application written in JavaScript that makes use of Backbone routing to navigate between major tasks. Each of those major tasks is implemented by one or more view managers (basically Backbone Views which implement a view container and controller for multiple views that work on a defined set of data). We allow the user to make a set of edits and explicitly decide when to save them to the server, rather than push every edit to the server as it happens. We don’t want the user to accidentally lose those edits before they are saved. Some of the ways that could occur are:

  • User selects an in-application navigation that triggers Backbone routing
  • User refreshes URL or types a new URL
  • User closes the browser window or tab

Approach

The basic approach I took was inspired by Kevin Yao’s post, Page Closing Confirmation with jQuery Custom Event. I liked his idea of listening for the window ‘beforeunload’ event and triggering a custom event that any interested party (in our case, the view managers) could respond to. However, I wanted to use Backbone for the events rather than plain JQuery, and I needed to catch in-application navigation as well.

Catching the ‘beforeunload’ event

The following code is called as part of our application overall setup.

$(window).bind('beforeunload', function(){
  var status = {
    unsavedEdits: false
  };

  Backbone.trigger('myapp:navigate', status);
  if (status.unsavedEdits) {
    return status.msg || "There are unsaved edits. Continuing will discard these edits.";
  }
});

Line 1 listens for the ‘beforeunload’ event which happens when the user closes the tab, reloads or enters a new URL.

Lines 2-5 set up a ‘status’ object in the event will be used by the view managers to communicate if they have unsaved edits, and to provide a message to be displayed.

Line 6 triggers a custom event ‘myapp:navigate’, using the Backbone object as a global event bus.

Lines 7-9 checks whether any of the ‘myapp:navigate’ handlers have set the status.unsavedEdits flag. If so, it returns a non-empty string, which the browser may or may not choose to display (Firefox does not – see WindowEventHandlers.onbeforeunload).

Communicating if there are unsaved edits

The following code is in the view managers:

this.listenTo(Backbone, "myapp:navigate", function(status){
  if (this.unsavedEdits()){
    status.unsavedEdits = true;
    status.msg = "There are unsaved edits to XXX. Continuing will discard these edits.";
  }
});

Line 1 listens for the custom ‘myapp:navigate’ event on the global event bus. When it occurs, lines 2-5 check whether there are unsaved edits, and if so, set the status.unsavedEdits flag, and optionally a message to be displayed.

Catching in-application navigation

To catch in-application navigation, I added code to the router to check for unsaved edits before routing to a new URL. Arguably, it may be preferable to intercept any navigation events before they reach the router. However, I opted to handle the logic in a single place, rather than adding logic wherever there is navigation.

The following code is in a routine that gets triggered for any route change resulting in a change of view managers (i.e. where there is a potential for unsaved edits to be lost).

var status = { 
  unsavedEdits: false
};
Backbone.trigger('myapp:navigate', status);

if(status.unsavedEdits) {
  var confirmed = window.confirm(status.msg || "There are unsaved edits. Continuing will discard these edits.");
  if (!confirmed){
    var history = this.history;
    if (history.length > 1){
      this.navigate(history[history.length-2], {replace: true});							
     }
  }
}

Lines 1-6 are the same as before… set up a status object, trigger a custom ‘myapp:navigate” event on the global event bus, and check whether the unsavedEdits flag has been set.

If the flag is set, line 7 uses a windows confirmation dialog to warn the user and get their response.

If the user does not want to proceed, we now have the issue that the URL has been added to the history and needs to be removed (or else subsequent navigation to that URL may be disabled). Lines 9-12 achieve this by replacing the newly added URL with its predecessor, using navigate with the replace option.

Getting started with Chef report and exception handlers

This post is for people who want to use or write their first Chef report handler, but aren’t sure where to begin. I first attempted to write a handler just after I learned how to write basic Chef recipes. It was hard, because I only had a rudimentary understanding of Chef mechanics, and I was learning Ruby as I went. This article would have got me started faster. However, in the end you are going to be writing Ruby, and probably you’ll need to get deeper into Chef too.

The three types of handler

There are three types of handler:

  • Start handler – Runs at the beginning of a chef-client run
  • Report handler – Runs at the end of a chef-client run, after all of the recipes have successfully completed
  • Exception handler – Runs at the end of a chef-client run if it exits with an error code

I am going to gloss over start handlers: they are less common and somewhat more complex because you have to get the handler in place before the chef-client run happens (so you can’t distribute them in a recipe).

Report handlers are useful for gathering information about a chef-client run (e.g. what cookbook versions were run, what resources were updated). Exception handlers are useful to capture information about failed runs, or to perform cleanup on exception (e.g. cleaning up frozen filesystem resources).

Using the built-in handlers

The most basic place to start is being able to run one of the built-in handlers. At current time of writing, there are two handlers built in to the chef-client: the json_file handler, and the error_report handler. The benefit of starting here is you don’t have to worry about how to get the handler code onto the nodes that you want to run them on – they’re distributed with the chef-client.

Running the error_report handler

Let’s start with the error_report handler. To run this, all you need to do is use the ‘chef_handler’ cookbook and add the following into a recipe in your runlist:

include_recipe "chef_handler"

chef_handler "Chef::Handler::ErrorReport" do
  source "chef/handler/error_report"
  action :enable
end

After a chef-client run, this will create a file called ‘failed_run_data.json’ in the chef-client cache (typically ‘/var/chef/cache’) on the node it is running on.

Despite its name, this handler can be useful whether or not the run fails. Assuming your run succeeded, here’s what you’ll find in the ‘failed_run_data.json’ file.

{
"node": {
  "name": "m2",
  "chef_environment": "_default",
  "json_class": "Chef::Node",
  "automatic": {
  "kernel": {
  "name": "Linux",
...

The JSON data starts off with details about the node attributes, similar to what you get with ‘knife node show m2 -l -Fjson'.

  "success": true,
  "start_time": "2014-08-31 20:43:53 +0000",
  "end_time": "2014-08-31 20:44:28 +0000",
  "elapsed_time": 34.995100522,

It then lists some basic details about the chef-client run.

  "all_resources": [
    {
      "json_class": "Chef::Resource::ChefHandler",
      ...
    }
  ],
  "updated_resources": [
    {
      "json_class": "Chef::Resource::ChefHandler",
      ...
    }
  ],

The next JSON elements describe all of the resources that were part of the chef-client run, and which were updated.

  "exception": null,
  "backtrace": null,
  "run_id": "53e02623-1bc9-4b33-a08d-eeb89936feca"

And then finally there is information about the exception that occurred (which is null in this case, a successful run).

Running error_report handler when an exception occurs

Let’s create an exception by adding an invalid resource to the recipe:

include_recipe "chef_handler"

execute "brokenthing"

chef_handler "Chef::Handler::ErrorReport" do
  source "chef/handler/error_report"
  action :enable
end

The chef-client run will fail with an error:

Errno::ENOENT: No such file or directory - brokenthing

But if you copied my recipe, the error report won’t have been updated! Why?

The problem is that the failing ‘execute’ resource happened before the error report handler was enabled. If you move the ‘execute’ resource after the chef_handler resource, the error report will be created. So one takeaway is that it is good practice to define handlers in a recipe that you put at the start of the runlist. But that’s not all that you want to do.

If the handler is the first resource encountered in the run, then it will report any errors happening when subsequent resources are executed. But sometimes exceptions happen before this, in what’s called the ‘compile’ phase (see About the chef-client Run). This is the phase where the chef-client constructs a list of all of the resources and actions that it is meant to perform, which it then executes in the ‘converge’ phase. For a much deeper explanation of this, see The chef resource run queue.

What we want to do is to enable the error report handler as early as possible in the compile phase, so it can catch errors occurring during this phase too. Here’s how to do that:

include_recipe "chef_handler"

execute "brokenthing"

chef_handler "Chef::Handler::ErrorReport" do
  source "chef/handler/error_report"
  action :nothing
end.run_action(:enable)

The end.run_action(:enable) tells Chef to do the “enable” action immediately on encountering the resource (i.e. during ‘compile’). The action :nothing tells Chef that it does not need to do anything during the ‘converge’ phase (as its already been enabled).

With this change, now you will find that the end of the error report has exception and backtrace information:

  "exception": "Errno::ENOENT: execute[brokenthing] (testapp::handlers line 11) had an error: Errno::ENOENT: No such file or directory - brokenthing",
  "backtrace": [
    "/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/mixlib-shellout-1.4.0/lib/mixlib/shellout/unix.rb:320:in `exec'",
    "/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/mixlib-shellout-1.4.0/lib/mixlib/shellout/unix.rb:320:in `block in fork_subprocess'",
...

Only running error_report on exception

Perhaps you do not want to run the error_report handler on every run, just when an exception occurs. To do this, we override the default supports attribute on the resource to specify it should be used with exceptions only:

chef_handler "Chef::Handler::ErrorReport" do
  source "chef/handler/error_report"
  action :nothing
  supports :exception=>true
end.run_action(:enable)

Replace :exception with :report if you only want the handler to run when the run is successful.

Running the json_file handler

The json_file handler is like the error_report handler, but puts the results into a timestamped file. The following chef_handler resource will result in data being written to /var/chef/reports/chef-run-report-20140831204047.json, for a run that started at 20:40:47 on 31st August 2014.

chef_handler 'Chef::Handler::JsonFile' do
  source 'chef/handler/json_file';
  arguments :path => '/var/chef/reports'
  action :nothing
end.run_action(:enable)

This example illustrates how to pass parameters into a handler.

You can also choose to use the ‘json_file’ recipe in the chef_handler cookbook to achieve the same result.

An alternative – using the client.rb file

An alternative (as described in the Chef docs) to enabling the handler using the chef_handler resource is to add the following to the client.rb file (or solo.rb file):

require 'chef/handler/json_file'
report_handlers << Chef::Handler::JsonFile.new(:path => "/var/chef/reports")
exception_handlers << Chef::Handler::JsonFile.new(:path => "/var/chef/reports")

Using a custom handler

The next step is to use a custom handler, i.e. one that is not built into the chef-client. The extra dimension here is that you need to get the handler code onto the node before it is run. You can do this by putting your handler source in the ‘files’ directory of your cookbook and using a cookbook_file resource to transfer it to the node, e.g.:

cookbook_file "#{node["chef_handler"]["handler_path"]}/your_handler.rb" do
  source "your_handler.rb"
  owner "root"
  group "root"
  mode 00755
  action :nothing
end.run_action(:create)

The expression #{node["chef_handler"]["handler_path"]} gives you the directory in which the chef_handler resource expects to find your handler. As previously, we run the cookbook_file resource immediately to ensure the handler file is created during the compile phase, before the handler itself is run. If the handler doesn’t run until the converge phase, you can replace the last two lines with:

  action :create
end

The chef_handler::default recipe can also be used to transfer handlers to the target node. You will need to make a copy of the chef_handlers cookbook and place your handlers in the ‘files/default/handlers’ directory of that cookbook (or copy the code from the default recipe into your handler cookbook).

To enable this handler, you would then define a chef_handler resource that refers to the transferred handler file:

chef_handler 'YourHandlerModule::YourHandler' do
  source "#{node["chef_handler"]["handler_path"]}/your_handler.rb";
  action :nothing
end.run_action(:enable)

For a real example you can try using, see Julian Dunn’s cookbook version handler.

Writing your own handler

To write a handler, you need to create a Ruby class that inherits from Chef::Handler and has a report method:

module YourHandlerModule
  class YourHandler < Chef::Handler
    def report
       # Handler code goes here
    end
  end
end

Put this code in the ‘your_handler.rb’ file in the ‘files’ directory of your handler cookbook.

Within the handler you can write arbitrary Ruby code, and you can use ‘run_status’ information available from the chef-client run. ‘run_status’ is basically the same information that is output by the ‘error_report’ handler. The information can be accessed through the following methods in the Ruby code:

  • data – a hash containing the run status
  • start_time, end_time, elapsed_time – times for the chef-client run
  • updated_resources, all_resources – the resources in the chef-client run
  • success?, failed? – methods that indicate if the chef-client run succeeded or failed

In the handler, you can access these directly or via the run_status object, e.g. ‘run_status.success?’ is equivalent to ‘success?’.

So for example, I can write the following handler:

require 'chef/log'

module TestApp
  module Handlers
     class UpdatedResources < Chef::Handler
      
      def report
        
        if success?
          Chef::Log.info('Running Updated Resources handler after successful chef-client run')
        else
          Chef::Log.info('Running Updated Resources handler after failed chef-client run')
        end

        Chef::Log.info('Updated resources are:')
        updated_resources.each do |resource|
          Chef::Log.info(resource.to_s)
        end
      end     
    end
  end
end

In the above, I use the success? method to test whether the run succeeded or not, and the updated_resources to loop through each resource updated during the run, and print it out as a string.

If you run ‘chef-client -linfo’ on the target node, you will see output similar to:

[2014-09-01T14:48:02+00:00] INFO: Running Updated Resources handler after successful chef-client run
[2014-09-01T14:48:02+00:00] INFO: Updated resources are:
[2014-09-01T14:48:02+00:00] INFO: chef_handler[Chef::Handler::JsonFile]
[2014-09-01T14:48:02+00:00] INFO: chef_handler[Chef::Handler::ErrorReport]
[2014-09-01T14:48:02+00:00] INFO: chef_handler[TestApp::Handlers::UpdatedResources]
  - TestApp::Handlers::UpdatedResources
Running handlers complete

In this case, the only updated resources in the chef-client run are the three chef_handlers in my handlers recipe. ‘chef_handler[TestApp::Handlers::UpdatedResources]’ is how the UpdatedResources handler is represented as a string.

You can also use the ‘to_hash’ method in place to ‘to_s’ in the above, to see what information is available about the resource. If you did this, you would see something like the following:

2014-09-01T14:54:25+00:00] INFO: {:name=>"TestApp::Handlers::UpdatedResources", :noop=>nil, 
:before=>nil, :params=>{}, :provider=>nil, :allowed_actions=>[:nothing, :enable, :disable], 
:action=>[:nothing], :updated=>true, :updated_by_last_action=>false, 
:supports=>{:report=>true, :exception=>true}, :ignore_failure=>false, :retries=>0, :retry_delay=>2, 
:source_line=>"/var/chef/cache/cookbooks/testapp/recipes/handlers.rb:41:in `from_file'", 
:guard_interpreter=>:default, :elapsed_time=>0.000480376, :resource_name=>:chef_handler, 
:cookbook_name=>"testapp", :recipe_name=>"handlers", 
:source=>"/var/chef/handlers/testapp_handlers.rb", :arguments=>[], 
:class_name=>"TestApp::Handlers::UpdatedResources"}

From this, you might decide just to print out the name of the resource, e.g.:

        updated_resources.each do |resource|
          Chef::Log.info("Updated resource with name: " + resource.to_hash[:name])
        end

Don’t expect ‘to_hash’ to work in all cases – it depends on whether it has been implemented in the relevant Ruby class. And be aware it may not be the best way to access the data. For example, the above can better be achieved using the name method on the resource, i.e. ‘resource.name’ in place of ‘resource.to_hash[:name]’. Over time, you’ll want to get comfortable with using debuggers to look at the data and reading the chef source code to understand how best to access it.

As well as ‘run_status’, you also have access to the run_context. I’ve included links to a couple of examples using the run context below.

Chef Handler with a parameter

If you want to pass a parameter into your handler, you need to add an initialize method to the handler, as below. This allows you to set the arguments attribute in chef_handler, which is passed to the initialize method as a parameter called ‘config’.

    class UpdatedResourcesToFile < Chef::Handler

      def initialize(config={})
        @config = config
        @config[:path] ||= "/var/chef/reports"
        @config
      end
            
      def report
        File.open(File.join(@config[:path], "lastrun-updated-resources.json"), "w") do |file|
          updated_resources.each do |resource|
            file.puts(resource.to_s)
          end
        end       
      end     
    end

What the above initialize method does is store the parameters in an attribute of the handler class called ‘@config’, and also sets a default value for the ‘path’ parameter. The ‘@’ indicates an attribute and means that the value is available to other methods in the class. The ‘report’ method can then access the path parameter using ‘@config[:path]’.

To enable the above handler, put something like the following in your handler recipe. You will also need to add a resource to create the path, if it doesn’t already exist.

chef_handler "TestApp::Handlers::UpdatedResourcesToFile" do
  source "#{node["chef_handler"]["handler_path"]}/testapp_handlers.rb"
  arguments :path => '/tmp'
  action :nothing
  supports :report=>true, :exception=>true
end.run_action(:enable)

In the above, I am overriding the default path value so that the file is written out to ‘/tmp’ instead.

Examples of handlers

Here are some examples of handlers that you may find useful: