Here’s a simple example of using knife-topo with Chef to manage the versions of software deployed in a multi-node topology. This post assumes you have Vagrant and ChefDK installed.
The example topology that we’ll work with consists of three nodes as shown below:
However, in this post, we’ll only define and deploy two of the nodes using knife-topo, Chef and Vagrant. I’ll introduce the third node in a later blog to illustrate some more complex uses of the topology JSON (such as conditional attributes). You can also look at the full example in the test-repo in knife-topo’s github repository.
Describing the topology
Define the nodes
The first step is to describe the topology that we want. Below is a minimal topology JSON file describing the two nodes in the topology (‘test1’). The ‘name’ property for the nodes defines the node name that will be used in Chef. The ‘ssh_host’ property specifies the address to use in bootstrapping the nodes.
{
"name": "test1",
"nodes": [
{
"name": "appserver01",
"ssh_host": "10.0.1.3"
},
{
"name": "dbserver01",
"ssh_host": "10.0.1.2"
}
]
}
This is sufficient to allow knife-topo to bootstrap the appserver and dbserver nodes, so that they will be managed by Chef. You can try this out by following the instructions later to download a test repo, run Vagrant and chef-zero, however, the results may be rather underwhelming. A data bag describing the topology will be created, Chef will be bootstrapped onto the two nodes, and they will register themselves with the server, but that’s all. You will also get warnings during bootstrap that the nodes have no runlists.
Define the runlists
Let’s fix the missing runlists now:
{
"name": "test1",
"nodes": [
{
"name": "appserver01",
"ssh_host": "10.0.1.3",
"run_list": [
"recipe[apt]",
"recipe[testapp::appserver]",
"recipe[testapp::deploy]"
]
},
{
"name": "dbserver01",
"ssh_host": "10.0.1.2",
"run_list": [
"recipe[apt]",
"recipe[testapp::db]"
]
}
]
}
The recipes in these runlistsinstall the software (mongodb, nodejs, and a test application) on the nodes, and are provided in the test repo, along with a Berksfile to manage the dependencies. However, they do not specify what versions of the software should be installed. If you try running the example now, the latest versions will be chosen. But we want to get control of what’s on our topology. We can do this by defining the software versions as attributes in an environment cookbook specific to our topology.
Defining specific software versions as attributes in an environment cookbook
Here’s the first addition that’s needed to deploy specific software versions. We add a “cookbook_attributes” section, specify the name of the environment cookbook (‘testsys_test1’) and the attribute file we want (‘softwareversion’), and the necessary version attributes to install NodeJS version 0.10.28, and MongoDB version 2.6.1.
{
"name": "test1",
"nodes": [
...
],
"cookbook_attributes": [
{
"cookbook": "testsys_test1",
"filename": "softwareversion",
"normal": {
"nodejs": {
"version": "0.10.28",
"checksum_linux_x64": "5f41f4a90861bddaea92addc5dfba5357de40962031c2281b1683277a0f75932"
},
"mongodb": {
"package_version": "2.6.1"
}
}
}
]
}
The second change is to add the environment cookbook to the runlists for the two nodes:
{
"name": "test1",
"nodes": [
{
"name": "appserver01",
"ssh_host": "10.0.1.3",
"run_list": [
"recipe[apt]",
"recipe[testapp::appserver]",
"recipe[testapp::deploy]",
"testsys_test1"
]
},
{
"name": "dbserver01",
"ssh_host": "10.0.1.2",
"run_list": [
"recipe[apt]",
"recipe[testapp::db]",
"testsys_test1"
]
}
],
"cookbook_attributes": [
...
]
}
The topology JSON is now ready to use – go ahead and try it.
A word of warning if you ran knife-topo with runlists but no specific software versions – there’s currently a bug in the mongodb cookbook that means it can’t handle downgrades. So you may want to destroy the virtual machines (‘vagrant destroy’) and start again (or see troubleshooting for an alternative).
Running the example
These instructions may be enough to get you started. If you want more details or encounter problems, see these instructions and the knife-topo readme.
Setting up the environment
Install Vagrant and ChefDK if you do not already have them. Neither are required for knife-topo, but this post uses them.
Install knife-topo using gem. You may need to use ‘sudo’.
gem install knife-topo
To get the test repo, it’s easiest to download and unzip the latest knife-topo release from github (replace ‘0.0.7’ with the latest release number):
unzip knife-topo-0.0.7.zip -d
The test-repo gives you a multi-node Vagrantfile similar to what that I described in my previous post. Use that to create the virtual machines:
cd ~/knife-topo-0.0.7/test-repo
vagrant up
When the virtual machines have started (this can take a while the first time), you can start chef-zero. If you have chefDK, you can use the embedded chef-zero, or you can install chef-zero as a gem. Here’s how to start the embedded chef-zero on Ubuntu:
/opt/chefdk/embedded/bin/chef-zero -H 10.0.1.1
Importing the pre-requisites
Leave chef-zero running, open a new terminal and go to the test-repo, then upload the required cookbooks using berkshelf:
cd ~/knife-topo-0.0.7/test-repo
berks install
berks upload
Save your topology file as “mytopo.json” in the test-repo, then import your topology into your Chef workspace:
knife topo import mytopo.json
Do not name the file “test1.json” or it will cause an error later on when we load the test1 topology from the data bag file (knife looks for data bag files in the current directory BEFORE it looks in the data bag directory).
If you are using the final topology JSON, ‘knife-topo import’ will have generated some useful artifacts for you: in particular, an attribute file in the topology cookbook containing the specified software versions. This file is in “knife-topo-0.0.7/test-repo/cookbooks/testsys_test1/attributes/softwareversion.rb” and should look like:
# # THIS FILE IS GENERATED BY THE KNIFE TOPO PLUGIN - MANUAL CHANGES WILL BE OVERWRITTEN # # Cookbook Name:: testsys_test1 # Attribute File:: softwareversion.rb # # Copyright 2014, YOUR_COMPANY_NAME # normal['nodejs']['version'] = "0.10.28" normal['nodejs']['checksum_linux_x64'] = "5f41f4a90861bddaea92addc5dfba5357de40962031c2281b1683277a0f75932" normal['mongodb']['package_version'] = "2.6.1"
Bootstrapping the topology
knife topo create test1 --bootstrap -xvagrant -Pvagrant --sudo
This command uploads the topology cookbook, creates the test topology in the Chef server and bootstraps all nodes that provide an ‘ssh_host’. After it finishes, you should have a working two node topology with the specified software versions. The test application welcome screen is at: http://localhost:3031
Graphs and Chef and Javascript - oh my! 