= Introducing cfengine =

Good configuration is a seriously hard problem. Worse, it just seems to get
harder and harder as time goes on. Managing a mixture of old and new computers
and making sure they all do what they're supposed to do becomes harder
exponentially and is known to cause hair loss, high blood pressure,
social deficiencies, and in some cases, death.
Anchor's ninja-sysadmin Jamie Wilkinson has the
answers; ''and'' he's managed to keep our computer's configurations humming
along nicely. In this series of articles, he'll let you in on some secret server
kung-fu that's keeping our network of systems manageable.


== Wot Anchor's Doing With ''cfengine'' ==

[[http://www.anchor.com.au/dedicated-hosting/dedicated-servers.py|Dedicated servers]] that are maintained manually require a lot of effort, and as the
number of systems grows, the amount of effort required grows at least linearly
with them; probably more than linear, due to interactions between servers --
but without the manpower to keep up that maintenance, a growing network has a
lot of entropy and quickly falls into disrepair.

Every seasoned sysadmin has their toolkit of scripts that have accumulated
over the years, built out of necessity to combat the eventual heat death of
control over the network: checklists of configuration tweaks that get applied
to each new server, either manually or progressively more automatically as the
admin gets older and values their time more.  In recent years, people have
started to codify as best practice what the old Bastards have known for a
long time: ''computers like boring, repetitive work, and humans don't''.

== So what is it? ==

''cfengine'' is one tool that gives the Pimply Faced Youth a head start
when it comes to devising a way to apply your tweaks to a range of systems.
It takes as input a description of a goal state for the systems, and the system
itself, and then makes changes to get the system into the goal state.  This
technique is known as "convergence".


There are several papers and websites that do a much better job of introducing
automatic systems administration that I ever could, so at this point I will
refer you, kind reader, to some of my favourites:


 * These two [[http://www.infrastructures.org/bootstrap/papers.shtml|papers on automated systems administration]] are definitely worth a read if you want to get a handle on the theory, or a treatise on how to design and build your new infrastructure from the ground up.  
 * [[http://www.iu.hio.no/~mark/research/immune/|Computer Immunology]] from the author of ''cfengine'', describing the system as a reactive organism, much like your own immune system.
 * And finally, the [[http://cfengine.org/|''cfengine'']] website itself, with the [[http://www.cfengine.org/docs/cfengine-Tutorial.html|tutorial]] and [[http://www.cfengine.org/docs/cfengine-Reference.html|reference manual]]

== How it works ==
''cfengine'' is a collection of programs that share the responsibility
of distributing configuration programs to servers, running the
configuration program, and collecting statistics about the server for
use in configuration programs.

The distribution service, ''cfservd'' runs on a central host,
normally known as the <em>gold</em> server.  ''cfservd'' can be
configured to restrict access to certain IPs, or ranges of IPs, and to
certain files on the gold server.

''cfengine''s client server model uses authenticated,
encrypted communication, similar to the way SSH works -- except that
both ends of the connection must verify their identity to each other.
This means that the administrator must get a copy of the host keys
from each machine to the gold server, and the gold server's key to
each client machine.  There is a mechanism in ''cfengine'' to
trust a connection by its IP address and then perform a key exchange,
however.

The ''cfagent'' program does the work of running a
''cfengine'' program and performing the configuration changes.
It can be run manually, or periodically from cron, or via the
''cfexecd'' service.  ''cfexecd'' is a wrapper program that
runs ''cfagent'' periodically and mails the output to the
designated sysadmin.  I have found it to have no benefit over cron,
and so ''cfexecd'' is run with the ''-F'' option half-hourly
from cron, which means ''cfagent'' is run once.
So, every half hour, cron launches ''cfexecd'', which launches
''cfagent'', which reads its local copy of the configuration
program.  ''cfagent'' then tries to connect to the gold server,
and if it can, checks to see if there are new versions of the
configuration program available.  If it cannot connect, it continues
with the existing copy of the configuration program.
If the connection succeeds (i.e. both ''cfagent'' and
''cfservd'' recognise each other and allow the connection to
continue) and there are new programs to download, then it retreives
them all and begins the process of checking and configuring the
machine.

The configuration programs are broken up into sections describing
various types of checks that can be made.  The running process list
can be checked for a program name, filesystems can be checked for free
space, file and directory existence, permissions and ownership.  Files
can be edited to ensure they match a certain pattern.
''cfagent'' performs these checks according to the
<em>actionsequence</em> defined in the configuration program.

A key part of ''cfengine'' is its <em>classes</em>.  A class
describes a property of the machine, such as 'it runs Debian' or 'it
has at least one IDE disk' or 'it is a webserver'.  Machines can
belong to many classes, and a class may contain only one machine.
Each machine has a hardcoded list of classes that it belongs to,
and classes that are defined by the program, and classes that are
dynamically set based on the behaviour of the program.  A class that
says 'restart the FTP server' may be defined if the configuration file
for the FTP service is modified.
The hostname of a machine is also a class; in this way machines can
be grouped.  A webserver class may be defined as a list of hostnames
-- the hosts who should be configured as webservers.

So, ''cfengine'' is a powerful tool that can simplify the
maintenance of a lot of similar (and sometimes, vastly different)
machines by providing a simple way of distributing common
configurations and applying those changes in an intuitive way.  The
core set of commands are flexible enough to tackle most administrative
tasks and the concept of classes is useful in keeping configurations
simple.

In future articles here on Anchor's Shipyard, I'll be going into
more detail about the structure of the Anchor configurations, and
delving into some useful techniques for some common, and some
difficult, tasks that I've encountered since rolling out the first
cfengine-controlled hosts at Anchor.


Author : Jamie Wilkinson

== Related links ==
 * [[http://infrastructures.org]]
 * [[http://www.cfengine.org/]]