Editing configuration files with cfengine

Now we've got a fairly solid template for checking a service, we want to be able to configure it. This time Jamie Wilkinson takes a look at cfengine's editfiles section and describes a neat way to cope with migrating existing managed servers with some slightly different configurations to the state we want them in.

After having used cfengine at a previous job, one of the first tasks I set to at Anchor was automating our new server builds by tying together Red Hat's kickstart and cfengine. We had a long list of configuration steps documented, which was a good starting point; the majority of tasks required were simple things such as ensuring a service was enabled and a certain configuration option was set in a particular way.

It was shortly after that when I then tried to migrate the existing servers into this new automated adminisration infrastructure that I discovered the simple cfengine inputs I had created were not sufficient to ensure confidence in convergence to the corresponding correct configuration.

For example, the operation "setting option X in file Y to the value Z", a common idiom when configuring daemons, took the form of an editfiles script:

  { Y
    LocateLineMatching '^X =.*'
    ReplaceLineWith 'X = Z'
  }

but this doesn't cope when option X doesn't already exist in

configuration file Y; while

  { Y
    AppendIfNoSuchLine 'X = Z'
  }

may work well for new systems that don't have X already specified in file Y, but breaks when we are migrating from a system that does have X set by

adding an additional line (which may cause undefined behaviour in the application); and

if the value of X is to change for some reason (say you want all new machines to use SSH protocol version 2 only, but leave the existing ones with both protocol 2 and 1) then we need to add a lot of editfiles

scripts, one for each combination, which becomes more
- unmanageable as the number of configurations grows.

Remember: The problem I am trying to solve is the

scalability of managing configurations, not push the problem
- from one domain to another...

It's true we could fill our cfengine inputs with lots of commands to delete lines which we recognise are deprecated, but that's a lot of overhead, especially when the value of X changes from Q to R to S to Z:

  { Y
    DeleteLinesMatching 'X = Q'
    DeleteLinesMatching 'X = R'
    DeleteLinesMatching 'X = S'
    AppendIfNoSuchLine 'X = Z'
  }

Without comments, this is meaningless to everyone, and within a week and a few beers it's meaningless to the author too. Even with comments, old deprecated configurations just seem to lie around and, without additional systems to check that those configurations are gone, one can never be sure that all the systems are up to date (for one reason or another, systems always seem to be switched off, or have cfengine disabled (but that's another story)).

editfiles scripts are quite tedious and cumbersome, and while they are relatively easy to read, their readability will decrease as the size of the script increases; and the less a human can understand in one reading, the more likely there's an error in the script. The solution then is to come up with a simple template for editing a configuration file which adds the key if necessary and then sets it to the correct value if it is not:

  { Y
    BeginGroupIfNoLineMatching '^X =.*'
      Append 'X ='
    EndGroup
    LocateLineMatching '^X =.*'
    ReplaceLineWith 'X = Z'
  }

That will make sure a key X exists in file Y, and then once we've ensured

it's there, will set it to the correct value.

The key difference between these two approaches is that the simple approach

does not scale well when a variety of permutations of an initial configuration

are presented to cfengine. The second will cope with all forms of that configuration.

An advantage of this method is that we can now define a cfengine

variable called "$(Y_X)" (my shorthand for "configuration option X in server Y, as a way of eliminating namespace conflicts in
- variables) and then you can do something like this:

control:

  Y_X = ( Z )

editfiles:

  { Y
    BeginGroupIfNoLineMatching '^X =.*'
      Append 'X ='
    EndGroup
    LocateLineMatching '^X =.*'
    ReplaceLineWith 'X = $(Y_X)'
  }

Wow. That means we only have to set a single cfengine variable in

the control section and the right thing will happen, so now we can split up the control section with various classes, and not have to duplicate the work in the editfiles section for each possible class:

control:

  class1::

    Y_X = ( Z )

  class2::

    Y_X = ( R )

editfiles:

  { Y
    BeginGroupIfNoLineMatching '^X =.*'
      Append 'X ='
    EndGroup
    LocateLineMatching '^X =.*'
    ReplaceLineWith 'X = $(Y_X)'
  }

That's probably enough abstract examples for now. I'll finish off now with the classes, control, and editfiles sections from ssh.cf.

classes:

  sshd_server = ( any )

  # use ssh protocol 2 only
  ssh_v2_only = ( anchor_secure )

control:

  ssh_v2_only::

    ssh_protocol = ( 2 )

  !ssh_v2_only::

    ssh_protocol = ( 2,1 )

editfiles:

  sshd_server::

    # fiddle with sshd settings
    { /etc/ssh/sshd_config
      Backup 'off'

      BeginGroupIfNoLineMatching '^X11Forwarding.*'
        Append 'X11Forwarding'
      EndGroup
      ResetSearch 1
      LocateLineMatching '^X11Forwarding.*'
      BeginGroupIfNoMatch '^X11Forwarding no$'
        ReplaceLineWith 'X11Forwarding no'
      EndGroup
      ResetSearch 1

      BeginGroupIfNoLineMatching '^Protocol.*'
          Append 'Protocol'
      EndGroup
      ResetSearch 1
      LocateLineMatching '^Protocol.*'
      BeginGroupIfNoMatch '^Protocol ${ssh_protocol}$'
          ReplaceLineWith 'Protocol ${ssh_protocol}'
      EndGroup
      ResetSearch 1

      BeginGroupIfNoLineMatching '^SyslogFacility.*'
        Append 'SyslogFacility'
      EndGroup
      ResetSearch 1
      LocateLineMatching '^SyslogFacility.*'
      BeginGroupIfNoMatch '^SyslogFacility AUTHPRIV$'
        ReplaceLineWith 'SyslogFacility AUTHPRIV'
      EndGroup
      ResetSearch 1

      BeginGroupIfNoLineMatching '^LogLevel.*'
        Append 'LogLevel'
      EndGroup
      ResetSearch 1
      LocateLineMatching '^LogLevel.*'
      BeginGroupIfNoMatch '^LogLevel INFO$'
        ReplaceLineWith 'LogLevel INFO'
      EndGroup
      ResetSearch 1

      DefineClasses 'sshd_restart'
    }

Note the use of the anchor_secure class for the machines that should be configured with SSH protocol v2 only. We also define a class sshd_restart to restart the SSH service after the configuration file has been changed. And to finish off, add the editfiles section to main.cf:

    actionsequence = (
        files
        editfiles
        processes
        shellcommands
    )

Author : Jamie Wilkinson

Editing configuration files with cfengine

Related links