Server Management with IPMI

Some of our recent dedicated servers have what is known as an "IPMI card" installed which facilitates additional control and management over the server. IPMI stands for Intelligent Platform Management Interface and basically is a standard for dedicated server management. It integrates with the server hardware to provide in-depth monitoring and control independent of the software installed on the server. It has the ability to provide:

  • Monitoring of voltages, fans, temperatures, power supply status
  • Monitoring of chassis and intrusion sensors
  • Power supply control (off, on, etc)
  • Serial console / virtual KVM access (including BIOS access)

Installation

The IPMI card is a direct add-on to the motherboard, installed in an IPMI slot. It provides access to the IPMI features over the network using (depending on the model of card) the first network interface of the motherboard or a separate independent network interface of its own. In both cases it is completely independent of the OSs usage of the network interface.

IMPORTANT: when setting a password for the IPMI admin account, it should not be longer than 16 characters. Newer IPMI cards and versions of IPMIView don't have this restriction.

AOC-1UIPMI-B

  1. Power off the machine and physically install the card.
  2. Boot the machine from the supplied software CD, and upgrade the motherboard BIOS with the appropriate flash image as per the instructions.
  3. Boot into the OS and run the ipnmac software (which is available for Windows and Linux) in order to assign TCP/IP details to the IPMI card.
  4. No software is required to be running on the machine after this point unless you wish to integrate Graceful Power Control on the machine (OS-assisted shutdown).

AOC-SIM1U-3B

  1. Note the MAC address on the label on the IPMI card.
  2. Power off the machine and physicall install the card.
  3. Boot up the machine. It will say something in Engrish about waiting for the card to initialise for the first time, which will take a couple of minutes, then the system will reboot.
  4. There are some BIOS options which should probably be standardised on for the IPMI card - ["FIXME"]
  5. Boot into the Linux OS and use the Linux version of ipnmac from the IPMI CD that came with the card
  6. Set the MAC address you noted from the label on the card, and the IP address you have allocated to the IPMI card.
  7. Test pinging to the card from a machine on the same subnet, to verify it is now active.
  8. Use IPMIView to continue configuration

Usage

There are several utilities that are supplied on the CDROM:

  • IPMIView (Windows, Java, Linux)
  • ipnmac (DOS, Linux)
  • ipmicli (Linux)
  • KCSTool (DOS)

Some IPMI cards also have a built in webserver that allows access via HTTP or HTTPS (and to some extent SSH).

Install the JRE if you don't already have it, and then run:

cd ~/IPMI_CDROM/IPMI_Solution/Jar/Administrator
./IPMIView20.sh

You can use the Discover IPMI Device menu option or just add the system with the relevant IP details. To connect, the default login/password details are both ADMIN which should be changed and entered in the server passwords list. The password supposedly is restricted to a maximum of 16 characters. Once connected, you have access to the following:

  • System Event Log (hardware only, e.g. power supply failure)
  • Sensors (fans, temperatures, voltages, chassis and PSU sensors)
  • Power Control (graceful shutdown controls, power up/down/cycle etc) and IPMI card reset
  • IPMI card TCP/IP settings, SNMP settings (which should be altered immediately, e.g. the community name and SNMP trap receivers)
  • Users (access control)
  • Text Console (serial console access - requires BIOS settings to allow access on first serial port, and OS settings e.g. mgetty to be listening on first serial port at 19200/direct).

Preparing to run, and running IPMIView

  1. Install the Sun Java Runtime Edition. Under Ubuntu, it is packaged in the multiverse repository. Run the following after adding that repo:
    apt-cache update
    apt-get install sun-java5-jre
  2. Open up outbound (& related) UDP port 623 to the IPMI traffic. You are running a firewall, aren't you?

  3. Copy the entire IPMIView application from the CD and edit the IPMIView20.sh file. You will need to customise the java command to reference the real path to the java binary on your system. E.g. /usr/lib/jvm/java-1.5.0-sun/bin/java

  4. Run the IPMIView20.sh script.

Ezemode connections

There's probably better ways to do this (ie, saving your config locally) but here's the alias I use:

# Connect to IPMI
alias ipmi='/usr/lib/jvm/java-1.5.0-sun-1.5.0.13/jre/bin/java -jar 
~IPMI_CDROM/IPMI_Solution/Jar/Administrator/IPMIView20.jar'

Ubuntu is known for overwriting the java binary with every update.

If you're running Ubuntu Hardy, you'll want to prepend the following to that command:

env LIBXCB_ALLOW_SLOPPY_LOCK=1

as per https://bugs.launchpad.net/ubuntu/+bug/193061