I kid you not, we just received this in a piece of marketing guff from our favourite enterprise vendor.

“Industry analysts predict that Linux and Windows will soon dominate the operating system space. How you respond to this is critical.”

Meanwhile, industry analysts predict that more than 98% of the population will be consuming oxygen by 2010.

Reducing the memory usage in your VPS/VM can be a great way to free up some resources to handle more requests, users or some other metric of win.

By default at Anchor we provision our Red Hat & Cent OS VPS servers with a trim memory usage profile by disabling a lot of unneeded services at install time. We do this by using Trogdor (our hardware/software burninator) and Puppet.

So just what services do we disable, if they exist on the new VPS?

• gpm
• netfs
• pcmcia
• sgi_fam
• yum-updatesd
• pcscd
• rhnsd
• xfs
• hald
• hcid or sdpd
• hpiod or hpssd.py
• dbus-daemon
• cupsd

You can also reap performance gains by changing how you serve content. For example you can use a cut down high performance web server (nginx or lighttpd) to serve all static content, such as images. Then use an Apache process to handle your dynamic requests.

Tuning Apache is deserving of an article all to itself, however some hard and fast rules are:

• Disable all unnecessary modules.

• Work out the per process memory usage, and set your max clients to a suitable number taking into account the available memory and other system daemons.

• Disable htaccess if you do not need it.

PHP is much the same as Apache, disable what you do not need. This goes for any service or application with many optional components.

Knowing the state of your disk IO latency in VMware ESX can help you pre-empt performance & capacity issues before the occur. There are a few guidelines you should keep in mind. These notes are directed towards people using directly attached storage.

• Write latency should be 0, because you have that fancy battery backed controller caching writes, right?
• Read latency should be under 8ms.
• Use the smallest stripe size possible for your RAID array setting. This helps keep random IO performance acceptable at the cost of some sequential performance.
• Do not virtualise very heavy random IO workloads on shared arrays, other guest VMs wont like you for it.
• Unless you have a very compelling reason not too, use RAID 10.

Some other notes, specific to Linux guests are:

• Mount file systems with noatime and nodiratime, this will help reduce random IO.
• Allocate enough memory to have some buffers.
• Do anything possible to stop your VM swapping heavily (see point above).

As with any system, having great monitoring and performance trending allows for you to have an excellent overview of your infrastructure. Even if you don’t have external systems for performance trending, the VMware Infrastructure client with a few tweaks will display the data you want to see.

1. Login to the VI Client.
2. Click on an object in the left navigation tree.
3. Click on the performance tab at the top of the main display pane.
4. Click the “Change Chart Options” button
5. Select the Disk chart option from the left expanding menu.
6. Now change the counters, pick the Latency counters and Number counters, un-ticking the KBps  counters.
7. Save the chart settings as disk-latency.

Now you can view in real time what is happening with your disk IO on the VMware ESX server. If you are more familiar with using a command line and Linux, you can SSH in to the ESX COS and use the command esxtop to view disk performance information.

1. Launch esxtop (as root)
2. Press “v”
3. Press “s” and then “1″

Now you can see the per VM disk usage counters, with a 1 second sample period.

These rules of thumb are also applicable to Xen and Hyper-V.

What is dstat?  dstat is a versatile replacement for vmstat, iostat and ifstat.

Dstat allows you to view all of your system resources instantly. Eg, You can compare disk usage in combination with interrupts from your HDD controller, or compare the network bandwidth numbers directly with the disk throughput (in the same interval).

No blog post pimping a cool tool would be complete without the obligatory screen shot.

dstat default output