Dual-stack IPv6/IPv4 as standard on new US deployments

Published November 22nd, 2011 by Barney Desmond

The focus for this post is obviously about our IPv6 deployment plans, but I’d like to take a small detour through our US presence on the way there.

Anchor’s networking and automation gurus have been hard at work preparing our new kit over in the US, and the day we go live is fast approaching. In the process we’ve had literally zero personal presence over there, not one plane ticket was bought. That we can get away with this is mostly thanks to two things: Equinix and DRACs (Dell’s remote-management interface).

Equinix

One of the reasons we went with Equinix is their high level of support, which goes nicely with Anchor’s approach to business.

The servers were dropshipped direct to the LA3 datacentre where staff unpacked them and racked them up according to our instructions. Once the DRACs were plugged in it was our turn to get excited.

DRACs

DRACs let you do stuff remotely, pretty much anything short of implementing a big red self destruct button. Wanna boot an OS install image from 12,000km away? You can do that.

With a little bit of shuffling, we’ve bootstrapped a new environment over there, ready to build high-availability VMs and servers.

The core of Anchor's LAX1 presence, all gigabit and redundant up the wazoo

IPv6

Which brings us to the fun stuff. The question of offering IPv6 has been on the cards for four or five years now, but to date there hadn’t been enough of a business case for us to commit to. The LAX1 POP changes this.

Anchor’s initial offerings will be focused on our VPS product, which will have full dual-stack connectivity from day 0. For those customers with IPv6 aspirations, we’re ready for you. For everyone else, we hope its enabled-by-default status will drive some interest.

As a small mark of this commitment, we’ve given our DRACs IPv6 addresses only. The practical upshot of this is that it forces us to ensure that all our infrastructure supports IPv6 – routing, switching, DNS, the works. This is popularly called “eating your own dogfood”, we wouldn’t sell something that we don’t use ourselves.

Why IPv6?

For those who don’t follow the IPv4 to IPv6 transition, the rest of this post will summarise Anchor’s perspective specifically.

It’s generally agreed that we’re going to run out of the IPv4 addresses that we know and love, and quite soon. Estimates vary, but unless there’s a revolutionary change in our use of addresses it looks like we’ll hit the wall within about three years.

IPv6 support in the backbone of the internet is already well established, but the real challenge is pushing it all the way to end-users. IPv6 requires big changes through the entire technology stack: routers, switches, firewalls, DNS, servers, operating systems, application configurations and more, it’s all there.

This makes it extremely costly to retrofit existing systems. ISPs have a nasty chicken-and-egg problem because of that – given that demand will only be driven by availability, it’s much easier to do nothing instead of committing to big spending with uncertain returns.

Even assuming that an ISP wanted to sell IPv6 to their consumers, there’s very few content providers doing IPv6. Due to aforementioned lack of demand, content providers would be crazy to present IPv6-only content and lose the whole IPv4 market. Given this, it ultimately looks like the shift to IPv6 will be driven by “external” forces.

As an example, big companies like Google are in a position to push for adoption (try visiting http://ipv6.google.com/), and in theory even offer IPv6-only content that enough users would hassle their ISP for. On the other end of the hypothetical stick, if hosting providers increase the cost (technical and monetary) of IPv4 hosting, it could push enough content to IPv6 to grind through a transition.

These are just two examples, but hopefully they illustrate the difficulty and significance of the problem.

0
Comments

Evil hack to make arrow and SysReq keys work with a Dell iDRAC KVM and Linux desktop

Published March 18th, 2011 by egalanos

So you’re trying to use the arrow keys in a remote server console with the iDRAC KVM. The keys not working and it’s driving you mad? Or perhaps that mission critical server is experiencing pain and you need to make use of the magic sysrq key to debug it?

Unfortunately, for a long long time, this has not worked if you are using Linux on your desktop.

Making it work

Just follow the steps below and you should be good to go.

  1. Download idrac.tar and unpack it:
  2. wget http://www.anchor.com.au/blog/wp-content/uploads/2011/03/idrac.tar
    tar xvf idrac.tar
  3. Build and install it:
  4. cd idrac
    make
    make install

All done! It should take effect the next time you launch a virtual console.

If you want to make use of magic SysReq, make sure that it is disabled on your local workstation! (otherwise you may get a bit of a surprise…)

cat /proc/sys/kernel/sysrq

If the output is not zero, then edit /etc/sysctl.conf and make sure that it contains

kernel.sysrq = 0

then (as root) run sysctl -p

Under the hood

When you are logged into an iDRAC and you click on Launch Virtual Console, your browser downloads a dynamically generated JNLP file for a Java Web Start based KVM application. The Java Web Start tool (typically /usr/bin/javaws) processes the .jnlp file and proceeds to download the KVM software from the iDRAC and run it. The KVM software makes a connection back to the iDRAC on the standard VNC port (5900) (with the single use credentials that were provided to it in the .jnlp file).

At this point, you could easily be mistaken into thinking, Ah, VNC, that’s got to work well right. Such a simple thing and all“. Unfortunately you would be mightily wrong :( .

Whilst the iDRAC is using the standard VNC port, it appears that the implementation has been somewhat customised. Normally under the VNC protocol, a client will send key events to a remote server using standard X Window System keysym values to represent the key. It appears that, for some unknown reason, that the iDRAC client instead sends raw keycodes which it is able to read through the use of native code (this is why the problem isn’t seen on other platforms). Raw keycodes being constant can not be depended on by an application.

As you can guess by now, the keycodes in Linux have changed thus causing the assumptions to break. In the past, Linux distributions used the kbd keyboard driver. Around 2008, most Linux distributions switched to the much nicer evdev input driver system which unifies events from different device types, handles hot plug, and generally makes things sane. A consequence of this change however was  that some keyboard keycodes ended up changing. The end result is the broken arrow keys (plus other less frequently used keys).

The work around above adds more evil to the equation by making a small pre load library to change the behaviour of an X11 function which gets events to silently rewrite the keycode. It currently only remaps the arrow keys and print screen (used for SysReq). When using it, the arrow keys will cease to work in the KVM application menus but that isn’t such a worry. Anyway, have a read of the source code if you need to change anything. Good luck!

1
Comment