These days, anti-virus protection for e-mail is fairly thorough, and nobody’s really swapping floppies full of 16 colour games at recess. Malware authors have moved on to new and more fertile ground — embedding their junk in web pages, and relying on browser exploits to gain access to computers. Of course, with this method, you can only get infected if you actually visit a page that has an infestation, so the malware authors have two options: either entice you to visit their sites, or modify existing websites that users will visit in the course of their day — legitimate sites that people know and trust, but with a little added infection.

Enticing people to a whole dodgy site is usually just a matter of providing something people love to look at and sticking it in search engines. Since the attacker has to have a stable, identifiable presence for the search engines to direct users to, that can also be used by anti-malware lists like stopbadware.org to protect web users, so this isn’t a particularly effective means of attack, and is waning somewhat in popularity. Far more effective is infecting a legitimate website with some form of malware. How does it happen, though? In our experience, there are four vectors for infection:

1. Brute-force password guessing, where the attacker has a botnet they control to just repeatedly try lots and lots of usernames and passwords. They’re bound to get lucky sooner or later.
2. Some sort of web-based exploit, typically a vulnerability in the web application that allows the attacker to run code of their choosing; this is then either used directly to edit files, or bootstrapped into sufficient access to edit files via another method.
3. Password “scraping”, where the attacker gets direct access to the FTP password for your site. This can either be some sort of malware on the workstation of the web developer (or someone else related to the management of the website) that gets the password off the local machine (in a saved password file, or via a keylogger), or else via the “lost password” functionality provided by the hosting provider. Once the attacker has the FTP password for the site, they are free to login to the live site and make whatever changes they like.
4. Direct modification of the website code on the client-side computer, relying on the developer not to notice it and then upload the compromised content to the live site. We recently had our first “confirmed” case of this (where the web developer found the malicious modifications in their local copy), and they swear blind they didn’t download the HTML from the live site (which would bring the “infection” onto the local machine from the infected live site — which we’ve seen before, and categorise under vectors 1 and/or 2).

The countermeasures required to combat all these vectors boil down to a few simple precautions.

• Use strong passwords. (Protects against vector 1) Yes, they’re a pain to manage, but a weak password is just an open invitation to getting repeatedly and painfully owned. Of course, the strongest password is a keypair, which leads us to…
• Don’t use FTP. (Protects against vectors 1 and 3) The list of reasons for this is long, but for securing your website, FTP is a pain because you can only use passwords[1]. Switch to using SFTP (the file transfer component of the SSH protocol) and you can use public keys, which are, for all practical purposes, unguessable. You should also encrypt your keys with a passphrase, which means that even if the attacker does get access to your workstation and copies the key, it’ll be useless to them — unless they keylog your passphrase, which brings us to…
• Keep your workstation secure. (Vectors 3 and 4) It seems that attackers have realised that the weakest link in the website security chain is still the Windows desktop, and they’re increasingly hitting it as the first step in taking over websites (if you get the right workstation, you can get the credentials to hundreds or thousands of websites, because one web developer often works on many different sites). So, on any machine you connect to webservers from, you need to be doubly, triply sure that it’s rock solid — and that’s just a matter of following all the good advice out on the web. Antivirus, antispyware and firewall software, constantly running, well-configured, and kept up to date; keep up to date with your application patches, especially for your web browser, e-mail client, and core OS; don’t visit dodgy web sites; and so on.
• Protect your e-mail. (Vector 3) If someone can get access to your e-mail, they can also get access to your website, by using the password recovery feature (or impersonating you to your hosting company). If they delete the e-mails that are coming in before you notice them, you’ll never know what’s going on, and all the password changes and workstation security in the world won’t help you.
• Don’t use shared hosting. (Vectors 1 and 3). This might seem an odd thing to say, given that we sell shared hosting, but it is a legitimate way to reduce your vulnerability. If you use a dedicated server (including a VPS), you (or we) can configure it to only allow logins from certain IP addresses, rather than the entire Internet. This means that even if an attacker does get your password (or SSH key), via brute force or sniffing it off your workstation, they can’t login from their own machine because it won’t have an authorized IP address. On shared hosting, this configuration is impractical, because hundreds of people have legitimate access to the server, from a great many different IP addresses.
• Code responsibly. (Vector 2) It is said that “PHP is great, because its ease of use means that any idiot can produce a security hole — and most of them do”. Whilst this is a little derogatory to the many (several? few? one? PLEASE?) good PHP programmers out there, it is certainly fair to say that the capabilities of many people who write dynamic code aren’t up to the challenge of writing code that is exposed to the extremely hostile conditions that are the public Internet.

  Thus, if you are not familiar with the common security practices and problems with the language or environment that you are developing for, stop right now and go learn a little. There’s plenty of good information out there on the Internet from people who have learnt the lessons the hard way. Celebrate the benefits of literacy by learning from their mistakes rather than having to educate yourself by cleaning up an infected website. If you feel that isn’t something you can commit to, then please, for the sake of the Internet, find someone else to write the code.

• Keep your web applications patched. (Vector 2) Whilst some sites do use custom-built web applications, many sites choose to use a standard CMS or other application to manage their website. This is great, because hopefully someone else is taking a bit of responsibility for the security of the software, but that doesn’t do you any good if you don’t keep it up-to-date. Far, far too many people install a CMS once, then forget about it. Almost all of these applications have a vulnerability at some point, and not keeping them up to date is absolutely fatal, because once a vulnerability is found in a piece of software, an attacker can typically use Google to find all of the publicly-available instances of the vulnerable software, and quickly attack them all.

  This means that you need to keep yourself well-informed of any security updates for your off-the-shelf web applications. Subscribe to a relevant security announcements mailing list, or ensure that your vendor sends them to you. (If your commercial CMS vendor doesn’t have this ability, find a new CMS vendor).

Websites get compromised all the time, by a variety of methods. You should reinforce your defences, lest you’re the next target.

1. The first person to mention Kerberos or other unused-in-practice authentication schemes in a comment gets a free laughing at. If you think SFTP and SCP aren’t supported in widely used web development programs, try finding something that supports GSSAPI…

I found this earlier this evening; it’s a little old (~18 months), but that doesn’t make it any less relevant, so it’s a good read. There’s the odd inaccuracy here and there, but it’s solid stuff, and relevant to anyone writing webapp code to handle authentication.
http://www.matasano.com/log/958/enough-with-the-rainbow-tables-what-you-need-to-know-about-secure-password-schemes/

The article focuses heavily on one particular authentication methodology, because it’s something a lot of people do. After you read it, you’ll understand that it’s something a lot of people do poorly. It assumes a reasonable degree of knowledge about what you’re doing (ay, there’s the rub), but the best lesson to be learnt there is that you shouldn’t be writing that code! Just stop! Someone’s already done it better than you, and it’s been checked by a lot more people than you and your colleague, who reckons it “looks pretty secure”. If you’ve got time to read all the comments on the article you’ll find a rollercoaster of people saying, “wouldn’t it be secure if I just..?” Just don’t do it. Don’t write my security code, bro!

Just using a proper authentication mechanism means you can spend your valuable time on more important things.

• Like not building SQL queries in a piecemeal manner, using unsanitised user-supplied data
• Like not making your whole site world-writeable so you can handle file uploads (you don’t have to do this on our shared hosting servers)
• Like fixing your newsletter signup process so it requires double opt-in, saving you from filling your database with spurious accounts and causing massive headaches when you do a campaign mailout
• Like optimising your database schema for bitchin’ performance – do you retain a DBA on a six-figure salary to do this for you? Nah I didn’t think so, this is cheaper and much more fun
• Like checking that your code doesn’t spew 14 PHP Warns/Errors to the apache logs for every page access, and fill the partition
• Like trying to get nice round corners on your page elements

IPMI is short for Intelligent Platform Management Interface. Different companies have different names for the technology – Dell has this in their DRAC cards, HP calls this iLO, Sun calls it ALOM. IPMI basically gives you access to a bunch of diagnostic information and management controls for the server. The real killer feature is remote console. Because IPMI is largely independent of the rest of the system, you can get unimpeded access to everything on the server, regardless of what state it’s in. Just like The Matrix, you need to experience how great it is when you rescue a downed server from a BSOD, or a dopey prompt holding up the BIOS while booting.

You see that? IPMI just saved you from:

1. Walking all the way to the datacentre
2. Going through six proximity card doors and an airlock
3. Getting to your suite
4. Opening the racks
5. Finding the server
6. Hooking up a keyboard and monitor
7. Hitting F1 to get past that pesky “keyboard not found” error
8. Walking back to the office – congratulations, you just wasted your lunchtime

Supermicro has IPMI cards for their whole range of servers. They generally all do the same stuff, they just work in slightly different ways. But because they’re not entirely removed from the rest of the system, they can behave in strange ways…

After installing an IPMI card in one system, the video output to the monitor was noticeably poorer. Text was fuzzy and somewhat distorted. Removing the IPMI card fixed this problem immediately.

Some IPMI cards “piggyback” off an onboard network port. An IPMI card has its own IP address but needs a physical network connection. We’ve observed a piggyback-style IPMI card prevent the host from network-booting.

IPMI cards usually offer access to one or more serial ports on the host. Because this happens via the motherboard, we’ve had an instance of it interfering with use of the external serial port, even without wanting to use this feature on the IPMI card.

Of course, once you’ve got the IPMI card installed, it doesn’t count unless you can use it. Supermicro supplies software to configure the card, but it’s not exactly convenient. To actually use the card, there’s a java application you install and run on your workstation. Putting aside any weird issues you might have getting it to run on your version of the JVM, it’s really not fun to use. The interface is clunky and a bit of an eyesore.

Let us turn now to something much nicer, the IBM RSA-II slimline. The RSA (Remote Supervisor Adapter) card provides all the functionality you expect from a remote management card, through a very nice interface. The remote console is still a java client, but this time embedded in a normal webpage served from the card itself. I can SSH to the card, and everything in the interface is snappy (unlike APC’s managed power rails). It really is a nice unit, I love it. It’s also a very capable unit (with a pricetag to match). The board is about the size of a cheap video card (cf. Supermicro’s cards which seem to be mini-PCI form factor), running a 200MHz PowerPC chip, with good management interfaces for multiple users.

It’s when I was setting up these users, the day I got it, that I managed to lock myself out. As is good practice, you setup a strong password for the admin account. This is especially important for such a device exposed to the world, as they don’t have as much protection available. I put the nice, strong password into both password fields, checked all the settings, then called it a day. So you can imagine my surprise when I net tried to login and was rejected.

I was stumped; I’d taken extra care to ensure everything was correct, and now this. Well, not to worry, the card will just have to be hard reset. Somehow.

After searching online it became evident that this wouldn’t be easy. Not content to have something straightforward like an internal reset switch or some sort of magic handshake, IBM have made it necessary for you to run a firmware updater from within an OS. To add annoyance, you have to install the OS driver as well, so the app can talk to the RSA card. Some hours later I had Redhat on a disk, installed the ibmasm driver, and got the card reverted to factory default settings.

I then promptly locked myself out again.

It seems the RSA has a maximum password length of 15 characters. They even include this hard limit as part of the HTML form that you enter the password into, but it doesn’t help much when all you see on screen are bullet-points; you can’t tell how many characters you’ve entered, and browsers quietly drop characters once you hit the limit. A little reminder to the textbox would have been a nice touch, just to let you know that you’ve got a limit. Or just let you put in a password of arbitrary length and have a popup warning if it’s too long. They do that if it’s too simple, so why not for this?

At least it’s highly unlikely you’ll get bitten by this problem twice, but it’s a waste of time finding out the hard way.