For the lulz (Building Secure Websites)

Published June 28th, 2011 by Chris Collins

With all of the #antisec love going around, we felt was a good time to discuss some of the key principles in writing secure webcode. Today’s topic is unsanitised input.

A great piece of philosophy for designing secure systems is that any piece of information that comes from an external source is inherently untrustworthy. This applies quite strongly to the web – every detail in a web request comes from a user somewhere, and can easily be forged and specially crafted to manipulate poorly written code.

This is particularly notable in languages like PHP, where there is a low barrier to entry for writing code, but doesn’t often provide good frameworks or assistance to avoid being caught out by these problems.

Validate All Input

The first rule is to validate all input. You can’t trust anything that comes in from the request to conform to a certain format.

Just because your pretty HTML form only allows certain values to be sent back doesn’t mean that some nasty cracker somewhere can’t craft up a request to your webserver that doesn’t conform to this.

The way we deal with this is to clean these parameters, always. If we expect the parameter to be an integer, we ensure that the parameter only contains an integer, and treat everything else as an error. If it’s a string, we have to assume we don’t know how long that string is, or what sort of characters it may contain.

Now, I know some of you are thinking “Ah, but we can avoid all of this by using javascript to validate our forms!”. That doesn’t work as the javascript has no bearing on requests sent by our crafty cracker – they can only influence the result from a user using a web browser in the way you intended.

Tread with Care

With strings in particular, filtering the contents is not always an option – in these cases, we need to ensure that when we use the string elsewhere, its contents do not interfere with the other operation.

One of the most common places where this gets fouled up is in SQL queries.

A common (erroneous) construct you’ll find used is:

$query = sprintf("SELECT * FROM articles WHERE id="%s"', $_REQUEST['id']);
$myresults = mysql_query($query);

// do stuff with $myresults...

The problem with this is that $id is joined verbatim with the rest of the SQL query. If $id contains characters with special significance in SQL, such as the ” (double-quote) character, they will be interpreted as such.

If Little Johnny Cracker were to send our PHP page a request with id set to:

"; DELETE FROM articles; --

the next thing we know, our DB server no longer has anything in its articles table!

Copyright © Randall Munroe - xkcd.com

We can protect against this by modifying the string so that characters that are special to SQL are appropriately “escaped”. In this specific instance, there is a mysql_real_escape_string function to do this for us, but most other database libraries and other languages offer different ways to perform this important task.

If we amend our example above, the code becomes:

$query = sprintf('SELECT * FROM articles WHERE id="%s"',
            mysql_real_escape_string($_REQUEST['id'])) ;
$myresults = mysql_query($query) ;

// do stuff with $myresults...

This ensures that the contents of id will be interpreted correctly as part of the string being compared to id, and won’t contain characters that could be interpreted as part of the SQL command.

But this seems all too hard/too manual!

Now, you’re all bound to say, “There must be an easier way!”, and there usually is.

There are plenty of libraries and wrappers available which are aware of the problem with trying to prepare safe SQL queries and provide simple, convenient, parameter escaping to ensure that user input can be safely combined with your queries.

In PHP, there is the PEAR MDB library which can do the above safely. An example:

$sth = $db->prepare('SELECT * FROM articles WHERE id=?') ;
$myresult = $db->execute($sth, $_REQUEST['id']) ;

// do stuff with $myresults...

Other languages, such as Ruby, Python and Perl, all have their own safe and convenient methods for doing this too.

In Summary

Don’t trust the input into your web code. Make sure you never mix input with strings you’re handing off elsewhere, such as via exec, eval or to a database. And finally, use convenience methods for correcting/filtering user input when possible to make your life easy.

0
Comments

Implementing proper trust relationships in backup systems

Published June 21st, 2011 by Keiran Holloway

As many people may have read today, a certain web hosting company which operates in Australia suffered an attack which resulted in significant amounts of data loss. Not only was their live production data lost, but all backups were also unrecoverably lost in the process.

Whilst significant technical details have yet to be released as to the nature of these attacks, now is an opportunity as good as ever to explain how the trust relationships exist between the backup server and the system being backed up across our entire server infrastructure.

Our backup servers are separated with there being a minimal trust relationship between the backup server and the system being backed up:

  • All backups happen on an independent network which is inaccessible from the Internet.
  • The backup server can only read data from the machine being backed up (and not alter it).
  • The machine being backed up can only send new data and not alter the data that is already backed up.
  • Using this methodology should protect any attacks from the public, customer facing servers from damaging the independently stored backups.

    0
    Comments

    Secure encrypted storage for your hosted server, VM, VPS, cloud server or $BUZZ_WORD

    Published March 4th, 2011 by egalanos

    So you’ve just provisioned your shiny new OS instance with your host of choice, loaded in your confidential data and away you go without a worry in the world right? If your data consists only of captioned photos of cute furry animals, then all is well. Perhaps however, your data is worth just a wee bit more than that (not that we don’t ♥ cute furry animals!).

    Depending on your host and product used, your data could be sitting on anywhere from locally attached disks, a NAS/SAN or some clustered distributed block device/filesystem with no way to easily determine who has access to it, what snapshots exist, what will happen to failed media, etc. For certain customers with certain sensitive applications, that is simply not an acceptable risk.

    To protect your data at rest, the most reassuring option is to utilise disk encryption within your operating system where only you possess the key. Whilst this can present a bit of an operational challenge (i.e. how does the key get entered if the server is rebooted), the show stopper question is going to be “Can I actually run a production database with heavy I/O load on encrypted storage?”. This post seeks to answer that question or to at least help you figure it out for your circumstances.

    Database server

    Our test server is a Dell PowerEdge R410 with:

    • 2 × Intel Xeon E5620 CPUs
    • 16 GB of RAM
    • Dell PERC H700 hardware RAID controller
    • 2 × Dell 50GB SSDs in a hardware RAID 1 array for the database
    • 2 × 300GB 15K SAS in a hardware RAID 1 array for the operating system
    • Red Hat Enterprise Linux Server 6.0 (kernel 2.6.32-71.18.1.el6.x86_64)

    The interesting thing to note in this setup is that the CPUs contain hardware accelerated AES encryption via the AES instruction set (Intel’s marketing name is AES-NI). You can check if your CPU and kernel support this feature by running at the command line:

    grep aes /proc/cpuinfo

    if you get output, then congratulations, you have the feature supported!

    Encrypted storage

    The recommended way to do block device level encryption under Linux is to use the device mapper target dm-crypt. This is a straightforward component that is built-in to the Linux kernel and well supported in all modern Linux distributions. To set up full disk encryption, you should use your distribution’s installer when you initially provision your server. If you are not encrypting your entire OS (sans /boot) and data, then you will need to use the utility cryptsetup on the storage device you want to encrypt. For our testing, we are just encrypting the SSDs as follows:

    cryptsetup luksFormat /dev/sdb -c aes-xts-plain -s 512

    This command formats the storage device with a LUKS meta-data header. This header contains useful information including:

    • the fact that it contains encrypted data (rather than just random data)
    • a UUID
    • the  algorithm used
    • the key size
    • the key (securely encrypted with a passphrase that you provide at format time)
    • checksums and other useful bits

    For high security, we are using a 512 bit AES key in an XTS cipher mode (requires kernel >= 2.6.24). A small word of warning: whilst cryptsetup allows you to setup encryption on devices without any sort of identifying meta-data header, DON’T ever do that. If you need to hide the fact that you are using encryption there are better ways to do that. Contact us for details if you really want to go down that path.

    Once you have formatted the device, you then need to activate it by running the command:

    cryptsetup luksOpen /dev/sdb testing

    this will then make the unencrypted block device ready for you to use (i.e mkfs) as /dev/mapper/testing. To have the operating system setup the device at boot, you will need to put an entry into /etc/crypttab. Check the man page for details.

    Finally, for best performance you want the implementation of AES that is most optimised for your hardware. You can check /proc/crypto to see which algorithms and drivers are available and their priority (highest priority number indicates higher priority). In order of performance, you want AES-NI, AES-x86_64 and the generic AES implementation last. The highest priority crypto implementation at the time of an encrypted block device being activated (via cryptsetup luksOpen) is used. Higher priority drivers installed into a running kernel have no effect on active devices.

    Benchmarking

    The simple benchmark tool zcav is fine for our purposes. It only measures the sustained read/write speed. For an I/O device, this benchmark would normally be of limited value as the device bottleneck is usually with the rate of I/O requests that can be handled. For an encrypted block device, the bottleneck is instead in the throughput (i.e. the sustained read/write speeds) as the higher the throughput, the more work that the CPU has to do.

    Results

    The output from zcav was fed into gnuplot to generate the following graphs:

    Read benchmark results

    The obvious thing to note in this diagram is that the new AES instructions make a huge difference to performance. The impact of the encryption is relatively low and should be quite acceptable for a production system. An interesting thing to note is the relatively low performance of the RAID array. Being RAID 1, the RAID controller should have been able to balance the read requests over both drives to get about double what it is getting. I haven’t looked into the RAID settings though to see if this can be tuned.

    Write benchmark results

    The throughput here with the AES instructions in use is almost identical.

    Conclusion

    The results certainly look promising for even relatively high end work loads. One thing that is not measured here is the latency difference. It should be relatively straightforward to calculate the latency that will be added to every I/O request.

    The numbers above can also be massively improved upon with a newer kernel featuring the dm-crypt scalability patches (the benchmark above was limited to using 1 core out of 8!).

    The nice thing about using the built-in dm-crypt solution is that you can use it on physical as well as virtual machines. If you are after something a bit more turn-key, you may wish to look at a system with drives (and a controller and BIOS) that support self encrypting drives as an alternative.

    2
    Comments

    DIY RFID security system

    Published December 13th, 2010 by egalanos

    We just moved office last week to much larger premises!

    Whilst our office building already has excellent security as it houses a credit union, an additional security layer for our office floor can’t hurt (this is unrelated to our servers; they are housed in the secure Global Switch data centre). Now sure, we could easily just get a commercial off the shelf alarm system and be done with it, but it can be pretty fun and interesting to get your hands dirty and do a bit of DIY hacking.

    Using some simple components, we can quickly and easily build a basic RFID security system. One of the advantages of doing it ourselves (besides the fun!) is that we can integrate it into our standard monitoring and internal chat systems. The parts list for this project come from the excellent Australian electronics store Little Bird Electronics. The parts:

    UPDATE 7th March 2011:

    We thought it would be good to re-use the proximity cards we already carry around with us for this project, but we couldn’t get any of them to work with this reader.  There are a lot of proprietary standards out there and documentation on compatibility is really quite poor.  We are currently sourcing another reader that can handle a wider range of 125kHz proximity cards.

    For the curious, Indala FlexCard cards by HID Global use a totally proprietary air interface.

    Whilst doing some research, an awesome open source hackers RFID module, the Proxmark3 was discovered. A very nice learning tool.

    More posts to follow with details as the project progresses…

    1
    Comment

    PHP on shared hosting – doing it better

    Published September 11th, 2010 by Barney Desmond

    Large scale shared hosting with an out-of-the-box install of apache and PHP is a recipe for security-disaster; this is not news. The solution is to run each website’s code separately so they can’t affect each other. This is pretty common nowadays but it wasn’t always the case with many providers.

    Anchor’s been doing this for what must be about ten years now. That’s way longer than I’ve been employed here, but what with our tech-director-and-co-founder being busy stuntjumping his scooter over rows of parked cars, it’s fallen to me to write this one up.

    We use apache’s mod_suexec to run PHP scripts as though they’re CGI scripts, and it works great. There’s lots of guides out there about how to do this for yourself, but for us one of the most important things when deploying a solution is to ensure that it Just Works, for everyone. When we do it right, nothing breaks and noone notices a thing, because it works exactly as everyone expects. That’s what we’ve done.

    It’s also one of the reasons that apache isn’t going anywhere in a hurry. Newer tech like nginx is an absolute performance-demon compared to apache, but the barrier to entry is way too high if your goal is “throw the site on the server and make it work”. Everyone writes apps assuming they’ll be deployed to apache. As a hosting company, if you’re not offering something compatible you’d better have a good ace up your sleeve.

    So, we give you a rundown on how Anchor does PHP; it’s secure and it just works. There are many others like it, but this one is ours. If you’re an existing customer, we hope you’ve never had to think about it. If you’ve got any questions we’d love to hear them.

    0
    Comments

    Securing your codez from the wily exploit injectors

    Published November 23rd, 2009 by matt

    Remember the good old days, when Melissa and ILOVEYOU were the major virus threats, spreading via e-mail and causing all sorts of embarrassing conversations at work? Or maybe even earlier than that, when the only way you could get a virus was by engaging in risky sex? (I mean Software EXchange, of course… get your mind out of the gutter)

    These days, anti-virus protection for e-mail is fairly thorough, and nobody’s really swapping floppies full of 16 colour games at recess. Malware authors have moved on to new and more fertile ground — embedding their junk in web pages, and relying on browser exploits to gain access to computers. Of course, with this method, you can only get infected if you actually visit a page that has an infestation, so the malware authors have two options: either entice you to visit their sites, or modify existing websites that users will visit in the course of their day — legitimate sites that people know and trust, but with a little added infection.

    Enticing people to a whole dodgy site is usually just a matter of providing something people love to look at and sticking it in search engines. Since the attacker has to have a stable, identifiable presence for the search engines to direct users to, that can also be used by anti-malware lists like stopbadware.org to protect web users, so this isn’t a particularly effective means of attack, and is waning somewhat in popularity. Far more effective is infecting a legitimate website with some form of malware. How does it happen, though? In our experience, there are four vectors for infection:

    1. Brute-force password guessing, where the attacker has a botnet they control to just repeatedly try lots and lots of usernames and passwords. They’re bound to get lucky sooner or later.
    2. Some sort of web-based exploit, typically a vulnerability in the web application that allows the attacker to run code of their choosing; this is then either used directly to edit files, or bootstrapped into sufficient access to edit files via another method.
    3. Password “scraping”, where the attacker gets direct access to the FTP password for your site. This can either be some sort of malware on the workstation of the web developer (or someone else related to the management of the website) that gets the password off the local machine (in a saved password file, or via a keylogger), or else via the “lost password” functionality provided by the hosting provider. Once the attacker has the FTP password for the site, they are free to login to the live site and make whatever changes they like.
    4. Direct modification of the website code on the client-side computer, relying on the developer not to notice it and then upload the compromised content to the live site. We recently had our first “confirmed” case of this (where the web developer found the malicious modifications in their local copy), and they swear blind they didn’t download the HTML from the live site (which would bring the “infection” onto the local machine from the infected live site — which we’ve seen before, and categorise under vectors 1 and/or 2).

    The countermeasures required to combat all these vectors boil down to a few simple precautions.

    • Use strong passwords. (Protects against vector 1) Yes, they’re a pain to manage, but a weak password is just an open invitation to getting repeatedly and painfully owned. Of course, the strongest password is a keypair, which leads us to…
    • Don’t use FTP. (Protects against vectors 1 and 3) The list of reasons for this is long, but for securing your website, FTP is a pain because you can only use passwords[1]. Switch to using SFTP (the file transfer component of the SSH protocol) and you can use public keys, which are, for all practical purposes, unguessable. You should also encrypt your keys with a passphrase, which means that even if the attacker does get access to your workstation and copies the key, it’ll be useless to them — unless they keylog your passphrase, which brings us to…
    • Keep your workstation secure. (Vectors 3 and 4) It seems that attackers have realised that the weakest link in the website security chain is still the Windows desktop, and they’re increasingly hitting it as the first step in taking over websites (if you get the right workstation, you can get the credentials to hundreds or thousands of websites, because one web developer often works on many different sites). So, on any machine you connect to webservers from, you need to be doubly, triply sure that it’s rock solid — and that’s just a matter of following all the good advice out on the web. Antivirus, antispyware and firewall software, constantly running, well-configured, and kept up to date; keep up to date with your application patches, especially for your web browser, e-mail client, and core OS; don’t visit dodgy web sites; and so on.
    • Protect your e-mail. (Vector 3) If someone can get access to your e-mail, they can also get access to your website, by using the password recovery feature (or impersonating you to your hosting company). If they delete the e-mails that are coming in before you notice them, you’ll never know what’s going on, and all the password changes and workstation security in the world won’t help you.
    • Don’t use shared hosting. (Vectors 1 and 3). This might seem an odd thing to say, given that we sell shared hosting, but it is a legitimate way to reduce your vulnerability. If you use a dedicated server (including a VPS), you (or we) can configure it to only allow logins from certain IP addresses, rather than the entire Internet. This means that even if an attacker does get your password (or SSH key), via brute force or sniffing it off your workstation, they can’t login from their own machine because it won’t have an authorized IP address. On shared hosting, this configuration is impractical, because hundreds of people have legitimate access to the server, from a great many different IP addresses.
    • Code responsibly. (Vector 2) It is said that “PHP is great, because its ease of use means that any idiot can produce a security hole — and most of them do”. Whilst this is a little derogatory to the many (several? few? one? PLEASE?) good PHP programmers out there, it is certainly fair to say that the capabilities of many people who write dynamic code aren’t up to the challenge of writing code that is exposed to the extremely hostile conditions that are the public Internet.

      Thus, if you are not familiar with the common security practices and problems with the language or environment that you are developing for, stop right now and go learn a little. There’s plenty of good information out there on the Internet from people who have learnt the lessons the hard way. Celebrate the benefits of literacy by learning from their mistakes rather than having to educate yourself by cleaning up an infected website. If you feel that isn’t something you can commit to, then please, for the sake of the Internet, find someone else to write the code.

    • Keep your web applications patched. (Vector 2) Whilst some sites do use custom-built web applications, many sites choose to use a standard CMS or other application to manage their website. This is great, because hopefully someone else is taking a bit of responsibility for the security of the software, but that doesn’t do you any good if you don’t keep it up-to-date. Far, far too many people install a CMS once, then forget about it. Almost all of these applications have a vulnerability at some point, and not keeping them up to date is absolutely fatal, because once a vulnerability is found in a piece of software, an attacker can typically use Google to find all of the publicly-available instances of the vulnerable software, and quickly attack them all.

      This means that you need to keep yourself well-informed of any security updates for your off-the-shelf web applications. Subscribe to a relevant security announcements mailing list, or ensure that your vendor sends them to you. (If your commercial CMS vendor doesn’t have this ability, find a new CMS vendor).

    Websites get compromised all the time, by a variety of methods. You should reinforce your defences, lest you’re the next target.


    1. The first person to mention Kerberos or other unused-in-practice authentication schemes in a comment gets a free laughing at. If you think SFTP and SCP aren’t supported in widely used web development programs, try finding something that supports GSSAPI…

    1
    Comment

    On the importance of knowing how to pick secure passwords

    Published May 28th, 2009 by Barney Desmond

    We’ve already got solid advice on picking half decent passwords, but this advice from Zebra really takes the cake.

    zebrasafepassword

    0
    Comments

    Developers, you can stop your crap authentication schemes now. Please.

    Published May 22nd, 2009 by Barney Desmond

    I really like cryptography and security. I was lucky enough to take it as a subject at UNSW before I graduated.

    I found this earlier this evening; it’s a little old (~18 months), but that doesn’t make it any less relevant, so it’s a good read. There’s the odd inaccuracy here and there, but it’s solid stuff, and relevant to anyone writing webapp code to handle authentication.
    http://www.matasano.com/log/958/enough-with-the-rainbow-tables-what-you-need-to-know-about-secure-password-schemes/

    The article focuses heavily on one particular authentication methodology, because it’s something a lot of people do. After you read it, you’ll understand that it’s something a lot of people do poorly. It assumes a reasonable degree of knowledge about what you’re doing (ay, there’s the rub), but the best lesson to be learnt there is that you shouldn’t be writing that code! Just stop! Someone’s already done it better than you, and it’s been checked by a lot more people than you and your colleague, who reckons it “looks pretty secure”. If you’ve got time to read all the comments on the article you’ll find a rollercoaster of people saying, “wouldn’t it be secure if I just..?” Just don’t do it. Don’t write my security code, bro!

    Just using a proper authentication mechanism means you can spend your valuable time on more important things.

    • Like not building SQL queries in a piecemeal manner, using unsanitised user-supplied data
    • Like not making your whole site world-writeable so you can handle file uploads (you don’t have to do this on our shared hosting servers)
    • Like fixing your newsletter signup process so it requires double opt-in, saving you from filling your database with spurious accounts and causing massive headaches when you do a campaign mailout
    • Like optimising your database schema for bitchin’ performance – do you retain a DBA on a six-figure salary to do this for you? Nah I didn’t think so, this is cheaper and much more fun
    • Like checking that your code doesn’t spew 14 PHP Warns/Errors to the apache logs for every page access, and fill the partition
    • Like trying to get nice round corners on your page elements :)
    0
    Comments

    # mysql_secure_installation… Ya-ha-! (and ~/.my.cnf)

    Published March 31st, 2009 by Barney Desmond

    I was setting up mysql-server for a customer recently and noticed something interesting – there’s a helpful script included with mysql called mysql_secure_installation. We thought about that for a moment and had a chuckle. Okay, that was a little unfair; it’s no secret that we prefer to use Postgres wherever possible, but the idea of having a “make it all secure” script isn’t too bad an idea, as long as it doesn’t produce a false sense of security.

    Read the rest of this entry »

    0
    Comments

    SaaS (Security-scanning as a Service)

    Published December 9th, 2008 by Barney Desmond

    We’ve had some enquiries from customers recently regarding security compliance scans, the most popular of which is the PCI DSS. For those not in the know, this stands for the Payment Card Industry Data Security Standard. It is of course a fascinating topic, covering best-practice standards for processing and storage of customer information.

    The enquiries we get relate to a security scan carried out by an Approved Scanning Vendor (ASV). The usual report format is a list of potential “vulnerabilities” detected, with a severity rating of 1 to 5 assigned to each. Anchor’s shared hosting servers never have any problems with this, so the report reads like a missal of mundanity.

    TCP port 21 is open, an FTP service appears to be running! Crazy, I know…

    The thing is, this scan is really just one small part of a much larger framework. The core requirements of the PCIDSS don’t specify at all how the scan should be performed; it’s really about secure storage and transmission of data, and accountability and auditing.

    Do our customers’ applications really encrypt the data they store in the database? I don’t know, but it sure isn’t checked as part of the scan. Requirement 6 is “Develop and maintain secure systems and applications”. Mm-hmm, that’s a good idea…

    Security is really a commodity nowadays, a fact highlighted most perfectly in the vending of SSL certificates. In case you hadn’t guessed, the PCIDSS scans we’ve seen can proudly join the ranks. Thankfully there are scanners who really know where their towel is, looks good to me!

    http://www.scanlesspci.com/

    http://blogs.zdnet.com/security/?p=1114

    http://jeremiahgrossman.blogspot.com/2008/04/my-blog-is-pci-certified-by-scanless.html

    0
    Comments