We’ve already got solid advice on picking half decent passwords, but this advice from Zebra really takes the cake.

https://www.arin.net/knowledge/comic.html

Huh, this is a new one. It begins:

ATTN: Dear Beneficiary,…DID YOU SEND THEM TO COME GET YOUR FUND

Release/Transfer Notice for your due Funds (US$10,500,000:00).

Without taking much of your time, my name is Dr. Ramsey Goodman, the Honorable Paymaster General of The Federal Republic of Nigeria.

I’m apparently lucky to be contacted by the country’s Financial Monitoring Unit, who have detected the attempted diversion of funds from my account (the very hide of them!) to some other “secret account in the middle East”. Those bastard middle Easterians! I’m glad to have people like this looking out for me.

Before they can transfer the money to my proper account I apparently have to “stop all further contact or communication with every other person regarding your (ie. my) Funds”. They must be “down”</airquotes> with the kinds of tricks those nasty scammers use on unsuspecting victims.

Now if only I could figure out who to send my reply and gratitude to.

• The envelope sender is Mr Song Li <maile.1 AT osu.edu>
• The reply-to header says novellfrancesd2 AT yahoo.co.uk
• The email itself implores me to reply to transferdept_101 AT inmail24.com

O I am conflictuated!

We found this other Aussie hosting company. I know we’re not the cheapest providers in the market (nor do we want to be), but this makes us look positively philanthropic.

Their budget plan is the height of generosity, with 50KiB of diskspace included:
http://www.planethomepage.com.au/hosting/plans_sitesaver.htm

The top-shelf Fully Featured plan is also worth a gander:
http://www.planethomepage.com.au/hosting/plans_fullsite.htm

In all fairness, this is probably fine for someone who would never have a website anyway (eg. old-skool professions for which the internet isn’t relevant to business). For a nominal fee they’ll knock up a website for you (something we’re not interested in providing) and put it on the interwebs for you. Probably not a bad earner if you can find the customers.

The Bureau of Meteorology offers commercial services for those customers that might need it, which is a good thing I guess. There’s an online order form, which you can see here:

http://www.bom.gov.au/other/orderform.shtml

Hey, at least they didn’t print it, put it on a coffee table, photograph it,…

Not quite as zippy as most of the stuff we deploy, but no less admirable for it.
http://www.humanclock.com/webserver.php

I’m currently porting apache 1.3.14 to the venerable TI-89, it should be finished in about 5-6 weeks.

I really like cryptography and security. I was lucky enough to take it as a subject at UNSW before I graduated.

I found this earlier this evening; it’s a little old (~18 months), but that doesn’t make it any less relevant, so it’s a good read. There’s the odd inaccuracy here and there, but it’s solid stuff, and relevant to anyone writing webapp code to handle authentication.
http://www.matasano.com/log/958/enough-with-the-rainbow-tables-what-you-need-to-know-about-secure-password-schemes/

The article focuses heavily on one particular authentication methodology, because it’s something a lot of people do. After you read it, you’ll understand that it’s something a lot of people do poorly. It assumes a reasonable degree of knowledge about what you’re doing (ay, there’s the rub), but the best lesson to be learnt there is that you shouldn’t be writing that code! Just stop! Someone’s already done it better than you, and it’s been checked by a lot more people than you and your colleague, who reckons it “looks pretty secure”. If you’ve got time to read all the comments on the article you’ll find a rollercoaster of people saying, “wouldn’t it be secure if I just..?” Just don’t do it. Don’t write my security code, bro!

Just using a proper authentication mechanism means you can spend your valuable time on more important things.

• Like not building SQL queries in a piecemeal manner, using unsanitised user-supplied data
• Like not making your whole site world-writeable so you can handle file uploads (you don’t have to do this on our shared hosting servers)
• Like fixing your newsletter signup process so it requires double opt-in, saving you from filling your database with spurious accounts and causing massive headaches when you do a campaign mailout
• Like optimising your database schema for bitchin’ performance – do you retain a DBA on a six-figure salary to do this for you? Nah I didn’t think so, this is cheaper and much more fun
• Like checking that your code doesn’t spew 14 PHP Warns/Errors to the apache logs for every page access, and fill the partition
• Like trying to get nice round corners on your page elements

Recently I blogged about our new IPv4 address allocation. While we don’t need to start using it for a while as we have been conserving IP addresses quite well, and gave ourselves plenty of time before we actually need to use the new allocation, it is a good idea to check that it is accessible to the Internet at large.

Our new allocation is from the block 110.0.0.0/8 which was only allocated to the Asia-Pacific regional registry APNIC last November. Prior to it being allocated to APNIC, it would have been in a state affectionately known as “bogon” to network administrators. Bogons are network ranges that aren’t in use, and therefore can be safely ignored by all live networks on the Internet. There have been cases where spammers or other parties looking to conduct illegal activity on the Internet have attempted to use unallocated network ranges for various reasons, so most knowledgable network administrators will block all bogon networks. There are several projects such as the Team Cymru Bogon Reference which put together lists of current bogons to aid network administrators in this task.

The problem comes at the time of removing these bogons from the list. There are currently over 30000 active ASs (Autonomous Systems) on the IPv4 Internet, and effectively each of these must update their own bogon list (if they are not peering with an automatic service such as what Team Cymru provides). Not all network administrators are up to date on the IANA allocations so this process can take months. We are lucky enough to have an allocation from a brand new APNIC range – others are not so lucky and often will have been allocated a range that was previously used by spammers.

Faced with this situation, we’ve decided to try to find out exactly how reachable our new allocation is. I consulted the members of the NANOG mailing list, and pondered their suggestions. I’ve documented below the success of the various methods they suggested and a method which we thought up and decided to try as well.

Do Nothing

One member of the mailing list said:

IMHO, if a network doesn't either update filters based on IANA
notifications or follow Cymru BOGON, then they don't deserve to receive
traffic from your network  

The BOFH within me likes this response very much, but sadly I don’t think that response would be accepted by the boss… I’d also like to take more of an active role in determining our connectivity.

RIPE Debogon Prefix Reachability

http://www.ris.ripe.net/cgi-bin/debogon.cgi

The RIPE regional registry has this page which is effectively just a rudimentary looking glass allowing you to ping or traceroute to your new IP address space. Unfortunately I found dubious results when pinging from some of the routers listed, on all of our address ranges. I suspect not all routers are available or the script behind the page needs updating. If you only have a single address range it would be hard to figure out if results are correct.

RIPE also performs their own testing of de-bogoned address space and graphs the output of their reachability tests. This only really helps you if your allocation has come from RIPE though.

Looking Glasses

Similar to the above method, this involves advertising a small segment of IP space for testing and then using as many public looking glasses on the web as you can find to test connectivity. It is quite thorough, although very time consuming.

Notify network operator groups

One suggestion from the list was simply to post a message on NOG mailing lists and ask the participants to check their filters and optionally attempt to ping the address space in question. This requires participation from the network administrators on the lists, but my main reservation with this method is that if they are knowledgeable enough to subscribe to NANOG or a similar mailing list, they probably take care of their BGP filters anyway so this method probably won’t reveal too many misconfigurations.

Active testing from BGP data

I’m always interested in using BGP data for new and exciting things, so this was a good challenge. From our border routers we can assemble a list of endpoint ASs (as we’re not that interested in strictly transit ASs unless we spot a problem we can pin down to one of them) and pick at least one subnet advertised by each AS. Then we attempt to ping or in some other way communicate with one IP address on each of those subnets. We do this from a working IP on our existing IP allocation range.

We then take the results of that testing and perform the tests again from an IP address on our new allocation range. In theory if the range has been correctly debogoned we will see identical results (even if not all IPs are reachable). If there are discrepancies we can determine which ASs may have bogon-related issues and attempt to contact them.

Reusing most of the BGP dump manipulation script I wrote for my BGP Data Visualisation article, I was able to pull out a list of unique endpoint AS numbers and a subnet for each from our live BGP data within a few seconds. The wc utility tells me that there are 31056 unique AS numbers, which sounds about right based on the recent AS reports. Here is the perl script I used to generate the list of ASs and a subnet for each. You simply pipe the output of the “show ip bgp” command from your router into it and it will print one AS and one target subnet per line:

#!/usr/bin/perl

my %aslist;

while (<STDIN>) {
    # Skip the first 5 lines of header data
    if ($. < 6) { next; }
    chomp;

    # Skip any blank lines
    if ( m/^$/ ) { next; }

    # Skip lines without an AS path or subnet
    if ( m/ 0 (i|e|\?)$/ ) { next; }
    if ( m/^... / ) { next; }
    unless (m/ 0 / ) { next; }

    # Skip the last summary line
    if ( m/^Total number of prefixes.*$/ ) { next; }

    # Grab the AS path and subnet
    s/^...([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}(\/[0-9]{1,2})?).* 0 (.*) (i|e|\?)$/\1 \3/;
    s/(\{|\})//g;
    s/,/ /g;

    # Turn the AS path string into an array
    my @path = split(' ', $_);

    # Add classful subnet designations
    $path[0] =~ s/^([0-9]{1,3})\.0\.0\.0$/\1.0.0.0\/8/;
    $path[0] =~ s/^([0-9]{1,3}\.[0-9]{1,3})\.0\.0$/\1.0.0\/16/;
    $path[0] =~ s/^([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3})\.0$/\1.0\/24/;

    # Add last AS to our global list of ASs
    $aslist{$path[-1]}=$path[0];
}

while (($key, $value) = each(%aslist)){
    print "$key $value\n";
}

Since this was very much just a proof of concept I didn’t have much motivation to ensure absolute correctness or make the process as efficient as possible. Ideally I’d have the entire thing within the one script/program which intelligently pings multiple hosts in separate threads with some sort of limiting involved. Instead, I hacked up a couple of quick shell scripts; the first takes the list of ASs and subnets and passes them to the second script which is forked off for each pair. Forking off indiscriminately would lead to the process scheduler having a fit and the machine becoming unresponsive pretty quick so there is a quick check to make sure there aren’t more than 250 concurrent pings running before forking off another instance.

#!/bin/bash
cat AS | while read as subnet; do
        while [ `ps h -C ping | wc -l` -gt 250 ]; do
                sleep 60
        done

        /data/pingloop $as $subnet &
done

“AS” is the file with AS/subnet pairs.

#!/bin/bash
as=$1
subnet=$2
for ip in `/usr/bin/ipcalc $subnet 255.255.255.255 | grep Hostroute | awk '{print $2}'`; do
        ping -c 5 -i 0.2 -w 5 $ip >/dev/null 2>/dev/null
        if [ $? -eq 0 ]; then
                ping -I X.X.X.X -c 5 -i 0.2 -w 5 $ip >/dev/null 2>/dev/null
                if [ $? -eq 0 ]; then
                        echo "$as $ip reachable" >> output
                else
                        echo "$as $ip bogoned" >> output
                fi
                exit 0
        fi
done

In the above “pingloop” script, we hamfistedly generate a sequence of IPs on the target subnet and attempt to find one reachable IP address then ping it from our new allocation immediately after to see if it is reachable from both subnets.

The results came back in about 10 hours, which isn’t bad for some fairly non-aggressive ICMP reachability testing of effectively the entire IPv4 Internet. Out of 25446 ASs we were able to reach initially, 1716 couldn’t be reached from our new address space which works out to be around 6.7%. Not terrible, but not great either. From here, we’ll look at the ASs that couldn’t be reached and see if there are any patterns that suggest common upstreams need to update their filters.

One disadvantage to this method is raising the ire of network administrators. The amounts of ICMP traffic the scripts generate is pretty minimal but some networks have overly sensitive network monitoring that will trigger if you perform a sequential ICMP “scan” of their network. Of course, it wasn’t performed with malicious intent to really they have no cause to complain.

On DNS and GeoIP

While network-based bogon lists are the prime concern, you should also consider DNS resolver ACLs and GeoIP data. Many DNS administrators will maintain bogon lists in their configurations and these are probably updated even less frequently than BGP bogon lists. If you run into issues with nameservers on your new IP allocation range, you will know that someone out there hasn’t updated their BIND configuration. Similarly, a lot of web services utilise GeoIP to determine the location of a remote IP. By virtue of the allocation to APNIC, our new range is displayed as being in Australia, but it does not show a city or geographical coordinates. Sending an email to GeoIP with your details can rectify this problem.

I was shoulder-surfing a colleague today while they were trying to diagnose a webserver problem for a client. I noticed, certainly not for the first time, that the Apache error log message was filled with messages like “robots.txt not found” and “favicon.ico not found”. Surely these must be amongst the most frequently logged errors (if not the top two).

Multiplied by many hundreds of servers, with millions of hits per day, and you have a significant amount of disk space being taken up by these trivial messages. What’s more, any time you spend scrolling through the hordes of messages like these is time taken away from debugging the real problem, if it exists.

So please, be kind to your sysadmin and include a robots.txt and favicon.ico for your website. It makes sense from a search engine point of view and makes your website just a little bit prettier, so why not?

As mentioned on this blog a few times before, we’re committed to getting IPv6 happening at Anchor. While the live rollout date is probably still a while away, we have at least begun making some inroads on the progress. Today we received our IPv6 allocation from APNIC:

2407:7800::/32

That equates to about 2^96 IP addresses, roughly 10^28 or 79228162514264337593543950336. Quite a mind-boggling number. We’ll continue our research, documentation, testing and will let you know when we are ready to start handing out live addresses. Until then, if you are a customer or would like to be, please let us know you are interested in IPv6, as there are still not too many hosting companies who are using it. Amazing, given IPv4 addresses will run out in a couple of years at best.