Category

Technical

A gentle intro to bughunting

By | Technical | One Comment

A lot of the bughunting that we talk about here is pretty involved, and requires in-depth knowledge of the systems and conventions in play. It’s not exactly conducive to learning if you’re just trying to get started, so we thought we’d take the opportunity to walk through a small bug that we found the other day. It’s really basic, is limited to userspace, and only needs a couple of common tools. We’ll assume that you know a little bit of C and have used gdb to poke around your own code before, but are stuck when it comes to real-world problems. Something that we deploy heavily at Anchor is daemontools, written by Dan Bernstein (aka. DJB). It keeps services running and works in conjunction with a few other small utilities…

Read More

Extending PostgreSQL for fun: with cats

By | Technical | 2 Comments

Perhaps you’ve thought I wish I had more cats in my Postgres database before. We certainly have. Just the other day we were lamenting some of the differences between MySQL and PostgreSQL, particularly the way that MySQL has case-insensitive matching using the LIKE operator, while Postgres has LIKE and ILIKE. This got us thinking, it’d be amusing to have more vague (and hilariously unwieldy) operators, such as: SELECT * FROM foo WHERE a VAGUELY RESEMBLES b; Something like this isn’t too implausible. It’s similar to what a full-text search entails, but it’s far from trivial. This got us thinking: if we can’t easily do that, can we at least have some amusing query syntax? Yes we can!

Read More

A Milky Missive from our Manufactory of Mandates leaves us Mystified as to the Murky Motives of this Mischievous Mailer!

By | Company News, Technical | No Comments

My! We found ourselves Momentarily Mollified by these Magnificent Mouthfuls. Marvelous! Misaddressed? Nay! Mea culpa, the Mail was Manifestly Meant to Make its way to My Merry Mates. Most odd, the Mailer Made up their Mind to keep Mum on their identity and Mask themselves behind anonymity. Monkeyed with, our Mission of discovery was Mooted – not even a Mere Majuscule on their Materials, save for the postmark of our fair capital. No Malice in their intent, thus it Matters not. These shall Make a Mean Mocha come Morning.

Read More

Out-Tridging Tridge

By | Technical | 2 Comments

Andrew Tridgell’s rsync utility is widely used for pushing files around between servers. Anyone can copy files across the network, what makes rsync special is that it compares the files on each end and only transfers the differences, instead of pushing the whole file across the wire when only a few bytes need to be updated. We use rsync to backup servers every single day, though we recently found a few big files where rsync was going to take an eternity to perform what should’ve been several hours of work. So, we busted out the butterfly nets and went to catch us some bugs. rsync in a nutshell Most servers don’t change a great deal on a day to day basis, so transferring just the differences is a very smart…

Read More

Cloud Servers – cheap, fast & reliable?

By | Company News, General, Technical | One Comment

One of the things the cloud has taught us over the past few years is to expect outages. Lots of them. Many of those outages have been pretty big! This is because in the world of public cloud (aka. Infrastructure as a Service), running servers in reliable, physical data centres has been replaced with a new concept; that of running cloud servers in Availability Zones. Which is fine, so long as you understand the implications and can configure your website or application to operate optimally in this type of environment. Don’t get me wrong, cloud servers have a lot going for them. It’s just that while they solve many problems, they also introduce a few biggies of their own and you really should be across the pros and cons before jumping on the…

Read More

A most unusual temporal issue

By | Technical | No Comments

Most of us have suffered time-related woes, dealing with servers that just can’t keep their clock straight even with NTP. Well we’ve just come across a new one, and NTP isn’t going to save the day – what happens when you have multiple misbehaving clocks? We first noticed the problem when reloading the firewall on the host, a KVM server that had been virtualised from hardware. We use filtergen for ruleset generation, and one of its postreload scripts restarts fail2ban, an anti-bruteforce tool. One part of fail2ban’s initscript makes a call to sleep 1 to give things time to settle. We noticed that the invocation of sleep was taking more than a second, much more. Something wasn’t right and the automatic 15sec rollback was kicking in, having not received explicit…

Read More

Anchor’s new Sydney datacentre presence coming online

By | Technical | No Comments

Our network admin and ops manager were loading hardware into boxes last night before leaving the office. This is somewhat unusual, so I grabbed them to find out what was happening before they escaped. “We’re off to install SYD3” It’s finally here! We’ve been talking about it for a while, and now it’s imminent. We’ve picked up some space in Equinix’s shiny new SY3 datacentre, which is going to be a beachhead for Anchor’s expansion plans. Opening the SYD3 point of presence (POP) is a strategic maneuver. Connectivity will initially be a dark fibre to SYD1, essentially making SYD3 a remote territory that, at the network layer, will be fully internal to SYD1. Once things are settled in, SYD3 will light up with local transit providers and start doing fully…

Read More

Slick log aggregation with Kibana

By | Technical | No Comments

Log aggregation is awesome-cool, you already know this. There’s no shortage of solutions for doing this, and you’ve probably heard of a few like Splunk, Graylog, Logstash, Scribe. There’s plenty more that integrate and gather data in various ways, and you can always can roll your own. Having all your logs in one place makes it much easier to spot patterns and perform analysis. This is particularly useful when you have a cluster of machines doing roughly the same thing, as it makes anomalies more obvious and less prone to noise from false positives. One of our devs was itching for a frontend tie into the service-oriented architecture stuff that we’re building at the moment, particularly one that could be easily hacked up and modified, so he went looking and…

Read More

IPv6 NAT from a webhosting perspective

By | Technical | No Comments

IPv6 is something that’s been coming for a long time now, to address the shortage of IPv4 addresses. If you’re not familiar with the problem, it goes like this: Networked devices need addresses to find and talk to each other. When the internet was invented there was capacity to address up to 4 billion devices. We’re now running out of addresses for all the networked devices in the world. There are two established technological ways out of this problem: Be frugal and share addresses, this is called NAT Move to a system with capacity for many more addresses, this new system is called IPv6. It’s not compatible with the old IPv4 system, but it gives us about 50 octillion addresses for every person on the planet NAT is a really…

Read More

Meet Sharif – the whiz kid with an answer for everything

By | Company News, Technical | No Comments

Sharif Olorin has a reputation around the office for 3 things: his deadpan sense of humour, his numerous tattoos and piercings, and his obsession with Russian Caravan tea. So far, the best Russian Caravan he’s found is at the Glebe markets in Sydney’s Inner-West, an area he now calls home. In many ways, Sharif is stereotypical of inner-Western Sydney. He’s young, wicked-smart and tech-savvy. He doesn’t wear a suit to work and he cares more about getting the job done than whose responsibility it is. He has a lambda – the Greek letter λ often used in mathematics and science – tattooed on his right forearm, and even uses the symbol as his signature!

Read More