Andrew Tridgell’s rsync utility is widely used for pushing files around between servers. Anyone can copy files across the network, what makes rsync special is that it compares the files on each end and only transfers the differences, instead of pushing the whole file across the wire when only a few bytes need to be […]
Today we’re talking about our experience with btrfs, the next-gen Linux filesystem. btrfs has been maturing rapidly over the last few years and offers many compelling features for modern systems, so we’ve been putting it through its paces on some of our backup servers. How does it stack up? Read on! We chose to test […]
Redis has become one of the most popular “noSQL” datastores in recent times, and for good reason. Customers love it because it’s fast and fills a niche, and we love it because it’s well behaved and easy to manage. In case you’re not familiar with Redis, it’s a key-value datastore (not a database in the […]
One of our sysadmins has been cleaning up and auditing our backups recently, and we thought we’d look at some of the numbers involved and how they’ve changed over time. Several years ago we used tapes exclusively, and they were always taken offsite. We don’t have the numbers from that time, but every fully-managed server […]
PostgreSQL is awesome in many, many ways. One of these ways is in its use of a write-ahead log, or WAL, that enables: Atomicity Durability Replication (that isn’t insane, like MySQL’s) Backups And much much more! </televisionSalesman> We’re interested in backups today, because textual dumps for backups (the lowest common denominator for mysql and pgsql) […]
As many people may have read today, a certain web hosting company which operates in Australia suffered an attack which resulted in significant amounts of data loss. Not only was their live production data lost, but all backups were also unrecoverably lost in the process. Whilst significant technical details have yet to be released as […]
Our current rdiff and amanda backup server, KRAKEN, is almost full, so it was time to order a new one. After much wrangling, we finally received LEVIATHAN this morning. I was pushing for PHYREXIAN DREADNOUGHT personally, but LEVIATHAN is acceptable too; the upkeep effort of backup servers is pretty high after all.
Sydney suffered a nasty power outage in the CBD on Monday, which according to reports affected tens of thousands of homes and businesses. Curiously, some traffic lights on George street were blacked out while others just a block or two away were working fine. From a technical standpoint, a measure of diversity like that is […]