Tag

cluster Archives - AWS Managed Services by Anchor

Mar 02 2015
1

A Year of OpenStack

By | General | One Comment

In late 2013 we looked around the company and asked ourselves the questions any management team has to ask: what are we doing, where do we need to be, and what’s holding us back from getting there? Because Anchor had grown organically across many years, our internal procedures for infrastructure management were spread across a number of tools that weren’t scalable and efficient enough to keep up with the demands of new sales. Furthermore, they weren’t compatible with a product-based self-serve future. There are a number of pieces to addressing that, but certainly it would be tremendously useful to have API-driven software defined infrastructure. The recommendation, universally, was that we needed to consider OpenStack. We were astonished at the degree of rigour and testing that had gone into vetting changes (since the beginning, there have been no un-gated…

Read More
Mar 07 2013
1

Flying with Redeye

By | Technical | No Comments

Several months ago we talked about extending the functionality of Redis, the popular noSQL key-value datastore. Since then we’ve taken things a bit further and we think it’s worth sharing. The Customer needs to store a lot of data in Redis, a few hundred GB at last check, and growing. Redis’ single-threaded nature means it doesn’t scale vertically without bounds though, you can only process so many thousands of transactions per second before you burn up a whole CPU. The solution is to shard your data and scale out horizontally across more CPUs and more RAM, so those hundreds of GB now live on a cluster of about a dozen independent Redis instances. Shard boundaries can’t be easily adjusted, so we created a large number of shards initially and let…

Read More
Sep 20 2012
1

What’s the big idea with: Plug and play hypervisors?

By | Technical | No Comments

Here at the Anchor Internet Laboratory we’ve been discussing ideas for new deployments of our VPS infrastructure. One that we’re excited about is the idea of “plug and play” hardware. Plug and play what? Deploying more hardware capacity takes time. It needs to be burn-in tested, tracked in the asset management system, installed, configured, and integrated with other systems. It’s not difficult, it just takes time. We’ve got pretty much fully automated provisioning of new VPSes, but the hypervisors that run them need hands-on love. We think we can make this a lot better. We’ve been looking at Ceph for shared storage. The key benefit of shared storage for VPS infrastructure is that it decouples the running of VMs (on Compute nodes) from their disks (on Storage nodes). This would…

Read More
Sep 14 2012
1

A crash course in Ceph, a distributed replicated clustered filesystem

By | Technical | 3 Comments

We’ve been looking at Ceph recently, it’s basically a fault-tolerant distributed clustered filesystem. If it works, that’s like a nirvana for shared storage: you have many servers, each one pitches in a few disks, and the there’s a filesystem that sits on top that visible to all servers in the cluster. If a disk fails, that’s okay too. Those are really cool features, but it turns out that Ceph is really more than just that. To borrow a phrase, Ceph is like an onion – it’s got layers. The filesystem on top is nifty, but the coolest bits are below the surface. If Ceph proves to be solid enough for use, we’ll need to train our sysadmins all about Ceph. That means pretty diagrams and explanations, which we thought would…

Read More
Aug 01 2012
0

Application Clustering for High Availability

By | Technical | No Comments

The HA binge continues, today we’re talking about high availability through clustering – providing a service with multiple, independent servers. This differs from the options we’ve discussed so far because it doesn’t involve Corosync and Pacemaker. We’ll still be using the term “clustering”, but it’s now applied high up at the application level. There’s no shared resources within the cluster, and the software on each node is independent of other nodes. A brief description For this article we’re talking exclusively about naive applications that aren’t designed for clustering – they’re unaware of other nodes, and use an independent load-balancer to distribute incoming requests. There are applications with clustering-awareness built in, but they’re targeted at a specific task and aren’t generally applicable, so they’re not worth discussing here. Comparison with highly…

Read More
May 23 2012
0

Hunting down unexpected behaviour in Corosync’s IP address selection

By | Technical | No Comments

Update from 2012-05-24: The Corosync devs have addressed this and a patch is in the pipeline. The effect is roughly as described below, to build the linked list by appending to the tail, and preferring an exact IP address match for bindnetaddr (which was intended all along but got lost along the way). Rejoicing all round! We’ve been looking at some of Corosync’s internals recently, spurred on by one of our new HA (highly-available) clusters spitting the dummy during testing. What we found isn’t a “bug” per se (we’re good at finding those), but a case where the correct behaviour isn’t entirely clear. We thought the findings were worth sharing, and we hope you find them interesting even if you don’t run any clusters yourself. Disclaimer: We’d like to emphasise…

Read More