I trust the team at Anchor more than any other systems administrators I’ve ever worked with”
Tom Preston-Werner – Co-founder, GitHub
GitHub is all about collaboration, Social Networking meets Source Code Management. Since bursting onto the scene in 2008, they’ve grown to over two million users and hosting over three and a half million repositories of code.
When their existing hosting infrastructure was found wanting, they came to Anchor for a tailored solution that would keep up with their rapid growth while providing a fantastic experience for their users.
As a Rails app at the core, Github grew up on an agile hosting platform that worked well for them in the early phases of being a startup. It was affordable, run by people who knew Rails apps, and they could get more VMs setup when they wanted to test out new features or improvements.
The downside of this ad-hoc growth was that after about 18 months of being live, the site wasn’t running as well as it should have been. Things would slow down at peak times, and there was an outage of some sort almost every day.
A new architecture
Github came to Anchor because they heard we had the right team for the job, and it didn’t matter that we were an Australian company while they’re based in San Francisco. Github hires talent wherever they can find it, and this was no exception.
A massive overhaul was needed to get them onto a system that had actually been designed for the scale they were operating at, which wasn’t something their existing host could provide. “Make it happen”, they said, so we did.
The problem that Github needed fixed immediately was performance. Almost-daily outages due to lack of grunt and problems with the clustered filesystem were severely damaging Github’s reputation. The hardware we provisioned was a real shot in the arm, providing roughly 5x as much memory and 3x the CPU power to serve up requests. The improvement was widely noticed and applauded by Github’s users.
Github’s other major concern was their rate of growth. At the time they were getting roughly 400 new users every day and 1,000 new repositories. We designed an architecture with scalability in mind, explicitly planning for dozens more servers to come online as needed.
For such a scalable architecture, it has to be modular. We distributed logical functions across separate physical and virtual servers as appropriate, allowing individual tiers to scale independently of others. Another benefit of this is that it makes diagnosis and resolution of problems easier and faster, as they tend to be isolated to a small area.
Last but not least, we made heavy use of automation tools to ensure reliable and consistent configuration of servers. The hands-on time and administrative overhead for a new server build is reduced to a minimum. The consistency enforced by this approach means that additional servers can be brought online as demand grows, without worrying about whether the server has been correctly provisioned.
Where they are today
Anchor hasn’t just provided a new architecture for Github, they’ve also worked closely to support their growth and introduction of new features for their users. Github’s faith appears to be well-placed; Github now sees daily growth figures an order of magnitude larger than when they first came to Anchor, without breaking a sweat.
Since then they’ve grown to dozens of webservers and fileservers, along with a plethora of supporting servers. Late in 2011 Github reached such a size that it made sense to bring their sysadmin talent in-house. The straightforward transition into a system they could manage for themselves on an ongoing basis is a testament to the solidarity of Anchor’s original design.
Today, Github continues to grow strongly, and we expect that their architecture will serve them well into the future.