US POP: Bandwidth Vendor selection process complete

Published December 22nd, 2011 by Keiran Holloway

So on the back of our last blog post on this topic it is with some excitement to announce that we’ve signed up with a total of three network providers to provide our network connectivity in our US Point of Presence.

As mentioned in previous posts, there are essentially two networks that we will be provisioning.

1) Public network which is configured for high-availability and performance in mind. For this link we’ve provisioned two independent network suppliers:

- 100Mbps Fixed-cost link with Level 3
- 100Mbps Fixed-cost link with Hurricane Electric

Across our public network we will be doing all our own BGP routing using the Anchor AS18020 which allows us to completely control how our traffic is routed. Both of these connections are provided as a fibre hand-off which will give us the ability to rapidly increase this all the way up to 1Gbps on each link as necessary.

2) Out-of-bound network which will be used for the remote management of the infrastructure

- 10Mbps fixed cost link with Internap.

This link should provide the reliability necessary for us to continue to comprehensively manage the network from the other side of the world.

conveniently, both of these connections have already been provisioned and deployed which means we’re already operating all our Beta clients on a redundant network with plenty of excess capacity.

0
Comments

Preparing for the holiday season

Published December 13th, 2011 by Keiran Holloway

For online retailers, Christmas is a worrying time for customers, who will want to know when you’re able to ship to them and whether they can speak to a customer service rep if something goes awry.

Nothing will drive them away sooner than hard-to-find contact information. Bring it to the front of your site for the holiday period, and include the hours your customer service team will be available each day.

Be sure you’re clear with your customers about cut off dates for orders to be purchased, shipped, and delivered well in time for the holidays. Make it easier to find by using direct language like “Shipping Deadlines” or “Shipping Cutoff” rather than polite euphemisms like “Shipping Info” and “Shipping Details”.

Freeze your code

The holiday season is not the time to be making big changes to your code base. Consider freezing your code in the next few days while the full development team are still available and monitor the environment closely to ensure likely problems are identified and resolved before the team takes off for a well-earned break.

Triage and simplify

If you’re a retailer, social sites or sports media publisher and you’re expecting a higher than usual workload over the holidays, consider simplifying to reduce load, by temporarily switching off non-essential resource-intensive code, data or content.

For example, adjusting a retailer catalog for a few peak days could mean a smaller product catalogue, which would be served more quickly and might reduce server load. Removing or disabling resource intensive features/functions, can ensure system resources are dedicated to core functionality.

Check and double-check your monitoring

If a server fails in the forest and nobody hears about it, did it ever really fail? Most definitely yes, but if you don’t hear about it, you can’t take action and while it’s down it’s doing harm to your business. Its safe to assume your team will be making fewer manual checks of the system over the holidays, so now is the time to test and re-test your automated monitoring to make sure that it trips for the right events, and that notifications will be delivered to team members who understand they’ll be responsible for responding over the holidays.

It may also be worth asking us to investigate whether the monitoring tools you are using are negatively impacting site performance.


Backup or be sleighed

We wouldn’t let you get by without any backup strategy at all, but now is a good time to ensure that your backup routine will be appropriate for the holidays, that key staff know where and how to begin rollback and recovery, and that you’re confident you can get it done quickly with your available holiday team members. If in doubt, give the Anchor team a call.

Test or hit Santa’s naughty list

Test early and often to make sure there’s enough capacity available while you’re holidaying, and that loads are balanced correctly. 
Make sure predicted traffic levels are reviewed by all your internal stakeholders and confirmed as realistic.


Check everything twice, and check from front to back — connectivity, to firewall, to application and web servers. Check more than twice to catch that Murphy’s Law instance if you can.

Wishing you a safe, happy and 100% uptime this holidays!

0
Comments

Github forks their sysadmins!

Published December 12th, 2011 by Barney Desmond

With a proud tear in the eye but a heart full of excitement, Anchor announces a parting of ways with Github as their hosting management provider.

Since we first got our hands into their systems about two years ago, Github has grown. A lot. For our sysadmins it’s been a process of gradual but continuous change, and it’s an eye opener to step back and take in the sheer scale of it all. We’d like to share that with you, and also talk about what the changes mean at a higher level.

Taking stock

Late in 2009, our project lead Matt Palmer penned a few technical posts about the size of the new architecture. Comprising some 17 physical servers and a dozen or so VMs, this was a huge upgrade for the whole architecture at the time.

Today Github has 48 pieces of hardware, and about twice as many VMs, with more coming online all the time. There’s about five times as much repo data being hosted, too. And that’s after they figured out how to dedupe all those copies of XBMC that people keep forking.

We’d like to think that this is a testament to the solidarity of our design. It’s not flawless by any means, but it’s proven to be very robust, manageable, and scalable enough to keep pace with Github’s growth.

Self sufficiency

So where to now? A recent count says Github has over a million users, about an eight-fold increase, and their team has grown similarly from five to 47 54 55 56! (for the sake of comparison, Anchor has gone from 18 to 30 staff – our sysadmins are denser).

Github now employs three full-time sysadmins in addition to their customer-facing technical support team, which means around-the-clock coverage. Anchor’s management services let customers leverage our size to get a high level of service at an affordable cost, but as a massively technical company working on a large scale, it makes even more sense for Github to bring it all in-house.

This is something that’s been in the pipeline for a while, and it’s been a pleasantly painless process of handing over the reins. Github’s sysadmins now handle the regular stream of diskspace alerts and other blips on the radar, and new VM services that we know nothing about appear on a regular basis (seriously guys, what’s a “cheddar”?), all thanks to the fully-automated build systems that we developed when the project started.

We wish the Github crew good luck and smooth sailing for the future. They’ve been a great bunch to work with and we look forward to more Octocat adventures.

0
Comments

Getting a start in IT: Part 2

Published December 6th, 2011 by matt

(This is the second part in a two-part series on getting your first job in IT. Part 1 of the series covers getting a job in the first place; this article is about the sort of job you can expect when you’re starting out)

Working in IT, like anything in life, is a learning curve. You start out knowing nothing, and you slowly gain experience and knowledge that allows you to work at a higher level, on more difficult (and, hopefully, interesting) problems.

A difficult fact for many people to accept is that there is no substitute for real experience. TAFE (community college), University, and industry certifications are not equivalents. These academic credentials have their value, but they’re not an equivalent for experience. They’re like power mirrors or window tinting on a car — they add value, but power mirrors don’t get you anywhere by themselves — you need four wheels and an engine.

This means that everyone will “do their time” working at the entry-level jobs, one way or another. Whether you have any sort of relevant academic credentials or not, you will spend some time learning the ropes, at a job which you will probably feel isn’t what you signed up for (very little IT bears any resemblance to scenes from “The Social Network”).

Now, you might luck out, and your “entry level” job might be that you’re the entire IT staff for a small company (that’s what I did at while I was at Uni), in which case you’re both the printer monkey and the CTO, but it’s not an easy road to travel. You’ll be working your ring off just trying to learn enough to be able to do the practical basics of your job, and you will learn a huge pile of bad habits as there’s nobody with experience to tell you “no, don’t do that”.

This isn’t to say that every single person who has a job in IT spent 18 months as a gofer, and there’s no other option. I only said that you needed practical, hands-on experience, not necessarily a pay-check. There are a few ways of getting that experience that doesn’t involve a regular job as someone’s cable monkey, but they do require a certain amount of effort on your part. For development work, it’s fairly easy to get some experience by contributing to open source projects. You learn a lot of really essential skills: the most important of which is to write real code. No course I know of is even a half-decent substitute for writing real code. You also get plenty of exposure to the real technologies and processes used in software development, like revision control, testing, collaboration, communication, and compromise.

Sysadmins have a bit of a harder time because there’s not many public systems that you can administer as a hobby, but running your own server on a cheap VM teaches you an awful lot about the basics of systems administration (whether you learn good things or bad things is actually far less important).

These sorts of “hobbyist” things you can do often blur into actual work experience, too — running a server on the Internet for yourself and a few friends might get you a job through word of mouth setting up a static website and blog for a friend’s dad who’s running his own small business. He tells a friend of his who runs a bigger business, which leads to something else, and so on and so on. All of that’s good experience, and helps to demonstrate that you’ve got a bit more to offer than everyone else.

Even then, though, don’t get your hopes up. SAGE‘s Core Job Descriptions (PDF) defines a Junior Sysadmin as someone with one to three years of full-time experience as a systems administrator.

0
Comments

Getting a start in IT: Part 1

Published December 5th, 2011 by matt

Nobody in the IT industry was born with into it. At some point in the (distant?) past, every one of us was in the position of looking for (or falling into) our first job in the industry. I think us old hands forget how daunting that process was, the memory having been dimmed by the passage of time. So I’ve written a couple of posts (Part 2 will be coming out in a few days) to try and demystify the process a little, for people who might be looking to get their start.

(Obligatory pimpage: Anchor is often looking for good people; keep an eye on our jobs page for new positions as they come up)

Competition for good entry-level jobs in IT is always going to be tough. Whether it’s an employee-friendly market (where there are lots of jobs and not many people who want them), or an employer-friendly market (where there are few jobs and plenty of applicants), you will always be competing against other people who also want the job.

This isn’t a resume article — although I’m going to do one one day soon, because life’s too short to spend it filtering crap resumes. Instead, I’m going to talk about what you can actively do to make yourself a more attractive candidate, especially early in your career. Later on, your work history and job performance count for practically everything, but you can’t get a good work history without landing that first job, so let’s talk about that first.

There are two ways to “get ahead” in the IT industry when you’re starting out. By this, I mean get the “better” entry-level jobs, with a bit better pay and conditions and hopefully a better career progression path. The two methods have different target audiences amongst employers, so you can tune your approach somewhat depending on the sort of job you might like to land:

  • Go to University (College, for some). Consistently get astonishingly high marks, academic awards, scholarships, extra-curricular activities (“President of the University Computing Club”), etc. This will take you to the top of the list primarily at large corporate employers. It shows you’re a very hard worker and very smart (you can’t consistently get top marks without some of both), and it shows you are capable of focusing on tasks that you don’t necessarily find interesting.

    Not all of working life is doing the things you love; being able to stick with something that is important but boring is an important trait at any job, but it is particularly valuable in the corporate world, and hence is highly sought after (and well paid).

  • The other way to get noticed is to live la vida nerda: Basically, being a geek. That’s a very strong hiring point here at Anchor, and at many companies at the less corporate end of the industry.

    “Being a geek” means things like learning programming languages (especially non-mainstream ones) in your own time, because you find it interesting. Having a 19″ rack at home filled with cast-off network and server hardware, running a variety of OSes, which you can’t help but compulsively fiddle with because you just heard about a new OS that runs on anything and uses whalesong-over-IP as it’s native networking protocol. It might also mean explaining in an interview that the reason you didn’t get a good mark in some irrelevant subject is because you were working so hard on making your wyse-60 talk to the BeOS installation you had running on the spare Juniper router that you forgot that you were supposed to be doing the final exam (not that that ever happened to me, oh no)

Of course, the two aren’t mutually exclusive, and in many ways they go together to some degree. If you’re a hardcore geek, the chances are most IT subjects will be much easier, and you’ll get good marks easily. You’ll almost certainly still need to work hard at those subjects that don’t interest you, though, if you want to get top marks across the board — which is why employers look for consistently excellent marks, to make sure you’re not just a geek savant.

A side note on industry certifications: at best, all they will do for you is to increase the amount of theoretical trivia you know. While IT is mostly theoretical trivia, and having more of it can make you more effective, it’s the practical application of that trivia that is the job we do, and quite frankly no degree or industry certification teaches you the ability to get real things done in the real world.

This leads me nicely into the second half of this post, which will appear in a few days’ time, which is all about what your first job in IT will actually involve.

So, there’s the skinny. To get a decent corporate IT job, you’ll be best served by going to Uni and studying like crazy. To get a job at Anchor, good marks don’t hurt, but you get more points for being genuinely interested in technology. if you fit into neither of these categories, you will blend into the dozens, if not hundreds, of other applicants for every job vacancy that comes up on the big job boards, and you will have a hard time landing a good job in IT.

0
Comments

Signups open for beta test of Anchor’s US presence

Published November 23rd, 2011 by Barney Desmond

Good news, everyone!

Anchor’s US hosting infrastructure is ready for business. We’re not so brazen/naïve as to think that it’s perfect, but we’re pretty damned confident that it’s ready to go.

This is where you come in. Starting in the first week of December, we want to give you a free VPS for three months and see how you like it. We want you to use it, a lot, and we’re not charging for anything – not for the server, not for the bandwidth, nothing. It’s the legendary Anchor products and service, for free.

The server comes sans-management, so you’ve got root and are welcome to run whatever you like on them. If this sounds like your idea of fun, APPLY ONLINE NOW to let us know you’d like to be part of the trial.

Get in quick if you’re interested – we don’t overprovision these things, so we only have a limited number to give away. Applications are subject to acceptance of our beta agreement, and the offer is good for CentOS or Debian Linux only. Have at it!

Tags: , ,
Posted in FTW, Newsletter

 Leave a comment

0
Comments

Dual-stack IPv6/IPv4 as standard on new US deployments

Published November 22nd, 2011 by Barney Desmond

The focus for this post is obviously about our IPv6 deployment plans, but I’d like to take a small detour through our US presence on the way there.

Anchor’s networking and automation gurus have been hard at work preparing our new kit over in the US, and the day we go live is fast approaching. In the process we’ve had literally zero personal presence over there, not one plane ticket was bought. That we can get away with this is mostly thanks to two things: Equinix and DRACs (Dell’s remote-management interface).

Equinix

One of the reasons we went with Equinix is their high level of support, which goes nicely with Anchor’s approach to business.

The servers were dropshipped direct to the LA3 datacentre where staff unpacked them and racked them up according to our instructions. Once the DRACs were plugged in it was our turn to get excited.

DRACs

DRACs let you do stuff remotely, pretty much anything short of implementing a big red self destruct button. Wanna boot an OS install image from 12,000km away? You can do that.

With a little bit of shuffling, we’ve bootstrapped a new environment over there, ready to build high-availability VMs and servers.

The core of Anchor's LAX1 presence, all gigabit and redundant up the wazoo

IPv6

Which brings us to the fun stuff. The question of offering IPv6 has been on the cards for four or five years now, but to date there hadn’t been enough of a business case for us to commit to. The LAX1 POP changes this.

Anchor’s initial offerings will be focused on our VPS product, which will have full dual-stack connectivity from day 0. For those customers with IPv6 aspirations, we’re ready for you. For everyone else, we hope its enabled-by-default status will drive some interest.

As a small mark of this commitment, we’ve given our DRACs IPv6 addresses only. The practical upshot of this is that it forces us to ensure that all our infrastructure supports IPv6 – routing, switching, DNS, the works. This is popularly called “eating your own dogfood”, we wouldn’t sell something that we don’t use ourselves.

Why IPv6?

For those who don’t follow the IPv4 to IPv6 transition, the rest of this post will summarise Anchor’s perspective specifically.

It’s generally agreed that we’re going to run out of the IPv4 addresses that we know and love, and quite soon. Estimates vary, but unless there’s a revolutionary change in our use of addresses it looks like we’ll hit the wall within about three years.

IPv6 support in the backbone of the internet is already well established, but the real challenge is pushing it all the way to end-users. IPv6 requires big changes through the entire technology stack: routers, switches, firewalls, DNS, servers, operating systems, application configurations and more, it’s all there.

This makes it extremely costly to retrofit existing systems. ISPs have a nasty chicken-and-egg problem because of that – given that demand will only be driven by availability, it’s much easier to do nothing instead of committing to big spending with uncertain returns.

Even assuming that an ISP wanted to sell IPv6 to their consumers, there’s very few content providers doing IPv6. Due to aforementioned lack of demand, content providers would be crazy to present IPv6-only content and lose the whole IPv4 market. Given this, it ultimately looks like the shift to IPv6 will be driven by “external” forces.

As an example, big companies like Google are in a position to push for adoption (try visiting http://ipv6.google.com/), and in theory even offer IPv6-only content that enough users would hassle their ISP for. On the other end of the hypothetical stick, if hosting providers increase the cost (technical and monetary) of IPv4 hosting, it could push enough content to IPv6 to grind through a transition.

These are just two examples, but hopefully they illustrate the difficulty and significance of the problem.

0
Comments

The Internet is trying to kill you

Published November 21st, 2011 by David Basden

Here are my notes from a 5 minute lightning talk that I gave at the Open Source Developers Conference in Canberra last week. It went down pretty well, and it was a lot of fun giving it. It was targeted at web developers, and titled “The Internet is trying to kill you”.


Hi, my name is David. I work for a web hosting company called Anchor Systems as a sysadmin, and like all sysadmins I’ve been avoiding actual sysadmin work as much as possible, and have been writing some web apps.

So people writing web apps, Hey!

The internet hates you.

Now I’m just trying to get across one simple point, and that is that there are people on the internet who want to set your shiny new web app on fire just to see it burn. Maybe they want to do it for the lolz. Maybe they want attention. Maybe they want your data. Maybe they’re just plain bored. They want it all to end in fire.

And you’re next.

You don’t have to be big. You don’t have to be popular. Earlier this afternoon I spun up a clean VM on a free IP address that had never been used before, ever. Nothing even pointing to it in DNS. People were hitting port 80 and 443 in less than a minute. This is a fact of life: This is background noise.

I don’t want you to feel safe and comfortable when you’re writing web applications. I want you to be thinking like a card-carrying member of the tin-foil hat brigade. Because this is the way that you don’t end up on the front page of The Australian’s IT section.

Here are some things that you are hopefully already thinking about:

Forms

The data’s fine. I check it all in javascript!

It’s okay. I pass the metadata in an <input type=”hidden”> where it can’t be seen

Okay, if you’re thinking this, put down the laptop now.
Talk to the person next to you. Read a couple of books.

NEVER. EVER. trust the browser.

Don’t even assume that your HTTP requests are even coming from
web browsers. We learned that the hard way 20 years ago,

DoS attacks

Hahaha. Why do I have to care? Ping floods aren’t my problem

That SELECT you’re doing on your front page that takes half a second?
Try 10,000 connections a second. Try 100,000.

Don’t just think about your front page. Think all your pages. Think
pages that process form submissions. Think stuff that doesn’t cache
well. Start thinking about what happens when someone is trying to bring down your site just by looking at it or posting data to it.

How about SQL injection attacks? Noone’s going to bother, right?

Yes. Yes they are. They’ve automated it. They’re doing that stuff literally in their sleep. Then they’ve packaged it up and given it to a thousand of their friends that don’t even know what SQL is.

Check your data. Check it again. Get into the mindset that anything that comes from the outside world is malicious, and the malicious person is smarter than you are.

Okay, just quickly some other things:

  • Never assume that no-one is sniffing your network
  • Never assume the system you’re running on hasn’t been broken into
  • Never assume that people using your site are rational.In the time you can say “Why would they DO that?!”, they’re already doing it.

Always assume attackers are smarter than you are, have more resources, and much more experience than you do.

Be paranoid. Wear a tinfoil hat. Have fun.

Posted in FTW

 Leave a comment

0
Comments

US POP: Data Centre Facility Selection Process Complete

Published November 14th, 2011 by Keiran Holloway

After our usual negotiation process, it is with great pleasure to announce the Anchor has decided to partner with Equinix for data centre services in North America.

With over 98 data centres world-wide, Equinix are probably one of the most mature data centre vendors on the planet.

Given the extra advantage of having a presence here in Sydney where they’ve recently opened up their new facility – SY3 it was an easy selection as it also gave us direct access to Australian sales saff, based on local time zones.

The specific facility which we will be operating out of is their LA3 facility which is based within 3 miles of LAX, which makes it very accessible in the event that we need to physically attend the site.

Initially we will be commencing with a single rack in the data centre and be fitting this out with our core infrastructure and a number of hosts which we will be able to offer our initial hosting services.

At this point in time it is expected that we will be able to commence our public beta services towards the end of November. For a limited time period, a small number of clients will be able to have services hosts in the US on Anchor managed infrastructure — in return, we ask that you use the bandwidth/hardware extensively and provide feedback on various aspects of the service.

If this sounds like something that you would be interested in, please do not hesitate to email beta@anchor.com.au

1
Comment

The Internet Show – Give Away Congratulations

Published November 9th, 2011 by bsmith

As some of you may have noticed, either from our previous posts or attending the event, Anchor was at The Internet Show . While it wasn’t to the same scale as the expo prior to this, the enthusiasm and calibre of the attendees was fantastic.

While at The Internet Show Anchor presented a talk regarding ‘Why hosting management is important to your online business‘.

For illustrative purposes only, we were dressed better.

It was a fantastic opportunity, for US, to present the facts and benefits face to face with the next crop of online success stories. For everyone who made the effort to sit and watch (and ask questions especially!), thank you. Opportunities to interact with the public in such a candid and personal manner are few and far between. So if you were ever racked by indecision regarding attending a Anchor talk, we not only encourage you to attend, but ask some hard questions while you’re there.

Besides the talk, we unintentionally provided much amusement to attendees via these two trouble makers:

Floating fish that swim, who'd a thunk it?


It was however, with deep regret that Nemo, easily the favourite at the event, decided the ceiling was a far nicer location than the expo floor. At present we are unsure as to his fate, but we have his remote and his left fin to keep us company..

For those of you out there still confused about these things click here.

As a thank you for everyone who stopped by our booth, we also ran a prize draw. This time round, rather than an iPad2, we went for the latest craze. Not only did we update the prize option, we made it an option at the same time!

First prize: iPhone 4S

Shiny.

Second prize: A Harbour Cruise for two including dinner!

First place winner was Adam Stead from Reach Local! The timing was impeccable for Adam, as his trusty iPhone 3GS had been wounded in the course of duty (i.e. the screen was cracked).

The second place, Harbour Cruise, was won by Ben Davey from Mobile Nation! (We were assured that he isn’t prone to sea sickness).

A big thank you to everyone who stopped by and had a chat. We hope to see you all at the next ‘The Internet Show’ event.

1
Comment