The 800lb Gorilla Knows Where Your Website Lives

By February 11, 2009Company News, Technical

If you run a website for commercial purposes, you know that the only way it’s going to provide you with benefit is if people actually visit it. Regardless of what sort of site it is (brochure, company promotion, online store, etc), if nobody’s actually loading the pages, your site may as well not exist. One way or another, you need to drive people to your site.

There are many different ways to get people to visit your site, and different strategies work for different sites. TV advertising, for example, has become popular in the last couple of years, to entice people to just visit the site. Other traditional forms of advertising, as well as online banner or text ads, are also popular. Putting your website name on your stationary and vehicles, and promoting your website to your existing customers works well in certain markets, too.

However, I am fairly confident that regardless of what industry you’re in, search engines make up a fair portion of your incoming traffic. More than that, though, it’s almost certain that one search engine in particular is driving the majority of your search engine-sourced traffic: Google.

For example, the operators of the popular developer question-and-answer forum Stack Overflow recently published some statistics on their sources of traffic:

Currently, 83% of our total traffic is from search engines, or rather, one particular search engine:

Search Engine Visits
Google 3,417,919
Yahoo 9,779
Live 5,638
Search 2,961
AOL 1,274
Ask 1,186
MSN 1,177
Altavista 202
Yandex 191
Seznam 103
[…] Google delivers 350x the traffic to Stack Overflow that the next best so-called “search engine” does. Three hundred and fifty times!

These numbers are almost certainly skewed more heavily towards Google than your average website, because the sort of people who benefit from Stack Overflow (software developers) are also the sort of people most likely to use Google over another search engine, but even if 100 times more people in the general population used a different search engine (and let’s face it, that’s not particularly likely), Google would still account for three-and-a-half times the incoming traffic of the next best search engine.

As a result of this massive traffic skew, if your business relies on search engine traffic, the main search engine you need to be targeting is Google. While there are seemingly endless parades of shonky search engine optimisers who will submit your website to “thousands of search engines”, the simple fact is that if all these thousands of search engines are providing you with the same proportion of traffic as “Seznam” (the 10th search engine on Stack Overflow’s top 10 list), then you’d need to be listed on over 33,000 search engines to match Google’s traffic contribution. Or, you could just make sure Google likes you, instead, for far less effort and expense.

This reliance by the world’s online population on one search engine isn’t necessarily healthy, though. As Whimsley describes in his excellent article, Mr Google’s Guidebook, Google has fundamentally changed the way the Web works, and in many ways it now dictates how websites are designed and marketed. The very fact that we are talking about “making sure Google likes you” and optimising your website for the Google indexer strongly suggests that Google is, in fact, “more a master than a servant”.

Philosophical arguments aside, though, you can’t afford to ignore Google if you want your online business presence to succeed and work for you. What can you do?

First off, I’d like to discuss the use of professional SEO (Search Engine Optimisation). While there are a few firms out there who do a decent job, it is a huge market for lemons. It is incredibly difficult to assess the actual value that an SEO is going to give you, in advance.

As a technical person, I’ve dealt with implementing the recommendations of a lot of dodgy SEO people over the years, and it’s not pretty. A lot of what SEO “experts” recommend are things that Google themselves have specifically debunked, like the virtual hosts vs dedicated IP address myth. Other times it’s doing things that Google specifically warns against, like buying links to boost your pagerank.

In several cases, I’ve seen a client of mine hire the services of a shyster, who has done everything that Google advises against, to provide a short-term benefit. The customer’s site shoots to the top of the Google rankings, the customer is pleased, and pays the SEO a big chunk of money. Some short time (less than a day, in one infamous case) later, the customer’s site disappears from Google’s index entirely. The SEO doesn’t care — they’ve got their money and are onto the next victim — but the customer’s website reputation is in ruins, as Google has detected all of the dodgy work, and has blacklisted the site from their indexes. Cleaning up from this mess can cost you many thousands of dollars directly, as well as lost revenue from people not being able to find you. In many cases, it can easily kill your business completely.

The simple fact is that there’s no real secret to SEO. Google is quite open in many ways about how it ranks pages, and what benefits and harms a site’s rank. It has a whole part of it’s main site dedicated to disseminating information to webmasters about how to do better in the site rankings, and what to avoid doing. You don’t need a professional SEO to tell you these things — there’s nothing secret about it all, and not even anything particularly difficult. However, if you do decide to hire a professional to help, here’s a few tips to make sure you don’t end up doing more harm than good:

  • Avoid anyone who talks about “thousands of search engines”. While Google isn’t the only search engine out there, there isn’t more than a half dozen or so that actually matter on an individual basis. Most of the reputable work that is done to improve your ranking in these mainstream search engines will also automatically help other search engines, too.
  • If you know any other online business owners personally, ask them if they’ve had any SEO work done, and get recommendations. If their search rankings have been consistently improved for three to six months after the SEO has been paid and left, then there’s less likelihood that they’re a fly-by-night shyster, and they may be worth using for your business.
  • Never, ever let an SEO modify your site content directly. Not only might they do deeply disreputable things to your site’s content without giving you any way to easily check what they’ve done, but if their work conflicts with your site designer’s work or processes, it might cost you a lot of money to fix. Having the person who did your site layout and content work review any SEO recommendations can also act as a filter against the worst excesses of a bad SEO.
  • For every recommendation that an SEO makes, ask for a citation regarding the legitimacy of the recommendation. If the SEO can’t show you where on a search engine’s site it recommends doing a certain things, then the chances are it’s a dodgy practice.
  • Do some research of your own on any recommendation you feel might not be above board. Don’t take the SEO’s word for it that it won’t cause you problems down the line.
  • If an SEO says they’ve got “secret” information about how a search engine works, run like hell. Nobody’s better at keeping secrets than Google (they’ve got 10,000 employees, yet nobody outside the company has any idea how many servers they use — is that good secret-keeping, or what?). The chances of a given SEO really having secret information is very slim indeed, and even if they do, the search engines can always change the way they do things to punish your site for gaming the system. It’s just not worth it.
  • If possible, have a trusted technical person (such as your website designer, or your hosting company) review the recommendations of the SEO. While it might cost you an extra couple of hundred dollars to have this checking done, what is the cost to your business if your site was blacklisted by the major search engines for doing dodgy things?
  • Try and get a longer-term contract for an SEO’s services, one that involves periodic payments over a 3-6 month period after the initial optimisation work has been done. This will tend to discourage the shysters, as their business model is one of “do some quick work, boost rankings temporarily, grab the cash, and get out before the whole thing falls apart”. A trustworthy company is far more likely to be happy with a longer-term relationship.

Whether you hire a professional or go it alone, it’s good to educate yourself a little about what sort of things the search engines recommend. Some excellent resources on this subject include:

  • Google Webmaster Central — the start page for anything related to improving your site in Google’s eyes.
  • The Google Webmaster help center — a collection of helpful articles about the how, what, and why of designing a site to be Google-friendly.
  • The Google Webmaster blog — chock full of interesting articles and tips for webmasters.
  • The Google Webmaster Dashboard — a fantastic resource that lets you peer into all the information that Google has about your site, like how many links there from other parts of the web to the various pages on your site, whether the crawler has had problems finding some of your site info, how your sitemaps are helping, the effects of robots.txt changes, and removing pages from the index that you don’t want showing up in search results.
  • The Google Webmaster Forum — where you can ask questions of other website owners (and Google employees), and find out loads of useful information on topics that you’ve probably never even thought about.

Yes, all of those links are Google-specific, because Google makes all this info easily and clearly available, and you get the most bang for your buck by targetting Google. Most good site design ideas will help with other search engines, too, so following Google’s advice will benefit you in general.

2 Comments

  • sydneytint says:

    ok ive been sitting in front of computer till sometimes 1am most nights with my wife just trying to find what we think is a genuine webhosting site to use. We have just bought our domain name and have narrowed it down to around 10 Australian webhosting company’s that are very happy to take our money. we are very confused especially reading your comments on how difficult it is to pick the shyster from the good. We want to design our own website (we think) and constantly will update it with fresh posts etc: not sure how many GB needed per month and don’t know who will get us ranked higher on organic listings, for longer..
    Our business details have been listed on just about every free advertising web pages going, and some of these have put us top three or four on Google. Have been for months…. Cracker.com is one such site which places us highly.
    My question is which webhosting company’s do you recommend, as you seem to be the most knowledgable techy we have come across.
    Thanks sydneytint

  • tiddlelit says:

    @sydneytint, try [http://www.google.com/a google apps] it costs you nothing or less than you imagine. Get the advantages of the services of a reputable webhosting company when you’ve a proof of concept or ask them to help you get there.