How everyone is doing REST wrong

By February 21, 2013 Technical 3 Comments

For the last few months our internal development team has been working on integration between our various internal systems. As part of this work we’ve been developing a series of Web APIs to decouple the functionality from the frontend interfaces.

We have a couple of specific goals in mind. One is enhanced automation: once you have an API, you can write robots to do work for you. The other goal is being able to expose this to customers. Anchor made its name with top-notch support and comprehensive hands-off management, but there’s a growing number of customers who want to do things for themselves. If we give them a way to do that, it makes them happy. And then they can write their own robots to do things for them.

As we went about building our RESTful APIs, we discovered two things:

  1. It’s hard to do RESTful right
  2. Everyone’s doing it wrong – this means you!

The worst offenders are really RPC with some of REST’s HTTP semantics sticky-taped on. A classic example would be http://example.com/restApi?do=deleteCustomer&id=1234. This happens when developers read a couple of blog posts and get that they should map actions onto URLs, but go on to miss the point entirely.

A number of us here enjoy using Trello. It’s a great piece of productivity software, but their API design suffers from exactly this problem. The API is difficult to use because you spend a lot of effort focusing on which URLs to use and how to call them correctly. RESTful APIs should be all about the resources, not URLs.

Good developers are well ahead of this: their RESTful API makes proper use of HTTP verbs to act on resources, and has reams of supporting documentation to describe how to achieve all the desired functionality. This is where most APIs are at.

The problem now is those reams of documentation. The API can only be used if you have all the documentation at hand, so you can find the resource endpoints and know how to call them with the correct keys. Amazon Web Services is a bit like this – make no mistake, it’s a powerful API and the high level of complexity is necessary, but you’re left in the starting gate without that documentation.

It’s all about hyperlinks

The web was built on hyperlinks, and they’re just what an API needs. What we’ve done is rigorously define each type of resource, and then published that metadata as type-descriptors. They’re published just like regular resources, at a predefined entry point of the API.

This is the key, the REST client only needs to know one thing: the entrypoint URL. From there it discovers all the available types of resources, and receives hyperlinks that it can use to manipulate resources, and collections of resources.

Here’s what one of our type-descriptors looks like. The collection_url and item_url_template keys are sufficient for the client to perform basic operations against the API, while the other templates (conformant to RFC 6570) provide type-specific functionality. Human readable documentation is reachable at the provided help_url, which a rich client app could present to the end-user if desired.

{
        "_id": "user",
        "_type": "type-descriptor",
        "collection_url": "https://auth.api.anchor.net.au/users",
        "item_url_template": "https://auth.api.anchor.net.au/user/{_id}",
        "auth_user_url_template": "https://auth.api.anchor.net.au/authenticate{?identity,password}",
        "token_check_url_template": "https://auth.api.anchor.net.au/token_check/{token}",
        "search_url_template": "https://auth.api.anchor.net.au/user_search{?identity}",
        "help_url": "https://auth.api.anchor.net.au/swagger_doc"
}

The benefits of this can’t be overstated. A self-describing API means that:

  • A properly implemented client is agnostic to changes on the server side. One of the first pieces of code we wrote was a commandline client that would save us having to bang out raw HTTP requests by hand. We’ve never had to update the client, because its behaviour is driven by the type descriptors received from the API entrypoints.
  • Maintenance is easier because you aren’t locked into a monolithic API codebase. This makes it possible to expose independent systems through independent APIs.
    • You’re free to choose the language that works best with your backend system. We’re using a mix of Python, Ruby and JRuby wherever they’re best suited.
    • Separate teams can work on different APIs without treading on each other’s toes.
  • Scalability and stability issues are much easier to address. Individual APIs can be scaled up as necessary, and any problems are isolated to their own domain.

It’s easy to know if you’ve done it right: the API consumer never needs to care what the server is doing, everything Just Works.

This isn’t everything you need to make a good RESTful API, but it’ll get you most of the way there, and we look forward to showing it off in the near future.

If you’re interested in some deeper details of RESTful design, we think these pages are worth reading.

3 Comments

  • oliver says:

    You really can’t have a blog post on bad REST patterns without making copious reference to the Richardson Maturity Model[0]. Most of what you describe falls into level 0 – the swamp of POX.

    It would also be a sin not to mention the fantastic reference – REST In Practice[1]. Sadly it largely orients itself around Java and XML services but the concepts and descriptions are solid enough to apply to any language you use. That being said, I’m still a bit skeptical about the self-discoverability of the hyperlink method but perhaps that’s because I haven’t see a great implementation yet.

    Another great reference on how to maintain forward and backward compatibility of RESTful interfaces is Martin Fowler’s Consumer Driven Contracts[2]. It’s also just a good general idea for approaching how to loosely couple interdependent teams.

    [0] http://martinfowler.com/articles/richardsonMaturityModel.html
    [1] http://restinpractice.com/book/
    [2] http://martinfowler.com/articles/consumerDrivenContracts.html

  • Barney Desmond says:

    It’s interesting that you mention “levels”, we were actually discussing something just like that while drafting the post (without having read about RMM). We’ve found roughly the same distinction, but merged level 0 and level 1.

    We won’t claim that our implementation is great (yet!), but I’ve just added an example of the type-descriptor and some explanatory notes. Perhaps it’ll crystallise the utility of hyperlinks for you a bit.

  • oliver says:

    Perhaps I should clarify that. I find the notion of hyperlinks an extremely good way to make a RESTful interface discoverable, and provide a guided process through a workflow without the client having to know all of the steps ahead of time.

    But expressing that and having the client aware of how to find out that process seems a bit clunky to me. I guess you need to have a well-defined endpoint the client knows it can find schema definitions, or some well-defined nodes inside of each response that point the way to finding out what the next step is (and means).

    I see you are attempting this from the very beginning of the workflow with your type descriptors, but I guess my point is that a client doesn’t innately know what a help_url is, even if it knows how to get to it. Perhaps this is just the natural boundary of what the client code must be prepared for, and not something requiring further definition. It’s a interesting topic.

Leave a Reply

Ready to talk business? Send us a note.