Using Hypermedia in Web APIs
Like many companies, Depop makes significant use of Web APIs. Given the ubiquity of them within our company, we’re keen to ensure we get the best out of them.
Now, hypermedia is something we’re all familiar with. You’re looking at it now — this page is hypertext — text with embedded links that can take you to more text. Wonderful.
So why is it that so often our Web APIs ignore this technology? Sure, we might use HTTP to GET, PUT and POST our entities, which gets us out of the swamp of POX, but it still leaves us with problems. Problems we can often solve via the magic of links.
What Problems?
A typical CRUD based Web API acts like a database over HTTP. Instead of SELECT, INSERT and DELETE we have GET, POST and DELETE and the rest is largely data munging.
Say we tell our clients that we have a resource, e.g. an address.
They can GET the addresses for a user like this:
https://www.depop.com/addresses/{user_id}All they have to do is fill out the missing URI parameter.
To add a new address for the user, they can POST there too.
To edit or delete they have to know the specific address ID, then they can PUT, DELETE or GET to:
https://www.depop.com/addresses/42ab-89cf/{address_id}Just fill in the blanks.
But you see, this is a bit like your favourite news site listing headlines, then making you edit the URL in the location bar to get to the story. Who would put up with that?
On top of this:
- The clients have to know the IDs.
- The clients have to know the URI templates.
Passing around IDs is not so great, especially if they’re your actual database IDs. Sharing all those URI templates is even worse — it makes for strong coupling between client and server. Every template known to the client becomes part of your API and you cannot change them on the server side — ever — without breaking old clients.
Which is why we end up with URIs like this:
https://www.depop.com/example/v1/addresses/{user_id}
https://www.depop.com/example/v2/addresses/{user_id}
https://www.depop.com/example/v3/addresses/{user_id}to enable us to evolve our API without breaking old clients. Maintenance nightmare, especially if you change your APIs often!
But there is a better way.
Hypermedia
Before we look at hypermedia solutions, we need to talk about JSON, by which I mean application/json.
Here’s some example JSON:
{
"addresses": [
"https://www.depop.com/example/addresses/e5f1-119a/1",
"https://www.depop.com/example/addresses/e5f1-119a/3",
"https://www.depop.com/example/addresses/e5f1-119a/7"
]
}Just a simple object with a list of links inside it.
Except it’s not.
The JSON specification defines six basic types, none of which are links. The above items are strings.
Now, we could try parsing every string value in a JSON document looking for “http” but it wouldn’t get us far, especially when it comes to relative links.
We need something better.
HTML does have links. When you see:
<a href=”https://www.depop.com/example/addresses/e5f1-119a/1”>
My First Address
</a>You know that’s a link. The semantics of HTML say so.
Sadly, HTML does not make for a client friendly payload in our APIs. However, it turns out that there are lots of flavours of JSON with hyperlink semantics baked in. The one we have chosen to focus on is JSON:API.
Loosening the coupling
The key to loosening the coupling is for the server to provide links in the response body to the client. The client can then follow those links without having to know anything about composing URIs with IDs. Let’s look at an example¹ of a user’s address book:
{
"data": [
{
"id": "e1fc-0f9d",
"attributes": {
"display”: "12 Spalding Road, Harrogate, HG1 4SF"
},
"links" : {
"item": {
"href": "/example/addresses/a933/e1fc-0f9d"
}
}
},
{
"id": "2bc5–00dd",
"attributes": {
"display": “141 Shields Road, Sheffield, S12 9JX"
},
"links" : {
"item": {
"href": "/example/addresses/a933/2bc5–00dd"
}
}
}
],
"links": {
"collection": {
"href": "/example/addresses/a933"
}
}
}There’s plenty of JSON:API structure in there, but the key takeaway is that we have three links, two pointing to the addresses in the user’s address book and one at the collection itself.
Lifting the level of abstraction
Having the server return URLs for the client to follow partially solves the coupling problem — it allows the server to change those URLs at will, without necessarily breaking the client. The client still needs some additional information to be able to understand the semantics of these URLs. This additional information is conveyed in the form of relationships.
A link relation in JSON:API looks something like this:
"item": {
"rel": "item",
"type": "application/vnd.api+json",
"href": "/example/addresses/a933/e1fc-0f9d"
}The important items here are the relationship type: rel: item, and the resource type: type: application/vnd.api+json. The former tells the client what the link points to — in this case the details of an item from a collection; and the latter tells the client how to parse that information, as well as how to request it from the server.
The resource type is included within the Accept header for subsequent client requests for that resource, forming a key part of the content negotiation mechanism.
Profiles
So our clients can now use relationship and resource types to navigate our server responses. The collection of all these semantic guides is called a profile and is the cheat sheet a client needs to be able to understand what the elements of an API and its links mean.
An API should publish a profile to help the client developers understand how to navigate it. There are some standard relationship types that can be used, but you are free to make your own.
Having the client depend on the profile is how we free it from depending on URI templates. Now we can continue to honour the meaning of a link even when changing the link itself.
Adding in JSON Schema documents gives us even more flexibility — but that’s a discussion for another day.
Profiles can also be appended to the Accept header of an API request², giving us a form of versioning using content negotiation. This can be useful if the profile changes, through extension or evolving semantics.
To make use of this, the client must append the profile to the Accept header of its requests:
accept: application/vnd.api+json;
profile=”www.depop.com/profiles/address”Then the server must advertise its content likewise in the response:
content-type: application/vnd.api+json;
profile=”www.depop.com/profiles/address”Here, the client is telling the server that it understands the semantics of this particular profile. Should the profile evolve over time, the server can advertise different versions of the same link adhering to different profiles — or even that a link supports both.
Newer clients can then prefer the updated profile to take advantage of the newer functionality:
accept: application/vnd.api+json;
profile=”www.depop.com/profiles/address_v2”,
application/vnd.api+json;
profile=”www.depop.com/profiles/address”;q=0.9It should be noted when returning a response that may be cached, that the Vary header must include Accept; otherwise a client may be served a cached response for a profile other than the one that was requested.
Conclusion
What we end up with is a state machine over HTTP. This takes us to the pinnacle of the aforementioned Richardson Maturity Model and is what is meant by the term HATEOAS. Now we can truly say we have a REST API!
More importantly though, it frees our clients from all that template filling and frees our server from the strong coupling that made evolving the API so tricky. All that and it simplifies our clients too, allowing them to be more generic.
So, at Depop we’re aiming to adopt this style more often in our client facing Web APIs.
¹ Some required JSON:API attributes have been left out for clarity. See the specification for details.
² If supported by the media type in use. JSON:API does support them.
