Part 1 - API

Lions and tigers eh? This is the first installment of a three part series, as I find it easier to split things up into smaller, manageable pieces, I decided to apply it to this blog post as well. The first thing we will be covering is the API.

API

One of our many projects here at MLS is a content API we use internally to support our other other projects. Building a REST API can be a fairly straight forward process, there are many tools available to streamline the workflow, but it can also provide some interesting challenges that, if not prepared for, can cause some major headaches.

Once we had our tools in place, the API itself was pretty straight forward to build. We chose node.js for the language, and restify as the main framework.

var restify = require('restify')
  , server = restify.createServer()

server.get({path: '/', version: '0.0.1'}
, function(req, res, next) {
  return res.send(200)
})

server.listen(8080)

With just a few lines of code, we have the beginnings of an API. Of course there are many other steps involved, but codewise, we have an API with a single route ready to go. So, let’s talk about a few things that may improve development.

Versioning

One of the decisions made early on was to require versioning in our API, which has helped out tremendously in pushing new code forward while still supporting existing routes and functionality. With restify, we have the ability to version right out of the box, all we have to do is require the client always specify which version of the route they want for a requests:

server.use(function(req, res, next) {
  if (!req.headers['x-api-version'] && !req.headers['accept-version']) {
    return next(new restify.BadDigestError('InvalidVersion'))
  }
  return next()
}

We strongly encourage requiring a specific route version from your clients right from the beginning, if only to make it clear which version of a route is being used for both you and the client. Having a route default to the latest version may seem like a good thing at the time, but what happens when you make breaking changes between versions? Did that Friday afternoon deploy incidentally break other apps? This is something that should be considered when designing your API.

It may seem fairly trivial to have a route default to a version, and then document the behavior. However, nobody will read your documentation and you will still have angry users when those breaking changes arrive. The versioning system we use for our API is semver. If the client really wants to automatically be upgraded to the latest API version, they can send us an wildcard version (‘*’) with each request and get that behavior. But that means they have made an explicit choice and we’re ok with that.

Tracking

OK, versioning is great, and now you have multiple versions of a route, which means that at some point, hopefully, the older versions and supporting code can be removed. But how can you tell? We use Datadog to track metrics such as this. Using node-datadog, it is simple to add metrics throughout your code. We added specific metrics for requests to each route for a given version and API key. Now we can track which versions for which routes are being used, and by who.

var datadog = require('node-dogstatsd')
  , statsd = new datadog.StatsD('localhost', 8125)

// Setup the middleware to capture
server.use(function(req, res, next) {
  var tags = []
  tags.push('version:' + req.version(req))
  tags.push('path: ' + req.route.path)
  tags.push('username:' + myCustomGetUsername(req))
  statsd.increment('requests', 1, tags)
  return next()
})

This snippet shows how we collect stats and push them into Datadog. It doesn’t really matter how you get the information, as long as you find it and track it. Logging would also work just fine, the important part here is that we need a way to find out if anyone is using code paths marked for removal.

Marking a route or version as deprecated is great for transparency, but trying to enforce a specific time duration of a route’s continued life won’t always work with your client’s release schedule. How important is it to remove old code when you still have a large percentage of consumers using it? Getting the information about which routes and versions of your API are actively used will allow you to make these decisions more easily.

Here is an example graph of what Datadog can do:

The requests are from a development environment so the stats are a bit wonky. Datadog provides many ways to configure the graphs and tags sent by the app, use what works best for you.

Send Extras

While it is important to know how your API is getting used, it is also important to let your users know what you are doing with it. At MLS, as we push our API forward, we ensure that version numbers are consistent across all routes. Sometimes this means simply adding an accepted version to a route:

server.get({path: '/', versions: ['0.0.1', '0.0.2']}
, function(req, res, next) {
  return res.send(200)
})

Great, that was easy, but it would be useful for our users to know if they can upgrade to the new version. Let’s add some code and give our users more information:

server.use(function(req, res, next) {
  var v = req.route ? req.route.versions || [req.route.version] : null
  if (v && v.length) {
    res.header('x-api-versions-available', v.join(', ')) // 0.0.1, 0.0.2
    res.header('x-api-version', v[v.length - 1])         // 0.0.2
  }
  return next()
})

We also might want let the end user know that a route may no longer be available in the future, so let’s add a made up ‘x-api-deprecated’ header:

function deprecate(req, res, next) {
  res.header('x-api-deprecated', 'true');
  return next()
}

server.get({path: '/', version: '0.0.1'}
, deprecate
, function(req, res, next) {
  return res.send(200)
})

server.get({path: '/', version: '0.0.2'}
, function(req, res, next) {
  return res.send(200, 'Hello there!')
})

Self-documenting APIs are helpful, but they do not replace traditional docs. However, adding this kind of information to your API responses opens the door for your clients to add more intelligence in their own apps, like tracking which API calls can be easily upgraded without having to look back and forth between code and documentation, or accidentally skipping over a 1.1.0 and 1.0.1 version change.

So far, we’ve briefly touched on the subject of APIs, in the next segment, we will discuss API specification, more specifically, route specification, and take another step forward towards self-documentation.

Part 2 - Specifications

New MLS Mobile App for 2015

January 12, 2015

Open beta for new MLSsoccer.com

December 04, 2014 Hans Gutknecht

Standings Visualizations

October 30, 2014 Tom Youds