Cleaner, safer Ruby API clients with Kleisli

Written in Development by Txus Bach — January 22, 2015

As part of one of our current client projects, I've had to build a JSON API that (among other things) wraps the Geonames API nicely. And I say nicely because the Geonames API is not nice at all. Although quite fast, it is very inconsistent, poorly documented and sometimes even unstable.

It's certainly not the first time that I've written a client for an HTTP API, but this time around has been quite different. From the beginning I set out to address all the classic problems that bug me:

Error handling, namely nicely encapsulating the herd of different failure cases arising from the wrapped API.
Stability, without the pervasive nil checking and high cyclomatic complexity typically seen in such code.
Maintainability, as poor APIs tend to change in the most unforeseeable ways, forcing our client code to change with them.

Let's look at some code then, shall we.

The problem

I chose a very specific slice of the overall problem, one that best illustrates the approach I'm taking.

In our API, we deal with three kinds of resources: Countries, Regions and Cities. As you might have guessed, a country usually contains regions, and those usually contain cities.

We'll look at the specific case of requesting all the cities in a specific region. It'll look something like this:

def cities_in_region(region_id)
  # here goes the code
end

Sounds simple enough, right?

The constraints

As it turns out, there are many idiosyncrasies in the Geonames API aimed at making our life a living hell even in such a simple use case. The way we get cities out of a region is by performing a city search within a specific bounding box that represents the region's boundaries.

Just in this one simple case, these many things could prevent us from getting that much craved list of cities:

The region referenced by region_id may not exist
Even if it does, the Geonames API call to fetch it may return an error
The returned region may or may not have a bounding box associated with it
The Geonames API call to search within a bounding box may return an error
That same call might return a specific message saying that there are no cities within that bounding box

Of course, if any of this goes wrong, we would like to know the specific error and somehow notify our callers.

Off the top of your head you're probably imagining a long entangled method with a cyclomatic complexity of 6. Phew.

A naive approach

Just for one moment let's imagine a naive way of implementing this:

def cities_in_region(region_id)
  begin
    region = Regions.lookup(region_id)
  rescue GeoNames::ApiError => e
    raise “Error looking up region #{region_id}: #{e.message}”
  end

  if region
    if bounding_box = region.bounding_box
	  begin
	    response = GeoNames.search_cities(bounding_box)
		response.fetch(“cities”, []).map { |hash|
		  City.new(hash)
        }
	  rescue GeoNames::ApiError => e
	    raise “Error searching cities in #{bounding_box}: #{e.message}”
	  end
    else
	  raise “Region #{region_id} doesn't have a bounding box”
    end
  else
    raise “Couldn't find region #{region_id}”
  end
end

Jesus, huh? But we can do better.

A better approach

Fortunately, these kind of problems have already been solved more generally, and a way we can benefit from the possible solutions is by adding Kleisli to our project:

gem 'kleisli'

Step 0: Introducing Either

Think of an Either as a little box containing value that might take two different forms: a Right value when everything was right, and a Left error when something went wrong.

A good example is an API call, precisely — when we make such a call we expect either a good response with a useful value or a bad response with an error message.

Let's start by making our GeoNames low-level client use that instead of exceptions.

Step 1: Fixing the low-level client

Taking the GeoNames.search_cities method as an example, we'll make it return an Either value (a Right if everything went right, even if we got no cities, and a Left if anything went wrong, containing the error message).

require 'kleisli' # our new little toolbox!

module GeoNames
  def self.search_cities(bounding_box)
    response_body = low_level_get(
	  resource: “cities”,
	  bounding_box: bounding_box
    )
    error = response_body.fetch(“status”, {})[“message”]
	if cities = response_body[“cities”]
	  Right(cities)
	elsif error =~ /no cities/
	  Right([])
	else
	  Left(response_body[“status”][“message”])
	end
  rescue ApiError => e
    Left(“the GeoNames API errored: #{e.message}”)
  end

  def self.low_level_get(params)
    # the actual HTTP call returning the
    # parsed body of the response
  end
end

Look at the Rights and Lefts — in every possible case, we return a value as useful and concrete as possible. But the true power of Rights and Lefts is that they are both Either instances, sharing a common interface. Let's see some cool things we can do with them:

require 'kleisli'

nice_either = Right(123)
bad_either = Left(“error!!!”)

nice_either.fmap { |num| num.inc }
# => Right(124)
bad_either.fmap { |num| num.inc }
# => Left(“error!!!”)

nice_either.or { |err| raise “Something happened! #{err}” }
# => Right(123)
bad_either.or { |err| raise “Something happened! #{err}” }
# raises “Something happened! error!!!”

And there's more power! We can combine #fmap and #or:

unknown_either.fmap { |num| num.inc }.or { |err| raise “OUCH!” }

We can always extract or unbox the value inside a Right or a Left:

Right(100).value # => 100
Right(100).right # => 100
Right(100).left # => nil
Left(“error”).value # => “error”
Left(“error”).left # => “error”
Left(“error”).right # => nil

And our last trick is >-> (pronounced “bind”), also known as the coolest operator in the Ruby world. This little operator is similar to fmap but it is used when the block that we pass to it may return a Left if whatever if does fails:

def reject_higher_than_five(either_number)
  either_number >-> num {
    if num > 5
	  Right(num)
	else
	  Left(“too high. rejected!”)
	end
  }
end

reject_higher_than_five(Right(2))
# => Right(2)
reject_higher_than_five(Right(8))
# => Left(“too high. rejected!”)
reject_higher_than_five(Left(“error from a previous step”))
# => Left(“error from a previous step”)

A nifty extra: Maybe

Let's think about the fact that our regions may or may not have a bounding box. How would we model this case with an Either? Region#bounding_box would return either a Right(bounding_box) or a Left... what?

We don't care why a region doesn't have a bounding box. It is expected. It just is. That's why using an Either would be a waste. There's a better tool we can use — it's called a Maybe.

A Maybe, just like Either, comes in two flavours: Some and None. It wraps a value that may or may not be there, we don't care why. And its interface will seem very familiar — let's see it in action:

require 'kleisli'

Maybe(123).fmap(&:inc) # => Some(124)
Maybe(nil).fmap(&:inc) # => None()

Maybe(123).or { raise “will never raise” }.value
# => 123
Maybe(nil).or(“default”)
# => “default”

As with Either, >-> (pronounced “bind”) is especially useful for chains of uncertainty, such as reaching deeply inside a Hash:

maybe_c = Maybe(deeply_nested_hash[:a]) >-> a {
  Maybe(a[:b]) >-> b {
    Maybe(b[:c])
  }
}
# => Some(value) or None()

So, as you're probably thinking — Maybe is simpler than Either, and we can use it to indicate a value that may or may not be there, such as the return value of Region#bounding_box:

class Region
  def bounding_box
    Maybe(@bounding_box)
  end
end

Step 2: Update our client code

Now that GeoNames.search_cities returns an Either, we can use this common interface to save a lot of cyclomatic complexity.

After we update Regions.lookup to return an Either as well (not shown here), either with a Right(region) or a Left(error), and assuming our regions have a #bounding_box method that returns a Maybe just like we did in the previous section, our client code becomes much cleaner:

def cities_in_region(region_id)
  Regions.lookup(region_id)
    .or(Left(“Couldn't find region #{region_id}”))
    >-> region { region.bounding_box }
    .or(Left(“Region #{region_id} has no bounding box”))
    >-> bounding_box { GeoNames.search_cities(bounding_box) }
end

Now our method informs us in the most possible concrete way about what went wrong, while being a single expression, readable from top to bottom, and much more maintainable.

Unless we've made a typo somewhere else, we have the guarantee that this method will never return nil or raise an unexpected exception. It'll always return either a Right([city1, city2, …]) or a Left(“what went wrong”).

The more advanced readers might have noticed that we've implicitly interleaved the use of Eithers with a Maybe. This is possible because both share #or and #fmap, and it's very handy!

Conclusion

By using abstraction responsibly, and relying on general, simple but powerful ideas such as Either and Maybe, we've been able to make our code not only much more expressive and concise , but safer as well — in our 5-line version of the method, compared to the almost 20 lines of the naive approach (and their high cyclomatic complexity), there is simply less space for bugs to creep in. And being more declarative , it is easier to read and reason about, provided that you understand Either and Maybe.

On that note -- admittedly, at a first glance it may look unfamiliar to programmers who don't know about these constructs — fortunately, you've seen that they are rather simple and encapsulate ideas that we are very used to dealing with on a daily basis (absence of values, error handling).

If you find this interesting, you can learn more about Either, Maybe and other nifty tools in the Kleisli readme.

View all posts tagged as

Development Ruby