and the tools we extracted from our own services, and our move towards breaking our monolithic rails applications in to smaller services. So what is a micro service?
single application as a suite of small services, each running in its own process and communicating with lightweight mechanisms, often an HTTP resource API… http://martinfowler.com/articles/microservices.html — Martin Fowler Martin Fowler has an awesome article on some highly detailed descriptions of what is and isn’t a microservice. But essentially it’s a process which handles a singular defined piece of work, which another process sends to it over a communication channel, usually HTTP, upon which it does some work and returns a result.
Application Public API API Client Domain Logic Domain Services HTTP These are the building blocks of a microservice. We have a standalone process which another process can call to process some piece of data and handle the response. In our case this is over HTTP, on an internal network.
won’t focus on the architecture of the service itself, rather the communication components. The wheels, if you will. These are the most hotly contacted components and nearly always get reinvented.
Client Resolver Response Extraction Data Model Domain Logic Request Serialisation HTTP Interface When we take a look at the internals of the API client component, we see some common patterns, something to talk raw HTTP, something to resolve the requests for specific resources, and something to turn an object in to JSON or some other format.
Server Router Response Extraction Data Model Domain Logic Request Serialisation HTTP Interface The internals of an API server are pretty much the same. You might not be handling the extraction yourself, but you’ll more than likely be handling the serialisation.
Interface ActiveModelSerializers, JBuilder, OAT , rabl, to_json ActionPack, Sinatra ActiveResource Faraday, Net::HTTP, Rack So across the client and server we have a mix of common tools which pretty much everyone ties together. If you’re using Rails most of this can be handled out of the box, but that’s not the case with Sinatra, for example. We were using ActiveModelSerializers, but we had to make a lot of internal modifications to support the responses we wanted to produce.
Request Serialisation HTTP Interface Kartograph ResourceKit Kartograph Faraday So across the client and server we have a mix of common tools which pretty much everyone ties together. If you’re using Rails most of this can be handled out of the box, but that’s not the case with Sinatra, for example.
'droplet', scopes: [:read] property :id, scopes: [:read] property :name, scopes: [:read, :create] end end lib/mappings/droplet_mapping.rb https://github.com/digitalocean/droplet_kit/blob/master/lib/droplet_kit/mappings/droplet_mapping.rb Kartograph does this using Mappings. Here’s an example of a really simple mapping, taken from DropletKit, our official API client for the DigitalOcean API.
'droplet', scopes: [:read] property :id, scopes: [:read] property :name, scopes: [:read, :create] end end lib/mappings/droplet_mapping.rb https://github.com/digitalocean/droplet_kit/blob/master/lib/droplet_kit/mappings/droplet_mapping.rb A kartograph mapping is simply a ruby class composed of the Kartograph DSL
'droplet', scopes: [:read] property :id, scopes: [:read] property :name, scopes: [:read, :create] end end lib/mappings/droplet_mapping.rb https://github.com/digitalocean/droplet_kit/blob/master/lib/droplet_kit/mappings/droplet_mapping.rb In this example we’re specifying two representations of our object. One for read and one for create. So you can see we only produce a root element on read, and not on create. We also don’t include the id attribute when serialising for create.
=> {"name": "Sammy"} DropletMapping.representation_for(:read, droplet) # => {"droplet": {"name": "Sammy", "id": 1}} Serialising an object We can call `representation_for` and produce a representation for this object based on the given scope. So we get a root-less object for making requests, and an object with a root for responding to the client. So one of these would be used in the client, and one on the server.
# => Droplet droplets_json = '{"droplets": [{"name": "Sammy", "id": 1}]}' DropletMapping.extract_collection(droplets_json, :read) # => [Droplet] Deserialisation, or Extraction as we call it, works exactly the same. We pass in some json, and receive an object according to our mapping and scope, in this case a Droplet object.
data structures Production Ready™ Powers DigitalOcean API V2, DropletKit, and internal clients for DigitalOcean DNS. { kartograph } digitalocean/kartograph And that’s kartograph. It has no dependencies, supports caching with custom expiration, handles complicated data structures, and it’s used in production.
itself, instead it will talk to any HTTP library you provide which responds to http verbs like get and post. Faraday works well here. You can also give it a test double.
:get # get is assumed if this is omitted path '/droplets/:id' handler(200) { |response| DropletMapping.extract_single(response.body, :read) } end action :all, 'GET /droplets' do handler(200) { |body| DropletMapping.extract_collection(body, :read) } end end end lib/resources/droplet_resource.rb Here’s how we would make a resource which talks to our Droplets API endpoint. We define two actions, all and find.
object) } handler(201) { |response| DropletMapping.extract_single(response.body, :read) } end We also make POST like resources easy to compose by providing a body and giving you the ability to transform it on the fly in to JSON.
|req| req.adapter :net_http end resource = DropletResource.new(connection: conn) My favourite part, using it. So we just defined an API client for talking to the droplets endpoint, let’s use it.
req.adapter :net_http end resource = DropletResource.new(connection: conn) all_droplets = resource.all single_droplet = resource.find(id: 123) create = resource.create(Droplet.new) The we can directly call those actions we defined earlier as methods, and behind the scenes everything is handled for us.
with resource_kit to bring in the helpers RSpec.describe DropletResource, resource_kit: true do it 'handles a 201 with response body' do expect(described_class).to handle_response(:create). with(status: 201, body: ‘{“droplets”:[{…}]}’) do |handled| expect(handled).to all(be_kind_of(Droplet)) end end end https://github.com/digitalocean/resource_kit/tree/master/lib/resource_kit/testing And finally the third step is to test it. We provide some easy to use helpers for declaratively writing specs.
Production Ready™ Powers DropletKit ResourceKit digitalocean/resource_kit ResourceKit only depends on Faraday, I’m not 100% sure it needs to, so if that becomes a problem I’m sure we could remove it. You don’t need to use all of ResourceKit, only what you need.