Building Composable Services Noah Kantrowitz

What? Why? How? Perils? Press Start This talk is going to cover four aspects of composable services. What they are, why to build them, how to build them, and what common pitfalls to avoid.

What? World 1-1

What is a Function? Let's start at the beginning.

f(x, y) = z A function, in the mathematical sense is some operation that takes inputs and produces an output.

What is Idempotence? So then what is an idempotent function?

f(x) = f(f(x)) Idempotence means that a function's output doesn't change when you run it twice. This is important when talking about operations that might be run 1-or-more times. If you click "log in" twice, you should get the same result as if you clicked it once, this is idempotence.

What is Composable? Then what does composable mean?

f(x) g(x) f(g(x)) Composability is a property of functions, where you can use the output of one function as an input to another.

req_user(x) user_id(x) user_id(req_user(x)) This is already a common pattern in many applications, and maps nicely to an object oriented style.

post('http://login') get('http://search') The fun part is applying this idea across a network. Rather than having one big application with code modules for each task, build lots of little services that talk to each other.

Why? World 1-2

Availability The single biggest reason to use this style is the fault tolerance. A failure can bring down some of the services, but the rest will continue to operate as best they can. If your search service goes down, the search box will be disabled, but users should still be able to log in.

Scaling You can also scale up services more easily. If your search service gets overloaded, simply launch more of them behind a load balancer.

Testing The difficulty of testing a service goes up exponentially as the service gets bigger and has more interacting features. Small, isolated services lead to easier testing and thus often better test coverage.

Logistics As long as APIs are agreed upon between services, the deployment and operations of one need not impact the others. In many organizations, these are handled by different teams so this leads to a natural separation of concerns.

How? World 1-3

Frameworks Storage Rich Data Discovery Resilience Containers Level Up Select a Skill There are a lot of little things that contribute to successful microservices.

µ-frameworks While anything can be used to build a small, self-contained service, some tools are easier than others. I will focus on HTTP and REST-ish tools for most of this talk as they are the most common.

Flask (Python) Sinatra (Ruby) Express (JavaScript) The three most popular frameworks in their respective languages are Flask, Sinatra, and Express. These all share a simple API, basic URL routing, and minimal integration with things like an ORM or HTML rendering library. As most of our services will be making HTTP queries to other services and rendering results as JSON, this saves on unnecessary complexity.

ZeroMQ nanomsg ProtoBufs Cap'nProto While HTTP and JSON are the most common formats used, you should know about a few of the alternatives. ZeroMQ and nanomsg provide a more compact wire protocol than HTTP, and Protocol Buffers and Cap'n Proto provide a more compact message serialization than JSON.

Data Storage (aka state) I mentioned most of your services will consume data from other services, but eventually some information does need to be stored somewhere. Just as we build models to wrap the database to control database access, in a composable world we make model services. This helps keep the surface area between the services and the databases to minimum.

AP Database While the specifics of different databases are beyond the scope of this talk, the decentralized nature of this style does mesh very well with AP databases like Riak and Cassandra.

Cache is the enemy The conventional wisdom in many web development circles is to cache early and cache often. I am here to tell you down this path lies madness. Each of those caches is really a new database to worry about, and all the earlier issues with service/storage interactions apply again. If something must be cached, perhaps due to being very slow to compute but too big to store ahead of time, just like with the database there should be a model service that wraps the cache and hides it from the rest.

Rich Data {id: me, cart: http...} Rather than passing around very large data structures, you can break it into more manageable chunks and include links to them. Having the links included in the data rather than simply included tokens or opaque identifiers helps keep all the logic for accessing that bit of data in the service that manages it.

Hypermedia APIs And as a natural extension of rich data, hypermedia APIs provide some structures for common problems, like related objects and pagination.

Service Discovery Now you have two services that want to communicate. How do they know where to find each other? Service discovery provides a way for cooperating services to locate each other.

Self-Organization One of the most important properties in any distributed system is self-organization, the ability of the system to shape itself to some extent. This allows the system to effectively route around minor failures, like a load balancer removing servers that are failing a health check.

DNS nslookup('login') DNS is one of the earliest forms of service discovery. In modern systems this can take one of two approaches, either use multiple records and round-robin on the client side or have the name map to a load balancer like HAProxy and have services register themselves with it. For the former approach, often this means that new services must be registered manually by an admin, and the latter means you need some other system to register with the load balancer. In cloud platforms that offer DNS or load balancer APIs, this can still be quite powerful.

ZooKeeper CP Database If you want more fine-grained control over service registration and discovery, ZooKeeper is the most widely used tool for cluster management. It also enables higher-level operations like leader election and ephemeral registration.

Etcd Serf Consul Archaius Many services have grown in the shadow of ZooKeeper. As with databases, the specifics are beyond this talk, but be sure to check out all the options as each makes different tradeoffs and offers different APIs.

Resilience Building microservices doesn't automatically make them fault tolerant, but it does make it a lot easier than with a single, monolithic application.

Timeouts Idempotent Retries The two most important things in making a resilient services are careful control over network timeouts and ensuring that operations are idempotent. This allows you to detect failure quickly, and then simply repeat the failed operation as needed. Some operations can be naturally idempotent, like deleting a record.

post('chpw', nonce: 314) Others need explicit idempotence checks, such as checking update times or nonces to avoid race conditions.

Any service can be down Above all else, always have a strategy for dealing with any service being down. If it was a critical dependency of your service then perhaps you just send back an error message, but degrade gracefully where possible. As before, better to have the search box disabled than the whole site be down. This is the very essence of composable systems.

Async Messaging Queues When possible, use asynchronous messages instead of direct calls. This allows the queue to serve as a buffer between producer and consumer during failures and keeps latency down on the response to the user.

AMQP Kafka Two quick tool recommendations, AMQP and RabbitMQ in particular are the gold standard in queueing systems. Kafka is newer but has an impressive feature set and user base.

Containers On the operational side, microservices have some unique requirements.

Less RAM Less Problems Compared to traditional applications, these tend to use far fewer resources.

LXC Jails Zones Docker/Mesos This pairs very nicely with low-overhead virtualization systems like Linux's LXC, FreeBSD's Jails, and Solaris' Zones. Docker and Mesos both offer higher-level interfaces to these technologies, though both are complex topics to say the least.

Security Isolation Using these containerization systems allows putting very hard boundaries between each service, which helps with a defense-in-depth strategy. A single vulnerable service is less likely to cascade to others.

Immutable Deployment While not a requirement for it, containers also play nicely with immutable deployment. This is the idea that once launched, a container is a read-only object. In turn, this allows for powerful techniques like rolling deploys and rapid-response auto-scaling.

Logical Boundaries As your graph of services gets bigger, it is common to find clusters of logically-related services that share an internal support service. Just as public/private functions work in a monolithic application, you can use subnet boundaries and other network-level controls to enforce system boundaries with microservices.

Perils? World 1-4

Cascade Failures One common issue is cascade failures and overloads. This is especially prevalent when caching slow operations, after a deploy the cache will get flooded with requests and may not be able to handle them all. To address this you can build back-pressure in to the system. If a service can't handle incoming requests it can unregister from service discovery or signal things calling it to wait before trying again.

Poor Visibility The move from one application to many can impact visibility in to the status of the system and hinder debugging.

Health & Metrics Central Logging Dashboards While important in any infrastructure, having solid monitoring and log aggregation is absolutely critical with microservices. Specialized reporters and dashboards like Sentry and Graphite can provide important overviews of system health.

Complex Deployments Similarly deploying many microservices requires a lot more coordination than a single codebase. Orchestration tools like Fabric and RunDeck can help with this, though training and documentation are still important, especially for cross-team services.

Minimalist Self-organizing Fault-tolerant Just keep these three goals in mind and you'll be well on your way to building better services and better APIs.

Thank You

Noah Kantrowitz @kantrn coderanger.net Questions?