Why Sidecars? Moving the Stack out of the Monolith
Sidecars present engineers with an opportunity to add complex systems functionality to simple microservices while only implementing them once. Learn how.
Poirier Sensu, Inc. Hi friends! My name is Greg Poirier, and I work at Sensu on the open source monitoring platform. I’ve spent the last 15 years building systems for ISPs, *air quotes* “big” data companies, the US government, and SaaS providers. Today, I want to talk about one of my favorite infrastructure design patterns: the sidecar.
2016 Sidecars extend and enhance the main container. In this case, the “main container” is the actual application. But I want to frame our discussion of sidecars a little so that they maybe make a little more sense in the context of a holistic architecture. First, though…
talk that are gonna make some people angrily @ me on Twitter or something after this, and I want to preface their use with a quick and simple explanation of my intentions.
one architecture is better than another. I don’t know your requirements. I don’t know anything about your product, and I’m not religious about architecture or design patterns.
super expensive choose your own adventure book that only ends when you run out of money. So, my suggestion is to make choices that mean you don’t run out of money trying to engineer your way out of a hole you dug yourself.
sprawling organization is now faced with addressing the mountains of technical debt that made you so successful. You: 1) Find more tires (pg. 9,345,160) 2) Reorganize people and code in a way that makes microservices make sense. (pg. 263,400) But maybe you’re reading along in your choose your own adventure book, and you find yourself at this page.
building out your brave new services oriented future. After doing so, you start to have conversations about how to break apart the codebase in a way that makes sense.
you see immediately, “Ah yes! Of course! I see how this easily decomposes into services. Why don’t we reorganize our teams around this very, very obvious decomposition so that all of our development efforts are aligned with a great new vision.” And you get to work right away on some new architecture diagrams.
services: - HTTP for transport - Rate limiting to provide backpressure to noisy internal users - A sensible authentication and authorization layer that allows for security and accountability - Uniform request logging across all services in a structured logging format that helps with request tracing through the new system - A core service discovery mechanism that makes it easy to find the nearest copy of another service - A configuration system to deliver durable configuration data to applications - And a standardized metrics format and storage mechanism Everyone’s really happy about this new world, but then a few teams get together, and start to think, “Hey wait! We could build some of these things once, and then every can benefit from that work!” And now it’s time to make a decision.
like he’s trying to make a decision. In this one, he’s clearly trying to decide between some shared libraries or reusable external processes that add functionality to each of your organization’s deployed applications! Wait. What was that last sentence, that sounds familiar!
organization devotes adequate resources to the building and maintenance of some shared sidecars that help developers rapidly iterate on business logic. At this point, I want to step back, though, because did you notice how I haven’t even said containers yet?
a locally deployed nginx reverse proxy. That is totally a sidecar. Nginx can provide your service with buffering, caching, rate limiting, and load balancing. That’s additional functionality that you didn’t have to implement yourself.
that does something we need it to do, and by deploying it alongside another application, we add functionality to the application without having to build it ourselves.
is going to be about shared libraries. It’s true, shared libraries provide code reuse as well, but there are a couple of key factors here. First, sidecars don’t care what language your service is written in. Second, sidecars are runtime dependencies, not build dependencies. Runtime dependencies can be built, tested, and deployed in isolation. So while you may have to deploy your sidecar to the world for a critical update, you won’t have to deploy the whole world. Solid integration and regression testing for your sidecars allow for a high confidence that they can be deployed safely in isolation of the services that depend on them. And that’s the key to being successful.
sure that your sidecars adhere to a well-specified interface, and you need to test that interface with every build. Even if it’s Nginx, the way that applications interact with that process needs to be tested.
Twitter moment is: “Why does this have to live on the same machine as my service? Why can’t I have a pool of other hosts make this functionality available for every service?” Well, you can go that route, and many companies do so successfully, but I prefer the sidecar approach for its next set of benefits. In order to talk about those benefits, let’s consider these two choices.
our imaginary architecture, configuration could have been a client library provided for applications, but we’ve decided to pull it out of the monolithic stack into a sidecar. If you’ve never used or built a library like this, all it effectively does is call out to an external key-value store like Zookeeper, Consul, or Etcd to get configuration data at runtime. It might even support hot reloading of configuration data by watching those values and restart or reconfiguring the application when they change.
left we have what it would look like if there was a centralized configuration service that applications spoke to, we have the customer and order services talking directly to an external configuration service. Again, this could be the key-value store directly or we could have fronted it by some kind of service providing a simpler interface. On the right, we’ve introduced a sidecar that acts as a smart proxy to our persistent data store backing the configuration service. Their implementations aren’t significantly different—the sidecar would behave pretty similarly to an external configuration service, BUT! You could implement identical interfaces, if you so wanted. There’s a very important, key difference here: the sidecar is local and shares a the physical location of the customer and order services, respectively. So instead of requiring network traversal, the two processes can communicate via any means available to the host locally. In our example, we’re going to have them communicate via the filesystem.
methods that it accepts, put and get, to a path. Your HTTP requests to the fictional HTTP configuration service would look like those with some JSON in the body. Our reimagined configuration service that is built using a sidecar might look a little different.
will provide a CLI that lets you put the contents of a file into a specified key in our key-value store. The CLI is also what’s running in the sidecar, just with different arguments.
is localized to this one artifact that the maintainers of the configuration service ship to your laptop and to production. Regardless of the interfaces exposed, this affords us considerable flexibility when implementing the configuration service, because now we can do fancier things! For example, we can cache a local copy of the configuration. Or we can make the sidecar poll our key-value store and update the local copy on disk whenever the value changes.
different than a client library talking to our configuration service? Well, what if someone working on a new service puts configuration retrieval too early in startup. Let’s pretend for a second that they’re not using whatever framework has been provided for the lingua franca of your organization. They’ve had to write a service in Go, and everything else there is in Java. They whip together a new service that talks to a centralized configuration service, but they put that action really early in startup—before any other dependencies are satisfied. Now let’s pretend that right after they talk to the configuration service, they panic trying to test for their first runtime dependency. The service restarts, causing it to poll the configuration service again. And again. And again. Once a millisecond or so. And now your configuration service is DOS’d. Even with the rate limiting you’ve put in place, it’s taking up sockets, because the process is leaving sockets around in FIN_WAIT on your config service hosts. Well.
sidecar, the service would just be opening and reading a file on disk. The sidecar is happily doing its thing, polling the key-value store occasionally or waiting on events saying the value has changed. And those are our final, two, key benefits of sidecars:
between processes and localize them to a single host. Your application may abuse the local copy of the sidecar only, and requests to the sidecar do not have to traverse the network. And, a local process has many more communication primitives to work with than a remote process. Every process having its own instance of our fictional configuration service means that no single service or process can dominate the pool of shared resources.
talk, I promised some practical examples. So I wandered the earth looking for some super sweet sidecars. Some of these were pointed out to me by friends, and I even wrote one myself in like a quick second as an example of a sidecar with a well-defined, tested interface for a configuration provider.
via Netflix’s rich suite of JVM client libraries over an HTTP interface for an application. This is exactly the kind of approach to sidecar development that I’m advocating for today. They have a well-defined, well-formed software stack that they want to leverage across multiple programming languages. So they just pulled the stack right out and built a sidecar service for applications that can’t use client libraries.
like Google. The next sidecar I want to talk about is this oauth2 proxy. This is really cool, because it’s a such great example of an external process that provides a very specific functionality to your application. In this case, a complete OAuth2 workflow, and I don’t know if you’ve ever added oauth to an application, but this is sort of an annoying thing to do over and over again.
library, but with this sidecar, the experience of implementing the OAuth2 workflow for applications is standardized without having to figure out how to do it for whatever language I’m developing in. I can easily ship a completely uniform OAuth experience across every externally-facing application.
totally sweet sidecar that I love. Have you ever wanted to just throw an nginx proxy in front of your project and have it manage its own SSL certificates with LetsEncrypt. Yeah, well, there you go. Now you can, and it’s totally easy. This does use docker-gen though, which requires access to the Docker socket, so there’s room for improvement there, but this is totally workable immediately for experimentation.
seen the stuff form buoyant.io, I highly encourage you to look at linkerd, linkerd-tcp which was recently announced, and namerd. Linkerd service-to-service routing sidecars that were built by the some of the team members that helped build Finagle at Twitter. This is a powerful construct, because it allows you to centralize all of your service discovery, load balancing, and failure handling in a single location.
take a look at metadataproxy. You can use metadataproxy for a few different things. I’ve used it for local development of applications that I want to talk to the EC2 metadata service. In a container runtime environment where you’re deploying with Docker, you could actually route requests from containers to the metadataproxy with firewall rules and have metadataproxy deliver STS credentials for specific services. If you’re using ECS, this isn’t necessary, because ECS provides per-container IAM roles already, but keep this in mind if you’re building a container platform.
grepory
lycorptech_jp
maroon1st
rainerhahnekamp
thomasvitale
kyohmizu
cybozuinsideout
manarobot
nakamuuu
8maki
toru_kubota
sansantech
taishin
moore
zakiwarfel
kneath
swwweet
3n
tammielis
mza
tammyeverts
aki_iinuma
maltzj
mojombo
paulrobertlloyd
