Microservices and Docker at Scale: the PB&J of Modern Application Delivery

Presented at DevOps Enterprise Summit 2016 (DOES16), November 7-9, San Francisco.

Docker has matured, and is expanding from being predominantly used in the Build/Test stages to Production deployments. Similarly, microservices are expanding from being used mostly for green-field web services to being used in the enterprise - as organizations explore ways to decompose their monolith to support faster release cycles.

Running microservices-based applications in a containerized environment makes a lot of sense – both for Build and Test, as well as from a runtime perspective in Production.

Docker and Microservices are natural companions, forming the foundation for modern application delivery. However, managing microservices and large-scale Docker deployments pose unique challenges for enterprise IT.

This talk discusses the modern requirements for building, deploying and operating microservices on a large-scale Docker-ized infrastructure. It covers:

• Patterns for microservices-based architectures
• What makes Docker such a good fit for microservices
• How to operationalize Docker orchestration, auto-scaling and DR in large-scale Production environments – recommended typologies and patterns
• Best practices for Docker configuration and registry management to ensure consistency throughout the pipeline and stability in Production
• Tips for integrating containers into complex existing environments, supporting the hybrid state of microservices and monolithic applications
• Tips and tooling for monitoring and managing clusters of containers as a single distributed application
• Putting an end to Shadow-IT: how can IT enable Dev and Ops to use Docker for both microservices and traditional application releases - while ensuring visibility, security, governance and compliance

Transcript

1. electric-cloud.com #DOES16 Microservices and Docker at Scale Anders Wallgren | CTO, Electric Cloud @anders_wallgren The PB&J of Modern Application Delivery
2. electric-cloud.com #DOES16 What Am I Talking About? • Microservices: what’s cool about them • Containers: what’s cool about them • Why are they often mentioned in the same sentence? • Challenges to good outcomes • Challenges at enterprise scale • Best practices to improve outcomes
3. electric-cloud.com #DOES16 Things I’m Not Trying To Say/Imply/Spend Time On • Microservices should be used by everyone, for everything • If it’s not containerize-able, it’s crap • How to solve all the tricky microservices problems in just one line of Haskell • “My Kubernetes cluster beat up your Docker DataCenter and took its lunch money and gave it to Mesos”
4. electric-cloud.com #DOES16 Why Microservices? • Divide and conquer complex distributed applications • Loose coupling; each service can: • Use the tooling that’s appropriate for the problem • Scale independent of other services • Have its own lifecycle independent of other services • Easier to scale teams and scale applications • Smaller more autonomous teams are more productive • Better resource utilization
5. electric-cloud.com #DOES16 Why Containers? • Containers provide a well-defined, isolated runtime environment • *-as-code: Dockerfile, docker images, repository • Isolated: toolchain & dependencies are in the container • Can be immutable • Environment consistency: ship the environment instead of “artifact + giant readme” • Move the container through the pipeline instead of the artifact • Per-container overhead << per-VM overhead • Scale up & down much easier/faster – seconds instead of minutes • Better utilization • Container hosting services provide cool/necessary features • Auto scaling, monitoring, resiliance • Some orchestration
6. electric-cloud.com #DOES16 Why Microservices in Containers? • 2002: One service per metal box • “I remember my first dual-core box, too!” • “Why is that 32-core server idle all the time? Can I have it?” • 2007: Hypervisor + 1 VM + Multiple services in that VM • “Yeah, can’t run ServiceA and ServiceB side by side, conflicting versions of…” • “Yeah, we did that until ServiceC filled up /tmp and took down ServiceD" • “Yeah, we tend to run ServiceE by itself once we’re past QA” • 2012: Hypervisor + Multiple VMs + 1 Service in each VM • “Yeah, each VM OS has a copy of that in memory, so…” • 2013: Containers: run multiple services in isolation without the OS overhead
7. electric-cloud.com #DOES16 The PB&J Part • Microservice: one process that solves a single problem • Container: a process and its associated dependencies with minimum overhead • Decent overlap between common challenges and best practices
8. electric-cloud.com #DOES16 Challenges
9. electric-cloud.com #DOES16 Challenges Common to Both • Complex tools for solving complex problems • It isn’t necessarily easier to do it this way… • Orchestration of the software pipeline – more moving parts • Increased testing complexity • Increased deployment complexity • Monitoring, logging, remediation are critical and more difficult • Diverse toolchains, architectures, and environments • How do you deploy updates to an application that has microservices running in containers as well as legacy components?
10. electric-cloud.com #DOES16 Microservices Can Be Challenging • Distributed Systems Are Hard • Service composition is tricky to get right, can be expensive to change • Inter-process failure modes have to be accounted for • Abstractions look good on paper but beware of bottlenecks • State management – transactions, caching, and other fun things • Team-per-service or Silo-per-service? + Conway’s Law • Legacy apps: Rewrite? Ignore? Hybrid? • Good system comprehension is key • Your service might be small, but how large is its deployment footprint?
11. electric-cloud.com #DOES16 Microservice Challenges (continued) • “Every outage is like a murder mystery!” • “But my service relies on version X of ServiceA and now I’m down” • “Wait, how many processes do I need to start for this test?” • “So then requestId 0xf00dfee8 in the log on ms-app-642-prod becomes messageId 1125f34c-e34e-11e2-a70f-5c260a4fa0c9 on ms-route-669-prod?” • Network issues – bandwidth, latency, reliability
12. electric-cloud.com #DOES16 Containers Can Be Challenging • State/volume/data management • “Wait, I can’t even write a log file inside the container file system?” • “The auditors asked for logs, but we removed those containers” • Security • What do you mean it runs as root? • Containers are black boxes to opsec: less control, less visibility inside the container, existing tools may not be container-savvy • How do you secure every random image that comes out of dev? • Monitoring: what does it mean when containers come and go all the time? • What hosting service should I use? • Ecosystem is moving very fast • Sprawl: it’s not just for VMs anymore
13. electric-cloud.com #DOES16 Challenges at Enterprise Scale • Technology maturity • Forrester 2015 Container Challenges • Security, Variable performance, Integration with existing tools & processes • Large Fortune 20 Bank: 6,000 applications • Motivated by container efficiencies, ease of scaling, ease of replicating • Maybe only 20% of apps are appropriate for containerization • Not rewriting everything, but new development follows container-friendly approaches • “At least one of everything” can complicate life • Governance/audit can get more complicated • Bi-modal/multi-modal software delivery: monoliths, micro-services, and everything in between, for the foreseeable future
14. electric-cloud.com #DOES16 Best Practices
15. electric-cloud.com #DOES16 Common Best Practices • Your Automated Software Pipeline Is Your Friend™ • Ideally, one platform handles all your software delivery • How’s your test coverage? • Are your tests automated? Really automated? • Self-service automation/ChatOps approaches • Reduce onboarding time, waiting, complexity • Service discovery • Consistent logging & monitoring output across services/containers • Does your monitoring/perf tool help you track a request through the system even as it bounces between different services?
16. electric-cloud.com #DOES16 Microservices Best Practices • Develop domain knowledge • Avoid premature decomposition • If starting from scratch, keep it modular and split things up as your understanding of the problem domain evolves • Be rigorous in handling failures (consider using, e.g. Hystrix to bake in better resiliency) • Be flexible about staffing and organization
17. electric-cloud.com #DOES16 Microservices Best Practices (continued) • Pay attention to transaction boundaries and foreign-key relationship as they’ll make it harder to decompose • Do you need ACID for that? • Consider event-based techniques to decrease coupling further • “Be conservative in what you do, be liberal in what you accept from others” • Independent delivery pipeline for each service • Test independent of other services • Static analysis, security validation, perf testing • Integration pipelines further downstream
18. electric-cloud.com #DOES16 Container Best Practices • Use an enterprise container registry • Container-native monitoring • State: what kinds and where and how to persist it? • Business data, reference data, compliance data, logs, diagnostics • There are open source and commercial offerings to help • Security • Sign & scan images, validate libraries, etc. • Harden the container environment as well • Drop privileges early, use fine-grained access control so it’s not all root • Be smart about credentials (container services can help) • Your Automated Software Pipeline Is Your Friend™
19. electric-cloud.com #DOES16 Container Best Practices (continued) • What’s running where and why? • Understand platform dependencies that might limit you • Avoid image bloat • Immutable images? • Do you really need an SSH daemon in that image?
20. electric-cloud.com #DOES16 https://neo4j.com/blog/managing-microservices-neo4j/
21. electric-cloud.com #DOES16 Thank you! Questions?