Zero Downtimes with Faulty Solutions

In this presentation we'll discuss what is a health check, why are they crucial when handling multiple docker containers of a service and how to use them with the new Docker Engine v1.13 using Docker Swarm. We'll use as an example a faulty solution with and without health checks and see the benefits they provide.

Transcript

1. Zero Downtimes with Faulty Solutions Health Checks for Microservices to the Rescue by Dimitris Kapanidis
2. +
3. @spiddy Dimitris Kapanidis • Founder and Senior Consultant at Harbur.io • Organizer of Docker BCN Meetup • Organizer of Kubernetes BCN Meetup • Member of Docker Captains • Member of Google Developer Experts
4. Help modernize enterprise development workflows focusing on containers as first-class citizens 2 years running Docker Containers in Production
5. @spiddy Have you ever had an outage?
9. “Outage is due to a cascaded bug in one of our infrastructure components” Twitter 21 June 2012
10. @spiddy SYSTEMS CAN AND WILL FAIL • Human Error • 3rd-party Service Outage • Distributed Denial of Service (DDoS) • Introduced Bugs • Many Many Other Reasons timeline outage
11. @spiddy DESIGN SYSTEMS FOR RESILIENCY Chaos Monkey randomly terminates VM instances and containers that run inside of your production environment. Exposing engineers to failures more frequently incentivizes them to build resilient services.
12. @spiddy WHAT TO DO WHEN SYSTEM FAILS • Monitor ◦ We need to identify failures ASAP • React ◦ We need to fix the system ASAP ◦ Page Ops during midnight
13. @spiddy A Health Check is a process that monitors the health of the application WHAT IS A HEALTH CHECK - Is web server running? - Is application working? - Is database connected?
14. @spiddy App Health Check Production Monitoring CENTRALIZED MONITORING App B Health Check
15. @spiddy EMBEDDED HEALTH CHECK LOGIC App Production Monitoring /health Health Check App B Health Check /health
16. @spiddy HEALTH CHECKS IN CONTAINERS App Health Check Container on startup every 3 secs ( starting | healthy | unhealthy )
17. @spiddy HEALTH CHECKS WITH SWARM Container Container Container Service (unhealthy) (starting) (starting)
18. @spiddy https://github.com/spiddy/coin Icon by: https://opengameart.org/users/galangpiliang
19. @spiddy spiddy/coin - Each instance on startup it flips a coin (50% change) - On success: - It always respond 200 on a GET request - On failure: - I always respond 500 on a GET request
20. @spiddy
21. @spiddy HEALTHCHECK instruction • HEALTHCHECK [OPTIONS] CMD command Check container health by running a command inside the container --interval=DURATION (default: 30s) --timeout=DURATION (default: 30s) --retries=N (default: 3)
22. @spiddy Thank You!