It's all about microservices!

Live recording: https://www.youtube.com/watch?v=pVFZOC32ON4

These days it's all about microservices: how to build and run them efficiently. Igor is Head of Software Development at Currency Solutions where he's focused on helping the business to choose right technologies. He's been in the industry for 13 years working in telecom, web hosting, digital consultancies and fintech.

Transcript

1. It's all about microservices Engineering challenges of running Docker Swarm Igor Karpovich
2. Disclaimer Neither whales nor gophers were harmed
3. The Goal Decrease time to market, increase engineering speed and and product quality. Increase number of engineers involved, improve cross-team communication. Decrease number of incidents, improve incident resolution time.
4. The Way Disposable products: - Immutable infrastructure - Containers - Microservices
5. The WHY • Experiment fast and safe • Be happy to make mistakes and learn • See anomalies and improvements at scale
6. Version Zero First microservice (2015) • REST API only • Monitoring coverage • Plugged into core app • Traditional deployment • Written in PHP
7. Swarm Out of the box • Service discovery • Networking (mesh routing and custom networks) • Volumes (EBS/EFS in AWS) • Orchestration with Compose • Configs, secrets
8. Swarm Missing bits • Job runner • Cron scheduler • Autoscaling • Access control • Request tracing
9. How do we do the rest?
10. Docker API! import "github.com/docker/docker/client" func main() { client, err := client.NewClientWithOpts(client.FromEnv) stack := DeployStack(client, ymlPath) PollStackTasks(stack) }
11. Cron scheduler func main() { cron := NewScheduler() cron.on("*/10 * * *", func(task CronTask, ctx context.Context){ err := SetDistributedLock(task, ctx) if err != nil { return errors.New("Oops!") } return ExecuteTask(task, ctx) // Emit event to a message bus }) }
12. Autoscaling • Docker for AWS addresses syncing between EC2 ASG and Swarm • Workers autoscaling: calculating reserved resources per node • Services autoscaling: relying operational metrics to scale services via Docker API
13. Access control • IAM-like model • Treat services as users • JWT and SAML • 2FA • Regular token audit • SDK to plug into ecosystem
14. Goblin Microservice framework
15. Some of Goblin's features • Runtime (module) management • Config management • Storage & Cache • Transport (HTTP Server/Client, WS router) • Monitoring client and metrics (Gin, WS router, go runtime) • Logging (context data injection, sentry.io integration) • Healthchecks • Tracing
16. Healthchecks as metrics • Docker native support • Expose metrics • Push to InfluxDB • Visualisation
17. Distributed tracing Trace everything! • HTTP/WS Server • HTTP Client • SQL • NoSQL • MQ • Events • Emails
18. Platform toolkit • Clusters self-deployment (traefik, CI, monitoring, logging, tracing) • GitOps • Secret management • Service wide event broadcast • Change Audit • Multi-route email delivery • Monitoring (InfluxDB, Grafana, Slack, TV dashboards)
19. Questions! Twitter: @ikarpovich