Container based application Design Real Practices

talked at #dockertokyo #dockerbday

1. © - BASE, Inc. Container-based Application Design Real Practices . . #dockertokyo - @hgsgtk
2. © - BASE, Inc. Talk Summary Keep each components is Decoupled Keep Balance with development efficiency Use Cloud-based Design Principles
3. © - BASE, Inc. About me Kazuki Higashiguchi Twitter / GitHub : @hgsgtk Server-side Engineer BASE BANK, Inc.
4. © - BASE, Inc. Agenda .Configuration .Logging .Building Application .Monitoring
5. Configuration
6. © - BASE, Inc. Features of Configuration • “Runtime” Data (Necessary at startup) • Different for each environment • A lot of values
7. © - BASE, Inc. Requests for configuration • Manage hierarchically • Make configurations easy to understand • Externalize configuration • Get rid of configurations from Application • Minimal accessibility • Configurations includes credentials
8. © - BASE, Inc. Related Principles • “ . CONFIGURATION, CREDENTIALS, AND CODE” in “Beyond the Twelve-Factor App” • “IMAGE IMMUTABILITY PRINCIPLE(IIP)” in “Principles of container-based application design (redhat)”
9. © - BASE, Inc. . CONFIGURATION, CREDENTIALS, AND CODE (WIP) • Get rid of configurations from Code • Not include in Version Control System • “Treat Your Apps Like Open Source” • Environment variables are considered the best practice for externalized configuration, • Externalized configurations
10. © - BASE, Inc. Extra: Beyond the Twelve-Factor App • Best practices of cloud-native application • published by Pivotal • https://content.pivotal.io/blog/ beyond-the-twelve-factor-app • Original is “The Twelve-Factor App” • https:// factor.net/ • Original + New = factors
11. © - BASE, Inc. Extra: Principles of container-based application design (redhat) • Design Principles of container- based application • published by red-hot • https://www.redhat.com/en/ resources/cloud-native- container-design-whitepaper
12. © - BASE, Inc. Configuration: Practical example • AWS ECS(Fargate) • Configurations are stored in Parameter Store • Encryption using Key Management Store • See also: • https://devblog.thebase.in/ entry/ / / /
13. © - BASE, Inc. Configuration: Real Issues • Depends on Features provided by Cloud vendor • If well-integrated features, become “Simple” • If Not, may become “Complicated”
14. © - BASE, Inc. For example: Fargate & Parameter Store • A example to use Parameter Store as externalized configuration store • Before Platform Version . • Each container access Parameter Store via API • Write entrypoint.sh by myself
15. © - BASE, Inc. entrypoint.sh #!/usr/bin/env bash set -e # request multiple times export DB_HOST=$(aws ssm get-parameters --name /prd/ database/host --query "Parameters[0].Value" --region ap- northeast-1 --output text) export DB_PORT=$(aws ssm get-parameters --name /prd/ database/port --query "Parameters[0].Value" --region ap- northeast-1 --output text) exec "$@"
16. © - BASE, Inc. For example: Fargate & Parameter Store • After Released . • “AWS Fargate Platform Version . Adds Secrets Support” • Just write task definition (and set IAM role) • Become well-integrated features. • It makes our application “Simpler”
17. Logging
18. © - BASE, Inc. Requests for Logging • Realtime • Know “What is happening now” • Availability • “No” Missing Log • Searching • Make identifying the cause of problem “Easy”
19. © - BASE, Inc. Related Principles • “ . LOGS” in “Beyond the Twelve-Factor App”
20. © - BASE, Inc. “ . LOGS” • Treat Log as Event Stream • Not depends on File System • Log on STDOUT/STDERR • Use tools (ex. ElasticSearch, Logstash, Kibana) to satisfied the requirements of aggregation, processing, storage of logs
21. © - BASE, Inc. LOGGING: Practical Examples • Application writes logs to STDOUT/STDERR • Fargate -> CloudWatchLogs(CWL) -> S -> ElasticSearch & Kibana • See also https://qiita.com/tomy rider/items/ aa dd
22. Build Application
23. © - BASE, Inc. Requests for Build Application • Dependency Management • Manage Application Dependency • Work in Difference environments • Work in Local, QA, Stating, Production
24. © - BASE, Inc. Application Components • Runtime Engine • Code • Dependencies • Configuration
25. © - BASE, Inc. Container Application • Container includes • Runtime Engine • Code • Dependencies • Configuration is injected at startup
26. © - BASE, Inc. For environments, Same Container
27. © - BASE, Inc. Related Principles • “ . DEPENDENCY MANAGEMENT” in “Beyond the Twelve-Factor App” • “ . DESIGN, BUILD, RELEASE, RUN” in “Beyond the Twelve-Factor App”
28. © - BASE, Inc. Build Application: Practical example • Use case • API developed by Go • One Dockerfile for deploy • Build image provide to environments
29. © - BASE, Inc. Build Application: Real Issues • Best practices: same container provides ALL environments • However, in Local environments, Development efficiency is also important
30. © - BASE, Inc. Build Application: Real Issues • One Dockerfile for deploy • Another Dockerfile for developer • Live reloading • See also • https://go- talks.appspot.com/ github.com/hgsgtk/ gocon -lt/realize.slide#
31. © - BASE, Inc. Find the suitable balance for you • “Docker for local” • -> weakens the merit • -> strengthen developers • Necessity to find suitable balance
32. Monitoring
33. © - BASE, Inc. Requests for Monitoring • Check Health • Monitor application is healthy • Keep Watching Metrics • Keep watching Metrics
34. © - BASE, Inc. Related Principles • “HIGH OBSERVABILITY PRINCIPLE (HOP)” in “Principles of container-based application design (redhat)”
35. © - BASE, Inc. HIGH OBSERVABILITY PRINCIPLE (HOP) • Treat Container like “Black box” • Provide API for various status check • Activity status • Preparation status • Log important events on STDOUT/STDERR
36. © - BASE, Inc. Health endpoint pattern • One of application monitoring patterns • Provide HTTP endpoints in the application • convey the health of the application • See also: “Practical Monitoring—Effective Strategies for the Real World” • http://shop.oreilly.com/product/ .do
37. © - BASE, Inc. Mackerel Container Agent • Monitor “Inside Container” • CPU, Memory, Network etc • Run as a task / pod sidecar • See also: https://mackerel.io/ja/docs/entry/ howto/container-agent
38. © - BASE, Inc. Monitoring: Practical example /.health_check endpoint return “OK” See also: https://devblog.thebase.in/entry/ / / / -> % curl -i http://localhost/.health_check HTTP/1.1 200 OK Content-Type: application/json; charset=utf-8 Date: Mon, 04 Mar 2019 13:24:23 GMT Content-Length: 15 {"status":200}
39. © - BASE, Inc. Monitoring: Practical example /.health_check endpoint also return “NG” See also: https://devblog.thebase.in/entry/ / / / -> % curl -i http://localhost/.health_check HTTP/1.1 503 Service Unavailable Content-Type: application/json; charset=utf-8 Date: Mon, 04 Mar 2019 13:35:38 GMT Content-Length: 106 {"status":503,"message":"failed to get connection database","string":"sql: connection is already closed"}
40. © - BASE, Inc. Talk Summary Keep each components is Decoupled Keep Balance with development efficiency Use Cloud-based Design Principles
41. © - BASE, Inc. Extra: Related Presentations • “Container-based Application Design Reference and Practice” at Docker Tokyo Meetup • https://speakerdeck.com/hgsgtk/container-based-application-design- reference-and-practice-number-dockertokyo • “Docker Go development environment starting with realize” at Go Conference Autumn • https://go-talks.appspot.com/github.com/hgsgtk/gocon -lt/ realize.slide#
42. © - BASE, Inc. Extra: Related Blogs • “ECS(Fargate)でコンテナアプリケーションを動かすための設定情報の扱 い⽅” • https://devblog.thebase.in/entry/ / / / • “アプリケーション監視のパターン「Health エンドポイントパターン」 を実践する | 書籍『⼊⾨ 監視 ―モダンなモニタリングのためのデザイン パターン』を読んで” • https://devblog.thebase.in/entry/ / / /
43. © - BASE, Inc. Extra: Related Books • 『⼊⾨ 監視 モダンなモニタリングのためのデザインパターン』 https://www.oreilly.co.jp/books/ /
44. © - BASE, Inc. Extra: Related Documents • Beyond the Twelve-Factor App • https://content.pivotal.io/blog/beyond-the-twelve-factor-app • Principles of container-based application design (redhat) • https://www.redhat.com/en/resources/cloud-native-container-design- whitepaper (English Edition) • https://www.redhat.com/ja/resources/cloud-native-container-design- whitepaper (Japanese Edition) • AWS Cloud Design Patterns • http://en.clouddesignpattern.org/index.php/Main_Page
45. © - BASE, Inc. Extra: Related Documents • “AWS Fargate Platform Version . Adds Secrets Support” • https://aws.amazon.com/about-aws/whats-new/ / /aws- fargate-platform-version- - -adds-secrets-support/?nc =h_ls