Container based application Design Real Practices

talked at #dockertokyo #dockerbday

Transcript

1. © - BASE, Inc.
   Container-based
   Application Design
   Real Practices
   . . #dockertokyo - @hgsgtk
   

   View Slide

2. © - BASE, Inc.
   Talk Summary
   Keep each components is Decoupled
   Keep Balance with development efficiency
   Use Cloud-based Design Principles
   

   View Slide

3. © - BASE, Inc.
   About me
   Kazuki Higashiguchi
   Twitter / GitHub : @hgsgtk
   Server-side Engineer
   BASE BANK, Inc.
   

   View Slide

4. © - BASE, Inc.
   Agenda
   .Configuration
   .Logging
   .Building Application
   .Monitoring
   

   View Slide

5. Configuration
   

   View Slide

6. © - BASE, Inc.
   Features of Configuration
   • “Runtime” Data (Necessary at startup)
   • Different for each environment
   • A lot of values
   

   View Slide

7. © - BASE, Inc.
   Requests for configuration
   • Manage hierarchically
   • Make configurations easy to understand
   • Externalize configuration
   • Get rid of configurations from Application
   • Minimal accessibility
   • Configurations includes credentials
   

   View Slide

8. © - BASE, Inc.
   Related Principles
   • “ . CONFIGURATION, CREDENTIALS, AND
   CODE” in “Beyond the Twelve-Factor App”
   • “IMAGE IMMUTABILITY PRINCIPLE(IIP)” in
   “Principles of container-based application
   design (redhat)”
   

   View Slide

9. © - BASE, Inc.
   . CONFIGURATION, CREDENTIALS, AND CODE (WIP)
   • Get rid of configurations from Code
   • Not include in Version Control System
   • “Treat Your Apps Like Open Source”
   • Environment variables are considered the best
   practice for externalized configuration,
   • Externalized configurations
   

   View Slide

10. © - BASE, Inc.
    Extra: Beyond the Twelve-Factor App
    • Best practices of cloud-native
    application
    • published by Pivotal
    • https://content.pivotal.io/blog/
    beyond-the-twelve-factor-app
    • Original is “The Twelve-Factor
    App”
    • https:// factor.net/
    • Original + New = factors
    

    View Slide

11. © - BASE, Inc.
    Extra: Principles of container-based application
    design (redhat)
    • Design Principles of container-
    based application
    • published by red-hot
    • https://www.redhat.com/en/
    resources/cloud-native-
    container-design-whitepaper
    

    View Slide

12. © - BASE, Inc.
    Configuration: Practical example
    • AWS ECS(Fargate)
    • Configurations are stored in
    Parameter Store
    • Encryption using Key
    Management Store
    • See also:
    • https://devblog.thebase.in/
    entry/ / / /
    

    View Slide

13. © - BASE, Inc.
    Configuration: Real Issues
    • Depends on Features provided by Cloud vendor
    • If well-integrated features, become “Simple”
    • If Not, may become “Complicated”
    

    View Slide

14. © - BASE, Inc.
    For example: Fargate & Parameter Store
    • A example to use Parameter Store as
    externalized configuration store
    • Before Platform Version .
    • Each container access Parameter Store via API
    • Write entrypoint.sh by myself
    

    View Slide

15. © - BASE, Inc.
    entrypoint.sh
    #!/usr/bin/env bash
    set -e
    # request multiple times
    export DB_HOST=$(aws ssm get-parameters --name /prd/
    database/host --query "Parameters[0].Value" --region ap-
    northeast-1 --output text)
    export DB_PORT=$(aws ssm get-parameters --name /prd/
    database/port --query "Parameters[0].Value" --region ap-
    northeast-1 --output text)
    exec "$@"
    

    View Slide

16. © - BASE, Inc.
    For example: Fargate & Parameter Store
    • After Released .
    • “AWS Fargate Platform Version . Adds
    Secrets Support”
    • Just write task definition (and set IAM role)
    • Become well-integrated features.
    • It makes our application “Simpler”
    

    View Slide

17. Logging
    

    View Slide

18. © - BASE, Inc.
    Requests for Logging
    • Realtime
    • Know “What is happening now”
    • Availability
    • “No” Missing Log
    • Searching
    • Make identifying the cause of problem “Easy”
    

    View Slide

19. © - BASE, Inc.
    Related Principles
    • “ . LOGS” in “Beyond the Twelve-Factor App”
    

    View Slide

20. © - BASE, Inc.
    “ . LOGS”
    • Treat Log as Event Stream
    • Not depends on File System
    • Log on STDOUT/STDERR
    • Use tools (ex. ElasticSearch, Logstash, Kibana)
    to satisfied the requirements of aggregation,
    processing, storage of logs
    

    View Slide

21. © - BASE, Inc.
    LOGGING: Practical Examples
    • Application writes logs to STDOUT/STDERR
    • Fargate -> CloudWatchLogs(CWL) -> S ->
    ElasticSearch & Kibana
    • See also https://qiita.com/tomy rider/items/
    aa dd
    

    View Slide

22. Build Application
    

    View Slide

23. © - BASE, Inc.
    Requests for Build Application
    • Dependency Management
    • Manage Application Dependency
    • Work in Difference environments
    • Work in Local, QA, Stating, Production
    

    View Slide

24. © - BASE, Inc.
    Application Components
    • Runtime Engine
    • Code
    • Dependencies
    • Configuration
    

    View Slide

25. © - BASE, Inc.
    Container Application
    • Container includes
    • Runtime Engine
    • Code
    • Dependencies
    • Configuration is
    injected at startup
    

    View Slide

26. © - BASE, Inc.
    For environments, Same Container
    

    View Slide

27. © - BASE, Inc.
    Related Principles
    • “ . DEPENDENCY MANAGEMENT” in “Beyond
    the Twelve-Factor App”
    • “ . DESIGN, BUILD, RELEASE, RUN” in “Beyond
    the Twelve-Factor App”
    

    View Slide

28. © - BASE, Inc.
    Build Application: Practical example
    • Use case
    • API developed by Go
    • One Dockerfile for
    deploy
    • Build image provide to
    environments
    

    View Slide

29. © - BASE, Inc.
    Build Application: Real Issues
    • Best practices: same container
    provides ALL environments
    • However, in Local environments,
    Development efficiency is also
    important
    

    View Slide

30. © - BASE, Inc.
    Build Application: Real Issues
    • One Dockerfile for deploy
    • Another Dockerfile for
    developer
    • Live reloading
    • See also
    • https://go-
    talks.appspot.com/
    github.com/hgsgtk/
    gocon -lt/realize.slide#
    

    View Slide

31. © - BASE, Inc.
    Find the suitable balance for you
    • “Docker for local”
    • -> weakens the merit
    • -> strengthen developers
    • Necessity to find suitable balance
    

    View Slide

32. Monitoring
    

    View Slide

33. © - BASE, Inc.
    Requests for Monitoring
    • Check Health
    • Monitor application is healthy
    • Keep Watching Metrics
    • Keep watching Metrics
    

    View Slide

34. © - BASE, Inc.
    Related Principles
    • “HIGH OBSERVABILITY PRINCIPLE (HOP)” in
    “Principles of container-based application
    design (redhat)”
    

    View Slide

35. © - BASE, Inc.
    HIGH OBSERVABILITY PRINCIPLE (HOP)
    • Treat Container like “Black box”
    • Provide API for various status check
    • Activity status
    • Preparation status
    • Log important events on STDOUT/STDERR
    

    View Slide

36. © - BASE, Inc.
    Health endpoint pattern
    • One of application monitoring patterns
    • Provide HTTP endpoints in the application
    • convey the health of the application
    • See also: “Practical Monitoring—Effective
    Strategies for the Real World”
    • http://shop.oreilly.com/product/
    .do
    

    View Slide

37. © - BASE, Inc.
    Mackerel Container Agent
    • Monitor “Inside Container”
    • CPU, Memory, Network etc
    • Run as a task / pod sidecar
    • See also: https://mackerel.io/ja/docs/entry/
    howto/container-agent
    

    View Slide

38. © - BASE, Inc.
    Monitoring: Practical example
    /.health_check endpoint return “OK”
    See also: https://devblog.thebase.in/entry/ /
    / /
    -> % curl -i http://localhost/.health_check
    HTTP/1.1 200 OK
    Content-Type: application/json; charset=utf-8
    Date: Mon, 04 Mar 2019 13:24:23 GMT
    Content-Length: 15
    {"status":200}
    

    View Slide

39. © - BASE, Inc.
    Monitoring: Practical example
    /.health_check endpoint also return “NG”
    See also: https://devblog.thebase.in/entry/ /
    / /
    -> % curl -i http://localhost/.health_check
    HTTP/1.1 503 Service Unavailable
    Content-Type: application/json; charset=utf-8
    Date: Mon, 04 Mar 2019 13:35:38 GMT
    Content-Length: 106
    {"status":503,"message":"failed to get connection
    database","string":"sql: connection is already closed"}
    

    View Slide

40. © - BASE, Inc.
    Talk Summary
    Keep each components is Decoupled
    Keep Balance with development efficiency
    Use Cloud-based Design Principles
    

    View Slide

41. © - BASE, Inc.
    Extra: Related Presentations
    • “Container-based Application Design Reference and Practice”
    at Docker Tokyo Meetup
    • https://speakerdeck.com/hgsgtk/container-based-application-design-
    reference-and-practice-number-dockertokyo
    • “Docker Go development environment starting with realize”
    at Go Conference Autumn
    • https://go-talks.appspot.com/github.com/hgsgtk/gocon -lt/
    realize.slide#
    

    View Slide

42. © - BASE, Inc.
    Extra: Related Blogs
    • “ECS(Fargate)でコンテナアプリケーションを動かすための設定情報の扱
    い⽅”
    • https://devblog.thebase.in/entry/ / / /
    • “アプリケーション監視のパターン「Health エンドポイントパターン」
    を実践する | 書籍『⼊⾨ 監視 ―モダンなモニタリングのためのデザイン
    パターン』を読んで”
    • https://devblog.thebase.in/entry/ / / /
    

    View Slide

43. © - BASE, Inc.
    Extra: Related Books
    • 『⼊⾨ 監視 モダンなモニタリングのためのデザインパターン』
    https://www.oreilly.co.jp/books/ /
    

    View Slide

44. © - BASE, Inc.
    Extra: Related Documents
    • Beyond the Twelve-Factor App
    • https://content.pivotal.io/blog/beyond-the-twelve-factor-app
    • Principles of container-based application design (redhat)
    • https://www.redhat.com/en/resources/cloud-native-container-design-
    whitepaper (English Edition)
    • https://www.redhat.com/ja/resources/cloud-native-container-design-
    whitepaper (Japanese Edition)
    • AWS Cloud Design Patterns
    • http://en.clouddesignpattern.org/index.php/Main_Page
    

    View Slide

45. © - BASE, Inc.
    Extra: Related Documents
    • “AWS Fargate Platform Version . Adds Secrets Support”
    • https://aws.amazon.com/about-aws/whats-new/ / /aws-
    fargate-platform-version- - -adds-secrets-support/?nc =h_ls
    

    View Slide