Databases in containers

A demystifying about database within modern containers presented in https://www.pgconf.com.br

Transcript

1. Banco de dados em Containers
   Fernando Ike
   PGConf.Brasil 2018
   

   View Slide

2. Cache/Buffer
   

   View Slide

3. Cache
   An auxiliary memory from which high-speed retrieval is possible.
   https://en.oxforddictionaries.com/definition/cache
   

   View Slide

4. Buffer
   A temporary memory area in which data is stored while it is being processed or
   transferred, especially one used while streaming video or downloading audio.
   https://en.oxforddictionaries.com/definition/buffer
   

   View Slide

5. https://momjian.us/main/writings/pgsql/hw_performance/
   

   View Slide

6. View Slide

7. Database cache
   Filesystem cache
   Disk cache
   Store data (disk)
   

   View Slide

8. What’s is Virtual Machine?
   “...an efficient, isolated duplicate of a real computer machine...”
   Gerald Popek and Robert Goldberg
   

   View Slide

9. VM
   Infrastructure
   Hypervisor
   App A
   Artifact
   Bin/Libs
   App B
   Bin/Libs
   App C
   Bin/Libs
   Guest OS Guest OS Guest OS
   

   View Slide

10. What’s container?
    Container is a type of virtualization, more common call
    “operating-system-level virtualization”
    

    View Slide

11. Container
    Infrastructure
    Hypervisor
    Docker
    App A
    Artifact
    Bin/Libs
    App B
    Bin/Libs
    App C
    Bin/Libs
    

    View Slide

12. Ephemera
    Ephemera (singular: ephemeron) are any
    transitory written or printed matter not meant to
    be retained or preserved
    

    View Slide

13. Stateless application
    Applications don’t need to store anything in a persistent disk. Data
    application are storage in other part of architecture system
    

    View Slide

14. Stateful application
    Application that need to store “locally” data and
    need to recover that easy after an “interruption”
    

    View Slide

15. Cap Theorem
    

    View Slide

16. Union filesystem
    It allows files and directories of separate file systems, known
    as branches, to be transparently overlaid, forming a single
    coherent file system.
    

    View Slide

17. Container layers
    

    View Slide

18. Main filesytem for internal container data
    AUFS
    Overlayfs
    Overlayfs2
    Device-Mapper
    Bind/Mount
    

    View Slide

19. Databases operation
    Plan what type of workload to use
    Maintain, organize and “improve” dataset together application
    Install security fix releases
    Archive “old” data (sanitize)
    Upgrade Database version
    Tuning OS, Database and Application
    

    View Slide

20. What’s means in OS
    handle...
    sysctl
    /proc
    /sys
    limits
    overcommit
    

    View Slide

21. What’s means in the PostgreSQL
    Shared Buffers
    WAL
    Max_connections
    Effective_cache_size
    Checkpoint_segments
    Random_page_cost
    

    View Slide

22. Container Volumes
    They are a mechanism for persisting data generated and/or
    used by container
    

    View Slide

23. Why to use PostgreSQL within containers
    Lower footprint to start
    Easy to change postgresql.conf and use git to have history change (tracking)
    Less surface attack in the “application
    Can “restrict” Linux Capabilities within container
    “Small” database by microservice
    Use always a Container Storage
    

    View Slide

24. Tips
    One database per type of workload
    Take care to restrict network, consider restrict database access only own
    application
    Push database log to Monitoring system/service
    Clusters and data replication are more complex
    Database use intensive CPU and IO, think if there is another service with the same
    workload type
    Database is prepare to support two version of application (Deploy strategies without
    downtime)
    

    View Slide

25. version: "3"
    services:
    db:
    image: postgres
    environment:
    - POSTGRES_USER=postgres
    - POSTGRES_PASSWORD=postgress
    - POSTGRES_DB=postgres
    ports:
    - "5433:5432"
    volumes:
    - pgdata:/var/lib/postgresql/data
    networks:
    - postgres
    volumes:
    pgdata:
    networks:
    postgres:
    

    View Slide

26. Demo
    

    View Slide

27. Demo code
    https://gist.github.com/fike/158204255cfbc368f36fad66ccd999a7
    

    View Slide

28. ● Bruce Momjian - Performance tuning
    ● Docker Volumes
    ● Kubernetes Volumes
    ● Oxford Dictionaries
    ● Formal requirements for virtualizable third generation architectures - Gerald J. Popek and Robert P. Goldberg
    ● CAP Theorem and Distributed Database Management Systems
    References
    

    View Slide

29. // http://www.10deploys.com
    // fernando.ike@gmail.com
    // https://twitter.com/fernandoike
    // http://www.fernandoike.com
    // https://www.linkedin.com/in/fernandoike/
    Contacts
    

    View Slide