Databases in containers

A demystifying about database within modern containers presented in https://www.pgconf.com.br

Transcript

1. Banco de dados em Containers Fernando Ike PGConf.Brasil 2018
2. Cache/Buffer
3. Cache An auxiliary memory from which high-speed retrieval is possible. https://en.oxforddictionaries.com/definition/cache
4. Buffer A temporary memory area in which data is stored while it is being processed or transferred, especially one used while streaming video or downloading audio. https://en.oxforddictionaries.com/definition/buffer
5. https://momjian.us/main/writings/pgsql/hw_performance/
7. Database cache Filesystem cache Disk cache Store data (disk)
8. What’s is Virtual Machine? “...an efficient, isolated duplicate of a real computer machine...” Gerald Popek and Robert Goldberg
9. VM Infrastructure Hypervisor App A Artifact Bin/Libs App B Bin/Libs App C Bin/Libs Guest OS Guest OS Guest OS
10. What’s container? Container is a type of virtualization, more common call “operating-system-level virtualization”
11. Container Infrastructure Hypervisor Docker App A Artifact Bin/Libs App B Bin/Libs App C Bin/Libs
12. Ephemera Ephemera (singular: ephemeron) are any transitory written or printed matter not meant to be retained or preserved
13. Stateless application Applications don’t need to store anything in a persistent disk. Data application are storage in other part of architecture system
14. Stateful application Application that need to store “locally” data and need to recover that easy after an “interruption”
15. Cap Theorem
16. Union filesystem It allows files and directories of separate file systems, known as branches, to be transparently overlaid, forming a single coherent file system.
17. Container layers
18. Main filesytem for internal container data AUFS Overlayfs Overlayfs2 Device-Mapper Bind/Mount
19. Databases operation Plan what type of workload to use Maintain, organize and “improve” dataset together application Install security fix releases Archive “old” data (sanitize) Upgrade Database version Tuning OS, Database and Application
20. What’s means in OS handle... sysctl /proc /sys limits overcommit
21. What’s means in the PostgreSQL Shared Buffers WAL Max_connections Effective_cache_size Checkpoint_segments Random_page_cost
22. Container Volumes They are a mechanism for persisting data generated and/or used by container
23. Why to use PostgreSQL within containers Lower footprint to start Easy to change postgresql.conf and use git to have history change (tracking) Less surface attack in the “application Can “restrict” Linux Capabilities within container “Small” database by microservice Use always a Container Storage
24. Tips One database per type of workload Take care to restrict network, consider restrict database access only own application Push database log to Monitoring system/service Clusters and data replication are more complex Database use intensive CPU and IO, think if there is another service with the same workload type Database is prepare to support two version of application (Deploy strategies without downtime)
25. version: "3" services: db: image: postgres environment: - POSTGRES_USER=postgres - POSTGRES_PASSWORD=postgress - POSTGRES_DB=postgres ports: - "5433:5432" volumes: - pgdata:/var/lib/postgresql/data networks: - postgres volumes: pgdata: networks: postgres:
26. Demo
27. Demo code https://gist.github.com/fike/158204255cfbc368f36fad66ccd999a7
28. • Bruce Momjian - Performance tuning • Docker Volumes • Kubernetes Volumes • Oxford Dictionaries • Formal requirements for virtualizable third generation architectures - Gerald J. Popek and Robert P. Goldberg • CAP Theorem and Distributed Database Management Systems References
29. // http://www.10deploys.com // fernando.ike@gmail.com // https://twitter.com/fernandoike // http://www.fernandoike.com // https://www.linkedin.com/in/fernandoike/ Contacts