Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Databases in containers

Databases in containers

A demystifying about database within modern containers presented in https://www.pgconf.com.br

Fernando ike

August 03, 2018
Tweet

More Decks by Fernando ike

Other Decks in Technology

Transcript

  1. Banco de dados em Containers
    Fernando Ike
    PGConf.Brasil 2018

    View Slide

  2. Cache/Buffer

    View Slide

  3. Cache
    An auxiliary memory from which high-speed retrieval is possible.
    https://en.oxforddictionaries.com/definition/cache

    View Slide

  4. Buffer
    A temporary memory area in which data is stored while it is being processed or
    transferred, especially one used while streaming video or downloading audio.
    https://en.oxforddictionaries.com/definition/buffer

    View Slide

  5. https://momjian.us/main/writings/pgsql/hw_performance/

    View Slide

  6. View Slide

  7. Database cache
    Filesystem cache
    Disk cache
    Store data (disk)

    View Slide

  8. What’s is Virtual Machine?
    “...an efficient, isolated duplicate of a real computer machine...”
    Gerald Popek and Robert Goldberg

    View Slide

  9. VM
    Infrastructure
    Hypervisor
    App A
    Artifact
    Bin/Libs
    App B
    Bin/Libs
    App C
    Bin/Libs
    Guest OS Guest OS Guest OS

    View Slide

  10. What’s container?
    Container is a type of virtualization, more common call
    “operating-system-level virtualization”

    View Slide

  11. Container
    Infrastructure
    Hypervisor
    Docker
    App A
    Artifact
    Bin/Libs
    App B
    Bin/Libs
    App C
    Bin/Libs

    View Slide

  12. Ephemera
    Ephemera (singular: ephemeron) are any
    transitory written or printed matter not meant to
    be retained or preserved

    View Slide

  13. Stateless application
    Applications don’t need to store anything in a persistent disk. Data
    application are storage in other part of architecture system

    View Slide

  14. Stateful application
    Application that need to store “locally” data and
    need to recover that easy after an “interruption”

    View Slide

  15. Cap Theorem

    View Slide

  16. Union filesystem
    It allows files and directories of separate file systems, known
    as branches, to be transparently overlaid, forming a single
    coherent file system.

    View Slide

  17. Container layers

    View Slide

  18. Main filesytem for internal container data
    AUFS
    Overlayfs
    Overlayfs2
    Device-Mapper
    Bind/Mount

    View Slide

  19. Databases operation
    Plan what type of workload to use
    Maintain, organize and “improve” dataset together application
    Install security fix releases
    Archive “old” data (sanitize)
    Upgrade Database version
    Tuning OS, Database and Application

    View Slide

  20. What’s means in OS
    handle...
    sysctl
    /proc
    /sys
    limits
    overcommit

    View Slide

  21. What’s means in the PostgreSQL
    Shared Buffers
    WAL
    Max_connections
    Effective_cache_size
    Checkpoint_segments
    Random_page_cost

    View Slide

  22. Container Volumes
    They are a mechanism for persisting data generated and/or
    used by container

    View Slide

  23. Why to use PostgreSQL within containers
    Lower footprint to start
    Easy to change postgresql.conf and use git to have history change (tracking)
    Less surface attack in the “application
    Can “restrict” Linux Capabilities within container
    “Small” database by microservice
    Use always a Container Storage

    View Slide

  24. Tips
    One database per type of workload
    Take care to restrict network, consider restrict database access only own
    application
    Push database log to Monitoring system/service
    Clusters and data replication are more complex
    Database use intensive CPU and IO, think if there is another service with the same
    workload type
    Database is prepare to support two version of application (Deploy strategies without
    downtime)

    View Slide

  25. version: "3"
    services:
    db:
    image: postgres
    environment:
    - POSTGRES_USER=postgres
    - POSTGRES_PASSWORD=postgress
    - POSTGRES_DB=postgres
    ports:
    - "5433:5432"
    volumes:
    - pgdata:/var/lib/postgresql/data
    networks:
    - postgres
    volumes:
    pgdata:
    networks:
    postgres:

    View Slide

  26. Demo

    View Slide

  27. Demo code
    https://gist.github.com/fike/158204255cfbc368f36fad66ccd999a7

    View Slide

  28. ● Bruce Momjian - Performance tuning
    ● Docker Volumes
    ● Kubernetes Volumes
    ● Oxford Dictionaries
    ● Formal requirements for virtualizable third generation architectures - Gerald J. Popek and Robert P. Goldberg
    ● CAP Theorem and Distributed Database Management Systems
    References

    View Slide

  29. // http://www.10deploys.com
    // [email protected]
    // https://twitter.com/fernandoike
    // http://www.fernandoike.com
    // https://www.linkedin.com/in/fernandoike/
    Contacts

    View Slide