Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Introduction to Docker and Docker Compose

Avatar for Luong Vo Luong Vo
November 01, 2018

Introduction to Docker and Docker Compose

Avatar for Luong Vo

Luong Vo

November 01, 2018
Tweet

More Decks by Luong Vo

Other Decks in Programming

Transcript

  1. - Let’s look into Eh Avatar, an application to generate

    a beautiful avatar for you <3 - https://github.com/Thinkei/eh-avatar - Let’s setup and run the project Before we start
  2. Application Flow Web server 1) POST /avatars --data name=”Minh” Postgres

    Sidekiq server Redis 2) Save into DB 3) Send to sidekiq via Redis 4) Background job to generate avatar RMagick 5) Get /avatars/1
  3. Linux container technology - Linux containers contain applications in a

    way that keep them isolated from the host system that they run on. - Allow a developer to package up an application with all of the parts it needs, such as libraries and other dependencies - Make it easier to provide a consistent experience between development and production environment - Easy to deploy and replicate deployments
  4. Why Containers but not VM? - We don’t need an

    entire operating system, just some components to make everything up and run. - Container use host kernel to ensure the isolation and resource control, instead of full layer of visualization - Therefore, containers are - Fast in both starting up and operation - Lower memory footprint - Lightweight
  5. Linux container technology is not new - First form of

    container was Chroot, 1982 - FreeBSD Jails, 2000 - Solaris Zones, 2004 - Linux OpenVZ, 2005 - LXC, 2008 - Docker, 2013 - Built on LXC - Moved to libcontainer (March 2014) - Moved to runC (July 2015)
  6. What is Docker? - Container solution that provides full packages:

    - Image management - Resource Isolation - File System Isolation - Network Isolation - Change Management - Sharing - Process Management - Service Discovery (DNS since 1.10)
  7. Docker concepts Source code Dockerfile Build Docker image Push Image

    registry Host Pull Docker container 1 Docker container 2 Start
  8. Image vs Container? - Images are read-only. - An instance

    of an image is called a container. - You can have many running containers of the same image. - Images are created with the build command, and they'll produce a container when started with run.
  9. Docker concepts - Each image has a name and usually

    attached tags - All docker images with the same name are grouped - The most popular docker image registry is Docker Hub - https://hub.docker.com/r/library/postgres/tags/ - There are other public / private image registries from AWS, Google, etc.
  10. Practice 1: Bring up dependencies - Checkout branch docker_1 of

    the repository - Or view directly on this link: - https://github.com/Thinkei/eh-avatar/blob/docker_1/DOC KER_1.md
  11. Practice 1: Bring up dependencies - Install Docker on your

    local machine. https://docs.docker.com/install/ - Pull image of redis docker pull redis:latest - Pull image of postgres docker pull postgres:latest
  12. Practice 1: Bring up dependencies - Start redis docker run

    --name my-redis redis - Test the redis server by command redis-cli -h localhost - And it fails :troll:
  13. Practice 1: Bring up dependencies - Why couldn’t we connect

    to the redis server in postgres? + All containers are network isolated + It means that containers could not access others’ network and the host could not access containers’ network
  14. Docker network diagram (simplified) Host’s network interface (no port) Network

    interface A (open 6379) Container A: Redis (port 6379) Network interface B Container B Process Sidekiq Process redis-cli
  15. Practice 1: Bring up dependencies - Forward port 6379 of

    container to outside with port 6378 docker run --name my-redis -p 6378:6379 redis - Test the redis server by command redis-cli -h localhost -p 6378 - And it works
  16. Docker network diagram (simplified) Host’s network interface (open 6378) Network

    interface A (open 6379) Container A: Redis (port 6379) Network interface B Container B Process Sidekiq Process redis-cli
  17. Practice 1: Bring up dependencies - Bring up postgres docker

    run \ --name my-postgres \ -p 5433:5432 \ -e POSTGRES_PASSWORD=password \ -e POSTGRES_USER=username \ -e POSTGRES_DB=ehavatar \ postgres
  18. Practice 1: Bring up dependencies - Setup schema again DATABASE_URL=postgres://username:password@localhost:5433/

    ehavatar \ REDIS_URL=redis://localhost:6378 \ bundle exec ruby setup_schema.rb
  19. Practice 1: Bring up dependencies - Start web server on

    host machine DATABASE_URL=postgres://username:password@localhost:5433/ ehavatar \ REDIS_URL=redis://localhost:6378 \ bundle exec puma
  20. Practice 1: Bring up dependencies - Start sidekiq on host

    machine DATABASE_URL=postgres://username:password@localhost:5433/ ehavatar \ REDIS_URL=redis://localhost:6378 \ bundle exec bundle exec sidekiq -r ./config/environment.rb
  21. Practice 1: Bring up dependencies - Let’s turn off the

    postgres docker container and restart - Oops, all data is gone :’(
  22. Practice 1: Bring up dependencies - Let’s turn off the

    postgres docker container and restart - Oops, all data is gone :’( - What did happen? + All containers are file system isolated + All containers data are not mounted (linked) to the host. When it starts again, no data is retained
  23. Practice 1: Bring up dependencies - Try again with this

    command docker run \ --name my-postgres \ -p 5433:5432 \ -e POSTGRES_PASSWORD=password \ -e POSTGRES_USER=username \ -e POSTGRES_DB=ehavatar \ -e PGDATA=/var/data \ -v $(pwd)/tmp/data:/var/data \ postgres
  24. Docker File system diagram (simplified) Container A’s Virtual FIle system

    Container A: Postgres Host’s real file system /var/data /Users/ahihi/tmp/data
  25. Docker concepts - Dockerfile is a text document that contains

    all the commands a user could call on the command line to assemble an image. - It defines: - Is current image based on other image? If yes, what is it? - Dependencies installation commands - How to start the container of this image? - What environments are allowed to be passed in? - Which ports the container will expose? - etc.
  26. Practice 2: Build your own docker image - Checkout branch

    docker_2 - Or view online at https://github.com/Thinkei/eh-avatar/blob/docker_1/DOC KER_1.md
  27. Practice 2: Build your own docker image - Start Postgres

    and Redis container again we don't need to expose ports - Create a new file Dockerfile with the following content FROM ruby:2.4.0 RUN mkdir -p /app WORKDIR /app COPY Gemfile Gemfile.lock ./ RUN gem install bundler && bundle install --jobs 20 --retry 5 --without test COPY . ./ EXPOSE 9292 CMD ["bundle", "exec", "puma"]
  28. Practice 2: Build your own docker image - Build the

    docker image with command docker build -t ehavatar . - The building process fails. We fail to build RMgick gem. Need to add dependencies. Add this line to Dockerifle RUN apt-get update -qq --fix-missing && apt-get install -y libmagickwand-dev
  29. Practice 2: Build your own docker image - Build again.

    And it is sure to be successful. - Start web server and sidekiq docker run -p 9292:9292 eh-avatar docker run -p 9292:9292 eh-avatar bundle exec sidekiq -r ./config/environment.rb - And it fails again. We fail to connect to redis and postgres
  30. Practice 2: Build your own docker image - Using `link`

    to link the application containers to dependencies docker run \ --link my-redis \ --link my-postgres \ -e DATABASE_URL=postgres://username:password@my-postgres:5432/ehavatar \ -e REDIS_URL=redis://my-redis:6379 \ -v $(pwd)/tmp:/app/tmp \ -p 9292:9292 \ eh-avatar
  31. Practice 2: Build your own docker image - Using `link`

    to link the application containers to dependencies docker run \ --link my-redis \ --link my-postgres \ -e DATABASE_URL=postgres://username:password@my-postgres:5432/ehavatar \ -e REDIS_URL=redis://my-redis:6379 \ -v $(pwd)/tmp:/app/tmp \ eh-avatar bundle exec sidekiq -r ./config/environment.rb
  32. Docker Compose - A tool for defining and running multi-container

    Docker applications. - Run multiple isolated environments on a single host with a single command. - Better Development environments. - Easier build / run / scale.
  33. Best practice and pitfalls 1. Containers should be ephemeral 2.

    Specify a build context 3. Use a .dockerignore file 4. Use multi-stage builds 5. Avoid installing unnecessary packages 6. Minimize the number of layers 7. ….
  34. After this, do you become a master of Docker? -

    This is just a beginning :troll: - There are a lot of things haven’t been mentioned yet
  35. Q&A