Deploying Applications to the Docker Ecosystem - SVCC (Sunday) 2015

Slides from the Sunday (October 4th) version (some fixes and couple of new slides) of my talk at the Silicon Valley Code Camp. Has extra slides with links to some more details.

Transcript

1. Ted M. Young @jitterted
   https://about.me/tedmyoung
   Deploying Applications
   in the Docker Ecosystem
   

   View Slide

2. Ted M. Young @jitterted
   tedyoung@gmail.com http://about.me/tedmyoung
   Thank you for attending my talk, I hope that you learned something
   worthy of your time.
   Interested in learning more? Contact me about my upcoming on-line
   video learning course that goes into much more depth, and walks you
   through creating and deploying scalable applications.
   If you need help incorporating the Docker Ecosystem into your
   applications or Continuous Integration system, I am available for
   training and consulting.
   

   View Slide

3. Ted M. Young @jitterted
   tedyoung@gmail.com http://about.me/tedmyoung
   Here are some links to things I mentioned during my talk (missed something? Contact me!):
   Linux OSes to use for Docker Hosting:
   * CoreOS (ships with Docker and other tools): https://www.coreos.com
   * Ubuntu (14.04 or later)
   Distributed/High-Availability Configuration tools
   * Apache Zookeeper: https://zookeeper.apache.org
   * etcd (CoreOS): https://github.com/coreos/etcd
   Docker Container Hosting in the Cloud (just a sampling!):
   * Google Container Engine: https://cloud.google.com/container-engine/
   * Amazon EC2 Container Service: https://aws.amazon.com/ecs/
   * Tutum: https://www.tutum.co
   * Joyent: https://www.joyent.com
   

   View Slide

4. Ted M. Young @jitterted
   tedyoung@gmail.com http://about.me/tedmyoung
   Deploying Applications
   in the Docker Ecosystem
   Silicon Valley Code Camp – Sunday Edition
   

   View Slide

5. What do we know?
   

   View Slide

6. Container!
   * obligatory
   container
   picture
   

   View Slide

7. Runs natively
   only on Linux
   Soon: FreeBSD, Solaris,
   Windows Server
   Not OS X
   

   View Slide

8. Terms
   Some Definitions…
   

   View Slide

9. Image
   packaged filesystem
   $ docker images
   

   View Slide

10. View Slide

11. View Slide

12. Image Registry
    source of images
    $ docker pull
    registry/image:tag
    

    View Slide

13. Get images
    from the registry
    and run them
    command-line demo
    

    View Slide

14. Container
    process(es)
    (running or not)
    $ docker ps [-a]
    

    View Slide

15. Processes are running
    in memory
    

    View Slide

16. Processes are gone,
    memory state lost
    

    View Slide

17. Containers provide
    Isolation
    

    View Slide

18.  File system
     Process
     Network
     Memory
    all are isolated
    

    View Slide

19. The Operating
    System
    is shared
    

    View Slide

20.  File system
     Process
     Network
     Memory
    constrained
    

    View Slide

21. Environment
    Isolation
    

    View Slide

22. Environment
    Stability
    

    View Slide

23. Want Node.js
    version 0.10.38?
    

    View Slide

24. Also
    version 0.12.2?
    

    View Slide

25. On the SAME
    machine?
    

    View Slide

26. $ docker run node:0.10.38
    $ docker run node:0.12.2
    done.
    

    View Slide

27. VM vs. Container
    

    View Slide

28. Package once…
    Run everywhere
    

    View Slide

29. An Image
    ready-to-run
    packaged file system
    $ docker run ...
    

    View Slide

30. Let someone else
    build and I'll run it
    

    View Slide

31. Before Dockerization
    

    View Slide

32. Running Doc Generator Tool
    • Make sure you have Python
     And make sure it's Python 2.7
     No, not Python 3, it needs 2.7
    • Install pip
     $ python get-pip.py
     Did it work?
    • Install sphinx
     $ pip install sphinx
     argh…
    

    View Slide

33. After Dockerization
    

    View Slide

34. Running Doc Generator Tool
    $ docker run -v /mydocpath:/doc \
    minimum2scp/sphinx \
    "cd /doc; make html"
    

    View Slide

35. View Slide

36. Use of Image Registry
    

    View Slide

37. Docker Registry
    

    View Slide

38. Creating images
    Dockerfile
    

    View Slide

39. FROM nginx:1.7
    MAINTAINER Ted M. Young "tedyoung@gmail.com"
    RUN rm /etc/nginx/nginx.conf /etc/nginx/mime.types \
    && mkdir /etc/nginx/ssl
    COPY nginx.conf /etc/nginx/nginx.conf
    COPY mime.types /etc/nginx/mime.types
    COPY default /etc/nginx/sites-enabled/default
    COPY default-ssl /etc/nginx/sites-available/default-ssl
    # This will add the expanded tar file
    ADD www.tar /var/www
    EXPOSE 80 443
    VOLUME /var/log/nginx
    CMD ["nginx"]
    

    View Slide

40. Build the image
    command-line demo
    

    View Slide

41. Standing on the
    [images] of giants
    

    View Slide

42. FROM mongo:2.6
    ADD init-mongo.js /usr/local/bin/init-mongo.js
    # Define working directory (where commands below will be executed)
    WORKDIR /usr/local/bin
    ENTRYPOINT ["mongo"]
    CMD ["--host", "mongodb", "init-mongo.js"]
    

    View Slide

43. FROM debian:wheezy
    # add our user and group first to make sure their IDs get assigned consistently, regardless of whatever dependencies get added
    RUN groupadd -r mongodb && useradd -r -g mongodb mongodb
    RUN apt-get update \
    && apt-get install -y curl numactl \
    && rm -rf /var/lib/apt/lists/*
    # grab gosu for easy step-down from root
    RUN gpg --keyserver pool.sks-keyservers.net --recv-keys B42F6819007F00F88E364FD4036A9C25BF357DD4
    RUN curl -o /usr/local/bin/gosu -SL "https://github.com/tianon/gosu/releases/download/1.2/gosu-$(dpkg --print-architecture)" \
    && curl -o /usr/local/bin/gosu.asc -SL "https://github.com/tianon/gosu/releases/download/1.2/gosu-$(dpkg --print-architecture).asc" \
    && gpg --verify /usr/local/bin/gosu.asc \
    && rm /usr/local/bin/gosu.asc \
    && chmod +x /usr/local/bin/gosu
    ENV MONGO_RELEASE_FINGERPRINT DFFA3DCF326E302C4787673A01C4E7FAAAB2461C
    RUN gpg --keyserver pool.sks-keyservers.net --recv-keys $MONGO_RELEASE_FINGERPRINT
    ENV MONGO_VERSION 2.6.9
    RUN curl -SL "https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-$MONGO_VERSION.tgz" -o mongo.tgz \
    && curl -SL "https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-$MONGO_VERSION.tgz.sig" -o mongo.tgz.sig \
    && gpg --verify mongo.tgz.sig \
    && tar -xvf mongo.tgz -C /usr/local --strip-components=1 \
    && rm mongo.tgz*
    VOLUME /data/db
    COPY docker-entrypoint.sh /entrypoint.sh
    ENTRYPOINT ["/entrypoint.sh"]
    EXPOSE 27017
    CMD ["mongod"]
    

    View Slide

44. Pushing images
    to the registry
    

    View Slide

45. Image tag naming
    [registry/][owner/]name[:version]
    mongo
    mongo:2.6
    mongo:latest
    19hz/mongo-container
    private-registry/tyoung/mongodb:0.3.0
    

    View Slide

46. Pushing images
    to the registry
    $ docker push image_name
    

    View Slide

47. What Makes an
    Application?
    

    View Slide

48. Service
    API
    Web
    Server
    Databa
    se
    

    View Slide

49. Apps Don't
    Stand Alone
    Scaling Deployments
    

    View Slide

50. Service
    API
    Web
    Server
    Databa
    se
    network
    link
    network
    link
    

    View Slide

51. Service
    API
    Web
    Server
    Databa
    se
    shared
    namespace
    

    View Slide

52. Service
    API
    Web
    Server
    Databa
    se
    

    View Slide

53. Service
    API
    Web
    Server
    Databa
    se
    Cac
    he
    

    View Slide

54. Service
    API
    Web
    Server
    Databa
    se
    Cac
    he
    

    View Slide

55. docker-compose
    Static Deployment
    

    View Slide

56. nginx:
    image: nginx:latest
    links: apiservice
    apiservice:
    image: api-service:latest
    links:
    - logstash
    - redis
    - postgres
    - elastic
    logstash:
    image: logstash:latest
    links:
    - apiservice
    elastic:
    image: elasticsearch:latest
    postgres:
    image: …
    docker-compose.yml
    

    View Slide

57. Bring up the
    containers
    $ docker-compose up -d
    

    View Slide

58. Bring up the
    containers
    Service API
    Web Server
    Database
    Cach
    e
    $ docker-compose up -d
    

    View Slide

59. On one machine?
    

    View Slide

60. Multiple Nodes?
    

    View Slide

61. View Slide

62. Load Balancer
    

    View Slide

63. Load Balancer
    Messag
    e
    Queue
    Legacy
    Service
    Legacy
    Service
    

    View Slide

64. Container
    Deployment
    Dynamic
    

    View Slide

65. Cluster Management
    • Resource Manager
    • Job/Task Scheduler
    

    View Slide

66. Resource Manager
    • Tracks state of
     Node (host machine)
     CPU
     RAM
     Storage
    • Fulfills requests
    

    View Slide

67. Scheduler
    • Assigns workloads to machines
    • Decides which task/job/thing to start
    • When to start it
    • When to stop/kill it
    

    View Slide

68. Mesos: Meta Scheduler
    http://mesos.apache.org/
    

    View Slide

69. Marathon
    Distributed
    "initd"
    

    View Slide

70. Mesos: Meta Scheduler
    

    View Slide

71. Google Kubernetes
    (Greek for "helmsman")
    http://kubernetes.io
    

    View Slide

72. Lessons from
    Borg and Omega
    Borg: http://research.google.com/pubs/pub43438.html
    Omega: http://research.google.com/pubs/pub41684.html
    

    View Slide

73. API service
    log forwarder
    service monitor
    attached
    storage
    Pod: Unit of Placement
    

    View Slide

74. Node
    

    View Slide

75. Replication Controller
    "ensures that a specified
    number of pod 'replicas' are
    running at any one time"
    

    View Slide

76. Replication Controller
    "ensures that a specified number of pod
    'replicas' are running at any one time"
    •Too many? Kill it.
    

    View Slide

77. Replication Controller
    "ensures that a specified number of pod
    'replicas' are running at any one time"
    •Too many? Kill it.
    •Too few? Start some.
    

    View Slide

78. Replication Controller
    "ensures that a specified number of pod
    'replicas' are running at any one time"
    •Not enough? Start some.
    •Too many? Kill them.
    •Failed? Start more.
    

    View Slide

79. View Slide

80. Service: Access to Pods
    

    View Slide

81. View Slide

82. Final Questions?
    

    View Slide

83. Thanks!
    Happy to chat more about
    Docker anytime.
    

    View Slide

84. the end
    Twitter: @jitterted
    https://about.me/tedmyoung
    

    View Slide