Deploying Applications to the Docker Ecosystem - SVCC (Saturday) 2015

Slides from the Saturday version of my talk at the Silicon Valley Code Camp. Has extra slides with links to some more details.

Transcript

1. Ted M. Young @jitterted
   tedyoung@gmail.com http://about.me/tedmyoung
   Deploying Applications
   in the Docker Ecosystem
   

   View Slide

2. Ted M. Young @jitterted
   tedyoung@gmail.com http://about.me/tedmyoung
   Thank you for attending my talk, I hope that you learned something
   worthy of your time.
   Interested in learning more? Contact me about my upcoming on-line
   video learning course that goes into much more depth, and walks you
   through creating and deploying scalable applications.
   If you need help incorporating the Docker Ecosystem into your
   applications or Continuous Integration system, I am available for
   training and consulting.
   

   View Slide

3. Ted M. Young @jitterted
   tedyoung@gmail.com http://about.me/tedmyoung
   Here are some links to things I mentioned during my talk (missed something? Contact me!):
   Linux OSes to use for Docker Hosting:
   * CoreOS (ships with Docker and other tools): https://www.coreos.com
   * Ubuntu (14.04 or later)
   Distributed/High-Availability Configuration tools
   * Apache Zookeeper: https://zookeeper.apache.org
   * etcd (CoreOS): https://github.com/coreos/etcd
   Docker Container Hosting in the Cloud (just a sampling!):
   * Google Container Engine: https://cloud.google.com/container-engine/
   * Amazon EC2 Container Service: https://aws.amazon.com/ecs/
   * Tutum: https://www.tutum.co
   * Joyent: https://www.joyent.com
   

   View Slide

4. What do we know?
   

   View Slide

5. It's a Container
   * obligatory
   container
   picture
   

   View Slide

6. Runs natively
   only on Linux
   Soon: FreeBSD, Solaris,
   Windows Server
   Not OS X
   

   View Slide

7. Terms
   Some Definitions…
   

   View Slide

8. Image
   packaged filesystem
   $ docker images
   

   View Slide

9. View Slide

10. View Slide

11. Image Registry
    source of images
    $ docker pull
    registry/image:tag
    

    View Slide

12. Get images
    from the registry
    and run them
    command-line demo
    

    View Slide

13. Container
    process(es)
    (running or not)
    $ docker ps [-a]
    

    View Slide

14. Processes are running
    in memory
    

    View Slide

15. Processes are gone,
    memory state lost
    

    View Slide

16. Containers provide
    Isolation
    

    View Slide

17.  File system
     Process
     Network
     Memory
    all are isolated
    

    View Slide

18.  File system
     Process
     Network
     Memory
    constrained
    

    View Slide

19. Environment
    Isolation
    

    View Slide

20. Environment
    Stability
    

    View Slide

21. Want Node.js
    version 0.10.38?
    

    View Slide

22. Also
    version 0.12.2?
    

    View Slide

23. On the SAME
    machine?
    

    View Slide

24. $ docker run node:0.10.38
    $ docker run node:0.12.2
    done.
    

    View Slide

25. VM vs. Container
    

    View Slide

26. Package once…
    Run everywhere
    

    View Slide

27. An Image
    ready-to-run
    packaged file system
    $ docker run ...
    

    View Slide

28. Let someone else
    build and I'll run it
    

    View Slide

29. Before Dockerization
    

    View Slide

30. Running Doc Generator Tool
    • Make sure you have Python
     And make sure it's Python 2.7
     No, not Python 3, it needs 2.7
    • Install pip
     $ python get-pip.py
     Did it work?
    • Install sphinx
     $ pip install sphinx
     argh…
    

    View Slide

31. After Dockerization
    

    View Slide

32. Running Doc Generator Tool
    $ docker run -v /mydocpath:/doc \
    minimum2scp/sphinx \
    "cd /doc; make html"
    

    View Slide

33. Use of Image Registry
    

    View Slide

34. Creating images
    Dockerfile
    

    View Slide

35. FROM nginx:1.7
    MAINTAINER Ted M. Young "tedyoung@gmail.com"
    RUN rm /etc/nginx/nginx.conf /etc/nginx/mime.types \
    && mkdir /etc/nginx/ssl
    COPY nginx.conf /etc/nginx/nginx.conf
    COPY mime.types /etc/nginx/mime.types
    COPY default /etc/nginx/sites-enabled/default
    COPY default-ssl /etc/nginx/sites-available/default-ssl
    # This will add the expanded tar file
    ADD www.tar /var/www
    EXPOSE 80 443
    VOLUME /var/log/nginx
    CMD ["nginx"]
    

    View Slide

36. Build the image
    command-line demo
    

    View Slide

37. Standing on the
    [images] of giants
    

    View Slide

38. FROM mongo:2.6
    ADD init-mongo.js /usr/local/bin/init-mongo.js
    # Define working directory (where commands below will be executed)
    WORKDIR /usr/local/bin
    ENTRYPOINT ["mongo"]
    CMD ["--host", "mongodb", "init-mongo.js"]
    

    View Slide

39. FROM debian:wheezy
    # add our user and group first to make sure their IDs get assigned consistently, regardless of whatever dependencies get added
    RUN groupadd -r mongodb && useradd -r -g mongodb mongodb
    RUN apt-get update \
    && apt-get install -y curl numactl \
    && rm -rf /var/lib/apt/lists/*
    # grab gosu for easy step-down from root
    RUN gpg --keyserver pool.sks-keyservers.net --recv-keys B42F6819007F00F88E364FD4036A9C25BF357DD4
    RUN curl -o /usr/local/bin/gosu -SL "https://github.com/tianon/gosu/releases/download/1.2/gosu-$(dpkg --print-architecture)" \
    && curl -o /usr/local/bin/gosu.asc -SL "https://github.com/tianon/gosu/releases/download/1.2/gosu-$(dpkg --print-architecture).asc" \
    && gpg --verify /usr/local/bin/gosu.asc \
    && rm /usr/local/bin/gosu.asc \
    && chmod +x /usr/local/bin/gosu
    ENV MONGO_RELEASE_FINGERPRINT DFFA3DCF326E302C4787673A01C4E7FAAAB2461C
    RUN gpg --keyserver pool.sks-keyservers.net --recv-keys $MONGO_RELEASE_FINGERPRINT
    ENV MONGO_VERSION 2.6.9
    RUN curl -SL "https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-$MONGO_VERSION.tgz" -o mongo.tgz \
    && curl -SL "https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-$MONGO_VERSION.tgz.sig" -o mongo.tgz.sig \
    && gpg --verify mongo.tgz.sig \
    && tar -xvf mongo.tgz -C /usr/local --strip-components=1 \
    && rm mongo.tgz*
    VOLUME /data/db
    COPY docker-entrypoint.sh /entrypoint.sh
    ENTRYPOINT ["/entrypoint.sh"]
    EXPOSE 27017
    CMD ["mongod"]
    

    View Slide

40. Pushing images
    to the registry
    

    View Slide

41. Image tag naming
    [registry/][owner/]name[:version]
    mongo
    mongo:2.6
    mongo:latest
    19hz/mongo-container
    private-registry/tyoung/mongodb:0.3.0
    

    View Slide

42. Pushing images
    to the registry
    $ docker push image_name
    

    View Slide

43. What Makes an
    Application?
    

    View Slide

44. View Slide

45. Apps Don't
    Stand Alone
    Scaling Deployments
    

    View Slide

46. network
    link
    network
    link
    

    View Slide

47. shared
    namespace
    

    View Slide

48. View Slide

49. View Slide

50. View Slide

51. docker-compose
    Static Deployment
    

    View Slide

52. nginx:
    image: nginx:latest
    links: apiservice
    apiservice:
    image: api-service:latest
    links:
    - logstash
    - redis
    - postgres
    - elastic
    logstash:
    image: logstash:latest
    links:
    - apiservice
    elastic:
    image: elasticsearch:latest
    postgres:
    image: …
    docker-compose.yml
    

    View Slide

53. Bring up the
    containers
    $ docker-compose up -d
    

    View Slide

54. Bring up the
    containers
    $ docker-compose up -d
    

    View Slide

55. On one machine?
    

    View Slide

56. Multiple Nodes?
    

    View Slide

57. View Slide

58. View Slide

59. View Slide

60. Container
    Deployment
    Dynamic
    

    View Slide

61. Cluster Management
    • Resource Manager
    • Job/Task Scheduler
    

    View Slide

62. Resource Manager
    • Tracks state of
     Node (host machine)
     CPU
     RAM
     Storage
    • Fulfills requests
    

    View Slide

63. Scheduler
    • Assigns workloads to machines
    • Decides which task/job/thing to start
    • When to start it
    • When to stop/kill it
    

    View Slide

64. Mesos: Meta Scheduler
    

    View Slide

65. Marathon
    Distributed
    "initd"
    

    View Slide

66. Mesos: Meta Scheduler
    

    View Slide

67. Google Kubernetes
    (Greek for "helmsman")
    

    View Slide

68. Lessons from
    Borg and Omega
    

    View Slide

69. API service
    log forwarder
    service monitor
    attached
    storage
    Pod: Unit of Placement
    

    View Slide

70. Node
    

    View Slide

71. Replication Controller
    "ensures that a specified
    number of pod 'replicas' are
    running at any one time"
    

    View Slide

72. Replication Controller
    "ensures that a specified number of pod
    'replicas' are running at any one time"
    •Too many? Kill it.
    

    View Slide

73. Replication Controller
    "ensures that a specified number of pod
    'replicas' are running at any one time"
    •Too many? Kill it.
    •Too few? Start some.
    

    View Slide

74. Replication Controller
    "ensures that a specified number of pod
    'replicas' are running at any one time"
    •Too many? Kill it.
    •Too few? Start some.
    •Failured? Start more.
    

    View Slide

75. View Slide

76. Service: Access to Pods
    

    View Slide

77. View Slide

78. Final Questions?
    

    View Slide

79. Thanks!
    Happy to chat more about
    Docker anytime.
    

    View Slide

80. the end
    @jitterted
    tedyoung@gmail.com
    

    View Slide