Microplugins with Docker

Transcript

1. Microplugins DOCKER ******************* WITH *******************
2. Freelance « DevOps » JULIEN VEY ERIC BELLEMON ********************************************* at bywan @julienvey @haklop
3. BY NOW, WE THINK YOU KNOW WHAT IS DOCKER WHAT ?
4. DOCKER HOW ? Patterns *******************************************
5. MICROSERVICES
6. A MICROSERVICE IS USUALLY A PERSISTANT SERVICE YOU CAN ACCESS VIA AN API
7. GO GO PYTHON RUBY NODE JAVA JAVA PHP ANY LANGUAGE
8. UBUNTU REDHAT DEBIAN CENTOS FEDORA FEDORA DEBIAN CENTOS ANY DISTRIBUTION
9. AWS GCE COREOS ANY WHERE
10. WHY CONTAINERS AND NOT VMS? CONTAINERS ARE LIGHTWEIGHT SPAWN FAST USE LAYERS
11. DOCKER Tips n’ Tricks ******************************************* HOW ?
12. DOCKER IN DOCKER
13. THE DOCKER MODEL DOCKER DAEMON UNIX SOCKET DOCKER REMOTE API DOCKER CLI creates an which exposes the that can be used by the
14. FIRST APPROACH
15. DOCKER IN DOCKER IN DOCKER …
16. docker run --privileged -t -i jpetazzo/dind SPECIAL DOCKER IMAGE
17. docker run --privileged -t -i jpetazzo/dind PRIVILEGED MODE
18. docker run --privileged -t -i jpetazzo/dind PRIVILEGED MODE
19. IT RAISES SOME PROBLEMS INIT SYSTEM DISK SPACE CUSTOM IMAGE
20. SECOND APPROACH
21. DOCKER IN DOCKER IN DOCKER … DOCKER HOST DOCKER SOCKET
22. docker run -v /var/run/docker.sock:/var/run/docker.sock [your image] DOCKER SOCKET AS A DOCKER VOLUME
23. docker run -v /var/run/docker.sock:/var/run/docker.sock [your image] DOCKER SOCKET AS A DOCKER VOLUME
24. NO PRIVILEGED MODE ONLY ONE DOCKER DAEMON (INSTEAD OF ONE PER CONTAINER)
25. What about ? MICRO PLUGINS *********************************************
26. MICROSERVICE: A SMALL SERVICE RUNNING IN A CONTAINER
27. MICROPLUGIN: A SMALL PLUGIN RUNNING IN A CONTAINER
28. What is a ? Plugin Architecture *********************************************
29. A STANDARD INTERFACE A LOADING MECHANISM A PACKAGING MECHANISM A DISTRIBUTION MECHANISM
30. VAGRANT ******************* WITH ******************* A PLUGIN ARCHITECTURE RUBY EXAMPLE
31. A STANDARD INTERFACE require 'vagrant' module VagrantPlugins module Openstack class Provider < Vagrant.plugin('2', :provider) RUBY CLASSES
32. A LOADING MECHANISM RUBY DYNAMIC LOADING SOURCES INSTALLED IN VAGRANT BUNDLE DIR
33. A PACKAGING MECHANISM RUBY GEM PACKAGE CODE AND DOCUMENTATION DESCRIBE DEPENDENCIES
34. A DISTRIBUTION MECHANISM RUBYGEMS.ORG
35. ****************** AND ****************** PROS CONS
36. PLATFORM INDEPENDANT PACKAGING AND DISTRIBUTION ARE NOT SPECIFIC TO VAGRANT PROS
37. RUBY ONLY DEPENDENCY MANAGEMENT CAN BE HARD CONS
38. JENKINS ****************** WITH ****************** A PLUGIN ARCHITECTURE JAVA EXAMPLE
39. A STANDARD INTERFACE package jenkins.plugins.git; public abstract class AbstractGitSCMSource extends SCMSource { protected SCMRevision retrieve(…) protected void retrieve(…) } JAVA INTERFACES & CLASSES
40. A LOADING MECHANISM JAVA CLASSLOADING
41. A PACKAGING MECHANISM JPI/HPI = JENKINS/HUDSON PLUG IN CUSTOM PACKAGING (JPI/HPI)
42. UPDATES.JENKINS-CI.ORG A DISTRIBUTION MECHANISM JENKINS HAS ITS OWN
43. ****************** AND ****************** PROS CONS
44. PLATFORM INDEPENDANT COMPILED PROS
45. JVM ONLY DISTRIBUTION CONS SPECIFIC MECHANISM PACKAGING
46. Can we do DOCKER ? a Plugin Architecture *********************** WITH ***********************
47. Docker ****************** WITH ****************** A PLUGIN ARCHITECTURE
48. A STANDARD INTERFACE
49. A STANDARD INTERFACE DOCKER VOLUMES ├ config │ ├ config.json │ ├ key ├ src │ ├ main.go │ ├ main_test.go │ ├ Makefile DOCKER PLUGIN INPUT & OUPUT
50. A STANDARD INTERFACE DOCKER VOLUMES ├ config │ ├ config.json │ ├ key ├ src │ ├ main.go │ ├ main_test.go │ ├ Makefile DOCKER PLUGIN A DOCKER PLUGIN B ├ binaries │ ├ main_darwin_amd64 │ ├ main_linux_x86 INPUT & OUPUT
51. A STANDARD INTERFACE ENVIRONMENT VARIABLES docker run my_plugin -e URL=github.com -e DEPTH=3 INPUT ONLY
52. A STANDARD INTERFACE DOCKER EXIT CODE OUTPUT ONLY docker run my_image echo $?
53. A LOADING MECHANISM
54. A LOADING MECHANISM DOCKER RUN docker run my_plugin
55. A PACKAGING MECHANISM
56. A PACKAGING MECHANISM DOCKER BUILD DOCKER IMAGES DOCKERFILES
57. A DISTRIBUTION MECHANISM
58. A DISTRIBUTION MECHANISM DOCKER HUB HUB.DOCKER.COM FOR PUBLIC PLUGINS
59. A DISTRIBUTION MECHANISM ANY DOCKER REGISTRY FOR PUBLIC AND PRIVATE PLUGINS (OR DOCKER DISTRIBUTION NOW)
60. ****************** VS ****************** CLASSIC PLUGIN Docker ****************** VS ****************** ARCHITECTURE
61. NEED TO LEARN ONLY ONE LANGUAGE BUT TIED TO THIS LANGUAGE LANGUAGE AGNOSTIC PROGRAMMING LANGUAGE CLASSIC DOCKER
62. NEED TO BE WRAPPED CAN BE NATIVE LANGUAGE CAN BE CHOSEN ACCORDINGLY NATIVE TOOLS CLASSIC DOCKER
63. VERY SPECIALIZED INTERFACES HARD TO MAINTAIN RETRO-COMPATIBILITY SIMPLE INTERFACES BUT ALSO PERMISSIVE COMPATIBILITY CLASSIC DOCKER
64. MANUALLY IMPLEMENTED DOCKER ECOSYSTEM CAN BE TRICKY WITH VOLUMES SCALING CLASSIC DOCKER
65. GIT PLUGIN ******************* WITH ******************* AN EXAMPLE COMPARISON
66. THE COMPETITION RETRIEVE SOURCE CODE Git URL Git Branch SSH Private key Source Code ?
67. JENKINS
68. JENKINS ~600 LINES OF CODE TO DO ‘GIT CLONE’
69. JENKINS ~600 LINES OF CODE TO DO ‘GIT CLONE’
70. DOCKER ├ private-key ├ out │ ├ main.go │ ├ main_test.go │ ├ Makefile GIT PLUGIN CONTAINER $SCM_URL $SCM_BRANCH
71. DOCKER FROM debian:jessie RUN apt-get update \ && apt-get install -y git ADD run.sh /run.sh RUN echo "IdentityFile /private-key" \ >> /etc/ssh/ssh_config CMD ["/run.sh"] #!/bin/bash git clone "$SCM_URL" --recursive /out cd /out && git checkout "$SCM_BRANCH" Dockerfile run.sh
72. NOW THAT WE HAVE 1 PLUGIN, HOW CAN WE ORCHESTRATE MULTIPLE PLUGINS ?
73. BUILD PIPELINE EXAMPLE 1. FETCH SOURCE CODE 2. COMPILE & PACKAGE 3. TEST IT
74. BUILD PIPELINE EXAMPLE ORCHESTRATOR CONTAINER
75. GIT PLUGIN CONTAINER ├ private-key $SCM_URL $SCM_BRANCH BUILD PIPELINE EXAMPLE ORCHESTRATOR CONTAINER docker run git_plugin
76. GIT PLUGIN CONTAINER ├ private-key ├ src │ ├ Main.java │ ├ pom.xml $SCM_URL $SCM_BRANCH BUILD PIPELINE EXAMPLE ORCHESTRATOR CONTAINER docker run git_plugin
77. GIT PLUGIN CONTAINER ├ private-key ├ src │ ├ Main.java │ ├ pom.xml $SCM_URL $SCM_BRANCH BUILD PIPELINE EXAMPLE MAVEN PLUGIN CONTAINER ORCHESTRATOR CONTAINER docker run git_plugin docker run maven_plugin
78. GIT PLUGIN CONTAINER ├ private-key ├ src │ ├ Main.java │ ├ pom.xml $SCM_URL $SCM_BRANCH BUILD PIPELINE EXAMPLE MAVEN PLUGIN CONTAINER ├ output │ ├ main.war ORCHESTRATOR CONTAINER docker run git_plugin docker run maven_plugin
79. GIT PLUGIN CONTAINER ├ private-key ├ src │ ├ Main.java │ ├ pom.xml $SCM_URL $SCM_BRANCH BUILD PIPELINE EXAMPLE MAVEN PLUGIN CONTAINER SELENIUM PLUGIN CONTAINER ├ output │ ├ main.war ORCHESTRATOR CONTAINER docker run git_plugin docker run maven_plugin docker run selenium_plugin echo $?
80. GIT PLUGIN CONTAINER ├ private-key ├ src │ ├ Main.java │ ├ pom.xml $SCM_URL $SCM_BRANCH MERCURIAL INSTEAD OF GIT ? MAVEN PLUGIN CONTAINER SELENIUM PLUGIN CONTAINER ├ output │ ├ main.war ORCHESTRATOR CONTAINER docker run git_plugin docker run maven_plugin docker run selenium_plugin echo $?
81. HG PLUGIN CONTAINER ├ private-key ├ src │ ├ Main.java │ ├ pom.xml $SCM_URL $SCM_BRANCH MERCURIAL INSTEAD OF GIT ? MAVEN PLUGIN CONTAINER SELENIUM PLUGIN CONTAINER ├ output │ ├ main.war ORCHESTRATOR CONTAINER docker run hg_plugin docker run maven_plugin docker run selenium_plugin echo $?
82. HG PLUGIN CONTAINER ├ private-key ├ src │ ├ Main.java │ ├ pom.xml $SCM_URL $SCM_BRANCH MAVEN PLUGIN CONTAINER SELENIUM PLUGIN CONTAINER ├ output │ ├ main.war ORCHESTRATOR CONTAINER docker run hg_plugin docker run maven_plugin docker run selenium_plugin echo $? GRADLE INSTEAD OF MAVEN ?
83. HG PLUGIN CONTAINER ├ private-key ├ src │ ├ Main.java │ ├ pom.xml $SCM_URL $SCM_BRANCH GRADLE INSTEAD OF MAVEN ? GRADLE PLUGIN CONTAINER SELENIUM PLUGIN CONTAINER ├ output │ ├ main.war ORCHESTRATOR CONTAINER docker run hg_plugin docker run gradle_plugin docker run selenium_plugin echo $?
84. HG PLUGIN CONTAINER ├ private-key ├ src │ ├ Main.java │ ├ pom.xml $SCM_URL $SCM_BRANCH ANOTHER BUILD WORKFLOW ? GRADLE PLUGIN CONTAINER SELENIUM PLUGIN CONTAINER ├ output │ ├ main.war ORCHESTRATOR CONTAINER docker run hg_plugin docker run gradle_plugin docker run selenium_plugin echo $?
85. HG PLUGIN CONTAINER ├ private-key ├ src │ ├ Main.java │ ├ pom.xml $SCM_URL $SCM_BRANCH ANOTHER BUILD WORKFLOW ? GRADLE PLUGIN CONTAINER SELENIUM PLUGIN CONTAINER ├ output │ ├ main.war YOUR OWN ORCHESTRATOR CONTAINER docker run hg_plugin docker run gradle_plugin docker run selenium_plugin echo $?
86. Our Experiment BAZOOKa ******************* WITH *******************
87. Our Experiment BAZOOKa ******************* WITH *******************
89. CONCLUSION *************************************************
90. DOCKER MICROPLUGINS EASY TO SETUP EXTENSIBLE PERMISSIVE EASY TO DISTRIBUTE LANGUAGE AGNOSTIC
91. QUESTIONS?