Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Microplugins with Docker
Search
Julien Vey
April 17, 2015
Programming
5
970
Microplugins with Docker
Julien Vey
April 17, 2015
Tweet
Share
More Decks by Julien Vey
See All by Julien Vey
Du Dev à la Prod... et ensuite ?
julienvey
0
26
On-Demand CI/CD with Gitlab and Kubernetes
julienvey
0
37
From CI to CD in a microservice architecture
julienvey
1
65
Terraform
julienvey
1
260
OpenStack Solum Presentation
julienvey
0
130
Use OpenStack as a Vagrant Provider
julienvey
0
180
Docker and its Ecosystem
julienvey
8
1.3k
Introducing Solum: Application Development in OpenStack
julienvey
1
840
Docker, Get used to it !
julienvey
7
3.1k
Other Decks in Programming
See All in Programming
[SRE NEXT] 複雑なシステムにおけるUser Journey SLOの導入
yakenji
0
810
Streamlitで実現できるようになったこと、実現してくれたこと
ayumu_yamaguchi
2
230
レトロゲームから学ぶ通信技術の歴史
kimkim0106
0
140
状態遷移図を書こう / Sequence Chart vs State Diagram
orgachem
PRO
3
290
slogパッケージの深掘り
integral0515
0
160
顧客の画像データをテラバイト単位で配信する 画像サーバを WebP にした際に起こった課題と その対応策 ~継続的な取り組みを添えて~
takutakahashi
4
1.4k
No Install CMS戦略 〜 5年先を見据えたフロントエンド開発を考える / no_install_cms
rdlabo
0
370
AI Agent 時代のソフトウェア開発を支える AWS Cloud Development Kit (CDK)
konokenj
6
1k
AIともっと楽するE2Eテスト
myohei
9
3.2k
PHPUnitの限界をPlaywrightで補完するテストアプローチ
yuzneri
0
350
GPUを計算資源として使おう!
primenumber
1
290
Advanced Micro Frontends: Multi Version/ Framework Scenarios
manfredsteyer
PRO
0
110
Featured
See All Featured
How to Create Impact in a Changing Tech Landscape [PerfNow 2023]
tammyeverts
53
2.9k
Building Applications with DynamoDB
mza
95
6.5k
Agile that works and the tools we love
rasmusluckow
329
21k
Making the Leap to Tech Lead
cromwellryan
134
9.4k
I Don’t Have Time: Getting Over the Fear to Launch Your Podcast
jcasabona
33
2.4k
No one is an island. Learnings from fostering a developers community.
thoeni
21
3.4k
Visualization
eitanlees
146
16k
Embracing the Ebb and Flow
colly
86
4.8k
StorybookのUI Testing Handbookを読んだ
zakiyama
30
5.9k
Rails Girls Zürich Keynote
gr2m
95
14k
Large-scale JavaScript Application Architecture
addyosmani
512
110k
Measuring & Analyzing Core Web Vitals
bluesmoon
7
530
Transcript
Microplugins DOCKER ******************* WITH *******************
Freelance « DevOps » JULIEN VEY ERIC BELLEMON ********************************************* at
bywan @julienvey @haklop
BY NOW, WE THINK YOU KNOW WHAT IS DOCKER WHAT
?
DOCKER HOW ? Patterns *******************************************
MICROSERVICES
A MICROSERVICE IS USUALLY A PERSISTANT SERVICE YOU CAN ACCESS
VIA AN API
GO GO PYTHON RUBY NODE JAVA JAVA PHP ANY LANGUAGE
UBUNTU REDHAT DEBIAN CENTOS FEDORA FEDORA DEBIAN CENTOS ANY DISTRIBUTION
AWS GCE COREOS ANY WHERE
WHY CONTAINERS AND NOT VMS? CONTAINERS ARE LIGHTWEIGHT SPAWN FAST
USE LAYERS
DOCKER Tips n’ Tricks ******************************************* HOW ?
DOCKER IN DOCKER
THE DOCKER MODEL DOCKER DAEMON UNIX SOCKET DOCKER REMOTE API
DOCKER CLI creates an which exposes the that can be used by the
FIRST APPROACH
DOCKER IN DOCKER IN DOCKER …
docker run --privileged -t -i jpetazzo/dind SPECIAL DOCKER IMAGE
docker run --privileged -t -i jpetazzo/dind PRIVILEGED MODE
docker run --privileged -t -i jpetazzo/dind PRIVILEGED MODE
IT RAISES SOME PROBLEMS INIT SYSTEM DISK SPACE CUSTOM IMAGE
SECOND APPROACH
DOCKER IN DOCKER IN DOCKER … DOCKER HOST DOCKER SOCKET
docker run -v /var/run/docker.sock:/var/run/docker.sock [your image] DOCKER SOCKET AS A
DOCKER VOLUME
docker run -v /var/run/docker.sock:/var/run/docker.sock [your image] DOCKER SOCKET AS A
DOCKER VOLUME
NO PRIVILEGED MODE ONLY ONE DOCKER DAEMON (INSTEAD OF ONE
PER CONTAINER)
What about ? MICRO PLUGINS *********************************************
MICROSERVICE: A SMALL SERVICE RUNNING IN A CONTAINER
MICROPLUGIN: A SMALL PLUGIN RUNNING IN A CONTAINER
What is a ? Plugin Architecture *********************************************
A STANDARD INTERFACE A LOADING MECHANISM A PACKAGING MECHANISM A
DISTRIBUTION MECHANISM
VAGRANT ******************* WITH ******************* A PLUGIN ARCHITECTURE RUBY EXAMPLE
A STANDARD INTERFACE require 'vagrant' module VagrantPlugins module Openstack class
Provider < Vagrant.plugin('2', :provider) RUBY CLASSES
A LOADING MECHANISM RUBY DYNAMIC LOADING SOURCES INSTALLED IN VAGRANT
BUNDLE DIR
A PACKAGING MECHANISM RUBY GEM PACKAGE CODE AND DOCUMENTATION DESCRIBE
DEPENDENCIES
A DISTRIBUTION MECHANISM RUBYGEMS.ORG
****************** AND ****************** PROS CONS
PLATFORM INDEPENDANT PACKAGING AND DISTRIBUTION ARE NOT SPECIFIC TO VAGRANT
PROS
RUBY ONLY DEPENDENCY MANAGEMENT CAN BE HARD CONS
JENKINS ****************** WITH ****************** A PLUGIN ARCHITECTURE JAVA EXAMPLE
A STANDARD INTERFACE package jenkins.plugins.git; public abstract class AbstractGitSCMSource extends
SCMSource { protected SCMRevision retrieve(…) protected void retrieve(…) } JAVA INTERFACES & CLASSES
A LOADING MECHANISM JAVA CLASSLOADING
A PACKAGING MECHANISM JPI/HPI = JENKINS/HUDSON PLUG IN CUSTOM PACKAGING
(JPI/HPI)
UPDATES.JENKINS-CI.ORG A DISTRIBUTION MECHANISM JENKINS HAS ITS OWN
****************** AND ****************** PROS CONS
PLATFORM INDEPENDANT COMPILED PROS
JVM ONLY DISTRIBUTION CONS SPECIFIC MECHANISM PACKAGING
Can we do DOCKER ? a Plugin Architecture *********************** WITH
***********************
Docker ****************** WITH ****************** A PLUGIN ARCHITECTURE
A STANDARD INTERFACE
A STANDARD INTERFACE DOCKER VOLUMES ├ config │ ├ config.json
│ ├ key ├ src │ ├ main.go │ ├ main_test.go │ ├ Makefile DOCKER PLUGIN INPUT & OUPUT
A STANDARD INTERFACE DOCKER VOLUMES ├ config │ ├ config.json
│ ├ key ├ src │ ├ main.go │ ├ main_test.go │ ├ Makefile DOCKER PLUGIN A DOCKER PLUGIN B ├ binaries │ ├ main_darwin_amd64 │ ├ main_linux_x86 INPUT & OUPUT
A STANDARD INTERFACE ENVIRONMENT VARIABLES docker run my_plugin -e URL=github.com
-e DEPTH=3 INPUT ONLY
A STANDARD INTERFACE DOCKER EXIT CODE OUTPUT ONLY docker run
my_image echo $?
A LOADING MECHANISM
A LOADING MECHANISM DOCKER RUN docker run my_plugin
A PACKAGING MECHANISM
A PACKAGING MECHANISM DOCKER BUILD DOCKER IMAGES DOCKERFILES
A DISTRIBUTION MECHANISM
A DISTRIBUTION MECHANISM DOCKER HUB HUB.DOCKER.COM FOR PUBLIC PLUGINS
A DISTRIBUTION MECHANISM ANY DOCKER REGISTRY FOR PUBLIC AND PRIVATE
PLUGINS (OR DOCKER DISTRIBUTION NOW)
****************** VS ****************** CLASSIC PLUGIN Docker ****************** VS ****************** ARCHITECTURE
NEED TO LEARN ONLY ONE LANGUAGE BUT TIED TO THIS
LANGUAGE LANGUAGE AGNOSTIC PROGRAMMING LANGUAGE CLASSIC DOCKER
NEED TO BE WRAPPED CAN BE NATIVE LANGUAGE CAN BE
CHOSEN ACCORDINGLY NATIVE TOOLS CLASSIC DOCKER
VERY SPECIALIZED INTERFACES HARD TO MAINTAIN RETRO-COMPATIBILITY SIMPLE INTERFACES BUT
ALSO PERMISSIVE COMPATIBILITY CLASSIC DOCKER
MANUALLY IMPLEMENTED DOCKER ECOSYSTEM CAN BE TRICKY WITH VOLUMES SCALING
CLASSIC DOCKER
GIT PLUGIN ******************* WITH ******************* AN EXAMPLE COMPARISON
THE COMPETITION RETRIEVE SOURCE CODE Git URL Git Branch SSH
Private key Source Code ?
JENKINS
JENKINS ~600 LINES OF CODE TO DO ‘GIT CLONE’
JENKINS ~600 LINES OF CODE TO DO ‘GIT CLONE’
DOCKER ├ private-key ├ out │ ├ main.go │ ├
main_test.go │ ├ Makefile GIT PLUGIN CONTAINER $SCM_URL $SCM_BRANCH
DOCKER FROM debian:jessie RUN apt-get update \ && apt-get install
-y git ADD run.sh /run.sh RUN echo "IdentityFile /private-key" \ >> /etc/ssh/ssh_config CMD ["/run.sh"] #!/bin/bash git clone "$SCM_URL" --recursive /out cd /out && git checkout "$SCM_BRANCH" Dockerfile run.sh
NOW THAT WE HAVE 1 PLUGIN, HOW CAN WE ORCHESTRATE
MULTIPLE PLUGINS ?
BUILD PIPELINE EXAMPLE 1. FETCH SOURCE CODE 2. COMPILE &
PACKAGE 3. TEST IT
BUILD PIPELINE EXAMPLE ORCHESTRATOR CONTAINER
GIT PLUGIN CONTAINER ├ private-key $SCM_URL $SCM_BRANCH BUILD PIPELINE EXAMPLE
ORCHESTRATOR CONTAINER docker run git_plugin
GIT PLUGIN CONTAINER ├ private-key ├ src │ ├ Main.java
│ ├ pom.xml $SCM_URL $SCM_BRANCH BUILD PIPELINE EXAMPLE ORCHESTRATOR CONTAINER docker run git_plugin
GIT PLUGIN CONTAINER ├ private-key ├ src │ ├ Main.java
│ ├ pom.xml $SCM_URL $SCM_BRANCH BUILD PIPELINE EXAMPLE MAVEN PLUGIN CONTAINER ORCHESTRATOR CONTAINER docker run git_plugin docker run maven_plugin
GIT PLUGIN CONTAINER ├ private-key ├ src │ ├ Main.java
│ ├ pom.xml $SCM_URL $SCM_BRANCH BUILD PIPELINE EXAMPLE MAVEN PLUGIN CONTAINER ├ output │ ├ main.war ORCHESTRATOR CONTAINER docker run git_plugin docker run maven_plugin
GIT PLUGIN CONTAINER ├ private-key ├ src │ ├ Main.java
│ ├ pom.xml $SCM_URL $SCM_BRANCH BUILD PIPELINE EXAMPLE MAVEN PLUGIN CONTAINER SELENIUM PLUGIN CONTAINER ├ output │ ├ main.war ORCHESTRATOR CONTAINER docker run git_plugin docker run maven_plugin docker run selenium_plugin echo $?
GIT PLUGIN CONTAINER ├ private-key ├ src │ ├ Main.java
│ ├ pom.xml $SCM_URL $SCM_BRANCH MERCURIAL INSTEAD OF GIT ? MAVEN PLUGIN CONTAINER SELENIUM PLUGIN CONTAINER ├ output │ ├ main.war ORCHESTRATOR CONTAINER docker run git_plugin docker run maven_plugin docker run selenium_plugin echo $?
HG PLUGIN CONTAINER ├ private-key ├ src │ ├ Main.java
│ ├ pom.xml $SCM_URL $SCM_BRANCH MERCURIAL INSTEAD OF GIT ? MAVEN PLUGIN CONTAINER SELENIUM PLUGIN CONTAINER ├ output │ ├ main.war ORCHESTRATOR CONTAINER docker run hg_plugin docker run maven_plugin docker run selenium_plugin echo $?
HG PLUGIN CONTAINER ├ private-key ├ src │ ├ Main.java
│ ├ pom.xml $SCM_URL $SCM_BRANCH MAVEN PLUGIN CONTAINER SELENIUM PLUGIN CONTAINER ├ output │ ├ main.war ORCHESTRATOR CONTAINER docker run hg_plugin docker run maven_plugin docker run selenium_plugin echo $? GRADLE INSTEAD OF MAVEN ?
HG PLUGIN CONTAINER ├ private-key ├ src │ ├ Main.java
│ ├ pom.xml $SCM_URL $SCM_BRANCH GRADLE INSTEAD OF MAVEN ? GRADLE PLUGIN CONTAINER SELENIUM PLUGIN CONTAINER ├ output │ ├ main.war ORCHESTRATOR CONTAINER docker run hg_plugin docker run gradle_plugin docker run selenium_plugin echo $?
HG PLUGIN CONTAINER ├ private-key ├ src │ ├ Main.java
│ ├ pom.xml $SCM_URL $SCM_BRANCH ANOTHER BUILD WORKFLOW ? GRADLE PLUGIN CONTAINER SELENIUM PLUGIN CONTAINER ├ output │ ├ main.war ORCHESTRATOR CONTAINER docker run hg_plugin docker run gradle_plugin docker run selenium_plugin echo $?
HG PLUGIN CONTAINER ├ private-key ├ src │ ├ Main.java
│ ├ pom.xml $SCM_URL $SCM_BRANCH ANOTHER BUILD WORKFLOW ? GRADLE PLUGIN CONTAINER SELENIUM PLUGIN CONTAINER ├ output │ ├ main.war YOUR OWN ORCHESTRATOR CONTAINER docker run hg_plugin docker run gradle_plugin docker run selenium_plugin echo $?
Our Experiment BAZOOKa ******************* WITH *******************
Our Experiment BAZOOKa ******************* WITH *******************
None
CONCLUSION *************************************************
DOCKER MICROPLUGINS EASY TO SETUP EXTENSIBLE PERMISSIVE EASY TO DISTRIBUTE
LANGUAGE AGNOSTIC
QUESTIONS?