Slide 1
Slide 1 text
Dive into
BuildKit & LLB
with Go
Gopher Conference Tokyo 2019 Spring
April 12, 2019 @ po3rin
Slide 2
Slide 2 text
QBHF
ɾ#FDPNFBMJUUMFNPSFGBNJMJBSXJUI
CVJMEJOHDPOUBJOFSJNBHFCZSFBEJOH(P
ɾ"GPPUIPMEUPDSFBUFDPOUBJOFSSFMBUFE
UPPMTXJUI(P
Ὂ the benefits of this talk
Slide 3
Slide 3 text
@po3rin
Software Enginner @ Shiroyagi Corporation
QBHF
Golang / Python / Rust /
Docker / AWS / Elasticsearch
Slide 4
Slide 4 text
QBHF
Introduction
\
Slide 5
Slide 5 text
QBHF
.PCZJTBOPQFOGSBNFXPSLDSFBUFECZ
%PDLFSUPBTTFNCMFTQFDJBMJ[FE
DPOUBJOFSTZTUFNTXJUIPVUSFJOWFOUJOH
UIFXIFFM
What is Moby
Ὂ
IUUQTHJUIVCDPNNPCZ
Slide 6
Slide 6 text
QBHF
What is BuildKit
#VJME,JUJTBUPPMLJUGPS
DPOWFSUJOHTPVSDFDPEFUP
CVJMEBSUJGBDUTJOBO
F⒏DJFOU
FYQSFTTJWFBOE
SFQFBUBCMFNBOOFS
Ὂ
IUUQTHJUIVCDPNNPCZNPCZJTTVFT
Slide 7
Slide 7 text
QBHF
CVJMELJUE
CVJMEEFBNPO
CVJMEDUM
DPNNVOJDBUFT
XJUICVJMELJUE
H31$
buildctl & buildkitd
Ὂ
5IFCVJMELJUEEBFNPOTVQQPSUTUXPXPSLFSCBDLFOET0$* SVOD
BOEDPOUBJOFSE
Slide 8
Slide 8 text
QBHF
BuildKit is written in Go !!
Ὂ
Slide 9
Slide 9 text
QBHF
Reading Go source code of BuildKit
\
Slide 10
Slide 10 text
QBHF
A foothold to read Go source code (1)
Ὂ
CVJMEDUMVTFT
HJUIVCDPNVSGBWFDMJ
QBDLBHF
Slide 11
Slide 11 text
QBHF
A foothold to read Go source code (2)
Ὂ
CVJMEDUMCVJMEDBMMT
DMJFOU4PMWFNFUIPE8IBUJT
UIFBSHVNFOUDBMMFEMMC
Slide 12
Slide 12 text
QBHF
7FSUFY
&EHF
7FSUFY
"UUIFDPSFPG#VJME,JUJTBOFXMPX
MFWFMCVJMEEFpOJUJPOGPSNBUDBMMFE
--# MPXMFWFMCVJMEFS
--#SFBMJ[FTQBSBMMFMFYFDVUJPOPG
JOTUSVDUJPOTCZFYQSFTTJOH
EFQFOEFODZBNPOHJOTUSVDUJPOTPG
%PDLFSpMFBT%"( EJSFDUFEBDZDMJD
HSBQI
Ὂ LLB
Slide 13
Slide 13 text
QBHF
TPMWFSQEPQTQSPUP
DPOUBJOTUIFTQFDJpDBUJPO
PGUIF--#TUSVDUVSF
Slide 14
Slide 14 text
QBHF
FYBNQMF
Slide 15
Slide 15 text
QBHF
*OQVUTSFQSFTFOUJOQVUFEHFT
GSPNPUIFS0QT5IJT
SFQSFTFOUTUIF%"(
TUSVDUVSF
*OQVUT
Slide 16
Slide 16 text
QBHF
MMCEPUQBDLBHFMFUTZPV
UPDPOWFSU#VJME,JU--#
UPEPUMBOHVBHFUP
BOBMJ[F
Ὂ Look inside of LLB (1)
Slide 17
Slide 17 text
QBHF
Ὂ Look inside of LLB (2)
Slide 18
Slide 18 text
QBHF
CVJMELJUE
CVJMEDUM
H31$
buildctl & buildkitd
Ὂ
4PMWF
Slide 19
Slide 19 text
QBHF
5IF--#JTHFOFSBUFECZ
DPOWFSUJOHBOBSCJUSBSZJOQVU
'JOBMMZ
CVJMEEFBNPOPVUQVUTCVJME
BSUJGBDUTJOUIFTQFDJpFEFYQPSUFS
GPSNBU UIFSJHIUpHVSFFYQMBJOT
FYBNQMFPG%PDLFS
'SPOUFOE
--#
%PDLFSpMF
HSQD
XPSLFS CVJMELJUE
FYQPSU
%PDLFSJNBHF
DPOWFSU
Build procedure
Ὂ
Slide 20
Slide 20 text
QBHF
Generetes LLB From Dockerfile (1)
Ὂ
'SPOUFOE
--#
%PDLFSpMF
DPOWFSU
Slide 21
Slide 21 text
QBHF
'PSFYBNQMF
MMCEPUVTFT
%PDLFSpMF--#GVODJOUFSOBMMZ
Generetes LLB From Dockerfile (2)
Ὂ
Slide 22
Slide 22 text
QBHF
Inside Dockerfile2LLB
Ὂ
%PDLFSpMF
%PDLFSpMF"45
#VJMEBCMF4UBHFT
--#
'JSTUPG"MM
%PDLFSpMF1BSTFS
HFOFSBUFT%PDLFSpMF"45'SPN
UIFSF
JUDSFBUFT#--UISPVHI
WBSJPVTQSPDFTTFT
Slide 23
Slide 23 text
QBHF
%PDLFSpMF
%PDLFSpMF"45
#VJMEBCMF4UBHFT
--#
Slide 24
Slide 24 text
QBHF
Node Construction
Ὂ
Slide 25
Slide 25 text
QBHF
6TJOHUIJTGVODUJPO
ZPVDBOBMTP
DSFBUF%PDLFSpMF-JOUFSGSPN"45
Slide 26
Slide 26 text
QBHF
%PDLFSpMF
%PDLFSpMF"45
#VJMEBCMF4UBHFT
--#
Slide 27
Slide 27 text
QBHF
DMJFOUMMCQBDLBHFQSPWJEFTUIF--#$MJFOU"1*UP
HFOFSBUF--#5IJTQSPWJEFTCVJMEJOHCMPDLTUIJT.
5IFDPEFCFMPXJTBOFYBNQMFPGHFOFSBUJOHMMC
VTJOHMMCDMJFOU
%PDLFSpMF
%PDLFSpMF"45
#VJMEBCMF4UBHFT
--#
FYBNQMFHJUIVCDPNNPCZCVJMELJUFYBNQMFT
Slide 28
Slide 28 text
QBHF
Develop BuildKit LLB frontend with Go
\
Slide 29
Slide 29 text
QBHF
'PDVTJOHPOUIFCVJMEqPX
JGZPV
DBOHFOFSBUF--#
'SPOUFOEJT
HPPEGPSBOZUIJOH
'SPOUFOE
--#
%PDLFSpMF
HSQD
#VJMELJUE CVJMEEFBNPO
FYQPSU
%PDLFSJNBHF
DPOWFSU
Slide 30
Slide 30 text
QBHF
'SPOUFOE
--#
%PDLFSpMF
HSQD
#VJMELJUE CVJMEEFBNPO
FYQPSU
%PDLFSJNBHF
DPOWFSU
0SJHJOBM'SPOUFOE
*OPUIFSXPSET
JUJTQPTTJCMFUP
JNQMFNFOUBOEQVCMJTIUIF
PSJHJOBMGSPOUFOE
Slide 31
Slide 31 text
QBHF
(PDLFSpMFJTB#VJME,JU
--#GSPOUFOEUIBUDBO
HFOFSBUFNVMUJTUBHFCVJMU
(P"1*JNBHFTJOKVTUUISFF
MJOFT
Example of Original BuildKit LLB frontend
Ὂ
Slide 32
Slide 32 text
QBHF
'SPOUFOE
(PDLFSpMF
(PTFSWFS%PDLFSJNBHF
Slide 33
Slide 33 text
QBHF
Easy way to Create Custom Frontend
Ὂ
$SFBUFDPOWFSTJPOQSPDFTTGSPN
BSCJUSBSZGPSNBUUP--#
6TF(BUFXBZ"1*QSPWJEFECZ
CVJMELJU
#JOBSZJ[FUIFQSPDFTTBOEQVCMJTIJU
BTB%PDLFSJNBHF
1
2
3
Slide 34
Slide 34 text
QBHF
Use BuildKit Gateway API (1)
Ὂ
&YBNQMFPG$BMMJOH
3VO'SPN&OWJSPONFOUJONBJO
QBDLBHF
Slide 35
Slide 35 text
QBHF
#VJME'VODOFFETUP
HFUBDPOpHVSBUJPO
pMFJOPSJHJOBM
GPSNBU
DPOWFSUJUUP
--#
BOEQBTTJUUP
4PMWF
UPFYFDVUF
CVJMEXJUIXPSLFS
Use BuildKit Gateway API (2)
Ὂ
Slide 36
Slide 36 text
QBHF
--#$MJFOU"1*DPNCJOFT4PVSDFBOE3VOPQFSBUJPOT
3VO ʜ
3PPU
4UBUF &YFD4UBUF
MMC*NBHF ʜ
Create conversion process from arbitrary format to LLB (2)
Ὂ
Slide 37
Slide 37 text
"EE.PVOU ʜ
QBHF
"EE.PVOUBMMPXTNPVOUJOHPGpMFTGSPNPUIFS0Q"MJUUMFDPNQMJDBUFE
3VO ʜ
3PPU
3VO ʜ
Create conversion process from arbitrary format to LLB (3)
Ὂ
Slide 38
Slide 38 text
QBHF
.FSHFEpMF0QJO.BSDI
Create conversion process from arbitrary format to LLB (3)
Ὂ
Slide 39
Slide 39 text
QBHF
Binaryize the process and publish it as a Docker image
Ὂ
Slide 40
Slide 40 text
QBHF
'SPOUFOE
(PDLFSpMF
(PTFSWFS%PDLFSJNBHF
QPSJOHPDLFSMBUFTU
-PBEFEBTBDPOUBJOFSJNBHF
GSPNSFHJTUSZBOESVOJOB
TFDVSFTBOECPYKVTUBTZPVS
SFHVMBSDPOUBJOFST
MPBEJNBHFDPOpH
Slide 41
Slide 41 text
QBHF
Conclusion
\
Slide 42
Slide 42 text
QBHF
Conclusion
Ὂ
*GZPVDBOSFBE(P
ZPVDBOHFUBEFFQFSVOEFSTUBOEJOHPGUIF
DPOUBJOFSUFDIOPMPHZZPVBSFVTJOHDBTVBMMZ"OEZPVDBOIBDLUIF
JNBHFCVJMEVTJOH(P
Slide 43
Slide 43 text
QBHF
Reference
Ὂ
#VJME,JU3FQPTJUPSZ
IUUQTHJUIVCDPNNPCZCVJMELJU
#VJME,JUʹΑΔߴͰηΩϡΞͳΠϝʔδϏϧυ εϥΠυ
IUUQTXXXTMJEFTIBSFOFU"LJIJSP4VEBCVJMELJU
(FUUJOH4UBSUFE#VJME,JU CVJMEDUMCVJMELJUE)BOETPO
IUUQTHFPSHFNBDSPSFQPTUTHFUUJOHTUBSUFEXJUICVJMELJU
Slide 44
Slide 44 text
QBHF
Follow @po3rin !!
Ὂ
Slide 45
Slide 45 text
Dive into
BuildKit & LLB
with Go
Gopher Conference Tokyo 2019 Spring
April 12, 2019 @ po3rin
Slide 46
Slide 46 text
QBHF
Additional Talk
\
Slide 47
Slide 47 text
QBHF
Use BuildKit Gateway API (3)
Ὂ
3VO'SPN&OWJSPONFOUVTFT
--##SJHEHF$MJFOUJOUFSOBMZ
(BUFXBZ"1*
'SPOUFOE
3VOT*OEPDLFSDPOUBJOFS
Slide 48
Slide 48 text
QBHF
Use BuildKit Gateway API (4)
Ὂ
4PMWF
$MJFOU FYCVJMEDUM
$POUPMMFS
(BUFXBZ"1*
'SPOUFOE
3VOT*OEPDLFSDPOUBJOFS