JAWS-UG コンテナ支部 #13 で発表した資料です。re:Invent 2019 でオープンソースとして発表された Firecracker とは何なのか、AWS はなぜ軽量 VM を作り、私たちはそこから何を享受できるのかを考えます。
Firecracker ͱԿ͔AWS ͳͥܰྔ VM Λ࡞Γࢲ͔ͨͪͦ͜ΒԿΛڗडͰ͖Δͷ͔JAWS-UG ίϯςφࢧ෦ #13 Dec 21, 2018Ryo NAKAMARU, SUPINF Inc.
View Slide
SUPINF IncτϐοΫ2• AWS ͳͥ Firecracker Λ࡞ͬͨͷ͔• Firecracker ͱԿ͔• ࢲͨͪ Firecracker ʹԿΛظ͠ɺͲ͏ߩݙͰ͖Δͷ͔• ͜Ε͔Β Fargate Ͳ͏มΘΔͷ͔
SUPINF Inc3AWS ͳͥ Firecracker Λ࡞ͬͨͷ͔
SUPINF IncLambda ैདྷ EC2 ϕʔε4ηΩϡϦςΟඞਢʂ࣮ߦίϯςΩετɺ·ͣ EC2 Ͱ͠Α͏ɹHardwareɹHost OSɹHypervisorɹSandboxɹLambda runtime← EC2ɹGuest OSϚϧνςφϯτAWS ΞΧϯτؔʢϨϕϧʣɹOur codeɾݫີͰ࣮͋Δ ख๏ɾͳΔͰϦϦʔε ͔ͨͬͨ͠
SUPINF IncLambda ैདྷ EC2 ϕʔε5ඞવతʹ EC2 ΠϯελϯεͷىಈɺͦͷͨΊͷϦιʔε֬อ͕ى͜ΔɹɹɹɹɹɹλɹɹɹλɹɹɹλɹA ΞΧϯτɹ ɹB ΞΧϯτ C ΞΧϯτɾىಈ͕͍͜ͱ͕͋ΔɾϦιʔεར༻ʹແବ͕ଟ͍ͳͷͰɾɾɹɹɹλɹɹɹλ
SUPINF Inc࣍ੈͷαʔόʔϨεج൫͕΄͍͠6 EC2 Λىಈ͍ͯͯ͠ඇޮ• ىಈ࣌ؒ• Ϧιʔεར༻ޮ1 ϓϩηε͕ݫີʹͰ͖Ε͍͍
SUPINF IncFirecracker Ͱ࣮ݱ͢Δ৽ߏ7ܰྔͳ VM ͱɺߴͳ Hypervisor ͕͋ΕɾɾɹHardwareɹHost OSɹSandboxɹLambda runtimeɹOur codeϚϧνςφϯτؔɹHypervisorɹGuest OSɹHardwareɹHost OSɹHypervisorɹSandboxɹLambda runtimeɹGuest OSϚϧνςφϯτAWS ΞΧϯτؔʢϨϕϧʣɹOur codeݟ͠ʂ
SUPINF Inc8ɹɹɹɹɹλɹAɾฏۉىಈ࣌ؒͷॖɾϦιʔεΛ࠷େݶ༗ޮ׆༻Ͱ͖ΔFirecracker Ͱ࣮ݱ͢Δ৽ߏܰྔͳ VM ͱɺߴͳ Hypervisor ͕͋ΕɾɾɹɹɹλɹBɹɹɹλɹBɹɹɹλɹCɹɹɹλɹC..ɹ࣍ੈ HypervisorɹɹɹλɹD
SUPINF Inc2018 ݱࡏɺฏߦՔಇதɾɾʂ9EC2 ϞʔυɹHardwareɹHost OSɹHypervisorɹSandboxɹLambda runtimeɹGuest OSϚϧνςφϯτAWS ΞΧϯτؔɹOur codeɹHardwareɹHost OSɹSandboxɹLambda runtimeɹOur codeϚϧνςφϯτؔɹHypervisorɹGuest OSFirecracker Ϟʔυ➕
SUPINF Inc͜͜·Ͱͷ → youtube.com/watch?v=QdzV04T_kec10
SUPINF Inc11Firecracker ͱԿ͔
SUPINF Inc“ίϯςφͷͨΊͷ” QEMU ସͰ͢121 Firecracker ϓϩηεͰ 1 microVM Λ୲…ɹɹHardwareCPU, Storage,Network card, …ɹHost kernel KVMɹ ɹ ɹFirecrackerɹGuest kernelmicroVM͜Ε
SUPINF Inc13Firecracker VM Λىಈ͢Δͷ͕ࣄʂϢʔβʔ Lambda͏ͧʙLambda / Fargate ͜͏ಈ͍͍ͯΔʢໝʣ
SUPINF IncLambda / Fargate ͜͏ಈ͍͍ͯΔʢໝʣ14Ϣʔβʔ1. AWS APIαʔόʔϨε / ECSίϯτϩʔϧϓϨΠϯFirecracker VM Λىಈ͢Δͷ͕ࣄʂ* αʔόʔϨεଆઌͷಈըͰجຊߏΛެ։ࡁA Serverless Journey: AWS Lambda Under the Hoodhttps://www.youtube.com/watch?v=QdzV04T_kec
SUPINF Incɹ15ɹHost kernel KVMϢʔβʔFirecracker VM Λىಈ͢Δͷ͕ࣄʂϗετ͝ͱʹଘࡏ͢ΔͰ͋Ζ͏Ṗ Agent ʹ Lambda ͷൃՐΛґཔṖ Agent2. ൃՐґཔLambda / Fargate ͜͏ಈ͍͍ͯΔʢໝʣαʔόʔϨε / ECSίϯτϩʔϧϓϨΠϯ
SUPINF Incɹ16ɹHost kernel KVMϢʔβʔFirecracker VM Λىಈ͢Δͷ͕ࣄʂFirecrackerṖ Agent4. Firecracker ىಈ3. μϯϩʔυṖ Agent S3 / ECR ͔ΒϢʔβʔίʔυΛर͍ͭͭFirecracker ΛҰͭىಈʢ͜ͷϝϞϦ͕ 5 MiB ະຬʣLambda / Fargate ͜͏ಈ͍͍ͯΔʢໝʣαʔόʔϨε / ECSίϯτϩʔϧϓϨΠϯ
SUPINF Incɹ17ɹHost kernel KVMϢʔβʔFirecracker VM Λىಈ͢Δͷ͕ࣄʂFirecrackerṖ Agent5. VM ઃఆɾىಈࢦࣔϢʔβʔίʔυͷ࣋ͪࠐΈؚΊFirecracker ͕ VM ΛىಈʢVM Ͱ /sbin/init ϓϩηε͕ɹΔ·Ͱ͕ 125 ms ҎʣɹGuest kernelmicroVMApplicationLambda / Fargate ͜͏ಈ͍͍ͯΔʢໝʣαʔόʔϨε / ECSίϯτϩʔϧϓϨΠϯ
SUPINF Incͦͷଞಛͱɺͦͷඞવੑ18AWS ͕ҎԼͷػೳ܈Λ crosvm ʹՃ࣮ͨ͠ͷͳͥͩͱࢥ͍·͢ʁFirecrackerɹGuest kernelmicroVMApplicationɾࢦࣔ REST API Ͱड͚Δɾϝλσʔλ ཧɾσόΠεΤϛϡϨʔγϣϯɹɹ- ϨʔτϦϛοτ͋ΓɾPICs, IOAPIC & PITɾetc..
SUPINF Incଓ Firecracker ͱԿ͔19firecracker-containerd
SUPINF Incfirecracker-containerd (FC)20࣍ੈαʔόʔϨεج൫ͷຊ໋ʂʁ• Firecracker Λ ίϯςφඪ४༷ʹै͍ར༻͢Δ ͨΊͷΈ‣ ΠϯλʔϑΣΠεʹ Λར༻‣ VM ىಈϑΝΠϧͷ࣋ͪࠐΈඪ४తͳ͓࡞๏ʹै͏• Firecracker ීٴͷ͞Βʹઌɺͪ͜Β͕ຊ໋ߏͳͷͰײ‣ ͍ͣΕΑΓඒ͘͠ɺΑΓޮతͰɺΑΓ͍ߏʹcontainerd
SUPINF IncɹFC ܦ༝Ͱͷ Docker ίϯςφىಈ21ʙ ΑΓ҆શʹ Docker ίϯςφΛϗετ͢ΔͨΊʹ ʙFirecracker͍ͭΘΕΔ͔͝ʙ
SUPINF Incɹ22containerdFirecrackerECS / EKS ͳͲࢦࣔܥ౷FC ܦ༝Ͱͷ Docker ίϯςφىಈʙ ΑΓ҆શʹ Docker ίϯςφΛϗετ͢ΔͨΊʹ ʙ
SUPINF Incɹ23containerd FC snapshotterFirecrackerReadOnly ͳ DockerΠϝʔδ͔Β r/w Մೳɺ͔ͭ Firecracker ͷىಈ͢Δ VM ʹύεεϧʔՄೳͳσόΠεͱͯ͠εφοϓγϣοτΛ࡞ECS / EKS ͳͲࢦࣔܥ౷FC ܦ༝Ͱͷ Docker ίϯςφىಈʙ ΑΓ҆શʹ Docker ίϯςφΛϗετ͢ΔͨΊʹ ʙ
SUPINF Incɹ24FC runtimecontainerd FC snapshotterFirecrackerຊདྷ runc ͕σϑΥϧτͰ͋Δ runtime(containerd-shim) ΛFC runtime ʹ͢Δ͜ͱͰɺίϯςφΛΒͤΔ VM ىಈΛFirecracker ʹࢦࣔECS / EKS ͳͲࢦࣔܥ౷FC ܦ༝Ͱͷ Docker ίϯςφىಈʙ ΑΓ҆શʹ Docker ίϯςφΛϗετ͢ΔͨΊʹ ʙ
SUPINF Incɹɹ25FC runtimecontainerd FC snapshottermicroVMruncFC agentFirecrackerVM ͕ඵͰىಈECS / EKS ͳͲࢦࣔܥ౷FC ܦ༝Ͱͷ Docker ίϯςφىಈʙ ΑΓ҆શʹ Docker ίϯςφΛϗετ͢ΔͨΊʹ ʙ
SUPINF Incɹɹ26FC runtimecontainerd FC snapshottermicroVMruncFC agentFirecrackervsock ܦ༝Ͱ VM ͷΤʔδΣϯτʹɺrunc Ͱ Docker ίϯςφΛىಈ͢ΔΑ ͏ࢦࣔECS / EKS ͳͲࢦࣔܥ౷FC ܦ༝Ͱͷ Docker ίϯςφىಈʙ ΑΓ҆શʹ Docker ίϯςφΛϗετ͢ΔͨΊʹ ʙ
SUPINF Incɹɹ27FC runtimecontainerd FC snapshottermicroVMruncFC agentFirecrackerECS / EKS ͳͲࢦࣔܥ౷FC ܦ༝Ͱͷ Docker ίϯςφىಈʙ ΑΓ҆શʹ Docker ίϯςφΛϗετ͢ΔͨΊʹ ʙ
SUPINF IncFC ݱࡏͷঢ়گ28ઈࢍ։ൃதʂʂʂhttps://github.com/firecracker-microvm/firecracker/blob/master/docs/experimental-vsock.mdhttps://github.com/firecracker-microvm/firecracker-containerd/issues͜ͷঢ়ଶͰ͢Ͱʹຊ൪ར༻͞Ε͍ͯΔͱࢥ͍͍ɾɾ
SUPINF Incͱ͜ΖͰ͜ͷߏɺݟ֮͑͋Γ·ͤΜ͔ʁ29ɹɹFC runtimecontainerd FC snapshottermicroVMruncFC agentFirecrackerECS / EKS ͳͲࢦࣔܥ౷
SUPINF Incͱ͜ΖͰ͜ͷߏɺݟ֮͑͋Γ·ͤΜ͔ʁ30ɹɹFC runtimecontainerd FC snapshottermicroVMruncFC agentFirecrackerECS / EKS ͳͲࢦࣔܥ౷ݻ༗໊ࢺҧ͏ͷͷɺߏͱͯࣅ͍ͯΔ ..Kata Containers.. ??
SUPINF IncطଘιϑτΣΞͱͷकඋൣғΛൺֱ31ʢFirecracker ୯ͱ gVisor ͳΒซ༻Ͱ͖ͦ͏ʣɹHardwareɹHost OSɹHypervisorɹGuest OSɹSandboxɹApplicationgVisorFirecrackerFirecracker, KVM &firecracker+containerdQEMU, KVM &Kata Containers
SUPINF IncKata Containers QEMU ͱԿ͕ҧ͏ͷʁ32ίϯςφΛΒͤΔ͜ͱΛલఏʹߟ͑͠ɺɾ࠷ܰྔ / ࠷ʹͩ͜Θͬͯ·͢ɾηΩϡϦςΟʹͩ͜Θͬͯ·͢ɾAWS ͷج൫ʹඞཁͳػೳἧ͑·ͨ͠ʢҙ༁ʣͱͷ͜ͱhttps://firecracker-microvm.github.io/
SUPINF Inc33ࢲͨͪ Firecracker ʹԿΛظ͠Ͳ͏ߩݙͰ͖Δͷ͔
SUPINF IncOSS34• Firecracker ε‣ crosvm (Chromium OS ͷ VMM) ىݯ → Rust & ͖ࠩͬͱࠓޙ࠷খݶ‣ Υον͢Δʹͯ͠ GPU / Inferentia ͷΤϛϡϨʔγϣϯ͋ͨΓɾɾʁ• firecracker-containerd ε‣ ࢲͷ༧͕ਖ਼͚͠Εɾɾຊ໋ͳ্ʹ։ൃ్্ʂߩݙͷνϟϯεʂʂ‣ ͔ͬͪ͜͠ Go Ͱ͢ΑΈͳ͞ΜLambda / Fargate ج൫ΛΈΜͳͰ࡞ΕΔʂͳΜͳΒΦϯϓϨͰ͑ɾɾ
SUPINF IncFirecracker ͷ͡Ίํ35• GitHub Ͱ docs / issues / release notes ΛΈΑ͏‣ ΈΜͳͷࣦഊཤྺରॲ๏͕͢Ͱʹͨ͘͞Μ• Slack ʹ༗༻ͳใଟ͍• Nested Virtualization ڥԼͰಈ͘Αε‣ GCP ← f1.micro ͳΒແྉ͑ͯ $0.0076 / hour ʙ‣ Azureε
SUPINF IncFirecracker ͷ͡Ίํ361. ϕΞϝλϧ͔ωετ͞ΕͨԾڥͰ Linux ͷ KVM Λ༗ޮԽ2. ΧʔωϧͱϧʔτϑΝΠϧγεςϜΛͲ͔͔͜Β࣋ͬͯ͘Δ3. VM ͰωοτϫʔΫΛ͍͍ͨ / ύεεϧʔ͍ͨ͠σόΠε͕͋Ε४උ4. A Ͱ Firecracker Λىಈ5. B ͔Β REST API Λ௨ͯ͡ Firecracker ʹ VM ͷઃఆɾىಈΛࢦࣔ6. A ଆͰ VM ʹଓ͞ΕΔ
SUPINF Inc37͜Ε͔Β Fargate Ͳ͏มΘΔͷ͔
SUPINF IncFargate ج൫Λݕূͯ͠Έͨ38API ͷԠΛಡΈɺCreatedAt ͔Β PullStartedAt ʹͳΔ·Ͱͷ࣌ؒΛܭଌɹɹ// ҎԼͷεΫϦϓτͰ֤Ϧʔδϣϯ 5 ճࢼߦɹɹ$ when=$(date '+%Y%m%d%H%M') && mkdir “${when}"ɹɹ$ for region in us-east-1 us-east-2 us-west-2 \ɹɹɹɹɹɹeu-west-1 eu-central-1 ap -northeast-1 \ɹɹɹɹɹɹap -southeast-1 ap -southeast-2; doɹɹɹɹecs-task-runner --region "${region}" run alpine:3.8 --command env \ɹɹɹɹɹɹ--extended-output > "${when}/${region}".json 2>&1ɹɹdone
SUPINF IncFargate ج൫Λݕূͯ͠Έͨ39EC2 Ϟʔυʁ Firecracker Ϟʔυ͚ͩͲεέʔϧΞτͨ͠ʁ ఆࠔ ..
SUPINF Inc• Firecracker ͷਁಁʢFC ʹͳΔ͔ո͍͠..ʣ• ENI Ξλονʹվળͷஹ͠2019 ʹظͰ͖Δ͜ͱ40VM ࣗମͷىಈ & ENI ΞλονߴԽhttps://www.slideshare.net/AmazonWebServices/a-serverless-journey-aws-lambda-under-the-hood-srv409r1-aws-reinvent-2018
SUPINF IncSUPINF Ͱͷ࠷ۙͷ Fargate ར༻ྫ41• ຊ൪αΠτɺRedash ͳͲͷཧπʔϧ• CI: αʔϏεͱಉ VPC ͔Βͷ DB ϚΠάϨʔγϣϯ / API e2e ςετ‣ pottava/ecs-task-runner Ͱಉظతʹ Fargate Λར༻‣ πʔϧΛ Docker ʹ͢Ε CI ͱશ͘ಉ༷ͷςετΛϩʔΧϧͰ• Docker Πϝʔδͷ੬ऑੑνΣοΫ‣ Clair DB ͝ͱ Fargate ʹࡌͤͯͳ͍
Presented by
தؙ ྑ @pottava• CTO at SUPINF Inc• Solutions Architect at Rescale, Inc.• JAWS-UG ίϯςφࢧ෦ɺAI ࢧ෦ӡӦProfile43
SUPINF Inc44
Containerize your app!45• Ϋϥυ / ίϯςφ ΛڧΈʹͨ͠डୗ։ൃӡ༻ɺίϯαϧςΟϯά• 2015 ͔Β Docker ͷຊ൪ӡ༻Λ։࢝ɾ๛ͳ CI / CD ࣄྫ• εϐϯϑɺͱಡΈ·͢ɾɾ
SUPINF Inc46
Cloud HPC with47• Ϋϥυ HPC γϛϡϨʔγϣϯϓϥοτϑΥʔϜͷఏڙ• 2011 ॳ಄ʹઃཱɺPeter Thiel Microsoft ͔Βग़ࢿ• εέʔϥϒϧͳγϛϡϨʔγϣϯػցֶशΛʂ
͝੩ௌ͋Γ͕ͱ͏͍͟͝·ͨ͠ :)ࢀߟɿ• AWS re:Invent 2018: A Serverless Journey: AWS Lambda Under theHood ( https://www.youtube.com/watch?v=QdzV04T_kec )• The Nitro Project: Next-Generation EC2 Infrastructure - AWS OnlineTech Talks ( https://www.youtube.com/watch?v=eWFEJmsddV0 )• Firecracker: Secure and fast microVMs for serverless computing( https://github.com/firecracker-microvm/firecracker )• firecracker-containerd: firecracker-containerd enables containerdto manage containers as Firecracker microVMs ( https://github.com/firecracker-microvm/firecracker-containerd )re:Invent ͰूΊ͖ͯͨεςοΧʔΓ·͢ʙʂ
pottava
kilometer
kageshiron
tanakahisateru
j5ik2o
weddingpark
suzuki
ericgpks
meihei3
fabpot
fishofprey
yoshikouki
hiro_y
speakerdeck
andyhume
chrislema
jonrohan
sferik
keavy
schacon
phodgson
ufuk
keathley
garrettdimon
pauljervisheath