Firecracker ͱ͸Կ͔ AWS ͸ͳͥܰྔ VM Λ࡞Γ ࢲͨͪ͸͔ͦ͜ΒԿΛڗडͰ͖Δͷ͔ JAWS-UG ίϯςφࢧ෦ #13 Dec 21, 2018 Ryo NAKAMARU, SUPINF Inc.

SUPINF Inc τϐοΫ 2 • AWS ͸ͳͥ Firecracker Λ࡞ͬͨͷ͔ • Firecracker ͱ͸Կ͔ • ࢲͨͪ͸ Firecracker ʹԿΛظ଴͠ɺͲ͏ߩݙͰ͖Δͷ͔ • ͜Ε͔Β Fargate ͸Ͳ͏มΘΔͷ͔

SUPINF Inc 3 AWS ͸ͳͥ Firecracker Λ࡞ͬͨͷ͔

SUPINF Inc Lambda ͸ैདྷ EC2 ϕʔε 4 ηΩϡϦςΟ͸ඞਢʂ࣮ߦίϯςΩετɺ·ͣ͸ EC2 Ͱ෼཭͠Α͏ ɹ Hardware ɹ Host OS ɹ Hypervisor ɹ Sandbox ɹ Lambda runtime ← EC2 ɹ Guest OS Ϛϧνςφϯτ AWS ΞΧ΢ϯτ ؔ਺ ʢ෼཭Ϩϕϧʣ ɹ Our code ɾݫີͰ࣮੷͋Δ ෼཭ख๏ ɾͳΔ͸΍ͰϦϦʔε ͔ͨͬͨ͠

SUPINF Inc Lambda ͸ैདྷ EC2 ϕʔε 5 ඞવతʹ EC2 ΠϯελϯεͷىಈɺͦͷͨΊͷϦιʔε֬อ͕ى͜Δ ɹ ɹ ɹ ɹ ɹ ɹ λ ɹ ɹ ɹ λ ɹ ɹ ɹ λ ɹ A ΞΧ΢ϯτ ɹ ɹ B ΞΧ΢ϯτ C ΞΧ΢ϯτ ɾىಈ͕஗͍͜ͱ͕͋Δ ɾϦιʔεར༻ʹແବ͕ଟ͍ ͳͷͰɾɾ ɹ ɹ ɹ λ ɹ ɹ ɹ λ

SUPINF Inc ࣍ੈ୅ͷαʔόʔϨεج൫͕΄͍͠ 6 ౎౓ EC2 Λىಈ͍ͯͯ͠͸ඇޮ཰ • ىಈ࣌ؒ • Ϧιʔεར༻ޮ཰ 1 ϓϩηε͕ݫີʹ෼཭Ͱ͖Ε͹͍͍

SUPINF Inc Firecracker Ͱ࣮ݱ͢Δ৽ߏ੒ 7 ௒ܰྔͳ VM ͱɺߴ଎ͳ Hypervisor ͕͋Ε͹ɾɾ ɹ Hardware ɹ Host OS ɹ Sandbox ɹ Lambda runtime ɹ Our code Ϛϧνςφϯτ ؔ਺ ɹ Hypervisor ɹ Guest OS ɹ Hardware ɹ Host OS ɹ Hypervisor ɹ Sandbox ɹ Lambda runtime ɹ Guest OS Ϛϧνςφϯτ AWS ΞΧ΢ϯτ ؔ਺ ʢ෼཭Ϩϕϧʣ ɹ Our code ݟ௚͠ʂ

SUPINF Inc 8 ɹ ɹ ɹ ɹ ɹ λ ɹ A ɾฏۉىಈ࣌ؒͷ୹ॖ ɾϦιʔεΛ࠷େݶ༗ޮ׆༻Ͱ͖Δ Firecracker Ͱ࣮ݱ͢Δ৽ߏ੒ ௒ܰྔͳ VM ͱɺߴ଎ͳ Hypervisor ͕͋Ε͹ɾɾ ɹ ɹ ɹ λ ɹ B ɹ ɹ ɹ λ ɹ B ɹ ɹ ɹ λ ɹ C ɹ ɹ ɹ λ ɹ C .. ɹ ࣍ੈ୅ Hypervisor ɹ ɹ ɹ λ ɹ D

SUPINF Inc 2018 ೥຤ݱࡏɺฏߦՔಇதɾɾʂ 9 EC2 Ϟʔυ ɹ Hardware ɹ Host OS ɹ Hypervisor ɹ Sandbox ɹ Lambda runtime ɹ Guest OS Ϛϧνςφϯτ AWS ΞΧ΢ϯτ ؔ਺ ɹ Our code ɹ Hardware ɹ Host OS ɹ Sandbox ɹ Lambda runtime ɹ Our code Ϛϧνςφϯτ ؔ਺ ɹ Hypervisor ɹ Guest OS Firecracker Ϟʔυ ➕

SUPINF Inc ͜͜·Ͱͷ࿩ → youtube.com/watch?v=QdzV04T_kec 10

SUPINF Inc 11 Firecracker ͱ͸Կ͔

SUPINF Inc “ίϯςφͷͨΊͷ” QEMU ୅ସͰ͢ 12 1 Firecracker ϓϩηεͰ 1 microVM Λ୲౰ … ɹ ɹ Hardware CPU, Storage, Network card, … ɹ Host kernel KVM ɹ ɹ ɹ Firecracker ɹ Guest kernel microVM ͜Ε

SUPINF Inc 13 Firecracker ͸ VM Λىಈ͢Δͷ͕࢓ࣄʂ Ϣʔβʔ Lambda ࢖͏ͧʙ Lambda / Fargate ͸͜͏ಈ͍͍ͯΔʢໝ૝ʣ

SUPINF Inc Lambda / Fargate ͸͜͏ಈ͍͍ͯΔʢໝ૝ʣ 14 Ϣʔβʔ 1. AWS API αʔόʔϨε / ECS ίϯτϩʔϧϓϨΠϯ Firecracker ͸ VM Λىಈ͢Δͷ͕࢓ࣄʂ * αʔόʔϨεଆ͸ઌͷಈըͰجຊߏ੒Λެ։ࡁ A Serverless Journey: AWS Lambda Under the Hood https://www.youtube.com/watch?v=QdzV04T_kec

SUPINF Inc ɹ 15 ɹ Host kernel KVM Ϣʔβʔ Firecracker ͸ VM Λىಈ͢Δͷ͕࢓ࣄʂ ϗετ͝ͱʹଘࡏ͢ΔͰ͋Ζ͏ Ṗ Agent ʹ Lambda ͷൃՐΛґཔ Ṗ Agent 2. ൃՐґཔ Lambda / Fargate ͸͜͏ಈ͍͍ͯΔʢໝ૝ʣ αʔόʔϨε / ECS ίϯτϩʔϧϓϨΠϯ

SUPINF Inc ɹ 16 ɹ Host kernel KVM Ϣʔβʔ Firecracker ͸ VM Λىಈ͢Δͷ͕࢓ࣄʂ Firecracker Ṗ Agent 4. Firecracker ىಈ 3. μ΢ϯϩʔυ Ṗ Agent ͸ S3 / ECR ͔Β ϢʔβʔίʔυΛर͍ͭͭ Firecracker ΛҰͭىಈ ʢ͜ͷϝϞϦ͕ 5 MiB ະຬʣ Lambda / Fargate ͸͜͏ಈ͍͍ͯΔʢໝ૝ʣ αʔόʔϨε / ECS ίϯτϩʔϧϓϨΠϯ

SUPINF Inc ɹ 17 ɹ Host kernel KVM Ϣʔβʔ Firecracker ͸ VM Λىಈ͢Δͷ͕࢓ࣄʂ Firecracker Ṗ Agent 5. VM ઃఆɾىಈࢦࣔ Ϣʔβʔίʔυͷ࣋ͪࠐΈؚΊ Firecracker ͕ VM Λىಈ ʢVM ಺Ͱ /sbin/init ϓϩηε͕ ɹ૸Δ·Ͱ͕ 125 ms Ҏ಺ʣ ɹ Guest kernel microVM Application Lambda / Fargate ͸͜͏ಈ͍͍ͯΔʢໝ૝ʣ αʔόʔϨε / ECS ίϯτϩʔϧϓϨΠϯ

SUPINF Inc ͦͷଞಛ௃ͱɺͦͷඞવੑ 18 AWS ͕ҎԼͷػೳ܈Λ crosvm ʹ௥Ճ࣮૷ͨ͠ͷ͸ͳͥͩͱࢥ͍·͢ʁ Firecracker ɹ Guest kernel microVM Application ɾࢦࣔ͸ REST API Ͱड͚Δ ɾϝλσʔλ ؅ཧ ɾσόΠεΤϛϡϨʔγϣϯ ɹɹ- ϨʔτϦϛοτ͋Γ ɾPICs, IOAPIC & PIT ɾetc..

SUPINF Inc ଓ Firecracker ͱ͸Կ͔ 19 firecracker-containerd

SUPINF Inc firecracker-containerd (FC) 20 ࣍ੈ୅αʔόʔϨεج൫ͷຊ໋ʂʁ • Firecracker Λ ίϯςφඪ४࢓༷ʹै͍ར༻͢Δ ͨΊͷ࢓૊Έ ‣ ΠϯλʔϑΣΠεʹ Λར༻ ‣ VM ىಈ΍ϑΝΠϧͷ࣋ͪࠐΈ΋ඪ४తͳ͓࡞๏ʹै͏ • Firecracker ීٴͷ͞Βʹઌɺͪ͜Β͕ຊ໋ߏ੒ͳͷͰ͸ײ ‣ ͍ͣΕ͸ΑΓඒ͘͠ɺΑΓޮ཰తͰɺΑΓ଎͍ߏ੒ʹ containerd

SUPINF Inc ɹ FC ܦ༝Ͱͷ Docker ίϯςφىಈ 21 ʙ ΑΓ҆શʹ Docker ίϯςφΛϗετ͢ΔͨΊʹ ʙ Firecracker ͍ͭ࢖ΘΕΔ͔ ͝஫໨ʙ

SUPINF Inc ɹ 22 containerd Firecracker ECS / EKS ͳͲ ࢦࣔܥ౷ FC ܦ༝Ͱͷ Docker ίϯςφىಈ ʙ ΑΓ҆શʹ Docker ίϯςφΛϗετ͢ΔͨΊʹ ʙ

SUPINF Inc ɹ 23 containerd FC snapshotter Firecracker ReadOnly ͳ Docker Πϝʔδ͔Β r/w Մೳɺ ͔ͭ Firecracker ͷى ಈ͢Δ VM ʹύεε ϧʔՄೳͳσόΠεͱ ͯ͠εφοϓγϣοτ Λ࡞੒ ECS / EKS ͳͲ ࢦࣔܥ౷ FC ܦ༝Ͱͷ Docker ίϯςφىಈ ʙ ΑΓ҆શʹ Docker ίϯςφΛϗετ͢ΔͨΊʹ ʙ

SUPINF Inc ɹ 24 FC runtime containerd FC snapshotter Firecracker ຊདྷ runc ͕σϑΥ ϧτͰ͋Δ runtime (containerd-shim) Λ FC runtime ʹ͢Δ ͜ͱͰɺίϯςφΛ ૸ΒͤΔ VM ىಈΛ Firecracker ʹࢦࣔ ECS / EKS ͳͲ ࢦࣔܥ౷ FC ܦ༝Ͱͷ Docker ίϯςφىಈ ʙ ΑΓ҆શʹ Docker ίϯςφΛϗετ͢ΔͨΊʹ ʙ

SUPINF Inc ɹ ɹ 25 FC runtime containerd FC snapshotter microVM runc FC agent Firecracker VM ͕ඵͰىಈ ECS / EKS ͳͲ ࢦࣔܥ౷ FC ܦ༝Ͱͷ Docker ίϯςφىಈ ʙ ΑΓ҆શʹ Docker ίϯςφΛϗετ͢ΔͨΊʹ ʙ

SUPINF Inc ɹ ɹ 26 FC runtime containerd FC snapshotter microVM runc FC agent Firecracker vsock ܦ༝Ͱ VM ಺ ͷΤʔδΣϯτʹɺ runc Ͱ Docker ίϯ ςφΛىಈ͢ΔΑ ͏ ࢦࣔ ECS / EKS ͳͲ ࢦࣔܥ౷ FC ܦ༝Ͱͷ Docker ίϯςφىಈ ʙ ΑΓ҆શʹ Docker ίϯςφΛϗετ͢ΔͨΊʹ ʙ

SUPINF Inc ɹ ɹ 27 FC runtime containerd FC snapshotter microVM runc FC agent Firecracker ECS / EKS ͳͲ ࢦࣔܥ౷ FC ܦ༝Ͱͷ Docker ίϯςφىಈ ʙ ΑΓ҆શʹ Docker ίϯςφΛϗετ͢ΔͨΊʹ ʙ

SUPINF Inc FC ݱࡏͷঢ়گ 28 ઈࢍ։ൃதʂʂʂ https://github.com/firecracker-microvm/firecracker/blob/master/docs/experimental-vsock.md https://github.com/firecracker-microvm/firecracker-containerd/issues ͜ͷঢ়ଶͰ͢Ͱʹຊ൪ར༻ ͞Ε͍ͯΔͱ͸ࢥ͍೉͍ɾɾ

SUPINF Inc ͱ͜ΖͰ͜ͷߏ੒ɺݟ֮͑͋Γ·ͤΜ͔ʁ 29 ɹ ɹ FC runtime containerd FC snapshotter microVM runc FC agent Firecracker ECS / EKS ͳͲ ࢦࣔܥ౷

SUPINF Inc ͱ͜ΖͰ͜ͷߏ੒ɺݟ֮͑͋Γ·ͤΜ͔ʁ 30 ɹ ɹ FC runtime containerd FC snapshotter microVM runc FC agent Firecracker ECS / EKS ͳͲ ࢦࣔܥ౷ ݻ༗໊ࢺ͸ҧ͏΋ͷͷɺ ߏ੒͸ͱͯ΋ࣅ͍ͯΔ .. Kata Containers .. ??

SUPINF Inc طଘιϑτ΢ΣΞͱͷकඋൣғΛൺֱ 31 ʢFirecracker ୯඼ͱ gVisor ͳΒซ༻Ͱ͖ͦ͏ʣ ɹ Hardware ɹ Host OS ɹ Hypervisor ɹ Guest OS ɹ Sandbox ɹ Application gVisor Firecracker Firecracker, KVM & firecracker+containerd QEMU, KVM & Kata Containers

SUPINF Inc Kata Containers ΍ QEMU ͱ͸Կ͕ҧ͏ͷʁ 32 ίϯςφΛ૸ΒͤΔ͜ͱΛલఏʹߟ͑௚͠ɺ ɾ࠷ܰྔ / ࠷଎ʹͩ͜Θͬͯ·͢ ɾηΩϡϦςΟʹͩ͜Θͬͯ·͢ ɾAWS ͷج൫ʹඞཁͳػೳἧ͑·ͨ͠ʢҙ༁ʣ ͱͷ͜ͱ https://firecracker-microvm.github.io/

SUPINF Inc 33 ࢲͨͪ͸ Firecracker ʹԿΛظ଴͠ Ͳ͏ߩݙͰ͖Δͷ͔

SUPINF Inc OSS 34 • Firecracker ε ‣ crosvm (Chromium OS ͷ VMM) ىݯ → Rust & ࠩ෼͸͖ͬͱࠓޙ΋࠷খݶ ‣ ΢Υον͢Δʹͯ͠΋ GPU / Inferentia ͷΤϛϡϨʔγϣϯ͋ͨΓɾɾʁ • firecracker-containerd ε ‣ ࢲͷ༧૝͕ਖ਼͚͠Ε͹ɾɾຊ໋ͳ্ʹ։ൃ్্ʂߩݙͷνϟϯεʂʂ ‣ ͔͠΋ͬͪ͜͸ Go Ͱ͢ΑΈͳ͞Μ Lambda / Fargate ج൫ΛΈΜͳͰ࡞ΕΔʂͳΜͳΒΦϯϓϨͰ΋࢖͑ɾɾ

SUPINF Inc Firecracker ͷ͸͡Ίํ 35 • GitHub Ͱ docs / issues / release notes ΛΈΑ͏ ‣ ΈΜͳͷࣦഊཤྺ΍ରॲ๏͕͢Ͱʹͨ͘͞Μ • Slack ʹ΋༗༻ͳ৘ใଟ͍ • Nested Virtualization ؀ڥԼͰ΋ಈ͘Αε ‣ GCP ← f1.micro ͳΒແྉ࿮௒͑ͯ΋ $0.0076 / hour ʙ ‣ Azureε

SUPINF Inc Firecracker ͷ͸͡Ίํ 36 1. ϕΞϝλϧ͔ωετ͞ΕͨԾ૝؀ڥͰ Linux ͷ KVM Λ༗ޮԽ 2. ΧʔωϧͱϧʔτϑΝΠϧγεςϜΛͲ͔͔͜Β࣋ͬͯ͘Δ 3. VM ಺ͰωοτϫʔΫΛ࢖͍͍ͨ / ύεεϧʔ͍ͨ͠σόΠε͕͋Ε͹४උ 4. ୺຤ A Ͱ Firecracker Λىಈ 5. ୺຤ B ͔Β REST API Λ௨ͯ͡ Firecracker ʹ VM ͷઃఆɾىಈΛࢦࣔ 6. ୺຤ A ଆͰ VM ʹ઀ଓ͞ΕΔ

SUPINF Inc 37 ͜Ε͔Β Fargate ͸Ͳ͏มΘΔͷ͔

SUPINF Inc Fargate ج൫Λݕূͯ͠Έͨ 38 API ͷԠ౴ΛಡΈɺCreatedAt ͔Β PullStartedAt ʹͳΔ·Ͱͷ࣌ؒΛܭଌ ɹɹ// ҎԼͷεΫϦϓτͰ֤Ϧʔδϣϯ 5 ճࢼߦ ɹɹ$ when=$(date '+%Y%m%d%H%M') && mkdir “${when}" ɹɹ$ for region in us-east-1 us-east-2 us-west-2 \ ɹɹɹɹɹɹeu-west-1 eu-central-1 ap -northeast-1 \ ɹɹɹɹɹɹap -southeast-1 ap -southeast-2; do ɹɹɹɹecs-task-runner --region "${region}" run alpine:3.8 --command env \ ɹɹɹɹɹɹ--extended-output > "${when}/${region}".json 2>&1 ɹɹdone

SUPINF Inc Fargate ج൫Λݕূͯ͠Έͨ 39 EC2 Ϟʔυʁ Firecracker Ϟʔυ͚ͩͲεέʔϧΞ΢τͨ͠ʁ ൑ఆࠔ೉ ..

SUPINF Inc • Firecracker ͷਁಁʢFC ʹͳΔ͔͸ո͍͠..ʣ • ENI Ξλον໰୊ʹվળͷஹ͠ 2019 ೥ʹظ଴Ͱ͖Δ͜ͱ 40 VM ࣗମͷىಈ & ENI Ξλονߴ଎Խ https://www.slideshare.net/AmazonWebServices/a-serverless-journey-aws-lambda-under-the-hood-srv409r1-aws-reinvent-2018

SUPINF Inc SUPINF Ͱͷ࠷ۙͷ Fargate ར༻ྫ 41 • ຊ൪αΠτɺRedash ͳͲͷ؅ཧπʔϧ • CI: αʔϏεͱಉ VPC ಺͔Βͷ DB ϚΠάϨʔγϣϯ / API e2e ςετ ‣ pottava/ecs-task-runner Ͱಉظతʹ Fargate Λར༻ ‣ πʔϧΛ Docker ʹ͢Ε͹ CI ͱશ͘ಉ༷ͷςετΛϩʔΧϧͰ΋ • Docker Πϝʔδͷ੬ऑੑνΣοΫ ‣ Clair ͸ DB ͝ͱ Fargate ʹࡌͤͯ΋໰୊ͳ͍

Presented by

தؙ ྑ @pottava • CTO at SUPINF Inc • Solutions Architect at Rescale, Inc. • JAWS-UG ίϯςφࢧ෦ɺAI ࢧ෦ӡӦ Profile 43

SUPINF Inc 44

Containerize your app! 45 • Ϋϥ΢υ / ίϯςφ ΛڧΈʹͨ͠डୗ։ൃӡ༻ɺίϯαϧςΟϯά • 2015 ೥͔Β Docker ͷຊ൪ӡ༻Λ։࢝ɾ๛෋ͳ CI / CD ࣄྫ • εϐϯϑɺͱಡΈ·͢ɾɾ

SUPINF Inc 46

Cloud HPC with 47 • Ϋϥ΢υ HPC γϛϡϨʔγϣϯϓϥοτϑΥʔϜͷఏڙ • 2011 ೥ॳ಄ʹઃཱɺPeter Thiel ΍ Microsoft ͔Βग़ࢿ • εέʔϥϒϧͳγϛϡϨʔγϣϯ΍ػցֶशΛʂ

͝੩ௌ͋Γ͕ͱ͏͍͟͝·ͨ͠ :) ࢀߟɿ • AWS re:Invent 2018: A Serverless Journey: AWS Lambda Under the Hood ( https://www.youtube.com/watch?v=QdzV04T_kec ) • The Nitro Project: Next-Generation EC2 Infrastructure - AWS Online Tech Talks ( https://www.youtube.com/watch?v=eWFEJmsddV0 ) • Firecracker: Secure and fast microVMs for serverless computing ( https://github.com/firecracker-microvm/firecracker ) • firecracker-containerd: firecracker-containerd enables containerd to manage containers as Firecracker microVMs ( https://github.com/ firecracker-microvm/firecracker-containerd ) re:Invent ͰूΊ͖ͯͨ εςοΧʔ഑Γ·͢ʙʂ