Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
3 state-of-the-art technologies in Linux and fu...
Search
KONDO Uchio
February 21, 2021
Technology
2
610
3 state-of-the-art technologies in Linux and future of the containers #SECKUN
2021.02.21 「新しいセキュリティビジネスキャリア」シンポジウム
KONDO Uchio
February 21, 2021
Tweet
Share
More Decks by KONDO Uchio
See All by KONDO Uchio
大規模レガシーテストを 倒すための CI基盤の作り方 / #CICD2023
udzura
5
2.3k
Ruby x BPF in Action / RubyKaigi 2022
udzura
0
210
Narrative of Ruby & Rust
udzura
0
180
開発者生産性指標の可視化 / pepabo-four-keys
udzura
3
1.6k
Talk of RBS
udzura
0
400
Re: みなさん最近どうですか? / FGN tech meetup in 2021
udzura
0
720
Dockerとやわらかい仮想化 - ProSec-IT/SECKUN 2021 edition -
udzura
2
690
Device access filtering in cgroup v2
udzura
1
800
"Story of Rucy" on RubyKaigi takeout 2021
udzura
0
750
Other Decks in Technology
See All in Technology
Alignment and Autonomy in Cybozu - 300人の開発組織でアラインメントと自律性を両立させるアジャイルな組織運営 / RSGT2025
ama_ch
1
2.3k
Accessibility Inspectorを活用した アプリのアクセシビリティ向上方法
hinakko
0
180
生成AI × 旅行 LLMを活用した旅行プラン生成・チャットボット
kominet_ava
0
150
Azureの開発で辛いところ
re3turn
0
240
【NGK2025S】動物園(PINTO_model_zoo)に遊びに行こう
kazuhitotakahashi
0
210
2025年のARグラスの潮流
kotauchisunsun
0
790
なぜfreeeはハブ・アンド・スポーク型の データメッシュアーキテクチャにチャレンジするのか?
shinichiro_joya
2
320
20250116_JAWS_Osaka
takuyay0ne
2
200
comilioとCloudflare、そして未来へと向けて
oliver_diary
6
440
I could be Wrong!! - Learning from Agile Experts
kawaguti
PRO
8
3.3k
PaaSの歴史と、 アプリケーションプラットフォームのこれから
jacopen
7
1.3k
「隙間家具OSS」に至る道/Fujiwara Tech Conference 2025
fujiwara3
6
6.3k
Featured
See All Featured
A Tale of Four Properties
chriscoyier
157
23k
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
smashingmag
251
21k
Faster Mobile Websites
deanohume
305
30k
A designer walks into a library…
pauljervisheath
205
24k
Designing Experiences People Love
moore
139
23k
Building a Modern Day E-commerce SEO Strategy
aleyda
38
7k
Statistics for Hackers
jakevdp
797
220k
Code Reviewing Like a Champion
maltzj
521
39k
Unsuck your backbone
ammeep
669
57k
Rebuilding a faster, lazier Slack
samanthasiow
79
8.8k
The Straight Up "How To Draw Better" Workshop
denniskardys
232
140k
Being A Developer After 40
akosma
89
590k
Transcript
ཁૉٕज़ͷstate of the art͔Βߟ͑Δ ۙ౻Ӊஐ࿕ / GMO Pepabo, Inc. 2021.02.21
ʮ৽͍͠ηΩϡϦςΟϏδωεΩϟϦΞʯγϯϙδϜ Linuxίϯςφͷະདྷ
GMOϖύϘגࣜձࣾ γχΞϓϦϯγύϧ ٕज़෦ٕज़ج൫νʔϜॴଐ ۙ౻Ӊஐ࿕ (@udzura)
ۙ౻Ӊஐ࿕ ུྺ • ࡾՏᅳͷਓɻچ٢ాൡߍ࣌शؗߴߍΛଔۀɺ౦ژେֶจֶ෦ຊޠຊจֶઐम՝ఔ ͷֶ࢜ଔʢ2007ʣɻ • ϚείϛͷࣾSEɺECαΠτ։ൃɺΦϯϥΠϯήʔϜ ։ൃͳͲΛܦͯ2013ΑΓݱ৬ɺಉʹԬҠॅɻ • RubyɺίϯςφɺΫϥυωΠςΟϒٕज़ͳͲͷίϛϡχςΟͰ
׆ಈɻஶॻʹʮWebͰ͑ΔmrubyγεςϜϓϩάϥϛϯάೖʯʢC&Rݚڀॴʣ • ͖ͳγεςϜίʔϧʢ࠷ۙʣ socketpair(2) ɻ
ࠓͷ͓ •ίϯςφͷཁૉٕज़ʹ͍ͭͯɺࢲͷߨٛͰʢͬ͘͟Γʣཧղ͞Εͨํ ͚ͷ༰Ͱ͢ɻཁૉٕज़ͷղઆࢀߟࢿྉΛͲ͏ͧɻ •ࢀߟ1: https://container-security.dev/ •ࢀߟ2: ʰίϯςφܕԾԽ֓ʱʢޱ, ΧοτγεςϜʣ •ۙͷΧʔωϧʹؚ·ΕΔ৽ٕज़ͷ͏ͪɺίϯςφʹؔ͢ΔͷΛ հ͠·͢ɻ
•͕࣌ؒ͋Εɺͦͷ্ͰίϯςφͷະདྷΛߟ͑·͢ɻ
cgroup v2
cgroup ͷ͓͞Β͍ •Linux Kernelͷجຊٕज़ͷҰͭɻ •ϓϩηεΛάϧʔϐϯά͠ɺͦͷάϧʔϓ୯ҐͰϦιʔεར༻ͷ੍ݶ Λ͔͚Δٕज़ɻCPUɺϝϞϦɺIOɺϓϩηε... IUUQTHJIZPKQBENJOTFSJBMMJOVY@DPOUBJOFST
cgroup v1ͷྫ •cgroupfs ͱ͍͏ϑΝΠϧγεςϜʹmkdir(2) read(2) write(2)ͳͲΛ࣮ ߦ͠ɺૢ࡞Λߦ͏
cgroup v2 •v1 ͷ͍͔ͭ͘ͷܽ - ओʹ੍ޚରʢίϯτϩʔϥʣ͝ͱʹσΟϨΫ τϦΛ͚ͳ͚Ε͍͚ͳ͍༷ - Λࠀ͘͢։ൃ͞Εͨ •େ͖ͳҧ͍ͱͯ͠ɺ
v1 ͰίϯτϩʔϥผʹσΟϨΫτϦ ΛϚϯ τɺݸผʹάϧʔϓʹॴଐͰ͖ͨͷʹର͠ɺv2ͰશίϯτϩʔϥΛ ·ͱΊͨҰͭͷσΟϨΫτϦͷΈΛϚϯτ͠ɺ·ͱΊͯάϧʔϓΛ ࡞͢ΔڍಈʹͳΔɻ •ίϯςφͱͯͪ͜͠Βͷํ͕߹͕͍͍ɻ
Unified hierarchy /sys/fs/cgroup /sys/fs/cgroup /group-a /group-b /cpu.* /memory.* /io.* ...
/cpu.* /memory.* /io.* ... /cpu /memory /blkio /group-a /group-b /group-a /group-c
ίϯςφϥϯλΠϜͰͷcgroupͷར༻ •ʢOCIܥͷʣϥϯλΠϜͰҎԼͷ2ͭͷઃఆ߲͕͋Δ •Cgroup Driver: ίϯςφʹׂΓͯΔcgroupΛͲ͏ίϯτϩʔϧ͢Δ͔ •cgroupfs: cgroupfsͷͷϑΝΠϧૢ࡞ •systemd: systemdʹΑΔཧ •Cgroup
Version: Ϧιʔε੍ݶʹ v1/v2 ͲͪΒΛར༻͢Δ͔ •/sys/fs/cgroup ʹͲͷϑΝΠϧγεςϜ͕Ϛϯτ͞ΕͯΔ͔Ͱఆ
v2 ͷ৽ػೳ •Unified Hierarchy •PSI(Pressure Stall Information) •eBPFͰcgoup IDͷऔಘ͕Մೳʹ •nsdelegate
(ඇಛݖίϯςφʹॏཁ) •clone3(2) Ͱಛఆͷcgroup෦ʹϓϩηε࡞͕Մೳʹ •ͳͲͳͲ...
e.g. PSI(Pressure Stall Information) •γεςϜશମɺ·ͨcgroup୯ҐͰར༻Ͱ͖Δෛՙͷࢦඪ •CPU, ϝϞϦ, IO Ͱ stall
ͨ͠୯Ґ࣌ؒͰͷׂ߹ ΛܭଌͰ͖Δ •e.g. 1ؒͰ45ඵؒɺάϧʔϓͷ ͋Δϓϩηε͕CPUىҼͰ Ԇͨ͠߹ɺcpu some: 75.00
e.g. eBPFͰͷτϥοΩϯάใ •bpf_get_current_cgroup_id(void) ϔϧύʔ •eBPFͷΠϕϯτ͕ى͖ͨλεΫ͕Ͳͷcgroup(v2)ʹॴଐ͍ͯ͠Δ͔ɺ ͦͷIDΛฦ͢ɻ
How cgroup-v2 and PSI Impacts Cloud Native? Uchio Kondo /
GMO Pepabo, Inc. 2019.07.23 CloudNative Days Tokyo 2019 Image from pixabay: https://pixabay.com/images/id-3193865/
eBPF per containers
eBPFٕज़ͱ •ϢʔβۭؒͰ࡞ͬͨϓϩάϥϜΛΧʔωϧͰಈ͔ٕ͢ज़ͷͻͱͭ •2012ʹseccompͷಋೖɺ2013ʹLinuxͷSDNͰͷԠ༻͕࣮͞ ΕɺͦΕҎ߱ख़͢Δ •ϑΟϧλϦϯά͕ಘҙʢtcpdump, seccomp, bpftraceʣ •ΧʔωϧͷใʹΞΫηεͰ͖Δ͕ɺةݥͳίʔυಈ͔ͳ͍ͳͲ ҆શੑ͕͋Δఔ୲อ͞Ε͍ͯΔ
eBPFͷԠ༻ྫ
ίϯςφͷeBPFτϨʔεઓུ •ઓུ͕͍͔ͭ͋͘Δ •Linux Namespace·ͨcgroup (v2)ͷใ͕ར༻Ͱ͖Δ
ྫ1: task_struct ͷใΛḷΔ •task_struct→nsproxy ͔Β namespaceͷใΛ औಘͯ͠ϑΟϧλ͢Δ ʢcxrayʣ IUUQTHJUIVCDPNNSUDDYSBZCMPCNBTUFSQLHUSBDFSPQFOPQFOHP--
ྫ2: NS/ϗετͰͷPIDΛൺֱ •BPFϓϩάϥϜͰऔಘͰ͖ͨ tidͱɺϗετͰͷtidΛ ൺֱ͠ɺҰக͠ͳ͚Ε ίϯςφͱఆ͢Δ ʢTraceeʣ • task_structґଘ IUUQTHJUIVCDPNBRVBTFDVSJUZUSBDFFCMPCNBJOUSBDFFUSBDFFCQGD-ɹ
ྫ3: cgroup helperΛར༻ IUUQTHJUIVCDPNVE[VSBDPQFODMPTFCMPCNBTUFSTSDCQGDPQFODMPTFCQGD
࣮ྫ •copenclose(8) •ۙ౻ͷPoC (BPF+Rust) •ϑϥάͰtask_struct/ cgroup v2 ID ΛΓସ͑
bpf_get_current_cgroup_id(void) を添えて Uchio Kondo / Container Runtime Meetup #3 ランタイムとcgroupの
xxxな関係 * Photo by Fukuoka City
seccomp
seccompͷ͓͞Β͍ •ϓϩάϥϜʹ͓͚ΔγεςϜίʔϧݺͼग़͠ΛϑΟϧλϦϯά͢Δ •γεςϜίʔϧͷҾͷ݅ʹΑͬͯࢦఆΛม͑ΒΕΔ •blacklist(denylist), whitelist(allowlist) ͳͲΛ࣮Ͱ͖Δ •ϑϥά͕ࡉ͔͘ଘࡏ͠ɺྫ͑γεςϜίʔϧͷauditϩάͷΈɺҙ ͷerrnoΛฦͤ͞ΔɺͳͲͷࢦఆ͕Ͱ͖Δ
seccompͷར༻(mruby)
User space notification •seccompʹΑΓγεςϜίʔϧݺͼग़͠Λݕ͠ɺͦͷڍಈΛϢʔβ ϥϯυͷϓϩάϥϜʹҕͶΔ͜ͱ͕Ͱ͖Δٕज़ •Linux 5.0 (2019/3) ͔Βͷಋೖ •அ͢Δ·ͰɺͦͷγεςϜίʔϧϒϩοΫ͢Δ
•e.g. LXCͰͷσόΠεΞΫηεͷ੍ޚ IUUQTHJIZPKQBENJOTFSJBMMJOVY@DPOUBJOFSTɹ
User space notification IUUQTHJIZPKQBENJOTFSJBMMJOVY@DPOUBJOFSTɹ • LXCͰֶͿίϯςφೖ ୈ47ճɹඇಛݖίϯςφͷՄೳੑΛ͛Δseccomp notifyػೳ ΑΓ
࣮ྫʢmrubyར༻ʣ •ҎԼͷΑ͏ͳ acceptor.rbΛ ༻ҙ͢Δ
࣮ྫʢmrubyར༻ʣ •ҎԼͷinvokerΛܦ༝ͯ͠ϓϩάϥϜΛ ىಈɺ listen(3) ΛݺͿ
listen(2) ͷىಈݕ •acceptor.rb ଆͷίϯιʔϧͰڐՄ/ېࢭΛ੍ޚՄೳɻ •ېࢭͨ͠Βͦͷ··ىಈࣦഊͯ͠invokerϓϩηε͕མͪΔ •ڐՄͨ͠ΒԿͳ͔͔ͬͨͷΑ͏ʹɺىಈΛܧଓͯ͠Ϧοεϯɻ
listen(2) ͷىಈݕ ېࢭ࣌ͷग़ྗ ڐՄ࣌ͷग़ྗ
Ԡ༻ʁ •ʮҙͷϥΠϒϥϦؔݺͼग़͠ʯͰϓϩηεΛఀࢭɺCRIU(*)ʹΑΓ ϓϩηεμϯϓΛ࡞͢Δ࣮ݧΛߦͬͨɻ •LD_PRELOAD + ϥούؔ + ʮԿ͠ͳ͍ʯsyscall + seccomp
IUUQTVE[VSBIBUFOBCMPHKQFOUSZ $IFDLQPJOUBOE3FTUPSF*O6TFSTQBDF ϓϩηεͷঢ়ଶΛอଘɺ͔ͦ͜Β࠶ੜ͢Δٕज़ IUUQTDSJVPSH.BJO@1BHF
ߟ
৽ٕज़ʹΑΓͰ͖Δ͜ͱ૿͑Δ͕... •৽ٕज़ͷʮग़ݱʯͱʮීٴʯͷλΠϛϯάζϨΔ •ͨͱ͑ cgroup v2ͷॳग़2013ɻ •2019 ~ 2020 ʹϥϯλΠϜͰͷରԠ͕ਐΜͩଆ໘ •पลͷπʔϧ͕ग़ݱ͢ΔͷͬͱઌͰ͋Ζ͏
•ग़ݱظʹ୯ମͰٕज़Λݕূ͠ɺʢηΩϡϦςΟؚΊʣͲ͏͍͏͕ ͋Δ͔ɺͲ͏͍͏Մೳੑ͕͋Δ͔ݕূ͢Δҙٛେ͖͍
eBPF Linuxͷجຊٕज़ʹͳΓͭͭ͋Δ •ద༻ൣғ͕ͲΜͲΜ·͍ͬͯΔ •τϨʔγϯάɺଳҬ੍ޚωοτϫʔΩϯάͷ΄͔ɺcgroup(v2) deviceͷఆɺLSM BPF programͳͲͳͲ... •ηΩϡϦςΟͷจ຺ͰτϨʔεɺࠪɺҟৗݕͱ͔ܽͤͳ͍ٕज़ ʹͳΔ͜ͱ͕૾͞ΕΔ •Ұͭͷprog
typeʹ৮͓͚ͬͯͩ͘Ͱײ͔֮Γͦ͏