Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
3 state-of-the-art technologies in Linux and future of the containers #SECKUN
Search
KONDO Uchio
February 21, 2021
Technology
2
560
3 state-of-the-art technologies in Linux and future of the containers #SECKUN
2021.02.21 「新しいセキュリティビジネスキャリア」シンポジウム
KONDO Uchio
February 21, 2021
Tweet
Share
More Decks by KONDO Uchio
See All by KONDO Uchio
大規模レガシーテストを 倒すための CI基盤の作り方 / #CICD2023
udzura
5
2.2k
Ruby x BPF in Action / RubyKaigi 2022
udzura
0
160
Narrative of Ruby & Rust
udzura
0
160
開発者生産性指標の可視化 / pepabo-four-keys
udzura
3
1.5k
Talk of RBS
udzura
0
360
Re: みなさん最近どうですか? / FGN tech meetup in 2021
udzura
0
670
Dockerとやわらかい仮想化 - ProSec-IT/SECKUN 2021 edition -
udzura
2
630
Device access filtering in cgroup v2
udzura
1
720
"Story of Rucy" on RubyKaigi takeout 2021
udzura
0
650
Other Decks in Technology
See All in Technology
Kaggle LEAP Competition Solution解説&振り返り
abeja
2
220
テストプロセスのなかに アクセシビリティ検査はどのように組み込まれるのか?
shift_evolve
1
180
事業価値と Engineering 2024年版
recruitengineers
PRO
21
4.8k
わたしの業務の中に住み着いたCacoo/Cacoo has taken up residence in my work routine
yukihirochiba
0
590
T2D3超えのBill Oneで学んだ組織運営:400名超えエンジニアリング組織のVPoEに挑む
sansantech
PRO
1
180
検索結果の品質向上
recruitengineers
PRO
1
530
サイボウズ 開発本部採用ピッチ / Cybozu Engineer Recruit
cybozuinsideout
PRO
9
40k
Go×LLMで新たなコード生成の可能性を探る / GolangDeveloperNight_Go×LLM
optim
0
120
新リポジトリを作るときフロントエンドで気をつけたかった5つのコト/ Five things I wanted to pay attention to on the front end when creating a new repository
kaminashi
3
400
実践オブジェクト指向設計 〜Javaで学ぶオブジェクト指向(応⽤)〜
recruitengineers
PRO
1
960
OSINT CTFを支える技術
xryuseix
1
400
20240811_若手エンジニアふんわりLT資料
tomi_t0mmy
0
260
Featured
See All Featured
Large-scale JavaScript Application Architecture
addyosmani
507
110k
Raft: Consensus for Rubyists
vanstee
134
6.5k
Practical Orchestrator
shlominoach
185
10k
Fashionably flexible responsive web design (full day workshop)
malarkey
399
65k
Writing Fast Ruby
sferik
623
60k
Designing the Hi-DPI Web
ddemaree
277
34k
The MySQL Ecosystem @ GitHub 2015
samlambert
250
12k
Making Projects Easy
brettharned
111
5.7k
Easily Structure & Communicate Ideas using Wireframe
afnizarnur
189
16k
I Don’t Have Time: Getting Over the Fear to Launch Your Podcast
jcasabona
26
1.8k
Rails Girls Zürich Keynote
gr2m
93
13k
WebSockets: Embracing the real-time Web
robhawkes
59
7.2k
Transcript
ཁૉٕज़ͷstate of the art͔Βߟ͑Δ ۙ౻Ӊஐ࿕ / GMO Pepabo, Inc. 2021.02.21
ʮ৽͍͠ηΩϡϦςΟϏδωεΩϟϦΞʯγϯϙδϜ Linuxίϯςφͷະདྷ
GMOϖύϘגࣜձࣾ γχΞϓϦϯγύϧ ٕज़෦ٕज़ج൫νʔϜॴଐ ۙ౻Ӊஐ࿕ (@udzura)
ۙ౻Ӊஐ࿕ ུྺ • ࡾՏᅳͷਓɻچ٢ాൡߍ࣌शؗߴߍΛଔۀɺ౦ژେֶจֶ෦ຊޠຊจֶઐम՝ఔ ͷֶ࢜ଔʢ2007ʣɻ • ϚείϛͷࣾSEɺECαΠτ։ൃɺΦϯϥΠϯήʔϜ ։ൃͳͲΛܦͯ2013ΑΓݱ৬ɺಉʹԬҠॅɻ • RubyɺίϯςφɺΫϥυωΠςΟϒٕज़ͳͲͷίϛϡχςΟͰ
׆ಈɻஶॻʹʮWebͰ͑ΔmrubyγεςϜϓϩάϥϛϯάೖʯʢC&Rݚڀॴʣ • ͖ͳγεςϜίʔϧʢ࠷ۙʣ socketpair(2) ɻ
ࠓͷ͓ •ίϯςφͷཁૉٕज़ʹ͍ͭͯɺࢲͷߨٛͰʢͬ͘͟Γʣཧղ͞Εͨํ ͚ͷ༰Ͱ͢ɻཁૉٕज़ͷղઆࢀߟࢿྉΛͲ͏ͧɻ •ࢀߟ1: https://container-security.dev/ •ࢀߟ2: ʰίϯςφܕԾԽ֓ʱʢޱ, ΧοτγεςϜʣ •ۙͷΧʔωϧʹؚ·ΕΔ৽ٕज़ͷ͏ͪɺίϯςφʹؔ͢ΔͷΛ հ͠·͢ɻ
•͕࣌ؒ͋Εɺͦͷ্ͰίϯςφͷະདྷΛߟ͑·͢ɻ
cgroup v2
cgroup ͷ͓͞Β͍ •Linux Kernelͷجຊٕज़ͷҰͭɻ •ϓϩηεΛάϧʔϐϯά͠ɺͦͷάϧʔϓ୯ҐͰϦιʔεར༻ͷ੍ݶ Λ͔͚Δٕज़ɻCPUɺϝϞϦɺIOɺϓϩηε... IUUQTHJIZPKQBENJOTFSJBMMJOVY@DPOUBJOFST
cgroup v1ͷྫ •cgroupfs ͱ͍͏ϑΝΠϧγεςϜʹmkdir(2) read(2) write(2)ͳͲΛ࣮ ߦ͠ɺૢ࡞Λߦ͏
cgroup v2 •v1 ͷ͍͔ͭ͘ͷܽ - ओʹ੍ޚରʢίϯτϩʔϥʣ͝ͱʹσΟϨΫ τϦΛ͚ͳ͚Ε͍͚ͳ͍༷ - Λࠀ͘͢։ൃ͞Εͨ •େ͖ͳҧ͍ͱͯ͠ɺ
v1 ͰίϯτϩʔϥผʹσΟϨΫτϦ ΛϚϯ τɺݸผʹάϧʔϓʹॴଐͰ͖ͨͷʹର͠ɺv2ͰશίϯτϩʔϥΛ ·ͱΊͨҰͭͷσΟϨΫτϦͷΈΛϚϯτ͠ɺ·ͱΊͯάϧʔϓΛ ࡞͢ΔڍಈʹͳΔɻ •ίϯςφͱͯͪ͜͠Βͷํ͕߹͕͍͍ɻ
Unified hierarchy /sys/fs/cgroup /sys/fs/cgroup /group-a /group-b /cpu.* /memory.* /io.* ...
/cpu.* /memory.* /io.* ... /cpu /memory /blkio /group-a /group-b /group-a /group-c
ίϯςφϥϯλΠϜͰͷcgroupͷར༻ •ʢOCIܥͷʣϥϯλΠϜͰҎԼͷ2ͭͷઃఆ߲͕͋Δ •Cgroup Driver: ίϯςφʹׂΓͯΔcgroupΛͲ͏ίϯτϩʔϧ͢Δ͔ •cgroupfs: cgroupfsͷͷϑΝΠϧૢ࡞ •systemd: systemdʹΑΔཧ •Cgroup
Version: Ϧιʔε੍ݶʹ v1/v2 ͲͪΒΛར༻͢Δ͔ •/sys/fs/cgroup ʹͲͷϑΝΠϧγεςϜ͕Ϛϯτ͞ΕͯΔ͔Ͱఆ
v2 ͷ৽ػೳ •Unified Hierarchy •PSI(Pressure Stall Information) •eBPFͰcgoup IDͷऔಘ͕Մೳʹ •nsdelegate
(ඇಛݖίϯςφʹॏཁ) •clone3(2) Ͱಛఆͷcgroup෦ʹϓϩηε࡞͕Մೳʹ •ͳͲͳͲ...
e.g. PSI(Pressure Stall Information) •γεςϜશମɺ·ͨcgroup୯ҐͰར༻Ͱ͖Δෛՙͷࢦඪ •CPU, ϝϞϦ, IO Ͱ stall
ͨ͠୯Ґ࣌ؒͰͷׂ߹ ΛܭଌͰ͖Δ •e.g. 1ؒͰ45ඵؒɺάϧʔϓͷ ͋Δϓϩηε͕CPUىҼͰ Ԇͨ͠߹ɺcpu some: 75.00
e.g. eBPFͰͷτϥοΩϯάใ •bpf_get_current_cgroup_id(void) ϔϧύʔ •eBPFͷΠϕϯτ͕ى͖ͨλεΫ͕Ͳͷcgroup(v2)ʹॴଐ͍ͯ͠Δ͔ɺ ͦͷIDΛฦ͢ɻ
How cgroup-v2 and PSI Impacts Cloud Native? Uchio Kondo /
GMO Pepabo, Inc. 2019.07.23 CloudNative Days Tokyo 2019 Image from pixabay: https://pixabay.com/images/id-3193865/
eBPF per containers
eBPFٕज़ͱ •ϢʔβۭؒͰ࡞ͬͨϓϩάϥϜΛΧʔωϧͰಈ͔ٕ͢ज़ͷͻͱͭ •2012ʹseccompͷಋೖɺ2013ʹLinuxͷSDNͰͷԠ༻͕࣮͞ ΕɺͦΕҎ߱ख़͢Δ •ϑΟϧλϦϯά͕ಘҙʢtcpdump, seccomp, bpftraceʣ •ΧʔωϧͷใʹΞΫηεͰ͖Δ͕ɺةݥͳίʔυಈ͔ͳ͍ͳͲ ҆શੑ͕͋Δఔ୲อ͞Ε͍ͯΔ
eBPFͷԠ༻ྫ
ίϯςφͷeBPFτϨʔεઓུ •ઓུ͕͍͔ͭ͋͘Δ •Linux Namespace·ͨcgroup (v2)ͷใ͕ར༻Ͱ͖Δ
ྫ1: task_struct ͷใΛḷΔ •task_struct→nsproxy ͔Β namespaceͷใΛ औಘͯ͠ϑΟϧλ͢Δ ʢcxrayʣ IUUQTHJUIVCDPNNSUDDYSBZCMPCNBTUFSQLHUSBDFSPQFOPQFOHP--
ྫ2: NS/ϗετͰͷPIDΛൺֱ •BPFϓϩάϥϜͰऔಘͰ͖ͨ tidͱɺϗετͰͷtidΛ ൺֱ͠ɺҰக͠ͳ͚Ε ίϯςφͱఆ͢Δ ʢTraceeʣ • task_structґଘ IUUQTHJUIVCDPNBRVBTFDVSJUZUSBDFFCMPCNBJOUSBDFFUSBDFFCQGD-ɹ
ྫ3: cgroup helperΛར༻ IUUQTHJUIVCDPNVE[VSBDPQFODMPTFCMPCNBTUFSTSDCQGDPQFODMPTFCQGD
࣮ྫ •copenclose(8) •ۙ౻ͷPoC (BPF+Rust) •ϑϥάͰtask_struct/ cgroup v2 ID ΛΓସ͑
bpf_get_current_cgroup_id(void) を添えて Uchio Kondo / Container Runtime Meetup #3 ランタイムとcgroupの
xxxな関係 * Photo by Fukuoka City
seccomp
seccompͷ͓͞Β͍ •ϓϩάϥϜʹ͓͚ΔγεςϜίʔϧݺͼग़͠ΛϑΟϧλϦϯά͢Δ •γεςϜίʔϧͷҾͷ݅ʹΑͬͯࢦఆΛม͑ΒΕΔ •blacklist(denylist), whitelist(allowlist) ͳͲΛ࣮Ͱ͖Δ •ϑϥά͕ࡉ͔͘ଘࡏ͠ɺྫ͑γεςϜίʔϧͷauditϩάͷΈɺҙ ͷerrnoΛฦͤ͞ΔɺͳͲͷࢦఆ͕Ͱ͖Δ
seccompͷར༻(mruby)
User space notification •seccompʹΑΓγεςϜίʔϧݺͼग़͠Λݕ͠ɺͦͷڍಈΛϢʔβ ϥϯυͷϓϩάϥϜʹҕͶΔ͜ͱ͕Ͱ͖Δٕज़ •Linux 5.0 (2019/3) ͔Βͷಋೖ •அ͢Δ·ͰɺͦͷγεςϜίʔϧϒϩοΫ͢Δ
•e.g. LXCͰͷσόΠεΞΫηεͷ੍ޚ IUUQTHJIZPKQBENJOTFSJBMMJOVY@DPOUBJOFSTɹ
User space notification IUUQTHJIZPKQBENJOTFSJBMMJOVY@DPOUBJOFSTɹ • LXCͰֶͿίϯςφೖ ୈ47ճɹඇಛݖίϯςφͷՄೳੑΛ͛Δseccomp notifyػೳ ΑΓ
࣮ྫʢmrubyར༻ʣ •ҎԼͷΑ͏ͳ acceptor.rbΛ ༻ҙ͢Δ
࣮ྫʢmrubyར༻ʣ •ҎԼͷinvokerΛܦ༝ͯ͠ϓϩάϥϜΛ ىಈɺ listen(3) ΛݺͿ
listen(2) ͷىಈݕ •acceptor.rb ଆͷίϯιʔϧͰڐՄ/ېࢭΛ੍ޚՄೳɻ •ېࢭͨ͠Βͦͷ··ىಈࣦഊͯ͠invokerϓϩηε͕མͪΔ •ڐՄͨ͠ΒԿͳ͔͔ͬͨͷΑ͏ʹɺىಈΛܧଓͯ͠Ϧοεϯɻ
listen(2) ͷىಈݕ ېࢭ࣌ͷग़ྗ ڐՄ࣌ͷग़ྗ
Ԡ༻ʁ •ʮҙͷϥΠϒϥϦؔݺͼग़͠ʯͰϓϩηεΛఀࢭɺCRIU(*)ʹΑΓ ϓϩηεμϯϓΛ࡞͢Δ࣮ݧΛߦͬͨɻ •LD_PRELOAD + ϥούؔ + ʮԿ͠ͳ͍ʯsyscall + seccomp
IUUQTVE[VSBIBUFOBCMPHKQFOUSZ $IFDLQPJOUBOE3FTUPSF*O6TFSTQBDF ϓϩηεͷঢ়ଶΛอଘɺ͔ͦ͜Β࠶ੜ͢Δٕज़ IUUQTDSJVPSH.BJO@1BHF
ߟ
৽ٕज़ʹΑΓͰ͖Δ͜ͱ૿͑Δ͕... •৽ٕज़ͷʮग़ݱʯͱʮීٴʯͷλΠϛϯάζϨΔ •ͨͱ͑ cgroup v2ͷॳग़2013ɻ •2019 ~ 2020 ʹϥϯλΠϜͰͷରԠ͕ਐΜͩଆ໘ •पลͷπʔϧ͕ग़ݱ͢ΔͷͬͱઌͰ͋Ζ͏
•ग़ݱظʹ୯ମͰٕज़Λݕূ͠ɺʢηΩϡϦςΟؚΊʣͲ͏͍͏͕ ͋Δ͔ɺͲ͏͍͏Մೳੑ͕͋Δ͔ݕূ͢Δҙٛେ͖͍
eBPF Linuxͷجຊٕज़ʹͳΓͭͭ͋Δ •ద༻ൣғ͕ͲΜͲΜ·͍ͬͯΔ •τϨʔγϯάɺଳҬ੍ޚωοτϫʔΩϯάͷ΄͔ɺcgroup(v2) deviceͷఆɺLSM BPF programͳͲͳͲ... •ηΩϡϦςΟͷจ຺ͰτϨʔεɺࠪɺҟৗݕͱ͔ܽͤͳ͍ٕज़ ʹͳΔ͜ͱ͕૾͞ΕΔ •Ұͭͷprog
typeʹ৮͓͚ͬͯͩ͘Ͱײ͔֮Γͦ͏