Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
dsync: Efficient Block-wise Synchronization of ...
Search
Yuuki Tsubouchi (yuuk1)
May 23, 2014
Technology
2
26k
dsync: Efficient Block-wise Synchronization of Multi-Gigabyte Binary Data
論文輪読会#4
ブロックデバイスレベルで実現するrsyncより高速なバックアップについて
Yuuki Tsubouchi (yuuk1)
May 23, 2014
Tweet
Share
More Decks by Yuuki Tsubouchi (yuuk1)
See All by Yuuki Tsubouchi (yuuk1)
AIスパコン「さくらONE」のLLM学習ベンチマークによる性能評価 / SAKURAONE LLM Training Benchmarking
yuukit
2
690
とあるSREの博士「過程」 / A Certain SRE’s Ph.D. Journey
yuukit
11
4.4k
eBPFを用いたAIネットワーク監視システム論文の実装 / eBPF Japan Meetup #4
yuukit
3
1.1k
クラウドのテレメトリーシステム研究動向2025年
yuukit
4
1.1k
博士論文公聴会: Scaling Telemetry Workloads in Cloud Applications: Techniques for Instrumentation, Storage, and Mining / PhD Defence
yuukit
1
260
博士学位論文予備審査 / Scaling Telemetry Workloads in Cloud Applications: Techniques for Instrumentation, Storage, and Mining
yuukit
1
2.1k
MetricSifter:クラウドアプリケーションにおける故障箇所特定の効率化のための多変量時系列データの特徴量削減 / FIT 2024
yuukit
2
300
工学としてのSRE再訪 / Revisiting SRE as Engineering
yuukit
19
14k
Cloudless Computingの論文紹介
yuukit
2
600
Other Decks in Technology
See All in Technology
GoでもGUIアプリを作りたい!
kworkdev
PRO
0
140
能登半島地震で見えた災害対応の課題と組織変革の重要性
ditccsugii
0
960
ガバメントクラウド(AWS)へのデータ移行戦略の立て方【虎の巻】 / 20251011 Mitsutosi Matsuo
shift_evolve
PRO
2
200
All About Sansan – for New Global Engineers
sansan33
PRO
1
1.2k
AI Agent Dojo #2 watsonx Orchestrateフローの作成
oniak3ibm
PRO
0
120
Performance Insights 廃止から Database Insights 利用へ/transition-from-performance-insights-to-database-insights
emiki
0
280
新規事業におけるGORM+SQLx併用アーキテクチャ
hacomono
PRO
0
290
【Kaigi on Rails 事後勉強会LT】MeはどうしてGirlsに? 私とRubyを繋いだRail(s)
joyfrommasara
0
270
スタートアップにおけるこれからの「データ整備」
shomaekawa
2
480
Contract One Engineering Unit 紹介資料
sansan33
PRO
0
8.8k
[Codex Meetup Japan #1] Codex-Powered Mobile Apps Development
korodroid
2
820
Railsの話をしよう
yahonda
0
140
Featured
See All Featured
Build your cross-platform service in a week with App Engine
jlugia
232
18k
Fashionably flexible responsive web design (full day workshop)
malarkey
407
66k
How To Stay Up To Date on Web Technology
chriscoyier
791
250k
Designing for Performance
lara
610
69k
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
stephaniewalter
285
14k
Into the Great Unknown - MozCon
thekraken
40
2.1k
How to Create Impact in a Changing Tech Landscape [PerfNow 2023]
tammyeverts
55
3k
Side Projects
sachag
455
43k
実際に使うSQLの書き方 徹底解説 / pgcon21j-tutorial
soudai
PRO
189
55k
Producing Creativity
orderedlist
PRO
347
40k
Designing for humans not robots
tammielis
254
26k
The Power of CSS Pseudo Elements
geoffreycrofte
79
6k
Transcript
dsync: Efficient Block-wise Synchronization of Multi Gigabyte Binary data Thomas
Knauth and Christof Fetzer, Technische Universität Dresden ! LISA’13 Best Paper Award id:y_uuki 2014/05/22 จྠಡձ#4
Agenda Backup Problem Implementation Device mapper Evaluation Discussion Conclusion
Better Backup • ωοτϫʔΫτϥϑΟοΫͷ࠷খԽ • νΣοΫαϜܭࢉͷCPUίετ͕ͳ͍ • σΟεΫͷread/writeͷ࠷খԽ • OSͷϖʔδΩϟογϡԚછͷ࠷খԽ
Backup ͷલఏ • ఆظతʹͳΔ͘සൟʹόοΫΞοϓ͍ͨ͠ • σʔλࣗମʹ΄ͱΜͲมߋ͕ͳ͍ • શσʔλྔʹରͯ͠มߋͷ͋ͬͨσʔλ ͔ᷮ
Problem • nc/scp: શσʔλίϐʔ • ωοτϫʔΫτϥϑΟοΫେ • 10Gbps Ethernet,100GB, 83
sec (1.2GB/s) • rsync: ͚ࠩͩసૹ • ࠩܭࢉͷͨΊʹશσʔλಡΈग़͠ඞཁ • νΣοΫαϜܭࢉͷCPUίετߴ͍ • OSͷΩϟογϡԚછ
rsync ᶃ ૹ৴ଆͱड৴ଆͷಉظ͍ͨ͠ϑΝΠϧΛݻఆϒϩοΫʹׂ ! ϒϩοΫ୯ҐͰࠩΛௐͯɺࠩͷ͋ΔϒϩοΫ͚ͩసૹ͍ͨ͠ ! ᶄ ֤ϒϩοΫʹରͯ͠νΣοΫαϜΛܭࢉ͠ɺϒϩοΫͷ༰ͷΘ ΓʹνΣοΫαϜ͚ͩΛૹ৴ !
ᶅ νΣοΫαϜΛൺֱ͕ͯࠩ͋͠Δ͔Ͳ͏͔ΛνΣοΫ ऑ͍νΣοΫαϜ(ܭࢉίετ)ͱڧ͍νΣοΫαϜ(ܭࢉίετߴ) ऑ͍νΣοΫαϜʢϩʔϦϯάνΣοΫαϜʣͰࠩͷ͋ΔϒϩοΫΛ ચ͍ग़ͯ͠ɺڧ͍νΣοΫαϜͰ࣮֬ʹࠩνΣοΫ
Idea • όοΫΞοϓ࣌ʹมߋՕॴΛܾఆ… • νΣοΫαϜΛൺֱ͢Δ͔͠ํ๏͕ͳ͍ • ࠷ॳ͔ΒมߋՕॴΛτϥοΩϯά͢Δ
Implementation • ϒϩοΫσόΠεϨϕϧͰมߋ͞Εͨϒϩο ΫͷτϥοΩϯάใΛอ࣋ • τϥοΩϯάใ: ϒϩοΫ͝ͱͷมߋ༗ແͷ ϑϥά ʢ1bit /
blockʣ • ߹ܭσʔλ: 4TiB -> ϑϥά: 128 MiB
Interface • ϢʔβεϖʔεΠϯλϑΣʔε • ֤σόΠε͝ͱʹ /proc ҎԼʹରԠ͢ΔϑΝΠϧ͕Ͱ͖Δ • /proc/mydev: ϒϩοΫ൪߸ͷϦετɻ͜Εʹॻ͖ࠐΉͱ
bit vector ॳظԽ͞ΕΔ • dmextract: มߋͷ͋ͬͨϒϩοΫͷநग़ stdout:ʢϒϩο Ϋ൪߸, σʔλʣ • dmextract mydev | ssh remotehost dmmerge /dev/ mapper/mydev
Block Device ϒϩοΫσόΠευϥΠό ετϨʔδσόΠε(HDD/SSD) ൚༻ϒϩοΫσόΠευϥΠό ΞϓϦέʔγϣϯ ϖʔδΩϟογϡ ϑΝΠϧγεςϜ ϑΝΠϧͷಡΈॻ͖ཁٻ ϑΝΠϧͱσόΠεͷϒϩοΫͱͷϚοϐϯά
*0ཁٻΛσόΠεʹదͨ͠ܗʹฒସ͑ͳͲ 3".ʹಡΈॻ͖σʔλΛΩϟογϡ ݸʑͷϋʔυΣΞʹ͋Θͤͨॲཧ ε τ Ϩ c δ σ ό Π ε ந Խ
Device mapper http://lc.linux.or.jp/lc2009/slide/T-02-slide.pdf ϒϩοΫσόΠευϥΠό ετϨʔδσόΠε(HDD/SSD) ൚༻ϒϩοΫσόΠευϥΠό ΞϓϦέʔγϣϯ ϖʔδΩϟογϡ ϑΝΠϧγεςϜ ϑΝΠϧͷಡΈॻ͖ཁٻ
ϑΝΠϧͱσόΠεͷϒϩοΫͱͷϚοϐϯά *0ཁٻΛσόΠεʹదͨ͠ܗʹฒସ͑ͳͲ 3".ʹಡΈॻ͖σʔλΛΩϟογϡ ݸʑͷϋʔυΣΞʹ͋Θͤͨॲཧ ε τ Ϩ c δ σ ό Π ε ந Խ %FWJDF NBQQFS ϒϩοΫͷಡΈॻ͖ཁٻΛ ͍Ζ͍Ζม
Device mapper (1) • ෳͷཧϒϩοΫσόΠεΛҰͭͷཧσόΠεͱͯ͠ ଋͶΒΕΔ • Mirror, Stripe, Snapshot
• RAID 0,1,5,10 • Snapshot: ཧσόΠεͷશมߋΛཧσόΠεʹϦμ ΠϨΫτ(Copy on Write) • ͋ͱͰཧσόΠεͱཧσόΠεΛϚʔδՄೳ • ཧσόΠε͔ΒόοΫΞοϓσόΠεʹϦϞʔτϦμ ΠϨΫτͰόοΫΞοϓ࡞ΕΔʁʁ
Device mapper (2) • 2ͭͷࢹ͕͔͚͍ͯΔ -> Snapshot Ͱແཧ • શมߋΛҰ࣌తʹཧσόΠε͕όοϑΝ͢Δඞ
ཁ͕͋Δ όοϑΝ͕͋;Εͯσʔλϩετ • ΦϦδφϧσʔλΛόοΫΞοϓઌͰϚʔδ͢Δ ඞཁ͕͋Δ • Device mapper ϒϩοΫมߋΛτϥοΩϯά͢Δ ͨΊͷશͯͷใΛͭ • liner mapping mode ࣌ͷ’map’ function
Architecture • ཧσόΠεʹରͯ͠τϥοΩϯά͢Δ͔ΘΓʹɺɹ ϧʔϓόοΫσόΠεʹରͯ͠τϥοΩϯά͢Δ • ϧʔϓόοΫσόΠε: ҰൠతͳϑΝΠϧΛϒϩοΫ σόΠεͰ͋Δ͔ͷΑ͏ʹѻ͏ͨΊͷػೳ ϒϩοΫσόΠε %FWJDFNBQQFS
ΞϓϦέʔγϣϯ ϒϩοΫσόΠε ϑΝΠϧγεςϜ ϧʔϓόοΫσόΠε %FWJDFNBQQFS ΞϓϦέʔγϣϯ
Data Structure • RAM্ʹมߋใΛ1ϒϩοΫ͋ͨΓ1bitͰͭ • 1bit ͷཁૉΛͭϒϩοΫͷྻ • ϝϞϦΞϩέʔγϣϯͷ •
kmalloc(), __get_free_pages(), vmalloc() • vmalloc() ͷΈ࣮֬ʹϝΨόΠτ୯ҐͰ֬อՄೳ • kmalloc() εϥϒΦϒδΣΫτ੍ݶ͕͋Δ(32MiB)ɺvmalloc ϖʔδ୯ҐͰ֬อ • ΦϯϝϞϦͳσʔλߏͳͷͰɺγϟοτμϯ࣌ʹτϥοΩϯ άใΛϩετ͢Δ • γϟοτμϯ࣌ʹτϥοΩϯάใΛσΟεΫʹॻ͖ࠐΜͰɺ ىಈ࣌ʹಡΈग़͢ • յΕ͍ͯΕϑϧಉظ
Evaluation (tools) • scp/nc • rsync • blockmd5sync • rsync
ͷϩʔϦϯάνΣοΫαϜͳ͠൛ • ZFS • features: ཧϘϦϡʔϜɺsnapshotɺ2ͭͷsnapshotͷࠩநग़ • ϒϩοΫσόΠεϨϕϧͩͱΞΫηεͰ͖ͳ͍ใ: /tmp ͚ͩແࢹ͢Δ • dsync • ϑΝΠϧγεςϜʹґଘ͠ͳ͍ • ϑΝΠϧγεςϜͷใ͕ͳ͍ͷͰ੍ݶ͋Δ (mtime ͳͲ) • νΣοΫαϜܭࢉͷ͔ΘΓʹɺϒϩοΫ͝ͱʹτϥοΩϯάεςʔλεΛͨ ͤΔ
Evaluation (Benchmarks) • 6-core AMD Phenom II processor • 2
TB spinning disk (Samsung HD204UI), • 128 GB SSD (Intel SSDSC2CT12) • εΠον͝͠ʹΪΨϏοτΠʔαωοτͰଓ
ಉظ࣌ؒ HDD/SSD rsync: 33min, dsync: 7 min )%% 44%
CPUར༻ STZODड৴ଆνΣοΫαϜ STZODૹ৴ଆνΣοΫαϜ ίΞ͍ͬͯΔ
ωοτϫʔΫଳҬ
Discussion • ৗʹ dsync > rsync • rsync dsync
ͷεʔύʔηοτ • rsync dsync ͱಉ͡Α͏ʹશͯͷߋ৽͞ΕͨϒϩοΫ Λread/transmit/merge͢Δ • rsync dsync ʹՃ͑ͯɺߋ৽ϒϩοΫΛܾఆ͢ΔͨΊ ʹɺ”શͯͷϒϩοΫ”Λreadɺchecksumܭࢉ͕ඞཁ • bit vector ͷߋ৽Φʔόϔου • ΦϯϝϞϦ͔ͩΒେͨ͜͠ͱͳ͍
Conclusion • ఆظతͳڊେόΠφϦσʔλͷޮతͳಉظํ ๏ͷఏҊ/࣮ • νΣοΫαϜΛܭࢉ͢ΔΘΓʹɺΦϯϥΠϯ ͰมߋΛτϥοΩϯά • Linux kernel
ͷ Device mapper֦ு • dmextract and dmmerge • rsycnc vs dsync, 32 min vs 7 min
ײ • rsync ͕͕͢͞ʹ͔Θ͍ͦ͏ͳͷͰɺసૹྔ͕͖͍ͯ ͘ΔΠϯλʔωοτܦ༝ͷಉظ࣌ؒൺֱ͕͋ΔͱΑ͞ ͦ͏
Linux 3.2 kernel module patch https://bitbucket.org/tknauth/devicemapper/