Rustでon-diskなB+Treeを作ったときの細かな話

Rust LT Online #3 で発表したスライドです
https://rust.connpass.com/event/209279/

Transcript

1. KOBA789 RustͰ
   on-diskͳB+TreeΛ 
 ࡞ͬͨͱ͖ͷ
   ࡉ͔ͳ࿩
2. KOBA789 ݱࡏਐߦܗͰແ৬ ‣ 3VTUྺ͸೥͘Β͍
   ‣ ී௨ʹ΢ΣϒαʔόʔΛॻ͍ͨΓ
   ‣ %#.4Λॻ͍ͨΓ
   ‣ ϒϥ΢βͰಈ͘ճ࿏$"%Λॻ͍ͨΓ 
 8"4.λʔήοτ
   ‣ ࣗ࡞ΩʔϘʔυͷ 
 ϑΝʔϜ΢ΣΞΛॻ͍ͨΓ 
 ͍ΘΏΔ૊ΈࠐΈ3VTU ͓͢͢Ίͷ࢓ࣄʹؔ͢Δ৘ใ͸ 
 ͍ͭͰ΋׻ܴ͍ͯ͠·͢ https://gihyo.jp/magazine/wdpress/archive/2021/vol122 ʲະঝ୚޿ࠂʳࡶࢽʹهࣄॻ͖·ͨ͠
3. RustͰ
   on-diskͳB+TreeΛ 
 ࡞ͬͨͱ͖ͷ
   ࡉ͔ͳ࿩ ࠓճͷλΠτϧ 
 ʢ࠶ܝʣ
4. RustͰ
   on-diskͳB+TreeΛ 
 ࡞ͬͨͱ͖ͷ
   ࡉ͔ͳ࿩ ͱ͸ʁ
5. RustͰ
   on-diskͳB+TreeΛ 
 ࡞ͬͨͱ͖ͷ
   ࡉ͔ͳ࿩ ͱ͸ʁ " ಺༰ΛσΟεΫʹӬଓԽͰ͖ͨΓɺ 
 ϝϞϦΑΓେ͖ͳ಺༰ΛσΟεΫ্Ͱѻ͑ͨΓ͢Δ 
 # 5SFF ˞
   ಠࣗͷఆٛͰ͢
6. ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ͜͏͍͏# 5SFFΛߟ͑Δ
7. ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ͱΓ͋͑ͣ࣍ݩʹͭͿ͢
8. σΟεΫʢϑΝΠϧʣ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ σΟεΫ͕͋Δͱ͢Δ
9. σΟεΫʢϑΝΠϧʣ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ σΟεΫʹೖͬͨ
10. σΟεΫʢϑΝΠϧʣ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ CPU # 5SFFΛૢ࡞͍ͨ͠
11. σΟεΫʢϑΝΠϧʣ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ CPU # 5SFFΛૢ࡞͍ͨ͠ σΟεΫΛ௚઀ಡΈॻ͖͢Δͷ͸஗͍͠ෆศ
12. ϝϞϦ σΟεΫʢϑΝΠϧʣ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ CPU ϝϞϦʹόοϑΝΛ༻ҙ
13. ϝϞϦ σΟεΫʢϑΝΠϧʣ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ CPU
14. ϝϞϦ σΟεΫʢϑΝΠϧʣ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ CPU ϊʔυͷҰ෦ΛϝϞϦ΁
15. ϝϞϦ σΟεΫʢϑΝΠϧʣ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ CPU ಡΈॻ͖ ೖΕସ͑ όοϑΝϓʔϧϚωʔδϟ
16. ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ Ұ෦ͷϊʔυͷΈ͕ϝϞϦʹ͋Δ# 5SFFͱ͍͑Δ ന͍ϊʔυ͸σΟεΫʹ͔͠ͳ͍
17. ϝϞϦ σΟεΫʢϑΝΠϧʣ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ CPU ಡΈॻ͖ ೖΕସ͑ ͱ͜ΖͰɺ
18. ϝϞϦ σΟεΫʢϑΝΠϧʣ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ CPU ಡΈॻ͖ ೖΕସ͑ ϚϧνίΞΛ׆༻͍ͨ͠Ͱ͢ΑͶʁ CPU CPU CPU
19. ϝϞϦ σΟεΫʢϑΝΠϧʣ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ CPU ར༻தͷϊʔυΛόοϑΝ͔Β௥͍ग़ͨ͠Γ͢Δͱ͍ͨ΁Μͳ͜ͱʹ CPU CPU CPU ΛಡΈ͍ͨɻΛ௥͍ग़ͦ͏ ଴ͯίϥ ϊʔυ ར༻த
20. ϝϞϦ σΟεΫʢϑΝΠϧʣ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ CPU ར༻தͷϊʔυΛόοϑΝ͔Β௥͍ग़ͨ͠Γ͢Δͱ͍ͨ΁Μͳ͜ͱʹ CPU CPU CPU ΛಡΈ͍ͨɻΛ௥͍ग़ͦ͏ ଴ͯίϥ ϊʔυ ར༻த ͜ͷ΁Μͷ 
 σʔλߏ଄ɾΞϧΰϦζϜΛ 
 3VTUͰͲ͏͢Δ͔ POEJTLಛ༗ͷ೉͕͋͠͞Δ
21. 3VTUͰPOEJTLͳ# 5SFFΛ࡞Δͱ͖ͷݒ೦ࣄ߲ w όοϑΝϓʔϧϚωʔδϟͷ 
 σʔλߏ଄ͱΞϧΰϦζϜ
    w # 5SFFͷϊʔυͷσʔλߏ଄
    w # 5SFFͷഉଞ੍ޚ
22. όοϑΝϓʔϧϚωʔδϟͷ 
 σʔλߏ଄ͱΞϧΰϦζϜ
23. ར༻தͷϊʔυΛอޢ͢Δʹ͸ $.6ͷߨٛͰ࢖ΘΕ͍ͯΔ#VT5VC $ ͱ͍͏࣮૷ͷ৔߹ ϊʔυʢϖʔδʣ಺ʹΧ΢ϯλΛ 
 ࣋ͬͯ؅ཧ͍ͯ͠Δ https://github.com/cmu-db/bustub/blob/master/src/include/storage/page/page.h ϐϯ 1JO ͸ʮར༻தʯͷҙຯ
    ˞3VTUͷ1JOͱ͸ؔ܎ͳ͍
24. 3VTUͰ͸Ͳ͏͢Δ͔ Ҋ
    $ ಉ༷ʹΧ΢ϯλͰอޢ w ڞ༗ϝϞϦΛॻ͖׵͑ΔͨΊɺ 
 ౰વͷΑ͏ʹVOTBGF͕ొ৔
    w ઈରʹόάΒͤΔࣗ৴͔͠ͳ͍
    w ͜ΕͳΒ$ Ͱॻ͚͹͍͍
    w ଈߦͰϘπ
25. 3VTUͰ͸Ͳ͏͢Δ͔ Ҋ
    3X-PDLͰอޢ w ར༻த͸4IBSFEMPDLΛ͓࣋ͬͯ͘
    w ௥͍ग़ͨ͘͠ͳͬͨΒ&YDMVTJWFMPDLΛऔΔ
    w (VBSEܥͷߏ଄ମ͸
    TUBUJD
    ͡Όͳ͍ͷͰ 
 औΓճ͕͠ѱ͍
26. 3VTUͰ͸Ͳ͏͢Δ͔ Ҋ
    "SDͰอޢ w "SD
    ʹ͸HFU@NVU NVU
    "SD5 ͱ͍͏ 
 ϝιου͕͋Δ
    w ࢀরΧ΢ϯλ͕ͷͱ͖ʹ͚ͩNVU
    5͕औΕΔ
    w όοϑΝϓʔϧͷʮϐϯʯͱ૬ੑ͕Αͦ͞͏ͳػೳ
    w ͨͩ͠Ϩγʔό͸NVU
    w όοϑΝϓʔϧશମΛอޢ͢Δ.VUFY͕͋ΔͷͰ 
 Մมࢀর͕औΕΔ
27. # 5SFFͷϊʔυͷ 
 σʔλߏ଄
28. # 5SFFͷϊʔυͷߏ଄ ૉ๿ʹ΍Δͱɺ͜͏
29. ⚠
    ϊʔυ͸σΟεΫʹ 
 ॻ͖ग़ͤͳ͚Ε͹ͳΒͳ͍ ΋ͪΖΜɺಡΈग़͠΋Ͱ͖ͳ͚Ε͹ͳΒͳ͍
30. 2
    ߏ଄ମΛσΟεΫʹ 
 ॻ͖ग़͍ͨ͠ͱ͖ɺͲ͏͢Δʁ
31. 2
    ߏ଄ମΛσΟεΫʹ 
 ॻ͖ग़͍ͨ͠ͱ͖ɺͲ͏͢Δʁ TFSEF
32. 2
    ߏ଄ମΛσΟεΫʹ 
 ॻ͖ग़͍ͨ͠ͱ͖ɺͲ͏͢Δʁ TFSEF γϦΞϥΠζޙͷσʔλ௕͕Θ͔Βͳ͍͠
    ޮ཰΋ѱ͍ͷͰμϝ
33. 2
    ߏ଄ମΛσΟεΫʹ 
 ॻ͖ग़͍ͨ͠ͱ͖ɺͲ͏͢Δʁ "
    ͦͷ··ॻ͖ग़ͤΔ 
 σʔλߏ଄Λ͕Μ͹࣮ͬͯ૷͢Δ
34. ͦͷ··ॻ͖ग़ͤΔσʔλߏ଄ ৗʹݻఆ௕ͷόΠτྻͷΈʹऩ·Δσʔλߏ଄ ϊʔυ ͖ͬ͞ొ৔ͨ͠ϊʔυ͘Μ
35. ͦͷ··ॻ͖ग़ͤΔσʔλߏ଄ ৗʹݻఆ௕ͷόΠτྻͷΈʹऩ·Δσʔλߏ଄ ,# ࣮͸͜͏͍͏େ͖͞
36. ͦͷ··ॻ͖ग़ͤΔσʔλߏ଄ ৗʹݻఆ௕ͷόΠτྻͷΈʹऩ·Δσʔλߏ଄ ,# ,FZ 7BMVF ͜Ε͸,FZ7BMVFͷϖΞ
37. ͦͷ··ॻ͖ग़ͤΔσʔλߏ଄ ৗʹݻఆ௕ͷόΠτྻͷΈʹऩ·Δσʔλߏ଄ ,# ,FZ 7BMVF ,FZ 7BMVF ,FZ 7BMVF ௕͞͸όϥόϥ
38. ͦͷ··ॻ͖ग़ͤΔσʔλߏ଄ ৗʹݻఆ௕ͷόΠτྻͷΈʹऩ·Δσʔλߏ଄ ,# ,FZ 7BMVF ,FZ 7BMVF ,FZ 7BMVF ͜ΕΛͦͷ··ෑ͖٧ΊΔͱ
39. ͦͷ··ॻ͖ग़ͤΔσʔλߏ଄ ৗʹݻఆ௕ͷόΠτྻͷΈʹऩ·Δσʔλߏ଄ ,# ,FZ 7BMVF ,FZ 7BMVF ,FZ 7BMVF Φϑηοτ͕౳ִؒͰ͸ͳ͍ͷͰೋ෼୳ࡧͰ͖ͳ͍
40. ͦͷ··ॻ͖ग़ͤΔσʔλߏ଄ ৗʹݻఆ௕ͷόΠτྻͷΈʹऩ·Δσʔλߏ଄ ,# ,FZ 7BMVF ,FZ 7BMVF ,FZ 7BMVF ࢓੾Γ௚͠
41. ͦͷ··ॻ͖ग़ͤΔσʔλߏ଄ ৗʹݻఆ௕ͷόΠτྻͷΈʹऩ·Δσʔλߏ଄ ,# ,FZ 7BMVF ,FZ 7BMVF ,FZ 7BMVF ,FZ7BMVF͸ޙΖʹ٧Ίͯ
42. ͦͷ··ॻ͖ग़ͤΔσʔλߏ଄ ৗʹݻఆ௕ͷόΠτྻͷΈʹऩ·Δσʔλߏ଄ ,# 1US ,FZ 7BMVF ,FZ 7BMVF ,FZ 7BMVF 1US 1US ݻఆ௕ͷϙΠϯλΛ಄ʹฒ΂Δ
43. ͦͷ··ॻ͖ग़ͤΔσʔλߏ଄ ৗʹݻఆ௕ͷόΠτྻͷΈʹऩ·Δσʔλߏ଄ ,# 1US ,FZ 7BMVF ,FZ 7BMVF ,FZ 7BMVF 1US 1US ͜ΕΛ3VTUͰॻ͘
44. ͱ͜ΖͰ͜ΕΛͲ͏΍ͬͯ 
 Ωϟετ͢Δͷ͔
45. std::mem::transmute
46. std::mem::transmute ͦΜͳةݥͳ΋ͷ࢖͏Θ͚ͳ͍
47. ศརDSBUF
    
    l[FSPDPQZz ΞϥΠϝϯτͳͲΛߟྀͯ҆͠શʹܕΩϟετͰ͖Δ w (PPHMFͷ'VDITJBͰ࢖ΘΕ͍ͯΔͬΆ͍
    w ࢖͍ํʹ͔ͳΓΫη͕͋Δ
    w υΩϡϝϯτ͸ۃΊͯগͳ͍
    w αϯϓϧίʔυ͸'VDITJBͷωοτϫʔΫελοΫ
48. LayoutVerifiedͱ͍͏ 
 εϚʔτϙΠϯλܦ༝Ͱૢ࡞͢Δ new_from_prefixϝιου͸ 
 ϔομͱϘσΟʹ෼͔Ε͍ͯΔ 
 σʔλߏ଄ʹ࢖͑Δ
49. 1US ,FZ 7BMVF ,FZ 7BMVF ,FZ 7BMVF 1US 1US OVN@TMPUT IFBEFS CPEZ
50. new_sliceϝιουͰ 
 ϙΠϯλͷεϥΠεΛ੾Γग़͍ͯ͠Δ
51. 1US ,FZ 7BMVF ,FZ 7BMVF ,FZ 7BMVF 1US 1US OVN@TMPUT QPJOUFST IFBEFS CPEZ
52. 1US ,FZ 7BMVF ,FZ 7BMVF ,FZ 7BMVF 1US 1US OVN@TMPUT QPJOUFST IFBEFS CPEZ ͜ͷਤΛΑ͘ݟͯ΄͍͠
53. 1US ,FZ 7BMVF ,FZ 7BMVF ,FZ 7BMVF 1US 1US OVN@TMPUT QPJOUFST IFBEFS CPEZ QPJOUFST͸CPEZΛआ༻͍ͯ͠Δ
54. 1US ,FZ 7BMVF ,FZ 7BMVF ,FZ 7BMVF 1US 1US OVN@TMPUT QPJOUFST IFBEFS CPEZ QPJOUFST͸CPEZΛआ༻͍ͯ͠Δ ͭ·Γɺ,FZ7BMVFϖΞΛมߋ͢Δͱ͖ʹ͸ 
 ͍͍ͪͪQPJOUFSTΛख์͞ͳ͚Ε͹ͳΒͳ͍ ͩΔ͍ 
 ʢ͔ͨ͠ͳ͠ʣ
55. # 5SFFͷഉଞ੍ޚ
56. ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ CPU CPU
57. ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ CPU CPU Modify Modify 💥 😩 😩
58. ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ CPU CPU ϩοΫ͸্͔ΒॱʹऔΔ
59. ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϩοΫ͸্͔ΒॱʹऔΔ CPU CPU
60. ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϩοΫ͸্͔ΒॱʹऔΔ CPU CPU
61. ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϩοΫ͕औΕͨΒɺ্ͷϩοΫ͸ख์ͯ͠΋Α͍ CPU CPU
62. ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ଞͷεϨου͕ಈ͚ͯฒྻ౓্͕͕Δ CPU CPU
63. ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ CPU CPU ಉ༷ʹଓ͚Δ
64. ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ Լͷϊʔυ͕ᷓΕͦ͏ͳ৔߹͸ɺ্ͷϩοΫΛख์ͯ͠͸ͳΒͳ͍ 
 ʢॻ͖ࠐΈͷ৔߹ʣ CPU CPU
65. ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ CPU CPU ͜ΕΛ3VTUͰ࣮૷͢Δ
66. # 5SFFͷഉଞ੍ޚ ૉ๿ʹ࣮૷
67. # 5SFFͷഉଞ੍ޚ ૉ๿ʹ࣮૷
68. # 5SFFͷഉଞ੍ޚ ૉ๿ʹ࣮૷ page: 
 Arc<RwLock<Page>> ro_page: 
 RwLockReadGuard<_, Page> next_page: 
 Arc<RwLock<Page>> ro_next_page: 
 RwLockReadGuard<_, Page> next_pageΛಈ͔͍͕ͨ͠ かといってRwLockReadGuardΛ 
 ख์͢Θ͚ʹ͸͍͔ͳ͍ ro_page͕pageΛɺ 
 ro_next_page͕next_pageΛ 
 ࢀর͍ͯ͠ΔͷͰಈ͔ͤͳ͍
69. # 5SFFͷഉଞ੍ޚ ૉ๿ʹ࣮૷ page: 
 Arc<RwLock<Page>> ro_page: 
 RwLockReadGuard<_, Page> next_page: 
 Arc<RwLock<Page>> ro_next_page: 
 RwLockReadGuard<_, Page> next_pageΛಈ͔͍͕ͨ͠ かといってRwLockReadGuardΛ 
 ख์͢Θ͚ʹ͸͍͔ͳ͍ ro_page͕pageΛɺ 
 ro_next_page͕next_pageΛ 
 ࢀর͍ͯ͠ΔͷͰಈ͔ͤͳ͍ ٧Έ
70. ͜Ε͸Α͋͘Δ4UBCMF"EESFTT໰୊ "SDͷࢦ͢ϙΠϯλ͸ෆมͳΜ͔ͩΒNPWFͤͯ͘͞Ε w PXOJOH@SFGΫϨʔτͳͲͰ 
 ղܾ͕ࢼΈΒΕ͍ͯΔ
    w ࠓճ͸3X-PDL3FBE(VBSEʹ 
 ಛघͳϝιουΛੜ΍ͯ͠ճආ
71. 0XOFE3X-PDL3FBE(VBSE ΊͬͪΌVOTBGF͚ͩͲ͔ͨ͠ͳ͍
72. ࠓ೔঺հͨ͠ίʔυ͸͍͍ͩͨ(JU)VCʹ͋Δ RQ w IUUQTHJUIVCDPN,0#"RQ
    w ϚϧνεϨου
    w ίʔυ͕ͱͯ΋Ԛ͍ SFMMZ w IUUQTHJUIVCDPN,0#"SFMMZ
    w γϯάϧεϨου
    w 8&# %#
    13&44
    7PMͷͨΊʹ 
 ॻ͖ԼΖͨ͠
    w ͪΐͬͱίʔυ͕៉ྷ
73. ·ͱΊ ಈతͳσʔλߏ଄Λ࣮૷͢Δͱ3VTUಛ༗ͷ೉͕ͯ͠͞Μ͜੝Γʂ w "SDHFU@NVUͷΑ͏ͳϚΠφʔͳ 
 ϝιουΛ஌͓ͬͯ͘ͱ໾ʹཱͭ
    w TUE΍ΫϨʔτΛ׆͔ͤΔΑ͏ʹ 
 σʔλߏ଄Λมܗ͢ΔεΩϧ͕ॏཁ
    w ίϯύΠϧΤϥʔΛோΊΔ͚ͩͰͳ͘ɺ 
 ϝϞϦͷ༷ࢠΛ૝૾͢Δ
    w ͋Δ΂͖ϝϞϦͷߏ଄͔Β 
 σʔλߏ଄Λٯࢉ͢Δͷ͕ίπ