Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Rustでon-diskなB+Treeを作ったときの細かな話

 Rustでon-diskなB+Treeを作ったときの細かな話

Rust LT Online #3 で発表したスライドです
https://rust.connpass.com/event/209279/

Bcbc80799bc32840662a366768318a9b?s=128

KOBA789

May 19, 2021
Tweet

Transcript

  1. KOBA789 RustͰ on-diskͳB+TreeΛ 
 ࡞ͬͨͱ͖ͷ ࡉ͔ͳ࿩

  2. KOBA789 ݱࡏਐߦܗͰແ৬ ‣ 3VTUྺ͸೥͘Β͍ ‣ ී௨ʹ΢ΣϒαʔόʔΛॻ͍ͨΓ ‣ %#.4Λॻ͍ͨΓ ‣ ϒϥ΢βͰಈ͘ճ࿏$"%Λॻ͍ͨΓ

    
 8"4.λʔήοτ  ‣ ࣗ࡞ΩʔϘʔυͷ 
 ϑΝʔϜ΢ΣΞΛॻ͍ͨΓ 
 ͍ΘΏΔ૊ΈࠐΈ3VTU ͓͢͢Ίͷ࢓ࣄʹؔ͢Δ৘ใ͸ 
 ͍ͭͰ΋׻ܴ͍ͯ͠·͢ https://gihyo.jp/magazine/wdpress/archive/2021/vol122 ʲະঝ୚޿ࠂʳࡶࢽʹهࣄॻ͖·ͨ͠
  3. RustͰ on-diskͳB+TreeΛ 
 ࡞ͬͨͱ͖ͷ ࡉ͔ͳ࿩ ࠓճͷλΠτϧ 
 ʢ࠶ܝʣ

  4. RustͰ on-diskͳB+TreeΛ 
 ࡞ͬͨͱ͖ͷ ࡉ͔ͳ࿩ ͱ͸ʁ

  5. RustͰ on-diskͳB+TreeΛ 
 ࡞ͬͨͱ͖ͷ ࡉ͔ͳ࿩ ͱ͸ʁ " ಺༰ΛσΟεΫʹӬଓԽͰ͖ͨΓɺ 
 ϝϞϦΑΓେ͖ͳ಺༰ΛσΟεΫ্Ͱѻ͑ͨΓ͢Δ

    
 # 5SFF ˞ಠࣗͷఆٛͰ͢
  6. ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ

    ͜͏͍͏# 5SFFΛߟ͑Δ
  7. ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ

    ͱΓ͋͑ͣ࣍ݩʹͭͿ͢
  8. σΟεΫʢϑΝΠϧʣ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ

    ϊʔυ σΟεΫ͕͋Δͱ͢Δ
  9. σΟεΫʢϑΝΠϧʣ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ

    ϊʔυ σΟεΫʹೖͬͨ
  10. σΟεΫʢϑΝΠϧʣ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ

    ϊʔυ CPU # 5SFFΛૢ࡞͍ͨ͠
  11. σΟεΫʢϑΝΠϧʣ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ

    ϊʔυ CPU # 5SFFΛૢ࡞͍ͨ͠ σΟεΫΛ௚઀ಡΈॻ͖͢Δͷ͸஗͍͠ෆศ
  12. ϝϞϦ σΟεΫʢϑΝΠϧʣ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ

    ϊʔυ ϊʔυ CPU ϝϞϦʹόοϑΝΛ༻ҙ
  13. ϝϞϦ σΟεΫʢϑΝΠϧʣ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ

    ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ CPU
  14. ϝϞϦ σΟεΫʢϑΝΠϧʣ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ

    ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ CPU ϊʔυͷҰ෦ΛϝϞϦ΁
  15. ϝϞϦ σΟεΫʢϑΝΠϧʣ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ

    ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ CPU ಡΈॻ͖ ೖΕସ͑ όοϑΝϓʔϧϚωʔδϟ
  16. ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ

    Ұ෦ͷϊʔυͷΈ͕ϝϞϦʹ͋Δ# 5SFFͱ͍͑Δ ന͍ϊʔυ͸σΟεΫʹ͔͠ͳ͍
  17. ϝϞϦ σΟεΫʢϑΝΠϧʣ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ

    ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ CPU ಡΈॻ͖ ೖΕସ͑ ͱ͜ΖͰɺ
  18. ϝϞϦ σΟεΫʢϑΝΠϧʣ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ

    ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ CPU ಡΈॻ͖ ೖΕସ͑ ϚϧνίΞΛ׆༻͍ͨ͠Ͱ͢ΑͶʁ CPU CPU CPU
  19. ϝϞϦ σΟεΫʢϑΝΠϧʣ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ

    ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ CPU ར༻தͷϊʔυΛόοϑΝ͔Β௥͍ग़ͨ͠Γ͢Δͱ͍ͨ΁Μͳ͜ͱʹ CPU CPU CPU ΛಡΈ͍ͨɻΛ௥͍ग़ͦ͏ ଴ͯίϥ ϊʔυ ར༻த
  20. ϝϞϦ σΟεΫʢϑΝΠϧʣ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ

    ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ CPU ར༻தͷϊʔυΛόοϑΝ͔Β௥͍ग़ͨ͠Γ͢Δͱ͍ͨ΁Μͳ͜ͱʹ CPU CPU CPU ΛಡΈ͍ͨɻΛ௥͍ग़ͦ͏ ଴ͯίϥ ϊʔυ ར༻த ͜ͷ΁Μͷ 
 σʔλߏ଄ɾΞϧΰϦζϜΛ 
 3VTUͰͲ͏͢Δ͔ POEJTLಛ༗ͷ೉͕͋͠͞Δ
  21. 3VTUͰPOEJTLͳ# 5SFFΛ࡞Δͱ͖ͷݒ೦ࣄ߲ w όοϑΝϓʔϧϚωʔδϟͷ 
 σʔλߏ଄ͱΞϧΰϦζϜ w # 5SFFͷϊʔυͷσʔλߏ଄ w

    # 5SFFͷഉଞ੍ޚ
  22. όοϑΝϓʔϧϚωʔδϟͷ 
 σʔλߏ଄ͱΞϧΰϦζϜ

  23. ར༻தͷϊʔυΛอޢ͢Δʹ͸ $.6ͷߨٛͰ࢖ΘΕ͍ͯΔ#VT5VC $ ͱ͍͏࣮૷ͷ৔߹ ϊʔυʢϖʔδʣ಺ʹΧ΢ϯλΛ 
 ࣋ͬͯ؅ཧ͍ͯ͠Δ https://github.com/cmu-db/bustub/blob/master/src/include/storage/page/page.h ϐϯ 1JO

    ͸ʮར༻தʯͷҙຯ ˞3VTUͷ1JOͱ͸ؔ܎ͳ͍
  24. 3VTUͰ͸Ͳ͏͢Δ͔ Ҋ$ ಉ༷ʹΧ΢ϯλͰอޢ w ڞ༗ϝϞϦΛॻ͖׵͑ΔͨΊɺ 
 ౰વͷΑ͏ʹVOTBGF͕ొ৔ w ઈରʹόάΒͤΔࣗ৴͔͠ͳ͍ w

    ͜ΕͳΒ$ Ͱॻ͚͹͍͍ w ଈߦͰϘπ
  25. 3VTUͰ͸Ͳ͏͢Δ͔ Ҋ3X-PDLͰอޢ w ར༻த͸4IBSFEMPDLΛ͓࣋ͬͯ͘ w ௥͍ग़ͨ͘͠ͳͬͨΒ&YDMVTJWFMPDLΛऔΔ w (VBSEܥͷߏ଄ମ͸TUBUJD͡Όͳ͍ͷͰ 
 औΓճ͕͠ѱ͍

  26. 3VTUͰ͸Ͳ͏͢Δ͔ Ҋ"SDͰอޢ w "SDʹ͸HFU@NVU NVU"SD5 ͱ͍͏ 
 ϝιου͕͋Δ w ࢀরΧ΢ϯλ͕ͷͱ͖ʹ͚ͩNVU5͕औΕΔ

    w όοϑΝϓʔϧͷʮϐϯʯͱ૬ੑ͕Αͦ͞͏ͳػೳ w ͨͩ͠Ϩγʔό͸NVU w όοϑΝϓʔϧશମΛอޢ͢Δ.VUFY͕͋ΔͷͰ 
 Մมࢀর͕औΕΔ
  27. # 5SFFͷϊʔυͷ 
 σʔλߏ଄

  28. # 5SFFͷϊʔυͷߏ଄ ૉ๿ʹ΍Δͱɺ͜͏

  29. ⚠ϊʔυ͸σΟεΫʹ 
 ॻ͖ग़ͤͳ͚Ε͹ͳΒͳ͍ ΋ͪΖΜɺಡΈग़͠΋Ͱ͖ͳ͚Ε͹ͳΒͳ͍

  30. 2ߏ଄ମΛσΟεΫʹ 
 ॻ͖ग़͍ͨ͠ͱ͖ɺͲ͏͢Δʁ

  31. 2ߏ଄ମΛσΟεΫʹ 
 ॻ͖ग़͍ͨ͠ͱ͖ɺͲ͏͢Δʁ TFSEF

  32. 2ߏ଄ମΛσΟεΫʹ 
 ॻ͖ग़͍ͨ͠ͱ͖ɺͲ͏͢Δʁ TFSEF γϦΞϥΠζޙͷσʔλ௕͕Θ͔Βͳ͍͠ ޮ཰΋ѱ͍ͷͰμϝ

  33. 2ߏ଄ମΛσΟεΫʹ 
 ॻ͖ग़͍ͨ͠ͱ͖ɺͲ͏͢Δʁ "ͦͷ··ॻ͖ग़ͤΔ 
 σʔλߏ଄Λ͕Μ͹࣮ͬͯ૷͢Δ

  34. ͦͷ··ॻ͖ग़ͤΔσʔλߏ଄ ৗʹݻఆ௕ͷόΠτྻͷΈʹऩ·Δσʔλߏ଄ ϊʔυ ͖ͬ͞ొ৔ͨ͠ϊʔυ͘Μ

  35. ͦͷ··ॻ͖ग़ͤΔσʔλߏ଄ ৗʹݻఆ௕ͷόΠτྻͷΈʹऩ·Δσʔλߏ଄ ,# ࣮͸͜͏͍͏େ͖͞

  36. ͦͷ··ॻ͖ग़ͤΔσʔλߏ଄ ৗʹݻఆ௕ͷόΠτྻͷΈʹऩ·Δσʔλߏ଄ ,# ,FZ 7BMVF ͜Ε͸,FZ7BMVFͷϖΞ

  37. ͦͷ··ॻ͖ग़ͤΔσʔλߏ଄ ৗʹݻఆ௕ͷόΠτྻͷΈʹऩ·Δσʔλߏ଄ ,# ,FZ 7BMVF ,FZ 7BMVF ,FZ 7BMVF ௕͞͸όϥόϥ

  38. ͦͷ··ॻ͖ग़ͤΔσʔλߏ଄ ৗʹݻఆ௕ͷόΠτྻͷΈʹऩ·Δσʔλߏ଄ ,# ,FZ 7BMVF ,FZ 7BMVF ,FZ 7BMVF ͜ΕΛͦͷ··ෑ͖٧ΊΔͱ

  39. ͦͷ··ॻ͖ग़ͤΔσʔλߏ଄ ৗʹݻఆ௕ͷόΠτྻͷΈʹऩ·Δσʔλߏ଄ ,# ,FZ 7BMVF ,FZ 7BMVF ,FZ 7BMVF Φϑηοτ͕౳ִؒͰ͸ͳ͍ͷͰೋ෼୳ࡧͰ͖ͳ͍

  40. ͦͷ··ॻ͖ग़ͤΔσʔλߏ଄ ৗʹݻఆ௕ͷόΠτྻͷΈʹऩ·Δσʔλߏ଄ ,# ,FZ 7BMVF ,FZ 7BMVF ,FZ 7BMVF ࢓੾Γ௚͠

  41. ͦͷ··ॻ͖ग़ͤΔσʔλߏ଄ ৗʹݻఆ௕ͷόΠτྻͷΈʹऩ·Δσʔλߏ଄ ,# ,FZ 7BMVF ,FZ 7BMVF ,FZ 7BMVF ,FZ7BMVF͸ޙΖʹ٧Ίͯ

  42. ͦͷ··ॻ͖ग़ͤΔσʔλߏ଄ ৗʹݻఆ௕ͷόΠτྻͷΈʹऩ·Δσʔλߏ଄ ,# 1US ,FZ 7BMVF ,FZ 7BMVF ,FZ 7BMVF

    1US 1US ݻఆ௕ͷϙΠϯλΛ಄ʹฒ΂Δ
  43. ͦͷ··ॻ͖ग़ͤΔσʔλߏ଄ ৗʹݻఆ௕ͷόΠτྻͷΈʹऩ·Δσʔλߏ଄ ,# 1US ,FZ 7BMVF ,FZ 7BMVF ,FZ 7BMVF

    1US 1US ͜ΕΛ3VTUͰॻ͘
  44. ͱ͜ΖͰ͜ΕΛͲ͏΍ͬͯ 
 Ωϟετ͢Δͷ͔

  45. std::mem::transmute

  46. std::mem::transmute ͦΜͳةݥͳ΋ͷ࢖͏Θ͚ͳ͍

  47. ศརDSBUFl[FSPDPQZz ΞϥΠϝϯτͳͲΛߟྀͯ҆͠શʹܕΩϟετͰ͖Δ w (PPHMFͷ'VDITJBͰ࢖ΘΕ͍ͯΔͬΆ͍ w ࢖͍ํʹ͔ͳΓΫη͕͋Δ w υΩϡϝϯτ͸ۃΊͯগͳ͍ w αϯϓϧίʔυ͸'VDITJBͷωοτϫʔΫελοΫ

  48. LayoutVerifiedͱ͍͏ 
 εϚʔτϙΠϯλܦ༝Ͱૢ࡞͢Δ new_from_prefixϝιου͸ 
 ϔομͱϘσΟʹ෼͔Ε͍ͯΔ 
 σʔλߏ଄ʹ࢖͑Δ

  49. 1US ,FZ 7BMVF ,FZ 7BMVF ,FZ 7BMVF 1US 1US OVN@TMPUT

    IFBEFS CPEZ
  50. new_sliceϝιουͰ 
 ϙΠϯλͷεϥΠεΛ੾Γग़͍ͯ͠Δ

  51. 1US ,FZ 7BMVF ,FZ 7BMVF ,FZ 7BMVF 1US 1US OVN@TMPUT

    QPJOUFST IFBEFS CPEZ
  52. 1US ,FZ 7BMVF ,FZ 7BMVF ,FZ 7BMVF 1US 1US OVN@TMPUT

    QPJOUFST IFBEFS CPEZ ͜ͷਤΛΑ͘ݟͯ΄͍͠
  53. 1US ,FZ 7BMVF ,FZ 7BMVF ,FZ 7BMVF 1US 1US OVN@TMPUT

    QPJOUFST IFBEFS CPEZ QPJOUFST͸CPEZΛआ༻͍ͯ͠Δ
  54. 1US ,FZ 7BMVF ,FZ 7BMVF ,FZ 7BMVF 1US 1US OVN@TMPUT

    QPJOUFST IFBEFS CPEZ QPJOUFST͸CPEZΛआ༻͍ͯ͠Δ ͭ·Γɺ,FZ7BMVFϖΞΛมߋ͢Δͱ͖ʹ͸ 
 ͍͍ͪͪQPJOUFSTΛख์͞ͳ͚Ε͹ͳΒͳ͍ ͩΔ͍ 
 ʢ͔ͨ͠ͳ͠ʣ
  55. # 5SFFͷഉଞ੍ޚ

  56. ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ

    CPU CPU
  57. ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ

    CPU CPU Modify Modify 💥 😩 😩
  58. ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ

    CPU CPU ϩοΫ͸্͔ΒॱʹऔΔ
  59. ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ

    ϩοΫ͸্͔ΒॱʹऔΔ CPU CPU
  60. ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ

    ϩοΫ͸্͔ΒॱʹऔΔ CPU CPU
  61. ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ

    ϩοΫ͕औΕͨΒɺ্ͷϩοΫ͸ख์ͯ͠΋Α͍ CPU CPU
  62. ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ

    ଞͷεϨου͕ಈ͚ͯฒྻ౓্͕͕Δ CPU CPU
  63. ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ

    CPU CPU ಉ༷ʹଓ͚Δ
  64. ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ

    Լͷϊʔυ͕ᷓΕͦ͏ͳ৔߹͸ɺ্ͷϩοΫΛख์ͯ͠͸ͳΒͳ͍ 
 ʢॻ͖ࠐΈͷ৔߹ʣ CPU CPU
  65. ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ ϊʔυ

    CPU CPU ͜ΕΛ3VTUͰ࣮૷͢Δ
  66. # 5SFFͷഉଞ੍ޚ ૉ๿ʹ࣮૷

  67. # 5SFFͷഉଞ੍ޚ ૉ๿ʹ࣮૷

  68. # 5SFFͷഉଞ੍ޚ ૉ๿ʹ࣮૷ page: 
 Arc<RwLock<Page>> ro_page: 
 RwLockReadGuard<_, Page>

    next_page: 
 Arc<RwLock<Page>> ro_next_page: 
 RwLockReadGuard<_, Page> next_pageΛಈ͔͍͕ͨ͠ かといってRwLockReadGuardΛ 
 ख์͢Θ͚ʹ͸͍͔ͳ͍ ro_page͕pageΛɺ 
 ro_next_page͕next_pageΛ 
 ࢀর͍ͯ͠ΔͷͰಈ͔ͤͳ͍
  69. # 5SFFͷഉଞ੍ޚ ૉ๿ʹ࣮૷ page: 
 Arc<RwLock<Page>> ro_page: 
 RwLockReadGuard<_, Page>

    next_page: 
 Arc<RwLock<Page>> ro_next_page: 
 RwLockReadGuard<_, Page> next_pageΛಈ͔͍͕ͨ͠ かといってRwLockReadGuardΛ 
 ख์͢Θ͚ʹ͸͍͔ͳ͍ ro_page͕pageΛɺ 
 ro_next_page͕next_pageΛ 
 ࢀর͍ͯ͠ΔͷͰಈ͔ͤͳ͍ ٧Έ
  70. ͜Ε͸Α͋͘Δ4UBCMF"EESFTT໰୊ "SDͷࢦ͢ϙΠϯλ͸ෆมͳΜ͔ͩΒNPWFͤͯ͘͞Ε w PXOJOH@SFGΫϨʔτͳͲͰ 
 ղܾ͕ࢼΈΒΕ͍ͯΔ w ࠓճ͸3X-PDL3FBE(VBSEʹ 
 ಛघͳϝιουΛੜ΍ͯ͠ճආ

  71. 0XOFE3X-PDL3FBE(VBSE ΊͬͪΌVOTBGF͚ͩͲ͔ͨ͠ͳ͍

  72. ࠓ೔঺հͨ͠ίʔυ͸͍͍ͩͨ(JU)VCʹ͋Δ RQ w IUUQTHJUIVCDPN,0#"RQ w ϚϧνεϨου w ίʔυ͕ͱͯ΋Ԛ͍ SFMMZ w

    IUUQTHJUIVCDPN,0#"SFMMZ w γϯάϧεϨου w 8&# %#13&447PMͷͨΊʹ 
 ॻ͖ԼΖͨ͠ w ͪΐͬͱίʔυ͕៉ྷ
  73. ·ͱΊ ಈతͳσʔλߏ଄Λ࣮૷͢Δͱ3VTUಛ༗ͷ೉͕ͯ͠͞Μ͜੝Γʂ w "SDHFU@NVUͷΑ͏ͳϚΠφʔͳ 
 ϝιουΛ஌͓ͬͯ͘ͱ໾ʹཱͭ w TUE΍ΫϨʔτΛ׆͔ͤΔΑ͏ʹ 
 σʔλߏ଄Λมܗ͢ΔεΩϧ͕ॏཁ

    w ίϯύΠϧΤϥʔΛோΊΔ͚ͩͰͳ͘ɺ 
 ϝϞϦͷ༷ࢠΛ૝૾͢Δ w ͋Δ΂͖ϝϞϦͷߏ଄͔Β 
 σʔλߏ଄Λٯࢉ͢Δͷ͕ίπ