分散アプリケーションにおける複数端末利用を考慮したプライベートデータの管理/WSA_1_kaleidoscope

 分散アプリケーションにおける複数端末利用を考慮したプライベートデータの管理/WSA_1_kaleidoscope

第一回 Web System Architecture 研究会 (WSA研)
http://websystemarchitecture.hatenablog.jp/entry/2017/12/17/133301

Cd3d2cb2dadf5488935fe0ddaea7938a?s=128

monochromegane

December 25, 2017
Tweet

Transcript

  1. ࡾ୐༔հ / Pepabo R&D Institute, GMO Pepabo, Inc. 2017.12.23 Web

    System Architecture ݚڀձ (WSAݚ) #1 ෼ࢄΞϓϦέʔγϣϯʹ͓͚Δ ෳ਺୺຤ར༻Λߟྀͨ͠ ϓϥΠϕʔτσʔλͷ؅ཧ
  2. ϓϦϯγύϧΤϯδχΞ ࡾ୐ ༔հ / @monochromegane 2 http://blog.monochromegane.com Yusuke Miyake ϖύϘݚڀॴ

    ݚڀһ
  3. 1. ͸͡Ίʹ 2. ෼ࢄΞϓϦέʔγϣϯʹ͓͚Δ
 ϓϥΠϕʔτσʔλ؅ཧͷ՝୊ 3. ख๏ͷఏҊ 4. ධՁ 5.

    ·ͱΊ 3 ໨࣍
  4. 1. ͸͡Ίʹ

  5. • ϒϩοΫνΣʔϯٕज़ͷొ৔ʹΑΓɺΠϯλʔωοτΛܦ༝ͨ͠ݸਓؒͰͷ௚ ઀ͷՁ஋ަ׵͕༰қͱͳΓͭͭ͋Δɻ • ͜Ε·ͰͷՁ஋ަ׵ͷ஥հͱͯ͠ΞϓϦέʔγϣϯ͔Βɺ௚઀ͷՁ஋ަ׵ϓϥο τϑΥʔϜͱͯ͠ͷΞϓϦέʔγϣϯ΁ • BitCoin, Ethereum, Stoj,

    Filecoin, OpenBazzar… 5 ௚઀ͷՁ஋ަ׵ϓϥοτϑΥʔϜͷొ৔
  6. • ͜ΕΒͷՁ஋ަ׵ϓϥοτϑΥʔϜ͸ࣗ཯͓ͯ͠Γɺதԝूݖతͳӡ༻΍αʔ όʔ͕ଘࡏ͠ͳ͍ɻ • શһ͕αʔόʔͰ͋ΓɺΫϥΠΞϯτͱͳΔ͜ͱͰՁ஋Λަ׵͠߹͏ • ैདྷܕͷWebαʔϏεʹର͢ΔύϥμΠϜγϑτ 6 ඇதԝूݖతͳΞϓϦέʔγϣϯͷ࣮૷ݕ౼ͷඞཁੑ •

    ඇதԝूݖతͳΞϓϦέʔγϣϯ࣮૷ͷݕ౼͕ඞཁ • ຊݚڀͰ͸ɺσʔλ؅ཧɺಛʹϓϥΠϕʔτσʔλͷऔΓѻ͍Λݕ౼
  7. 2. ෼ࢄΞϓϦέʔγϣϯʹ͓͚Δ ϓϥΠϕʔτσʔλ؅ཧͷ՝୊

  8. • ຊݚڀใࠂͰ͸ɺ௚઀ͷՁ஋ަ׵ΛՄೳͱ͢ΔɺதԝͷίϯτϩʔϧΛഉ͠ ͨɺࣗ཯෼ࢄܕͷγεςϜΛͻͱͭҎ্૊Έ߹ΘͤͨΞϓϦέʔγϣϯΛ෼ࢄ ΞϓϦέʔγϣϯͱఆٛ͢Δ 8 ෼ࢄΞϓϦέʔγϣϯ Centrized system Distributed system

    Distributed application
  9. • ෼ࢄΞϓϦέʔγϣϯͰ͸ΞϓϦέʔγϣϯ΍σʔλ͸෼ࢄωοτϫʔΫͷϊʔ υʹ෼ࢄ഑ஔ͞ΕΔɻ • σʔλಛੑʹ߹Θͤͨσʔλ؅ཧํࣜΛ૊Έ߹Θͤͯར༻͢Δ 9 ෼ࢄΞϓϦέʔγϣϯͱ༻్ผσʔλ؅ཧ 1. ΞϓϦέʔγϣϯʹͱͬͯॏཁͳऔҾ৘ใ͸ϒϩοΫνΣʔϯ΁ 2.

    औҾࣗମ͕ൃੜ͠ͳ͍Ϛελ৘ใ͸P2PϑΝΠϧγεςϜ΁ 3. ར༻ऀ͝ͱͷ৘ใ͸୺຤΁ 4. ΞϓϦέʔγϣϯ͸ϩʔΧϧͰ্هΛͭͳ͙ Blockchain P2P file system Local file system
  10. • ෼ࢄΞϓϦέʔγϣϯʹ͓͚Δσʔλ؅ཧํࣜͷଟ͘͸ɺඇதԝूݖతͳߏ੒ ͔Βෳ਺ͷ୺຤͔Βൺֱత༰қʹΞΫηεՄೳ • Ұํɺར༻ऀͷ৘ใ͸֤୺຤ʹอଘ͞ΕΔͨΊɺෳ਺୺຤Ͱ؀ڥΛἧ͑ΔͨΊ ʹ͸୺຤ؒͰͷσʔλΛڞ༗͕ඞཁ 10 ෼ࢄΞϓϦέʔγϣϯʹ͓͚ΔϓϥΠϕʔτσʔλڞ༗ͷ՝୊ • ैདྷͷσʔλڞ༗Ϟσϧͷݕ౼Λ௨ͯ͠ɺ෼ࢄΞϓϦέʔγϣϯʹ͓͚Δϓϥ

    Πϕʔτͳσʔλڞ༗ͷ՝୊Λࣔ͢ɻ
  11. ϑΝΠϧڞ༗γεςϜͱ σʔλϕʔε

  12. • தԝूݖతͳσʔλڞ༗Ϟσϧ • NFS΍DropboxʹΑΔαʔόʔΛܦ༝ͨ͠ϑΝΠϧڞ༗ • σʔλΛҰݩ؅ཧ͢Δσʔλϕʔεαʔόʔ • αʔόʔ্ͷίϯςϯπΛ૬ޓࢀরͤ͞ΔυΩϡϝϯτϕʔεγεςϜͱ͠ ͯͷWWW 12

    ϑΝΠϧڞ༗γεςϜͱσʔλϕʔε
  13. P2Pܕσʔλϕʔε

  14. • ดͨ͡ωοτϫʔΫΛ༻͍ͨσʔλڞ༗Ϟσϧ • Amazon DynamoDB΍Apache Cassandra • ϊʔυ௥Ճ΍σʔλΞΫηε͸தԝूݖతʹίϯτϩʔϧ • ดͨ͡ઐ༻ͷωοτϫʔΫͱͯ͠ߏஙɺӡ༻

    14 P2Pܕσʔλϕʔε
  15. P2Pܕ෼ࢄϑΝΠϧγεςϜ

  16. • ։͚ͨωοτϫʔΫΛ༻͍ͨσʔλڞ༗Ϟσϧ • P2PωοτϫʔΫΛ༻͍ͨ෼ࢄϑΝΠϧγεςϜ • ωοτϫʔΫࢀՃऀ͕σʔλΛࢀরՄೳ 16 P2Pܕ෼ࢄϑΝΠϧγεςϜ

  17. • IPFS (InterPlanetary File System) • σʔλͷ෼ࢄอଘͱσʔλ಺༰ʹجͮ͘ΞυϨογϯά • σΟϨΫτϦߏ଄Λѻ͑Δ •

    σʔλมߋ͕ΞυϨεมߋͱಉҙͳͷͰΞυϨεΛ؅ཧ͢Δඞཁ͕͋Δ • IPNS(InterPlanetary Name Service)ʹΑΔ೚ҙΞυϨεΛݻఆతʹࢀর 17 P2Pܕ෼ࢄϑΝΠϧγεςϜ - IPFS -
  18. P2Pܕ෼ࢄϑΝΠϧγεςϜ্ͷ σʔλετΞ

  19. • P2Pܕ෼ࢄϑΝΠϧγεςϜͷಛੑΛߟྀͨ͠σʔλڞ༗Ϟσϧ • σʔλʹجͮ͘ΞυϨογϯάʹΑΓ෦෼มߋ͕ߦ͑ͳ͍ͨΊɺมߋͷہ ॴԽ͕ඞཁ • ෳ਺୺຤Ͱར༻͢ΔͨΊσʔλڝ߹ͷճආ͕ඞཁ 19 P2Pܕ෼ࢄϑΝΠϧγεςϜ্ͷσʔλετΞ

  20. • OrbitDB • ૢ࡞ཤྺΛݸʑʹอଘ͢Δ͜ͱͰมߋΛہॴԽ • Մ׵ͳૢ࡞ͷΈΛೝΊΔ͜ͱͰॱෆಉͳσʔλΛड͚෇͚Δ͜ͱͰσʔλ ڝ߹ΛճආʢCRDTʣ • ཤྺͷ૿Ճʹ൐͍ࢀর࣌ͷΦʔόʔϔου͕ൃੜ͢Δ 20

    P2Pܕ෼ࢄϑΝΠϧγεςϜ্ͷσʔλετΞ
  21. 3. ෼ࢄΞϓϦέʔγϣϯʹ͓͚Δ ϓϥΠϕʔτσʔλ؅ཧख๏ͷఏҊ

  22. 1. ඇதԝूݖతͳߏ੒Ͱ͋Δ͜ͱ 2. ෼ࢄอଘ͞ΕΔσʔλʹରͯ͠ࢀর੍ݶΛ͔͚ΕΔ͜ͱ 3. ։͔Εͨීٴ͍ͯ͠ΔP2PܕωοτϫʔΫΛ༻͍Δ͜ͱ 4. ࠷৽ͷΞυϨεΛอ࣋·ͨ͸ղܾͰ͖Δ͜ͱ 5. มߋ͕ہॴԽ͞Ε͍ͯΔ͜ͱ

    6. มߋͷڝ߹ʹରԠͰ͖Δ͜ͱ 7. ࢀর࣌ͷΦʔόʔϔου͕গͳ͍͜ͱ 22 ෼ࢄΞϓϦέʔγϣϯʹ͓͚ΔϓϥΠϕʔτσʔλ؅ཧ
  23. Kaleidoscope Distributed peer-to-peer personal key-value store. https://github.com/monochromegane/kaleidoscope

  24. Kaleidoscopeͷ࢓૊Έ

  25. • P2P෼ࢄϑΝΠϧγεςϜ্ͷKey-Value Store • P2PωοτϫʔΫʹIPFSΛ࢖༻ • σʔλετΞ໊ɺΩʔ໊ΛσΟϨΫτϦͱ͠ɺ಺༰ΛϑΝΠϧͱ͢ΔσΟϨΫ τϦߏ଄ΛIPFS্ʹอଘ • σʔλ͸λΠϜελϯϓ౳ͷϝλσʔλͱ಺༰Λ߹Θͤͯ҉߸Խ

    • IPNSʹΑΔ࠷৽ΞυϨεͷղܾ 25 Kaleidoscopeͷ࢓૊Έ
  26. 26 Kaleidoscopeͷ࢓૊Έ QmFoo ├── __database_name ├── key1 │ └── value

    └── key2 └── value IPFS Kaleidoscope Application Encrypt / Decrypt Key <-> Directory IPNS Current address Name Hash Metadata + data Hash
  27. • ϑΝΠϧϕʔεࠩ෼ํࣜ • P2P෼ࢄϑΝΠϧγεςϜ্ʹσʔλετΞͷϑΝΠϧΛ௚઀഑ஔ • ΦϖϨʔγϣϯϕʔεࠩ෼ํࣜ (OrbitDB) • P2P෼ࢄϑΝΠϧγεςϜಛੑΛߟྀͨ͠มߋہॴԽΛཤྺ୯ҐอଘͰ࣮ݱ •

    σΟϨΫτϦϕʔεࠩ෼ํࣜ (Kaleidoscope) • ఏҊํࣜɻσʔλߏ଄ʹΑͬͯมߋہॴԽΛ࣮ݱ 27 ࠩ෼ߋ৽ͷํࣜ෼ྨ
  28. • ஋ͷߋ৽͸Ωʔ୯Ґʹ෼཭͞ΕɺσʔλετΞશମͷσʔλ༰ྔʹґଘ͠ͳ͍ • σΟϨΫτϦߏ଄ͷΞυϨεΛࢀর͢ΔͨΊɺཤྺࢀর͕ෆཁ • ஋ʹλΠϜελϯϓΛ࣋ͭ͜ͱͰɺσΟϨΫτϦߏ଄ͷ౷߹͕Մೳ • ॱෆಉͳՄ׵ͳૢ࡞ͱͯ͠දݱͰ͖Δ 28 σΟϨΫτϦϕʔεࠩ෼ํࣜʹΑΔ՝୊ͷղܾ

  29. Kaleidoscopeͷجຊతͳར༻ํ๏ 29 # Start IPFS daemon $ ipfs daemon #

    Start Kaleidoscope CLI $ kes > create my-db # => Create datastore key pair and Register directory on IPFS > > set key1 value1 # => Save encrypted data on IPFS > > get key1 value1 # => Get and decrypt data from directory on IPFS > > save # => Update IPNS by newest hash value
  30. KaleidoscopeͰͷ୺຤ؒσʔλڞ༗ (1) 30 $ kes > use my-db # =>

    Get newest hash value from IPNS → ݱঢ়ɺIPNSͷߋ৽ॲཧʹ਺ඵ͔͔ΔͨΊɺϦΞϧλΠϜͷಉظ͕೉͍͠
  31. KaleidoscopeͰͷ୺຤ؒσʔλڞ༗ (2) 31 # Start IPFS daemon with pubsub option

    $ ipfs daemon --enable-pubsub-experiment $ kes > sync # => Thereafter, operation history from other own terminal online is shared • ૢ࡞छผͱର৅ͱͳͬͨϑΝΠϧͷϋογϡ஋ΛPubSubܦ༝Ͱૹ৴ • ड৴ଆ͸ɺࣗ਎ͷૢ࡞ʹͦΕΒͷૢ࡞Λࠩ͠ࠐΈͳ͕Βॱ൪ʹॲཧ → ࣮ࡍͷӡ༻Ͱ͸λΠϜελϯϓ·Ͱߟྀͨ͠౷߹͕ඞཁ
  32. KaleidoscopeͰͷ୺຤ؒσʔλϚʔδ 32 ݱ࣌఺Ͱ͸ະ࣮૷ σΟϨΫτϦΛλΠϜελϯϓΛߟྀͯ͠Ϛʔδ ࡟আ࣌΋λΠϜελϯϓΛอ͓࣋ͯ͘͠ඞཁ͕͋Γͦ͏

  33. 4. ධՁ

  34. ࠩ෼ํࣜʹΑΔߋ৽࣌ؒͷൺֱ 34      .# .# .#

    .# .# .# .# .# .# .# pMFCBTFE PQFCBTFE EJSCBTFE • σʔλετΞͷ༰ྔ͝ͱͷ IPFS΁ͷొ࿥࣌ؒΛൺֱ • ϑΝΠϧϕʔεࠩ෼͸༰ྔʹൺ ྫͯ͠ొ࿥͕࣌ؒ૿Ճ • มߋΛہॴԽ͢ΔํࣜͰ͸༰ ྔʹґଘͤͣҰఆ • σΟϨΫτϦϕʔεࠩ෼ํࣜͰ ͸σΟϨΫτϦߋ৽෼ͷΦʔόʔ ϔου͕ݟΒΕΔ
  35. ࠩ෼ํࣜʹΑΔॳظىಈ࣌ؒͷൺֱ 35      .# .# .#

    .# .# .# .# .# .# .# pMFCBTFE PQFCBTFE EJSCBTFE • σʔλετΞͷ༰ྔ͝ͱͷॳ ظىಈ࣌ؒΛൺֱ • ϑΝΠϧϕʔεࠩ෼ɺΦϖϨʔ γϣϯϕʔεࠩ෼Ͱ͸༰ྔ·ͨ ͸ཤྺͷ૿Ճʹൺྫ͢Δ • σΟϨΫτϦϕʔεࠩ෼͸ৗʹ ࠷৽Λอ࣋͢ΔͨΊॳظىಈ ࣌ؒ͸ൃੜ͠ͳ͍
  36. • ະ࣮૷ • ΦϖϨʔγϣϯϕʔεࠩ෼ํࣜ͸શཤྺΛ࢖͏ͷʹର͠ɺσΟϨΫτϦϕʔε ࠩ෼ํࣜ͸࠷৽ͷ஋ಉ࢜ͷΈͰϚʔδΛߦ͏ͨΊɺ༏ҐͰ͋Δͱ૝ఆ 36 ࠩ෼ํࣜʹΑΔϚʔδ࣌ؒͷൺֱ

  37. 5. ·ͱΊ

  38. • ෼ࢄΞϓϦέʔγϣϯ࣮૷ʹ͓͚ΔϓϥΠϕʔτσʔλڞ༗ͷ՝୊Λݕ౼ͨ͠ • ैདྷํࣜͰ࠷΋ద͍ͯ͠ΔP2P෼ࢄϑΝΠϧγεςϜ্ͷσʔλετΞͷ՝୊ ΛσΟϨΫτϦϕʔεࠩ෼ํࣜͰղܾΛਤͬͨ • σʔλ༰ྔ΍ཤྺʹґଘ͠ͳ͍͜ͱΛݕূͨ͠ • σΟϨΫτϦϕʔεͷσʔλߏ଄͸֦ு΋ݟࠐΊΔ •

    ࠓޙ͸ಉظͱ·ʔ͡ͷ࣮૷ʹ͍ͭͯӡ༻ଇ͔Β஌ݟΛ൓өͯ͠ਐΊ͍ͨ 38 ·ͱΊ
  39. None