分散アプリケーションにおける複数端末利用を考慮したプライベートデータの管理/WSA_1_kaleidoscope

第一回 Web System Architecture 研究会 (WSA研)
http://websystemarchitecture.hatenablog.jp/entry/2017/12/17/133301

Transcript

1. ࡾ୐༔հ / Pepabo R&D Institute, GMO Pepabo, Inc. 2017.12.23 Web System Architecture ݚڀձ (WSAݚ) #1 ෼ࢄΞϓϦέʔγϣϯʹ͓͚Δ ෳ਺୺຤ར༻Λߟྀͨ͠ ϓϥΠϕʔτσʔλͷ؅ཧ
2. ϓϦϯγύϧΤϯδχΞ ࡾ୐ ༔հ / @monochromegane 2 http://blog.monochromegane.com Yusuke Miyake ϖύϘݚڀॴ ݚڀһ
3. 1. ͸͡Ίʹ 2. ෼ࢄΞϓϦέʔγϣϯʹ͓͚Δ
 ϓϥΠϕʔτσʔλ؅ཧͷ՝୊ 3. ख๏ͷఏҊ 4. ධՁ 5. ·ͱΊ 3 ໨࣍
4. 1. ͸͡Ίʹ
5. • ϒϩοΫνΣʔϯٕज़ͷొ৔ʹΑΓɺΠϯλʔωοτΛܦ༝ͨ͠ݸਓؒͰͷ௚ ઀ͷՁ஋ަ׵͕༰қͱͳΓͭͭ͋Δɻ • ͜Ε·ͰͷՁ஋ަ׵ͷ஥հͱͯ͠ΞϓϦέʔγϣϯ͔Βɺ௚઀ͷՁ஋ަ׵ϓϥο τϑΥʔϜͱͯ͠ͷΞϓϦέʔγϣϯ΁ • BitCoin, Ethereum, Stoj, Filecoin, OpenBazzar… 5 ௚઀ͷՁ஋ަ׵ϓϥοτϑΥʔϜͷొ৔
6. • ͜ΕΒͷՁ஋ަ׵ϓϥοτϑΥʔϜ͸ࣗ཯͓ͯ͠Γɺதԝूݖతͳӡ༻΍αʔ όʔ͕ଘࡏ͠ͳ͍ɻ • શһ͕αʔόʔͰ͋ΓɺΫϥΠΞϯτͱͳΔ͜ͱͰՁ஋Λަ׵͠߹͏ • ैདྷܕͷWebαʔϏεʹର͢ΔύϥμΠϜγϑτ 6 ඇதԝूݖతͳΞϓϦέʔγϣϯͷ࣮૷ݕ౼ͷඞཁੑ • ඇதԝूݖతͳΞϓϦέʔγϣϯ࣮૷ͷݕ౼͕ඞཁ • ຊݚڀͰ͸ɺσʔλ؅ཧɺಛʹϓϥΠϕʔτσʔλͷऔΓѻ͍Λݕ౼
7. 2. ෼ࢄΞϓϦέʔγϣϯʹ͓͚Δ ϓϥΠϕʔτσʔλ؅ཧͷ՝୊
8. • ຊݚڀใࠂͰ͸ɺ௚઀ͷՁ஋ަ׵ΛՄೳͱ͢ΔɺதԝͷίϯτϩʔϧΛഉ͠ ͨɺࣗ཯෼ࢄܕͷγεςϜΛͻͱͭҎ্૊Έ߹ΘͤͨΞϓϦέʔγϣϯΛ෼ࢄ ΞϓϦέʔγϣϯͱఆٛ͢Δ 8 ෼ࢄΞϓϦέʔγϣϯ Centrized system Distributed system Distributed application
9. • ෼ࢄΞϓϦέʔγϣϯͰ͸ΞϓϦέʔγϣϯ΍σʔλ͸෼ࢄωοτϫʔΫͷϊʔ υʹ෼ࢄ഑ஔ͞ΕΔɻ • σʔλಛੑʹ߹Θͤͨσʔλ؅ཧํࣜΛ૊Έ߹Θͤͯར༻͢Δ 9 ෼ࢄΞϓϦέʔγϣϯͱ༻్ผσʔλ؅ཧ 1. ΞϓϦέʔγϣϯʹͱͬͯॏཁͳऔҾ৘ใ͸ϒϩοΫνΣʔϯ΁ 2. औҾࣗମ͕ൃੜ͠ͳ͍Ϛελ৘ใ͸P2PϑΝΠϧγεςϜ΁ 3. ར༻ऀ͝ͱͷ৘ใ͸୺຤΁ 4. ΞϓϦέʔγϣϯ͸ϩʔΧϧͰ্هΛͭͳ͙ Blockchain P2P file system Local file system
10. • ෼ࢄΞϓϦέʔγϣϯʹ͓͚Δσʔλ؅ཧํࣜͷଟ͘͸ɺඇதԝूݖతͳߏ੒ ͔Βෳ਺ͷ୺຤͔Βൺֱత༰қʹΞΫηεՄೳ • Ұํɺར༻ऀͷ৘ใ͸֤୺຤ʹอଘ͞ΕΔͨΊɺෳ਺୺຤Ͱ؀ڥΛἧ͑ΔͨΊ ʹ͸୺຤ؒͰͷσʔλΛڞ༗͕ඞཁ 10 ෼ࢄΞϓϦέʔγϣϯʹ͓͚ΔϓϥΠϕʔτσʔλڞ༗ͷ՝୊ • ैདྷͷσʔλڞ༗Ϟσϧͷݕ౼Λ௨ͯ͠ɺ෼ࢄΞϓϦέʔγϣϯʹ͓͚Δϓϥ Πϕʔτͳσʔλڞ༗ͷ՝୊Λࣔ͢ɻ
11. ϑΝΠϧڞ༗γεςϜͱ σʔλϕʔε
12. • தԝूݖతͳσʔλڞ༗Ϟσϧ • NFS΍DropboxʹΑΔαʔόʔΛܦ༝ͨ͠ϑΝΠϧڞ༗ • σʔλΛҰݩ؅ཧ͢Δσʔλϕʔεαʔόʔ • αʔόʔ্ͷίϯςϯπΛ૬ޓࢀরͤ͞ΔυΩϡϝϯτϕʔεγεςϜͱ͠ ͯͷWWW 12 ϑΝΠϧڞ༗γεςϜͱσʔλϕʔε
13. P2Pܕσʔλϕʔε
14. • ดͨ͡ωοτϫʔΫΛ༻͍ͨσʔλڞ༗Ϟσϧ • Amazon DynamoDB΍Apache Cassandra • ϊʔυ௥Ճ΍σʔλΞΫηε͸தԝूݖతʹίϯτϩʔϧ • ดͨ͡ઐ༻ͷωοτϫʔΫͱͯ͠ߏஙɺӡ༻ 14 P2Pܕσʔλϕʔε
15. P2Pܕ෼ࢄϑΝΠϧγεςϜ
16. • ։͚ͨωοτϫʔΫΛ༻͍ͨσʔλڞ༗Ϟσϧ • P2PωοτϫʔΫΛ༻͍ͨ෼ࢄϑΝΠϧγεςϜ • ωοτϫʔΫࢀՃऀ͕σʔλΛࢀরՄೳ 16 P2Pܕ෼ࢄϑΝΠϧγεςϜ
17. • IPFS (InterPlanetary File System) • σʔλͷ෼ࢄอଘͱσʔλ಺༰ʹجͮ͘ΞυϨογϯά • σΟϨΫτϦߏ଄Λѻ͑Δ • σʔλมߋ͕ΞυϨεมߋͱಉҙͳͷͰΞυϨεΛ؅ཧ͢Δඞཁ͕͋Δ • IPNS(InterPlanetary Name Service)ʹΑΔ೚ҙΞυϨεΛݻఆతʹࢀর 17 P2Pܕ෼ࢄϑΝΠϧγεςϜ - IPFS -
18. P2Pܕ෼ࢄϑΝΠϧγεςϜ্ͷ σʔλετΞ
19. • P2Pܕ෼ࢄϑΝΠϧγεςϜͷಛੑΛߟྀͨ͠σʔλڞ༗Ϟσϧ • σʔλʹجͮ͘ΞυϨογϯάʹΑΓ෦෼มߋ͕ߦ͑ͳ͍ͨΊɺมߋͷہ ॴԽ͕ඞཁ • ෳ਺୺຤Ͱར༻͢ΔͨΊσʔλڝ߹ͷճආ͕ඞཁ 19 P2Pܕ෼ࢄϑΝΠϧγεςϜ্ͷσʔλετΞ
20. • OrbitDB • ૢ࡞ཤྺΛݸʑʹอଘ͢Δ͜ͱͰมߋΛہॴԽ • Մ׵ͳૢ࡞ͷΈΛೝΊΔ͜ͱͰॱෆಉͳσʔλΛड͚෇͚Δ͜ͱͰσʔλ ڝ߹ΛճආʢCRDTʣ • ཤྺͷ૿Ճʹ൐͍ࢀর࣌ͷΦʔόʔϔου͕ൃੜ͢Δ 20 P2Pܕ෼ࢄϑΝΠϧγεςϜ্ͷσʔλετΞ
21. 3. ෼ࢄΞϓϦέʔγϣϯʹ͓͚Δ ϓϥΠϕʔτσʔλ؅ཧख๏ͷఏҊ
22. 1. ඇதԝूݖతͳߏ੒Ͱ͋Δ͜ͱ 2. ෼ࢄอଘ͞ΕΔσʔλʹରͯ͠ࢀর੍ݶΛ͔͚ΕΔ͜ͱ 3. ։͔Εͨීٴ͍ͯ͠ΔP2PܕωοτϫʔΫΛ༻͍Δ͜ͱ 4. ࠷৽ͷΞυϨεΛอ࣋·ͨ͸ղܾͰ͖Δ͜ͱ 5. มߋ͕ہॴԽ͞Ε͍ͯΔ͜ͱ 6. มߋͷڝ߹ʹରԠͰ͖Δ͜ͱ 7. ࢀর࣌ͷΦʔόʔϔου͕গͳ͍͜ͱ 22 ෼ࢄΞϓϦέʔγϣϯʹ͓͚ΔϓϥΠϕʔτσʔλ؅ཧ
23. Kaleidoscope Distributed peer-to-peer personal key-value store. https://github.com/monochromegane/kaleidoscope
24. Kaleidoscopeͷ࢓૊Έ
25. • P2P෼ࢄϑΝΠϧγεςϜ্ͷKey-Value Store • P2PωοτϫʔΫʹIPFSΛ࢖༻ • σʔλετΞ໊ɺΩʔ໊ΛσΟϨΫτϦͱ͠ɺ಺༰ΛϑΝΠϧͱ͢ΔσΟϨΫ τϦߏ଄ΛIPFS্ʹอଘ • σʔλ͸λΠϜελϯϓ౳ͷϝλσʔλͱ಺༰Λ߹Θͤͯ҉߸Խ • IPNSʹΑΔ࠷৽ΞυϨεͷղܾ 25 Kaleidoscopeͷ࢓૊Έ
26. 26 Kaleidoscopeͷ࢓૊Έ QmFoo ├── __database_name ├── key1 │ └── value └── key2 └── value IPFS Kaleidoscope Application Encrypt / Decrypt Key <-> Directory IPNS Current address Name Hash Metadata + data Hash
27. • ϑΝΠϧϕʔεࠩ෼ํࣜ • P2P෼ࢄϑΝΠϧγεςϜ্ʹσʔλετΞͷϑΝΠϧΛ௚઀഑ஔ • ΦϖϨʔγϣϯϕʔεࠩ෼ํࣜ (OrbitDB) • P2P෼ࢄϑΝΠϧγεςϜಛੑΛߟྀͨ͠มߋہॴԽΛཤྺ୯ҐอଘͰ࣮ݱ • σΟϨΫτϦϕʔεࠩ෼ํࣜ (Kaleidoscope) • ఏҊํࣜɻσʔλߏ଄ʹΑͬͯมߋہॴԽΛ࣮ݱ 27 ࠩ෼ߋ৽ͷํࣜ෼ྨ
28. • ஋ͷߋ৽͸Ωʔ୯Ґʹ෼཭͞ΕɺσʔλετΞશମͷσʔλ༰ྔʹґଘ͠ͳ͍ • σΟϨΫτϦߏ଄ͷΞυϨεΛࢀর͢ΔͨΊɺཤྺࢀর͕ෆཁ • ஋ʹλΠϜελϯϓΛ࣋ͭ͜ͱͰɺσΟϨΫτϦߏ଄ͷ౷߹͕Մೳ • ॱෆಉͳՄ׵ͳૢ࡞ͱͯ͠දݱͰ͖Δ 28 σΟϨΫτϦϕʔεࠩ෼ํࣜʹΑΔ՝୊ͷղܾ
29. Kaleidoscopeͷجຊతͳར༻ํ๏ 29 # Start IPFS daemon $ ipfs daemon # Start Kaleidoscope CLI $ kes > create my-db # => Create datastore key pair and Register directory on IPFS > > set key1 value1 # => Save encrypted data on IPFS > > get key1 value1 # => Get and decrypt data from directory on IPFS > > save # => Update IPNS by newest hash value
30. KaleidoscopeͰͷ୺຤ؒσʔλڞ༗ (1) 30 $ kes > use my-db # => Get newest hash value from IPNS → ݱঢ়ɺIPNSͷߋ৽ॲཧʹ਺ඵ͔͔ΔͨΊɺϦΞϧλΠϜͷಉظ͕೉͍͠
31. KaleidoscopeͰͷ୺຤ؒσʔλڞ༗ (2) 31 # Start IPFS daemon with pubsub option $ ipfs daemon --enable-pubsub-experiment $ kes > sync # => Thereafter, operation history from other own terminal online is shared • ૢ࡞छผͱର৅ͱͳͬͨϑΝΠϧͷϋογϡ஋ΛPubSubܦ༝Ͱૹ৴ • ड৴ଆ͸ɺࣗ਎ͷૢ࡞ʹͦΕΒͷૢ࡞Λࠩ͠ࠐΈͳ͕Βॱ൪ʹॲཧ → ࣮ࡍͷӡ༻Ͱ͸λΠϜελϯϓ·Ͱߟྀͨ͠౷߹͕ඞཁ
32. KaleidoscopeͰͷ୺຤ؒσʔλϚʔδ 32 ݱ࣌఺Ͱ͸ະ࣮૷ σΟϨΫτϦΛλΠϜελϯϓΛߟྀͯ͠Ϛʔδ ࡟আ࣌΋λΠϜελϯϓΛอ͓࣋ͯ͘͠ඞཁ͕͋Γͦ͏
33. 4. ධՁ
34. ࠩ෼ํࣜʹΑΔߋ৽࣌ؒͷൺֱ 34      .# .# .# .# .# .# .# .# .# .# pMFCBTFE PQFCBTFE EJSCBTFE • σʔλετΞͷ༰ྔ͝ͱͷ IPFS΁ͷొ࿥࣌ؒΛൺֱ • ϑΝΠϧϕʔεࠩ෼͸༰ྔʹൺ ྫͯ͠ొ࿥͕࣌ؒ૿Ճ • มߋΛہॴԽ͢ΔํࣜͰ͸༰ ྔʹґଘͤͣҰఆ • σΟϨΫτϦϕʔεࠩ෼ํࣜͰ ͸σΟϨΫτϦߋ৽෼ͷΦʔόʔ ϔου͕ݟΒΕΔ
35. ࠩ෼ํࣜʹΑΔॳظىಈ࣌ؒͷൺֱ 35      .# .# .# .# .# .# .# .# .# .# pMFCBTFE PQFCBTFE EJSCBTFE • σʔλετΞͷ༰ྔ͝ͱͷॳ ظىಈ࣌ؒΛൺֱ • ϑΝΠϧϕʔεࠩ෼ɺΦϖϨʔ γϣϯϕʔεࠩ෼Ͱ͸༰ྔ·ͨ ͸ཤྺͷ૿Ճʹൺྫ͢Δ • σΟϨΫτϦϕʔεࠩ෼͸ৗʹ ࠷৽Λอ࣋͢ΔͨΊॳظىಈ ࣌ؒ͸ൃੜ͠ͳ͍
36. • ະ࣮૷ • ΦϖϨʔγϣϯϕʔεࠩ෼ํࣜ͸શཤྺΛ࢖͏ͷʹର͠ɺσΟϨΫτϦϕʔε ࠩ෼ํࣜ͸࠷৽ͷ஋ಉ࢜ͷΈͰϚʔδΛߦ͏ͨΊɺ༏ҐͰ͋Δͱ૝ఆ 36 ࠩ෼ํࣜʹΑΔϚʔδ࣌ؒͷൺֱ
37. 5. ·ͱΊ
38. • ෼ࢄΞϓϦέʔγϣϯ࣮૷ʹ͓͚ΔϓϥΠϕʔτσʔλڞ༗ͷ՝୊Λݕ౼ͨ͠ • ैདྷํࣜͰ࠷΋ద͍ͯ͠ΔP2P෼ࢄϑΝΠϧγεςϜ্ͷσʔλετΞͷ՝୊ ΛσΟϨΫτϦϕʔεࠩ෼ํࣜͰղܾΛਤͬͨ • σʔλ༰ྔ΍ཤྺʹґଘ͠ͳ͍͜ͱΛݕূͨ͠ • σΟϨΫτϦϕʔεͷσʔλߏ଄͸֦ு΋ݟࠐΊΔ • ࠓޙ͸ಉظͱ·ʔ͡ͷ࣮૷ʹ͍ͭͯӡ༻ଇ͔Β஌ݟΛ൓өͯ͠ਐΊ͍ͨ 38 ·ͱΊ