IPFS (惑星間ファイルシステム) のGo実装を読み解く

Gopher道場#5 卒業式＆LT大会 LT資料

Transcript

1. IPFS (࿭੕ؒϑΝΠϧγεςϜ) ͷGo࣮૷ΛಡΈղ͘ Gopherಓ৔ #5 ଔۀࣜ + LTେձ Jun 24 2019, Daigo Hamasaki @lfcd85
2. ࣗݾ঺հ - ᖛ࡚ େޗ (Daigo Hamasaki) @lfcd85
 - WebΤϯδχΞ (React, Rails, Vue.js / Scala, Laravel)
 - Ethereum ϒϩοΫνΣʔϯΛ࢖ͬͨWebΞϓϦ΍APIΛ։ൃத
 - ϚΠϒʔϜ: Rebuild.fm ͷաڈ์ૹΛௌ͖ͳ͕Βࢄา͢Δ
 - ݱࡏ 2017೥10݄ͿΜ·ͰḪߦ
 - ࣮຿Ϩϕϧͷ Go Λशಘ͍ͨ͠ͱࢥ͍Gopherಓ৔Λडߨ͠·ͨ͠
3. Go ͱϒϩοΫνΣʔϯք۾ Go ͸৭ʑͳϒϩοΫνΣʔϯք۾ͷϓϩδΣΫτʹ࢖ΘΕ͍ͯΔ
 - Go Ethereum (Ethereum ͷެࣜCLI)
 - IOST (DApps = ෼ࢄΞϓϦͷ։ൃ༻్ʹಛԽͨ͠৽ڵϒϩοΫνΣʔϯ)
 ɹ- ઌिൃද͞ΕͨFacebookͷ Libra ΋ Protobuf Ͱ Go Λ࢖༻ (ຊମ͸Rust) 
 - IPFS (InterPlanetary File System, ࿭੕ؒϑΝΠϧγεςϜ)
4. ࿭੕ - IPFS (࿭੕ؒϑΝΠϧγεςϜ)
 ☄
5. IPFS (InterPlanetary File System) ͱ͸ - P2P ͷϑΝΠϧڞ༗ωοτϫʔΫʗͦͷϓϩτίϧ
 - ͬ͘͟Γ BitTorrent ͷΑ͏ͳ΋ͷ
 - ϑΝΠϧͷ಺༰͕ϋογϡԽ͞ΕͯɺͦΕ͕ΞυϨεʹͳΔ
 - ಉ͡಺༰ͷϑΝΠϧͰ͋Ε͹ϋογϡʢʹΞυϨεʣ΋ಉҰʹͳΔ
 - Ethereum ͷ DApps ্Ͱը૾౳ͷେ༰ྔϑΝΠϧΛอଘ͢Δͷʹ࢖ΘΕ͕ͪ
 - Ethereum ࣗ਎͸େ༰ྔΛѻ͏ͷʹෆ޲͖ͳͷͰ
6. $ ipfs init initializing IPFS node at /Users/(username)/.ipfs generating 2048-bit RSA keypair...done peer identity: QmZcyLgzWejG4sdqiwvxtMxtf6oKnd7ELxxXj2asdByuVh to get started, enter: ipfs cat /ipfs/QmS4ustL54uo8FzR9455qaxZwuMiUhyvMcX9Ba8nUH4uVv/readme $ ipfs ls /ipfs/QmS4ustL54uo8FzR9455qaxZwuMiUhyvMcX9Ba8nUH4uVv QmZTR5bcpQD7cFgTorqxZDYaew1Wqgfbd2ud9QqGPAkK2V 1677 about QmYCvbfNbCwFR45HiNP45rwJgvatpiW38D961L5qAhUM5Y 189 contact QmY5heUM5qgRubMDD1og9fhCPA6QdkMp3QCwd4s7gJsyE7 311 help QmejvEPop4D7YUadeGqYWmZxHhLc4JBUCzJJHWMzdcMe2y 4 ping QmXgqKTbzdh83pQtKFb19SpMCpDDcKR2ujqk3pKph9aCNF 1681 quick-start QmPZ9gcCEpqKTo6aq61g2nXGUhM4iCL3ewB6LDXZCtioEB 1091 readme QmQ5vhrL7uv6tuoN9KeVBwd4PwfQkXdVVmDLUZuTNxqgvm 1162 security-notes $ ipfs cat /ipfs/QmZTR5bcpQD7cFgTorqxZDYaew1Wqgfbd2ud9QqGPAkK2V IPFS -- Inter-Planetary File system IPFS is a global, versioned, peer-to-peer filesystem. It combines good ideas from Git, BitTorrent, Kademlia, SFS, and the Web. It is like a single bit- torrent swarm, exchanging git objects. IPFS provides an interface as simple as the HTTP web, but with permanence built in. You can also mount the world at /ipfs. CLIΛݟͯΈΔ ࣗΒͷϊʔυ໊ (ϋογϡ) ϑΝΠϧ໊ͷΤΠϦΞε (ϋογϡ)
7. IPFS ͷ Go ࣮૷ΛಡΜͰΈΔ - CLI ΋ϓϩτίϧ΋ɺGoͰॻ͔Ε͍ͯΔ 
 - Gopherಓ৔Ͱಘͨ஌ݟΛ΋ͱʹ ipfs/go-ipfs ͷίʔυΛಡΜͰΈ·ͨ͠
8. go.mod ͕௕͍ go.mod 120ߦɺ go.sum 857ߦ
 (ུ)
9. go.mod ͔ΒΘ͔ͬͨ͜ͱ (1) - ipfs/go-ipfs ͸ଟ͘ͷ೿ੜϦϙδτϦΛ͍࣋ͬͯΔ
 - ͳ͔ͳ͔ͷڊେϓϩδΣΫτͳͷͰ͸
10. go.mod ͔ΒΘ͔ͬͨ͜ͱ (2) - P2P ؔ࿈ͷॲཧ͸ libp2p/go-libp2p ؔ࿈ϦϙδτϦʹґଘ͍ͯ͠Δ
 - IPFS͔Β೿ੜͨ͠ϥΠϒϥϦͰ͋Δ໛༷

11. go.mod ͔ΒΘ͔ͬͨ͜ͱ (3) - hashicorp/go-multierror ͕࢖ΘΕ͍ͯΔ
 - ؔ਺ͷ࠷ޙͰɺ֤Τϥʔνϟωϧ͔ΒͷΤϥʔΛ·ͱΊ͍ͯΔ

12. Makefile͕෼ׂ͞Ε͍ͯΔ - `GO111MODULE` ͸Ͳ͜Ͱࢦఆ͞Ε͍ͯΔͷ͔୳ͯ͠Έͨ
 - `Makefile` ͸ɺෳ਺ͷ `.mk` ϑΝΠϧʹ෼ׂޙΠϯϙʔτ͞Ε͍ͯΔ ↑͜͜
13. ύοέʔδߏ଄ΛݟͯΈΔ - *.go ϑΝΠϧ਺ ໿270 ʹର͠ɺ77ύοέʔδ
 - core/commands ౳ɺओཁͳύοέʔδ͸10ϑΝΠϧҎ্ͷ΋ͷ΋
 - ิॿతͳύοέʔδ͸୯ҰϑΝΠϧͷ΋ͷ΋ଟ͍
 2 | ipfs 4 | assets 2 | blocks/blockstoreutil 6 | cmd/ipfs 7 | cmd/ipfs/util 2 | cmd/ipfswatch 2 | cmd/seccat 2 | commands 2 | core 2 | core/bootstrap 43 | core/commands 2 | core/commands/cmdenv 1 | core/commands/dag 1 | core/commands/e 4 | core/commands/name 3 | core/commands/object 2 | core/commands/unixfs 13 | core/coreapi 1 | core/coreapi/test 4 | core/coredag 16 | core/corehttp 2 | core/corerepo 4 | score/coreunix 1 | core/mock 8 | core/node 1 | core/node/helpers 13 | core/node/libp2p 1 | coverage/main 5 | dagutils 4 | filestore 1 | filestore/pb 5 | fuse/ipns 2 | fuse/mount 5 | fuse/node 4 | fuse/readonly 4 | keystore 13 | namesys 1 | namesys/republisher 1 | namesys/republisher_test 1 | namesys/resolve 1 | namesys/resolve_test 5 | p2p 4 | pin 1 | pin/gc 2 | pin/internal/pb 5 | plugin 4 | plugin/loader 1 | plugin/plugins/badgerds 1 | plugin/plugins/flatfs 1 | plugin/plugins/git 1 | plugin/plugins/levelds 3 | provider 2 | provider/queue 2 | provider/simple 2 | provider/simple_test 3 | repo 1 | repo/common 5 | repo/fsrepo 1 | repo/fsrepo_test 4 | repo/fsrepo/migrations 1 | tar 1 | test/api-startup 1 | test/bench/bench_cli_ipfs_add 1 | test/bench/offline_add 1 | test/dependencies 1 | test/dependencies/go-sleep 1 | test/dependencies/go-timeout 1 | test/dependencies/iptb 1 | test/dependencies/ma-pipe-unidir 5 | test/integration 1 | thirdparty/assert 1 | thirdparty/cidv0v1 1 | thirdparty/dir 1 | thirdparty/notifier 1 | thirdparty/pollEndpoint 2 | thirdparty/unit 1 | thirdparty/verifbs ύοέʔδ͝ͱͷϑΝΠϧ਺ | ύοέʔδ໊
14. GoDoc΋ݟͯΈΔ - ιʔείʔυΛݩʹGoDoc͕ੜ੒͞Ε͍ͯΔ
15. ίϚϯυϥΠϯҾ਺ͷऔΓճ͠ - mainؔ਺͔Β `os.Args` Λ࢖ͬͯCLIʹ౉͞Ε͍ͯΔ
16. ܕΞαʔγϣϯͷ༻ྫ - interface{} ܕͰฦ͖ͬͯͨόʔδϣϯ৘ใΛ string ܕʹΩϟετ͢Δ༻๏
17. ܕεΠονͷ༻ྫ - path͔Βऔಘͨ͠ `f` ͷܕ͕ϑΝΠϧ͔Ͳ͏͔ͷ൑ఆ
18. ύχοΫΛ൐͏Must*ؔ਺ͷ༻ྫ - Prometheus ΫϥΠΞϯτ͔ΒͷMust*ؔ਺ݺͼग़͠Λൃݟ
 - Τϥʔॲཧͷ͓ߦّ͸Α͘ͳ͍͔΋ʁ
19. ςετपΓͰؾ͍ͮͨ͜ͱ - ͍͔ͭ͘ͷύοέʔδͰ͸ɺςετ༻ͷύοέʔδ͕ݸผʹ༻ҙ͞Ε͍ͯΔ
 - ྫ: - ౷߹ςετ΍ςετͷґଘ͢Δύοέʔδ͸ `test/*` ʹ഑ஔ͞Ε͍ͯΔ
 - ྫ: - ͜Ε͕͓ߦّͱͯ͠Ͳ͏ͳͷ͔͸൑அ͔ͭͣ… namesys/republisher -> namesys/republisher_test provider/simple -> provider/simple_test 1 | test/dependencies 1 | test/dependencies/go-sleep 1 | test/dependencies/go-timeout 1 | test/dependencies/iptb 1 | test/dependencies/ma-pipe-unidir 5 | test/integration
20. GoroutineपΓͷؾ͖ͮ - ͔ͳΓଟ͘ͷGoroutineʗฒྻॲཧؔ࿈ؔ਺͕࢖ΘΕ͍ͯΔ
 - ϦϙδτϦ಺Λ֤छؔ਺Ͱશจݕࡧͯ͠Έͨ݁Ռ: go func(): 70 matches across 41 files sync.Mutex: 12 matches across 8 files sync.WaitGroup: 15 matches across 12 files context.WithCancel: 39 matches across 25 files context.WithTimeout: 21 matches across 16 files
21. sync.WaitGroup ಡղͷࢼΈ - APIαʔόʔͰͷϦεφʔͷΫϩʔζॲཧʹ࢖ΘΕ͍ͯΔ Τϥʔνϟωϧͱ
 WaitGroup ΛཱͯΔ ֤Ϧεφʔ͝ͱʹ
 wg.Add(1) ͓͖ͯ͠ɺ
 Goroutine Ͱͷ Serve() ऴྃ࣌ʹ
 ฦΓ஋Λ errc Ͱड͚औΓ
 defer Ͱ wg.Done() ͢Δ ͢΂ͯͷϦεφʔ͕ऴྃ͢Δͱɺwg.Wait() ͷ଴ػ͕ղ͚ɺ
 Τϥʔνϟωϧ͕Ϋϩʔζ͞ΕΔ
22. http.ServeMux ΋େྔ౤ೖ͞Ε͍ͯΔ - - ServeOption ܕͷҾ਺ͱͯ͠ *http.ServeMux ͕ݺ͹ΕɺServe() ؔ਺ʹ౉͞Ε͍ͯΔ http.ServeMux: 26 matches across 10 files
23. Θ͔ͬͨ͜ͱɾ·ͱΊ - IPFS ͸ Go ͷ༷ʑͳཁૉΛ;ΜͩΜʹ׆༻͍ͯ͠Δ
 - go.mod, ܕεΠονʗܕΞαʔγϣϯ, Goroutine, http.ServeMux() …
 - ࿭੕ؒΛ໊৐Δ͚ͩͷ͜ͱ͸͋ͬͨ - ڊେͳOSSͷίʔυΛಡΜͰΈΔ͜ͱʹΑΔֶͼ͕͋ͬͨ
 - GoDoc, ಛʹύοέʔδ৘ใ͸શମΛ၆ᛌ͢Δͷʹॏཁ - Gopherಓ৔Λܦͯɺ࣮૷ʹՃ͑ɺଟ͘ͷίʔυΛಡղͰ͖ΔΑ͏ʹͳͬͨ
 - օ༷ͷϋΠϨϕϧͳ࣮૷ͷ਺ʑʹ৮ΕɺܹࢗΛ͍͖ͨͩ·ͨ͠
 - ࠓޙ΋͞Βʹ Go ΁ͷ஌ݟΛਂΊ͍͖͍ͯͨ