Future of Pinning - storing Threads and Buckets at Textile

Transcript

1. Storing Threads and Buckets
   at Textile.
   ADVANCED PINNING
   STRUCTURES
   Andrew Hill | @andrewxhill
   

   View Slide

2. hi! i’m
   @andrewxhill
   from textile.io
   

   View Slide

3. ThreadsDB
   The Hub
   Buckets Powergate
   

   View Slide

4. ThreadsDB
   

   View Slide

5. ThreadsDB
   A p2p database built on IPFS, Libp2p,
   IPNS, IPLD…
   • Easy MongoDB-like developer interface.
   • Modular ACL, Identity, CRDTs and more.
   • Encryption design that enables private sharing and
   3rd party pinning, relay, archival, etc.
   

   View Slide

6. ThreadsDB
   const Astronauts = await db.newCollectionFromObject('Astronauts', {
   ID: '',
   name: '',
   missions: 0,
   })
   db.on('Astronauts.**', update => {
   console.log(update) // subscribe to updates
   })
   await Astronauts.insert(
   { ID: '', missions: 2, name: 'Buzz Aldrin' },
   { ID: '', missions: 5, name: 'Christina Koch' },
   { ID: '', missions: 2, name: 'Sally Ride' },
   { ID: '', missions: 7, name: 'Jerry Ross' },
   )
   const all = Collection.find(
   { $or: [{ missions: { $gt: 2 } }, { name: 'Sally Ride' }] },
   { sort: { missions: -1 } }
   )
   for await (const { key, value } of all) {
   console.log(value)
   }
   • Every Thread (networking)
   has a Database
   • Every Database has 1-to-
   many Collections (schemas)
   • Collections have many
   instances (documents
   matching schemas)
   • Instances can be added,
   deleted, or updated.
   The database — simple
   

   View Slide

7. A Push
   B
   A Pull
   B
   A
   Z
   B
   Z
   Push based p2p
   Pull based p2p
   Push/Pull based
   Thread Service Pull
   Push
   docsend.com/view/gu3ywqi
   Z Z
   Networking — ThreadsDB keeps users connected
   

   View Slide

8. [ ]
   [ ]
   [ ] (Body, Key) (CID, Link)
   Encrypted (Service Key)
   hashed
   (CID)
   Meta
   Encrypted (Key) Encrypted
   (raw)
   (CID, Link)
   [ ]
   Meta
   (CID, Link)
   [ ]
   Meta
   (CID, Link)
   [ ]
   Meta
   ThreadsDB lets users do more with private data.
   (Read)
   docsend.com/view/gu3ywqi
   

   View Slide

9. [ ]
   [ ]
   [ ] (Body, Key) (CID, Link)
   Encrypted (Service Key)
   hashed
   (CID)
   Meta
   Encrypted (Key) Encrypted
   (raw)
   (CID, Link)
   [ ]
   Meta
   (CID, Link)
   [ ]
   Meta
   (CID, Link)
   [ ]
   Meta
   ThreadsDB lets users do more with private data.
   User Owned Service Ready
   (Read)
   

   View Slide

10. ThreadsDB
    Threads are dynamic pinsets!
    • Encrypted or open datasets.
    • One or many users collaborating.
    • Snapshots, Head tracking, Forking, Archives
    

    View Slide

11. ThreadsDB
    links:
    Whitepaper: docsend.com/view/gu3ywqi
    Golang: github.com/textileio/go-threads
    JavaScript: github.com/textileio/js-threads
    

    View Slide

12. Buckets
    

    View Slide

13. Buckets
    Dynamic folders over ThreadsDB
    • Collaborative, multi-user folders.
    • Automatically persist on IPFS via Textile Hub.
    • Push datasets to one or many remote IFPS peers.
    

    View Slide

14. Buckets
    Buckets are built on ThreadsDB!
    • Buckets are an Instance in a ThreadDB
    • Recall: Instance > Collection > Database > Thread
    • Instances have an ID that also happens to be an
    IPNS address…!
    • HTTP over Hub or any IPFS gateway with IPNS
    

    View Slide

15. Buckets
    

    View Slide

16. Buckets
    links:
    Docs: docs.textile.io/hub/buckets
    Blog: blog.textile.io/tag/buckets
    

    View Slide

17. The Hub
    

    View Slide

18. The Hub
    https://docs.textile.io/
    

    View Slide

19. The Hub
    Provide hosted IPFS and Thread
    services for devs and their users
    • Hosted IPFS and Threads Services
    • gRPC endpoints + JavaScript, Go, and Android(*)
    libraries.
    • Handle persistence, orchestration, relay.
    • HTTP, IPFS, and Libp2p endpoint
    

    View Slide

20. The Hub
    Provide hosted IPFS and Thread
    services for devs and their users
    • Developers can get an Account Key
    • Developers can give their users a User Key
    • Users/Developers can create Threads using their
    own PK
    • Provide DB and Storage for any PK identity
    

    View Slide

21. The Hub
    • Buckets and Threads offer new opportunities for
    pinning services to support interoperable, dynamic data
    formats.
    • Threads play nicely with pinning infrastructure but allow
    devs to disintermediate data<>user relationship.
    • Let us know!
    Dynamic, interoperable, and
    disintermediated.
    

    View Slide

22. ThreadsDB
    The Hub
    Buckets Powergate
    

    View Slide

23. Bigger Picture
    

    View Slide

24. Bigger Picture
    Textile was born in more normal times
    • If you are working on technology that brings
    people together.
    • If you are working on technology that keeps
    people safe.
    • If you are working on technology that brightens
    the future.
    We want to support you.
    

    View Slide

25. andrew@textile.io
    
    @andrewxhill
    Thank you!
    

    View Slide