Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Future of Pinning - storing Threads and Buckets at Textile

Future of Pinning - storing Threads and Buckets at Textile

Andrew W Hill

May 07, 2020
Tweet

More Decks by Andrew W Hill

Other Decks in Technology

Transcript

  1. Storing Threads and Buckets
    at Textile.
    ADVANCED PINNING
    STRUCTURES
    Andrew Hill | @andrewxhill

    View Slide

  2. hi! i’m
    @andrewxhill
    from textile.io

    View Slide

  3. ThreadsDB
    The Hub
    Buckets Powergate

    View Slide

  4. ThreadsDB

    View Slide

  5. ThreadsDB
    A p2p database built on IPFS, Libp2p,
    IPNS, IPLD…
    • Easy MongoDB-like developer interface.
    • Modular ACL, Identity, CRDTs and more.
    • Encryption design that enables private sharing and
    3rd party pinning, relay, archival, etc.

    View Slide

  6. ThreadsDB
    const Astronauts = await db.newCollectionFromObject('Astronauts', {
    ID: '',
    name: '',
    missions: 0,
    })
    db.on('Astronauts.**', update => {
    console.log(update) // subscribe to updates
    })
    await Astronauts.insert(
    { ID: '', missions: 2, name: 'Buzz Aldrin' },
    { ID: '', missions: 5, name: 'Christina Koch' },
    { ID: '', missions: 2, name: 'Sally Ride' },
    { ID: '', missions: 7, name: 'Jerry Ross' },
    )
    const all = Collection.find(
    { $or: [{ missions: { $gt: 2 } }, { name: 'Sally Ride' }] },
    { sort: { missions: -1 } }
    )
    for await (const { key, value } of all) {
    console.log(value)
    }
    • Every Thread (networking)
    has a Database
    • Every Database has 1-to-
    many Collections (schemas)
    • Collections have many
    instances (documents
    matching schemas)
    • Instances can be added,
    deleted, or updated.
    The database — simple

    View Slide

  7. A Push
    B
    A Pull
    B
    A
    Z
    B
    Z
    Push based p2p
    Pull based p2p
    Push/Pull based
    Thread Service Pull
    Push
    docsend.com/view/gu3ywqi
    Z Z
    Networking — ThreadsDB keeps users connected

    View Slide

  8. [ ]
    [ ]
    [ ] (Body, Key) (CID, Link)
    Encrypted (Service Key)
    hashed
    (CID)
    Meta
    Encrypted (Key) Encrypted
    (raw)
    (CID, Link)
    [ ]
    Meta
    (CID, Link)
    [ ]
    Meta
    (CID, Link)
    [ ]
    Meta
    ThreadsDB lets users do more with private data.
    (Read)
    docsend.com/view/gu3ywqi

    View Slide

  9. [ ]
    [ ]
    [ ] (Body, Key) (CID, Link)
    Encrypted (Service Key)
    hashed
    (CID)
    Meta
    Encrypted (Key) Encrypted
    (raw)
    (CID, Link)
    [ ]
    Meta
    (CID, Link)
    [ ]
    Meta
    (CID, Link)
    [ ]
    Meta
    ThreadsDB lets users do more with private data.
    User Owned Service Ready
    (Read)

    View Slide

  10. ThreadsDB
    Threads are dynamic pinsets!
    • Encrypted or open datasets.
    • One or many users collaborating.
    • Snapshots, Head tracking, Forking, Archives

    View Slide

  11. ThreadsDB
    links:
    Whitepaper: docsend.com/view/gu3ywqi
    Golang: github.com/textileio/go-threads
    JavaScript: github.com/textileio/js-threads

    View Slide

  12. Buckets

    View Slide

  13. Buckets
    Dynamic folders over ThreadsDB
    • Collaborative, multi-user folders.
    • Automatically persist on IPFS via Textile Hub.
    • Push datasets to one or many remote IFPS peers.

    View Slide

  14. Buckets
    Buckets are built on ThreadsDB!
    • Buckets are an Instance in a ThreadDB
    • Recall: Instance > Collection > Database > Thread
    • Instances have an ID that also happens to be an
    IPNS address…!
    • HTTP over Hub or any IPFS gateway with IPNS

    View Slide

  15. Buckets

    View Slide

  16. Buckets
    links:
    Docs: docs.textile.io/hub/buckets
    Blog: blog.textile.io/tag/buckets

    View Slide

  17. The Hub

    View Slide

  18. The Hub
    https://docs.textile.io/

    View Slide

  19. The Hub
    Provide hosted IPFS and Thread
    services for devs and their users
    • Hosted IPFS and Threads Services
    • gRPC endpoints + JavaScript, Go, and Android(*)
    libraries.
    • Handle persistence, orchestration, relay.
    • HTTP, IPFS, and Libp2p endpoint

    View Slide

  20. The Hub
    Provide hosted IPFS and Thread
    services for devs and their users
    • Developers can get an Account Key
    • Developers can give their users a User Key
    • Users/Developers can create Threads using their
    own PK
    • Provide DB and Storage for any PK identity

    View Slide

  21. The Hub
    • Buckets and Threads offer new opportunities for
    pinning services to support interoperable, dynamic data
    formats.
    • Threads play nicely with pinning infrastructure but allow
    devs to disintermediate data<>user relationship.
    • Let us know!
    Dynamic, interoperable, and
    disintermediated.

    View Slide

  22. ThreadsDB
    The Hub
    Buckets Powergate

    View Slide

  23. Bigger Picture

    View Slide

  24. Bigger Picture
    Textile was born in more normal times
    • If you are working on technology that brings
    people together.
    • If you are working on technology that keeps
    people safe.
    • If you are working on technology that brightens
    the future.
    We want to support you.

    View Slide

  25. [email protected]

    @andrewxhill
    Thank you!

    View Slide