Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Appbase - A realtime events-based document store

Appbase - A realtime events-based document store

Redisconf '15 talk: Using Redis to build a document store with a streaming events server.

Siddharth Kothari

March 05, 2015
Tweet

More Decks by Siddharth Kothari

Other Decks in Technology

Transcript

  1. Appbase:
    A Realtime events-based
    document index store
    - Siddharth Kothari
    @siddharthlatest

    View Slide

  2. Javascript foobar, Database enthusiast
    I know R+L = J
    CEO at appbase.io

    View Slide

  3. Why?

    View Slide

  4. Why?
    relational data vs linked data.

    View Slide

  5. Imagine…

    View Slide

  6. Imagine…
    1) Document store schema

    View Slide

  7. Imagine…
    1) Document store schema
    2) Dynamic References

    View Slide

  8. Imagine…
    1) Document store schema
    2) Dynamic References
    3) All JSON

    View Slide

  9. Hint:
    Is this MongoDB?

    View Slide

  10. Hint:
    It’s not MongoDB.
    This is Redis!

    View Slide

  11. Appbase:
    A Realtime events-based
    document index store

    View Slide

  12. Appbase:
    A Realtime events-based
    document index store
    1) Document store schema
    2) Dynamic References
    3) All JSON

    View Slide

  13. Appbase:
    A Realtime events-based
    document index store
    1) Document store schema
    2) Dynamic References
    3) All JSON
    4) Realtime events - Subscribe to updates

    View Slide

  14. Pulling a Mongo! Here’s how:
    Document store schema
    Collection

    View Slide

  15. Pulling a Mongo! Here’s how:
    Document store schema
    { “item”: “pencil”,
    “qty”: 500,
    “type”: “no.3”}
    Collection Document

    View Slide

  16. Pulling a Mongo! Here’s how:
    Embedded Documents
    {
    “item”: “pencil”,
    “qty”: 500,
    “type”: “no.3”,
    “foo”: {
    “embedded_foo”: “bar”
    }
    }
    Embedded Property

    View Slide

  17. Pulling a Mongo! Here’s how:
    Adding References
    {
    “item”: “pencil”,
    “qty”: 500,
    “type”: “no.3”,
    “foo”: {
    “embedded_foo”: “bar”
    },
    “/ref1”: “user/item#3”
    }
    Reference
    user
    item#3

    View Slide

  18. Pulling a Mongo! Here’s how:
    Adding References
    {
    “item”: “pencil”,
    …,
    …,
    “/ref1”: {
    “_collection”: “user”,
    “_id”: “item#3”,
    “ref_prop#1”: “something”
    }
    }
    Reference
    Adding the doc directly

    View Slide

  19. Beyond Document Store!
    Introducing Paths
    {
    “_id”: “pencil”,
    “_collection”: “stationary”,
    “foo”: {
    “embedded_foo”: “bar”
    },
    “/ref1”: “user/item#3”
    }
    Reference
    user
    item#3

    View Slide

  20. Beyond Document Store!
    Introducing Paths
    {
    “_id”: “pencil”,
    “_collection”: “stnry”,
    “foo”: {
    “embedded_foo”: “bar”
    },
    “/ref1”: “user/item#3”
    }
    Reference
    user
    item#3
    Two paths:
    1. /user/item#3
    2. /stnry/pencil/ref1

    View Slide

  21. Beyond Document Store!
    Introducing Paths
    Two paths:
    1. /user/item#3
    2. /stnry/pencil/ref1

    View Slide

  22. Beyond Document Store!
    Introducing Paths
    a
    b c
    d
    1. a/b/c/d,
    2. z/b/c/d,
    3. q/w/d
    4. …
    z
    q
    w

    View Slide

  23. So far…
    Document store with true references
    a
    b c
    d
    z
    q
    w
    {
    “_id”: “d”,
    “_collection”: “alpha”,
    “depth”: “4”
    }

    View Slide

  24. Taking advantage of true referencing
    Event subscriptions:

    View Slide

  25. {
    “_id”: “d”,
    “_collection”: “alpha”,
    “depth”: “4”
    }
    Taking advantage of true referencing
    a
    b
    z
    Event subscriptions:
    1.Property updates

    View Slide

  26. {
    “_id”: “d”,
    “_collection”: “alpha”,
    “depth”: “4”
    }
    Taking advantage of true referencing
    a
    b
    z
    c
    d
    q
    w
    Event subscriptions:
    1.Property updates
    2.Reference updates

    View Slide

  27. Open Source

    View Slide

  28. Open Source
    License: Apache 2
    v0.1 - Brave Souls

    View Slide

  29. Open Source
    appbaseio.github.io/appbase-redis
    License: Apache 2
    v0.1 - Brave Souls

    View Slide

  30. Live Demo

    View Slide

  31. Live Demo
    /users/user_542
    Create a new document

    View Slide

  32. Live Demo
    /users/user_542
    Create a new document
    Assign some properties

    View Slide

  33. Live Demo
    /users/user_542
    Create a new document
    Assign some properties
    {
    “em”: “[email protected]”,

    “points”: 5321,
    “/friends”: …
    }

    View Slide

  34. Live Demo
    /users/user_542
    Create a new document
    Assign some properties
    {
    “em”: “[email protected]”,

    “points”: 5321,
    “/friends”: …
    }
    Watch it Stream

    View Slide

  35. API Spec
    Comes with a HTTP API server

    View Slide

  36. API Spec
    Comes with a HTTP API server
    PATCH /{collection}/{document}/[path]
    GET /{collection}/{document}/[path]
    ?stream=true

    View Slide

  37. API Spec
    Comes with a HTTP API server
    PATCH
    GET
    https://github.com/appbaseio/
    appbase-redis/
    blob/master/
    index.js
    Entire Spec

    View Slide

  38. API Spec
    Request Body

    View Slide

  39. Roadmap to v1.0
    1. Battle Testing
    Unit Tests, Better Code
    Perf Tests

    View Slide

  40. Roadmap to v1.0
    1. Battle Testing
    Unit Tests, Better Code
    Perf Tests
    2. Scaling
    Use Pushpin = Message Queue + Streaming

    View Slide

  41. Roadmap to v1.0
    1. Battle Testing
    Unit Tests, Better Code
    Perf Tests
    2. Scaling
    Use Pushpin = Message Queue + Streaming
    3. Rich Querying
    More filters, aggregation queries

    View Slide

  42. Thank you!
    Questions?
    @siddharthlatest

    View Slide