Appbase - A realtime events-based document store

Redisconf '15 talk: Using Redis to build a document store with a streaming events server.

Transcript

1. Appbase:
   A Realtime events-based
   document index store
   - Siddharth Kothari
   @siddharthlatest
   

   View Slide

2. Javascript foobar, Database enthusiast
   I know R+L = J
   CEO at appbase.io
   

   View Slide

3. Why?
   

   View Slide

4. Why?
   relational data vs linked data.
   

   View Slide

5. Imagine…
   

   View Slide

6. Imagine…
   1) Document store schema
   

   View Slide

7. Imagine…
   1) Document store schema
   2) Dynamic References
   

   View Slide

8. Imagine…
   1) Document store schema
   2) Dynamic References
   3) All JSON
   

   View Slide

9. Hint:
   Is this MongoDB?
   

   View Slide

10. Hint:
    It’s not MongoDB.
    This is Redis!
    

    View Slide

11. Appbase:
    A Realtime events-based
    document index store
    

    View Slide

12. Appbase:
    A Realtime events-based
    document index store
    1) Document store schema
    2) Dynamic References
    3) All JSON
    

    View Slide

13. Appbase:
    A Realtime events-based
    document index store
    1) Document store schema
    2) Dynamic References
    3) All JSON
    4) Realtime events - Subscribe to updates
    

    View Slide

14. Pulling a Mongo! Here’s how:
    Document store schema
    Collection
    

    View Slide

15. Pulling a Mongo! Here’s how:
    Document store schema
    { “item”: “pencil”,
    “qty”: 500,
    “type”: “no.3”}
    Collection Document
    

    View Slide

16. Pulling a Mongo! Here’s how:
    Embedded Documents
    {
    “item”: “pencil”,
    “qty”: 500,
    “type”: “no.3”,
    “foo”: {
    “embedded_foo”: “bar”
    }
    }
    Embedded Property
    

    View Slide

17. Pulling a Mongo! Here’s how:
    Adding References
    {
    “item”: “pencil”,
    “qty”: 500,
    “type”: “no.3”,
    “foo”: {
    “embedded_foo”: “bar”
    },
    “/ref1”: “user/item#3”
    }
    Reference
    user
    item#3
    

    View Slide

18. Pulling a Mongo! Here’s how:
    Adding References
    {
    “item”: “pencil”,
    …,
    …,
    “/ref1”: {
    “_collection”: “user”,
    “_id”: “item#3”,
    “ref_prop#1”: “something”
    }
    }
    Reference
    Adding the doc directly
    

    View Slide

19. Beyond Document Store!
    Introducing Paths
    {
    “_id”: “pencil”,
    “_collection”: “stationary”,
    “foo”: {
    “embedded_foo”: “bar”
    },
    “/ref1”: “user/item#3”
    }
    Reference
    user
    item#3
    

    View Slide

20. Beyond Document Store!
    Introducing Paths
    {
    “_id”: “pencil”,
    “_collection”: “stnry”,
    “foo”: {
    “embedded_foo”: “bar”
    },
    “/ref1”: “user/item#3”
    }
    Reference
    user
    item#3
    Two paths:
    1. /user/item#3
    2. /stnry/pencil/ref1
    

    View Slide

21. Beyond Document Store!
    Introducing Paths
    Two paths:
    1. /user/item#3
    2. /stnry/pencil/ref1
    

    View Slide

22. Beyond Document Store!
    Introducing Paths
    a
    b c
    d
    1. a/b/c/d,
    2. z/b/c/d,
    3. q/w/d
    4. …
    z
    q
    w
    

    View Slide

23. So far…
    Document store with true references
    a
    b c
    d
    z
    q
    w
    {
    “_id”: “d”,
    “_collection”: “alpha”,
    “depth”: “4”
    }
    

    View Slide

24. Taking advantage of true referencing
    Event subscriptions:
    

    View Slide

25. {
    “_id”: “d”,
    “_collection”: “alpha”,
    “depth”: “4”
    }
    Taking advantage of true referencing
    a
    b
    z
    Event subscriptions:
    1.Property updates
    

    View Slide

26. {
    “_id”: “d”,
    “_collection”: “alpha”,
    “depth”: “4”
    }
    Taking advantage of true referencing
    a
    b
    z
    c
    d
    q
    w
    Event subscriptions:
    1.Property updates
    2.Reference updates
    

    View Slide

27. Open Source
    

    View Slide

28. Open Source
    License: Apache 2
    v0.1 - Brave Souls
    

    View Slide

29. Open Source
    appbaseio.github.io/appbase-redis
    License: Apache 2
    v0.1 - Brave Souls
    

    View Slide

30. Live Demo
    

    View Slide

31. Live Demo
    /users/user_542
    Create a new document
    

    View Slide

32. Live Demo
    /users/user_542
    Create a new document
    Assign some properties
    

    View Slide

33. Live Demo
    /users/user_542
    Create a new document
    Assign some properties
    {
    “em”: “s@a.io”,

    “points”: 5321,
    “/friends”: …
    }
    

    View Slide

34. Live Demo
    /users/user_542
    Create a new document
    Assign some properties
    {
    “em”: “s@a.io”,

    “points”: 5321,
    “/friends”: …
    }
    Watch it Stream
    

    View Slide

35. API Spec
    Comes with a HTTP API server
    

    View Slide

36. API Spec
    Comes with a HTTP API server
    PATCH /{collection}/{document}/[path]
    GET /{collection}/{document}/[path]
    ?stream=true
    

    View Slide

37. API Spec
    Comes with a HTTP API server
    PATCH
    GET
    https://github.com/appbaseio/
    appbase-redis/
    blob/master/
    index.js
    Entire Spec
    

    View Slide

38. API Spec
    Request Body
    

    View Slide

39. Roadmap to v1.0
    1. Battle Testing
    Unit Tests, Better Code
    Perf Tests
    

    View Slide

40. Roadmap to v1.0
    1. Battle Testing
    Unit Tests, Better Code
    Perf Tests
    2. Scaling
    Use Pushpin = Message Queue + Streaming
    

    View Slide

41. Roadmap to v1.0
    1. Battle Testing
    Unit Tests, Better Code
    Perf Tests
    2. Scaling
    Use Pushpin = Message Queue + Streaming
    3. Rich Querying
    More filters, aggregation queries
    

    View Slide

42. Thank you!
    Questions?
    @siddharthlatest
    

    View Slide