Storing all state on the server side

In this presentation I suggest some ways on how state handling can be improved in very large and data-intensive JavaScript applications. I gave this presentation at mloc.js 2015 in Budapest, Hungary on 19th of June 2015.

If you think this presentation could be useful for some of your friends, could you please consider clicking the share button above. I feel that many of the ideas mentioned in the deck are heavily bubbling under the mainstream of the JavaScript scene. It's just better for everyone, when more and more people in the JavaScript community get to hear about alternative architectual solutions. It's really the mix of people and ideas, which together has the power to move this community of ours forward. Thanx.

Transcript

1. Storing all state
   on the server side
   Jouni Kaplas
   @kaplas
   kaplas.net/blog
   

   View Slide

2. LEAN SERVICE CREATION
   @kaplas
   That’s
   mloc.js
   

   View Slide

3. LEAN SERVICE CREATION
   That’s
   Futurice
   @kaplas
   

   View Slide

4. LEAN SERVICE CREATION
   @kaplas
   That’s
   Tampere
   

   View Slide

5. LEAN SERVICE CREATION
   Application state
   BUSINESS DATA
   USER-
   GENERATED
   DATA
   CONFIGURATION
   DATA
   UI STATE
   DEVICE STATE
   DEVICE
   BROWSER
   USER ?
   @kaplas
   

   View Slide

6. LEAN SERVICE CREATION
   Question 1:
   What if the amount of business
   data is just huge?
   BUSINESS DATA
   @kaplas
   

   View Slide

7. LEAN SERVICE CREATION
   Question 2:
   What should we do for these
   other parts here?
   USER-
   GENERATED
   DATA
   CONFIGURATION
   DATA
   UI STATE
   DEVICE STATE
   DEVICE
   BROWSER
   USER ?
   @kaplas
   

   View Slide

8. LEAN SERVICE CREATION
   @kaplas
   Example:
   Cloud of a future
   connected home
   

   View Slide

9. LEAN SERVICE CREATION
   @kaplas
   How devices talk
   to each other on a
   physical level is
   not interesting
   

   View Slide

10. LEAN SERVICE CREATION
    @kaplas
    On a logical level
    though, those
    probably use a
    data bus for
    communication
    

    View Slide

11. LEAN SERVICE CREATION
    Situation
    •  The amount of data is huge
    •  It comes in real-time
    •  It is very dynamic by nature
    @kaplas
    

    View Slide

12. LEAN SERVICE CREATION
    What kind of web UI would be
    suitable for this type of system?
    @kaplas
    

    View Slide

13. LEAN SERVICE CREATION
    Traditional data fetching
    …has quite a lot of abstraction layers
    CONTROLLERS
    ROUTES
    ORM
    SCHEMAS
    DB
    V
    C
    M
    V
    C
    M
    V
    C
    M
    M
    M
    M
    HTTP
    REST API
    DTO
    PROXY
    @kaplas
    

    View Slide

14. LEAN SERVICE CREATION
    CONTROLLERS
    ROUTES
    ORM
    SCHEMAS
    DB
    V
    C
    M
    V
    C
    M
    V
    C
    M
    M
    M
    M
    HTTP
    REST API
    DTO
    PROXY
    @kaplas
    Traditional data fetching
    …may cache app state in many modules
    

    View Slide

15. LEAN SERVICE CREATION
    What we really
    just want
    DB
    V
    V
    V
    @kaplas
    

    View Slide

16. LEAN SERVICE CREATION
    So what could
    we do?
    CONTROLLERS
    ROUTES
    ORM
    SCHEMAS
    DB
    V
    C
    M
    V
    C
    M
    V
    C
    M
    M
    M
    M
    HTTP
    REST API
    DTO
    @kaplas
    

    View Slide

17. LEAN SERVICE CREATION
    1) Use a reactive
    data source
    CONTROLLERS
    ROUTES
    V
    C
    M
    V
    C
    M
    V
    C
    M
    M
    M
    M
    HTTP
    REST API
    DTO
    LOGIC
    LAYER
    DATA
    SOURCE
    (eg. a data bus)
    @kaplas
    

    View Slide

18. LEAN SERVICE CREATION
    2) Use global state
    CONTROLLERS
    ROUTES
    HTTP
    REST API
    DTO
    V
    C
    M
    V
    C
    V
    C
    LOGIC
    LAYER
    DATA
    SOURCE
    @kaplas
    

    View Slide

19. LEAN SERVICE CREATION
    3) Pushy backend
    connection
    CONTROLLERS
    ROUTES
    DTO
    V
    C
    M
    V
    C
    V
    C
    WS
    LOGIC
    LAYER
    DATA
    SOURCE
    @kaplas
    

    View Slide

20. LEAN SERVICE CREATION
    4) Use backend just
    as a data gateway
    DTO
    V
    C
    M
    V
    C
    V
    C
    WS
    SOCKET-2-DS
    ADAPTER
    LOGIC
    LAYER
    DATA
    SOURCE
    @kaplas
    

    View Slide

21. The problem of
    addressing
    Photo “Mailboxes in ivy” by Ryan McFarland (www.zieak.com) - CC Attribution license
    @kaplas
    

    View Slide

22. LEAN SERVICE CREATION
    Anything goes,
    really
    •  You just need to be able to identify a resource
    @kaplas
    

    View Slide

23. LEAN SERVICE CREATION
    Adressing with
    namespaces may provide
    even more flexibility
    Namespace Open sandbox
    @kaplas
    https://github.com/kaplas/memory-game/blob/master/gulpfile.js
    

    View Slide

24. LEAN SERVICE CREATION
    Web UI
    SOCKET-2-DS
    ADAPTER
    V
    C
    M
    V
    C
    V
    C
    WS
    @kaplas
    

    View Slide

25. LEAN SERVICE CREATION
    Web UI
    •  The amount of data is huge
    •  It comes in real-time
    •  It is very dynamic by nature
    ✓
    ✓
    ✓
    BUSINESS DATA ✓
    @kaplas
    ➡
    

    View Slide

26. Photo “Third Beach - La Push, Washington” by NullSynapse – CC Attribution license
    The web is getting pushy,
    and it has taken us way too
    long to realize that
    @kaplas
    

    View Slide

27. LEAN SERVICE CREATION
    Application state
    USER-
    GENERATED
    DATA
    CONFIGURATION
    DATA
    UI STATE
    DEVICE STATE
    DEVICE
    BROWSER
    USER ?
    @kaplas
    

    View Slide

28. LEAN SERVICE CREATION
    Why can not we
    store these as well?
    USER-
    GENERATED
    DATA
    CONFIGURATION
    DATA
    UI STATE
    DEVICE STATE
    DEVICE
    BROWSER
    USER ?
    @kaplas
    

    View Slide

29. MIND BLOWN
    

    View Slide

30. LEAN SERVICE CREATION
    Share unsaved changes
    Image from- http://johnpolacek.github.io/
    @kaplas
    

    View Slide

31. LEAN SERVICE CREATION
    Share unsaved changes
    Image from- http://johnpolacek.github.io/
    @kaplas
    Auto s
    Auto s
    Auto s
    

    View Slide

32. LEAN SERVICE CREATION
    Share unsaved changes
    Image from- http://johnpolacek.github.io/
    @kaplas
    Auto sync, yey!
    Auto sync, yey!
    Auto sync, yey!
    

    View Slide

33. LEAN SERVICE CREATION
    Live configuration of a system
    Big image from- http://johnpolacek.github.io/
    Tablet image by Icon Works on flaticon.com
    ✓ ✓ ✓
    @kaplas
    

    View Slide

34. LEAN SERVICE CREATION
    Live configuration of a system
    Big image from- http://johnpolacek.github.io/
    Tablet image by Icon Works on flaticon.com
    ✓ ✓
    @kaplas
    

    View Slide

35. LEAN SERVICE CREATION
    Live configuration of a system
    Big image from- http://johnpolacek.github.io/
    Tablet image by Icon Works on flaticon.com
    ✓
    @kaplas
    

    View Slide

36. LEAN SERVICE CREATION
    Live configuration of a system
    Big image from- http://johnpolacek.github.io/
    Tablet image by Icon Works on flaticon.com
    ✓ ✓
    @kaplas
    

    View Slide

37. LEAN SERVICE CREATION
    Inform about future changes
    Image from- http://johnpolacek.github.io/
    @kaplas
    

    View Slide

38. LEAN SERVICE CREATION
    Inform about future changes
    Image from- http://johnpolacek.github.io/
    “other user is
    editing this field”
    @kaplas
    I am editing…
    

    View Slide

39. There really are no limit on
    what you can do and store
    @kaplas
    

    View Slide

40. LEAN SERVICE CREATION
    Application state
    UI STATE
    @kaplas
    

    View Slide

41. UI events will cause (some)
    state that is outside of our
    global state pattern
    @kaplas
    

    View Slide

42. LEAN SERVICE CREATION
    “At least we had a
    beautiful architecture…”
    @kaplas
    That’s my own dog, Riksu, by the way J
    

    View Slide

43. LEAN SERVICE CREATION
    PROBLEM
    If UI events cause changes in the
    application state (like they usually
    should), we will get data roundtrips
    to the server.
    The app should be usable during
    those times as well.
    Internal state
    caused by UI events
    OUR STRATEGY
    Hybrid:
    Some UI style changes on how
    roundtripped data and
    user-generated data are
    displayed, so that those can not be
    mixed accidentally.
    User-generated state is stored as
    a small piece of temporary internal
    state inside the UI element in
    question.
    COMMON STRATEGIES
    Optimistic:
    Expect the roundtrip to succeed,
    fallback to previous state if it didn’t.
    This will cause internal state.
    Pessimistic:
    Do not expect anything, only
    update UI when the roundtrip has
    succeeded.
    @kaplas
    

    View Slide

44. •  Security needs more attention
    •  Handling of aggregated data can be challenging
    •  Scalability / Load balancing is a bit unresolved
    problem
    •  There is no safety net of type checking
    •  Logic layer has to be somewhere
    Cons:
    Reality check @kaplas
    

    View Slide

45. LEAN SERVICE CREATION
    Application state: Done!
    BUSINESS DATA
    USER-
    GENERATED
    DATA
    CONFIGURATION
    DATA
    UI STATE
    DEVICE STATE
    DEVICE
    BROWSER
    USER ?
    @kaplas
    

    View Slide

46. LEAN SERVICE CREATION
    Here,
    take these with you
    Ie. What ideas of this talk could hopefully
    help you in your own projects
    @kaplas
    

    View Slide

47. LEAN SERVICE CREATION
    Photo “The number 1” by Timothy Krause
    CC Attribution license
    Subscribe to
    your data
    @kaplas
    

    View Slide

48. LEAN SERVICE CREATION
    Photo “me and the number two...” by kate hiscock
    CC Attribution license
    Use the global
    state pattern
    @kaplas
    

    View Slide

49. LEAN SERVICE CREATION
    Use as
    lightweight
    abstractions
    as possible
    Photo “three” by Abulic Monkey
    CC Attribution license
    @kaplas
    

    View Slide

50. LEAN SERVICE CREATION
    Photo “20120213_1042” by Marco Ghitti
    CC Attribution license
    Try to find
    new creative
    ways to store
    and use your
    data
    @kaplas
    

    View Slide

51. LEAN SERVICE CREATION
    UI events will
    always be
    problematic
    Photo “day 5” by rosmary
    CC Attribution license
    @kaplas
    

    View Slide

52. That’s it. Thank you!
    Jouni Kaplas
    @kaplas
    kaplas.net/blog
    

    View Slide