Why Data Literals Matters? English version

English version

Transcript

1. Why Data Literals
   Matters?
   Jérémie Grodziski - @jgrodziski
   

   View Slide

2. At the
   Beginning of
   every Project
   I look for
   Two Things
   

   View Slide

3. Some Examples of
   Concepts from the Domain
   How can I obtain
   Immediate Feedback?
   

   View Slide

4. Some Examples of
   Concepts from the Domain
   Concrete
   and
   Real
   Visible and
   Understandable
   With Data !
   

   View Slide

5. How can I obtain
   Immediate Feedback?
   From the Software I
   build?
   From my
   Assumptions?
   

   View Slide

6. Why the Success of JSON?
   Concrete and
   Real
   Visible and
   Understandable
   Data!
   WYSIWYG
   JS is ubiquituous
   with browser
   

   View Slide

7. Usable directly or with a
   simple JSON.parse()
   

   View Slide

8. Where do you usually see your
   Data?
   Stdout?
   Debugger?
   Databases?
   Text Files (CSV, XML, JSON)?
   When you write code? rarely...
   

   View Slide

9. Data Literal
   « A notation for representing a fixed value in
   source code »
   Available in some
   languages for
   « composite » types
   Array, associative
   structure (map, vector,
   object, etc.)
   Available in every
   languages for
   « scalar » types
   Integer, Float, Boolean,
   String, Character
   

   View Slide

10. Some « Scalars »
    String
    Boolean
    Numbers 1 2 3 4.5
    Keyword :key1 :key2
    Symbol this that λ
    

    View Slide

11. Some Data Structures
    List
    Vector
    Map
    Set
    

    View Slide

12. …and lot of functions
    Alan J. Perlis, developed Algol language, 1st recipient of the Turing
    Award
    For other « Perlism »: http://www.cs.yale.edu/quotes.html
    « It is better to have 100 functions
    operate on 1 data structure than 10
    functions on 10 data structures »
    

    View Slide

13. Lot of functions
    Etc.
    Excerpt from clojure.org/cheatsheet
    

    View Slide

14. Tree Representation
    Two functions :
    Does this node has some child nodes?
    ⇒  returns a boolean
    How to retrieve the data structure of the
    child nodes?
    => Returns child nodes data structure
    

    View Slide

15. Tree Literal in JSON with a map and
    arrays
    Ex: JSON output of
    the tree command
    on my mac
    

    View Slide

16. Tree Literal in Clojure with map and
    arrays
    

    View Slide

17. Each node is an array
    [node-root child1 child2 …]
    More simple, a Tree Literal with an
    array of arrays
    tree-seq function in Clojure
    structure that tree and allows
    “walking” it with clojure.walk
    Viz done with github.com/ztellman/rhizome
    

    View Slide

18. Tree literal with more complex
    objects
    

    View Slide

19. Graph Representation
    Given a node
    Retrieve the list of
    identifiers of the
    adjacent nodes
    Two functions
    Retrieve the list of
    identifiers of the
    graph nodes
    

    View Slide

20. View Slide

21. A graphical representation of the graph is
    often necessary to « touch » it
    Viz done with github.com/ztellman/rhizome
    

    View Slide

22. Use Cases
    Of
    Data Literals
    At Design-time
    

    View Slide

23. Visual Representation of your
    problem – Bowling Game
    

    View Slide

24. Visual Representation of your
    problem – Chess Game
    

    View Slide

25. Visual Representation of your
    problem – Sudoku Game
    

    View Slide

26. Data Literals on steroids:
    Clojure EDN (Extended Data Notation)
    The Tagged Elements of Clojure separate the
    literal representation from the memory
    representation:
    #tag "valueString"!
    Instant
    UUID
    

    View Slide

27. Object defined by a thread of
    continuity and a particular
    identity
    Ex :
    The bank account n° 30003
    02367 000593287642 13,
    The credit card 8644 7023 0531 105
    The EventBrite ticket for
    SoftShake
    Domain-Driven Design
    Immutable object that
    represents a concept whose
    equality is based on all the
    attributes and has no
    identity
    Ex :
    The 42 EUR amount
    The UUID f4fbe22a-9d53-4359-
    adbb-57f7d21ed549
    The instant 21 october 2015
    at 16:29
    The distance 53 meters
    Value Object Entity
    

    View Slide

28. Tagged Data Literals are perfect for
    Value Object
    For each Value Object, add a literal
    representation as a « String » with
    valueOf(String)and toString() !
    methods!
    

    View Slide

29. What’s a state?
    State is « summarized » with a past participle that has a static and
    stable meaning (order paid, user connected, etc.)
    State influence the object behavior
    State implies that some Data Transformations occured during a
    state transition, triggered by an event
    State is associated with an identity, then together they form an
    entity, that represents the different states and transitions along the
    time
    A state is the relation between an entity and
    the values that she owns at the t instant;
    That state can change
    

    View Slide

30. State, Identity and Function
    State
    Identity
    State 1 State 2 State 3
    Function Function
    Event triggers Event triggers
    •  value 1a
    •  value 1b
    •  value 1c
    •  value 1a
    •  value 1b
    •  value 1c
    •  value 2a
    •  value 2b
    •  value 1a
    •  value 1b
    •  value 1c
    •  value 2a
    •  value 2b
    •  value 3a
    Publish
    Internal Event
    Publish
    Internal Event
    ...
    Time
    

    View Slide

31. Exanple with the Order entity
    Identity : Order N° 123456
    Checkout Paid Shipped
    pay ship
    Payment details
    from Payment
    Gateway received
    Carrier data
    q  Customer Data
    q  Product Lines
    (product + qty)
    q  Shipping Data
    States
    checkout
    q  Customer Data
    q  Product Lines
    (product + qty)
    q  Shipping Data
    q  Payment Data
    q  Customer Data
    q  Product Lines
    (product + qty)
    q  Shipping Data
    q  Payment Data
    q  Carrier Data
    Front website form
    submitted
    Order submitted
    event Order paid
    event Order shipped event
    Time
    

    View Slide

32. Les Data Literals sont également
    parfait pour visualiser les
    transitions d’états d’une Entity
    Identity
    State 1 State 2 State 3
    Function Function
    •  value 1a
    •  value 1b
    •  value 1c
    •  value 1a
    •  value 1b
    •  value 1c
    •  value 2a
    •  value 2b
    •  value 1a
    •  value 1b
    •  value 1c
    •  value 2a
    •  value 2b
    •  value 3a
    ...
    Time
    States
    

    View Slide

33. State transitions seen as
    functions
    Each « Snapshot » of the entity state at a
    particular instant can be displayed with a
    data literal
    A diff allows to visualize the differences
    between the prevState and the nextState
    provided by the function
    function(input): output!
    function(prevState, event): nextState!
    

    View Slide

34. setShippingMethod!
    increaseItemQty!
    Easy Diff
    

    View Slide

35. OOP done right with Clojure
    Identity State as values Function
    Method = function applied to an entity’s
    state, producing a new state (eventually)
    Entity = Identity + State
    Protocol = Interface = set of methods
    

    View Slide

36. During Testing and
    Debugging
    Use Cases
    Of
    Data Literals
    

    View Slide

37. TDD
    

    View Slide

38. Testing BDD
    In BDD scenarios (Behavior-Driven
    Development) the data can be expressed in
    a tabular form following the Gherkin
    grammar
    Some framework allows to input a Data
    Literal directly: cf framework BDD Spexec
    See speakerdeck.com/jgrodziski/anatomy-of-a-bdd-execution-library-in-clojure
    

    View Slide

39. Example of a BDD scenario
    with tabular data
    

    View Slide

40. Debugging
    Tools.trace is a clojure library that
    instrumentalize functions or a whole
    namespace and log functions input and output
    Combined with
    aprint (Awesome
    Print) it does
    miracles…almost no
    need of a debugger
    

    View Slide

41. Data Integration
    Use Cases
    Of
    Data Literals
    

    View Slide

42. Data Flow
    

    View Slide

43. Principles for Functions that fit
    Data Flows
    Pures
    Functions
    Immutable
    Data
    Structures
    seen as Data
    Literals
    Composables
    Steps (aka.
    Functions)
    Isolated Side
    Effects
    

    View Slide

44. Pure Function
    Always return the same result given the same
    arguments whatever the number of
    invocation (idempotence)
    Lead to no observable side effect (safe)
    Referential Transparency: Time doesn’t affect
    the result of a function, hence the function is
    transparent regarding time
    Allows « memoization », composition increase
    the testability
    

    View Slide

45. Immutable Data Structures
    The functional approach foster functions
    consuming values and returning new values
    The initial data structure is never modified
    Clojure provides an efficient implementation
    with Structural Sharing of Persistent Data
    Structures (values are shared between the old
    and new version of the structure)
    

    View Slide

46. Composable Functions
    Function 1
    Function 2
    Function 3
    state
    t0
    State
    t2
    State
    t1
    State
    t3
    

    View Slide

47. Isolated Side Effects
    Separate the data processing logic from
    the side effect (persistence, messaging, etc.)
    Place the non-pure functions at the
    beginning or end of the chain
    

    View Slide

48. Pure Functions or not?
    Pure Functions without
    side effects (Safe and
    Idempotent)
    •  Data Transformations
    •  Data structure shape (ex :
    map -> list)
    •  Values (transcoding)
    •  Creation of new data (ex:
    from computation)
    •  Decision : authorisation,
    result
    Non-pure Functions
    Impures with side effects
    (Non Safe or Non
    Idempotent)
    •  Safe, Non Idempotent :
    Data enrichment from a
    « mutable » external data
    source (ex : read from a
    database)
    •  Non Safe, Non
    Idempotent : Persistence
    (insert, update),
    Messaging
    

    View Slide

49. Data Pipeline
    Outside
    World
    Outside
    World
    Convert from External
    Repr. To Internal
    Enrich from
    external DataSource
    Validate
    Transform
    Transform
    Transform Operate
    (Persistence,
    Messaging)
    Pures Functions
    Get Data
    Non-pures
    Functions
    Non-pures
    Functions
    The non-pures functions
    are pushed to both ends
    of the pipeline
    

    View Slide

50. Domain Specific
    Language (DSL)
    Use Cases
    Of
    Data Literals
    

    View Slide

51. Domain-Specific Language
    A DSL with a data structure benefits from
    the powerful data manipulation functions of
    Clojure
    SQL
    HTML
    

    View Slide

52. Logs
    Use Cases
    Of
    Data Literals
    

    View Slide

53. Logs with Data Literals
    Store the contextual data of logs in a form
    that can be processed easily is a
    recommendation from the visualization
    tools (Splunk, Logstash)
    

    View Slide

54. Logs HTTP Requests with Data
    Literals
    Ring is the idiomatic library of Clojure for
    the web: each request and response are
    represented by a map processed by a
    handler
    

    View Slide

55. It’s all well and good that logs
    but…
    Can I replay a user interaction?
    

    View Slide

56. Logs HTTP Requests with Data
    Literals and the method
    invocation
    A simple eval allows
    to replay the whole
    set of HTTP requests
    

    View Slide

57. Clojure is an Homoiconic language
    Every Clojure clojure is a data structure of
    type List of the form: (verb a b)!
    Clojure is itself a Data Literal !
    Allows very powerful meta-programming
    « Code is Data, Data is Code »
    from homo - the same - and icon - representation -
    

    View Slide

58. Code is Data
    Vector
    List
    Symbols
    List
    

    View Slide

59. Food for Thoughts about Event
    Sourcing
    Event Sourcing means storing all the state
    transitions of an application like a series of
    events
    Replay is almost built-in … but is strongly
    dependant on the source code version that
    process the event at a particular momentnt t
    Can we store the source code which process an
    event besides the even itself?
    

    View Slide

60. Food for Thoughts about Event
    Sourcing
    Storing data in a textual or literal form is
    the most lasting one
    It’s easily readable both by a human AND a
    machine
    

    View Slide

61. And Feedback?
    A REPL or a notebook enables direct
    interaction with Data Literals
    And moreover enter « in the Zone », Be in
    the « Flow » where productivity is at its
    high
    cf Mihály Csíkszentmihályi
    

    View Slide

62. Takeaways
    Make your data VISIBLE! Touch it!
    Data express your DOMAIN
    Your Data Literals are your clay, your
    functions are your hands that shape it
    Make your functions composables
    Isolate Side-effects
    

    View Slide

63. « An algorithm must be seen to be
    believed »
    Donal Knuth
    « Source Code is static, Behavior is
    dynamic:
    Good programs make it easy to reason
    about their Behavior »
    Barbara Liskov
    

    View Slide

64. « Give me exemples »
    « Give me something concrete
    it’s my job to abstract it! »
    « Premature Abstraction is the
    Root of all Evil »
    « Give me a REPL! »
    

    View Slide

65. Thank you!
    Questions? Comments?
    @jgrodziski
    www.grodziski.com
    Get the slides:
    speakerdeck.com/jgrodziski/why-data-literals-matters-en
    

    View Slide

66. Bowling Game
    

    View Slide

67. TDD Bowling Game
    Domain : Bowling
    The game consists of 10 frames as shown above. In each frame the player has
    two opportunities to knock down 10 pins. The score for the frame is the total
    number of pins knocked down, plus bonuses for strikes and spares.
    A spare is when the player knocks down all 10 pins in two tries. The bonus for
    that frame is the number of pins knocked down by the next roll. ‘/’ denote a spare in
    the score sheet.
    A strike is when the player knocks down all 10 pins on his first try. The bonus
    for that frame is the value of the next two balls rolled. ‘X’ denote a striker in the
    score sheet.
    In the tenth frame a player who rolls a spare or strike is allowed to roll the extra
    balls to complete the frame (so 3 balls can be rolled in tenth frame).
    

    View Slide

68. Code Kata : Number
    theory 2, the return
    Now, let’s get back to the « number theory » stories for which we will
    rewrite but in a TDD manner.
    Then we’ll compare the two solution.
    

    View Slide