Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Why Data Literals Matters? English version

Why Data Literals Matters? English version

English version

Jérémie Grodziski

September 15, 2017
Tweet

More Decks by Jérémie Grodziski

Other Decks in Programming

Transcript

  1. Why Data Literals
    Matters?
    Jérémie Grodziski - @jgrodziski

    View Slide

  2. At the
    Beginning of
    every Project
    I look for
    Two Things

    View Slide

  3. Some Examples of
    Concepts from the Domain
    How can I obtain
    Immediate Feedback?

    View Slide

  4. Some Examples of
    Concepts from the Domain
    Concrete
    and
    Real
    Visible and
    Understandable
    With Data !

    View Slide

  5. How can I obtain
    Immediate Feedback?
    From the Software I
    build?
    From my
    Assumptions?

    View Slide

  6. Why the Success of JSON?
    Concrete and
    Real
    Visible and
    Understandable
    Data!
    WYSIWYG
    JS is ubiquituous
    with browser

    View Slide

  7. Usable directly or with a
    simple JSON.parse()

    View Slide

  8. Where do you usually see your
    Data?
    Stdout?
    Debugger?
    Databases?
    Text Files (CSV, XML, JSON)?
    When you write code? rarely...

    View Slide

  9. Data Literal
    « A notation for representing a fixed value in
    source code »
    Available in some
    languages for
    « composite » types
    Array, associative
    structure (map, vector,
    object, etc.)
    Available in every
    languages for
    « scalar » types
    Integer, Float, Boolean,
    String, Character

    View Slide

  10. Some « Scalars »
    String
    Boolean
    Numbers 1 2 3 4.5
    Keyword :key1 :key2
    Symbol this that λ

    View Slide

  11. Some Data Structures
    List
    Vector
    Map
    Set

    View Slide

  12. …and lot of functions
    Alan J. Perlis, developed Algol language, 1st recipient of the Turing
    Award
    For other « Perlism »: http://www.cs.yale.edu/quotes.html
    « It is better to have 100 functions
    operate on 1 data structure than 10
    functions on 10 data structures »

    View Slide

  13. Lot of functions
    Etc.
    Excerpt from clojure.org/cheatsheet

    View Slide

  14. Tree Representation
    Two functions :
    Does this node has some child nodes?
    ⇒  returns a boolean
    How to retrieve the data structure of the
    child nodes?
    => Returns child nodes data structure

    View Slide

  15. Tree Literal in JSON with a map and
    arrays
    Ex: JSON output of
    the tree command
    on my mac

    View Slide

  16. Tree Literal in Clojure with map and
    arrays

    View Slide

  17. Each node is an array
    [node-root child1 child2 …]
    More simple, a Tree Literal with an
    array of arrays
    tree-seq function in Clojure
    structure that tree and allows
    “walking” it with clojure.walk
    Viz done with github.com/ztellman/rhizome

    View Slide

  18. Tree literal with more complex
    objects

    View Slide

  19. Graph Representation
    Given a node
    Retrieve the list of
    identifiers of the
    adjacent nodes
    Two functions
    Retrieve the list of
    identifiers of the
    graph nodes

    View Slide

  20. View Slide

  21. A graphical representation of the graph is
    often necessary to « touch » it
    Viz done with github.com/ztellman/rhizome

    View Slide

  22. Use Cases
    Of
    Data Literals
    At Design-time

    View Slide

  23. Visual Representation of your
    problem – Bowling Game

    View Slide

  24. Visual Representation of your
    problem – Chess Game

    View Slide

  25. Visual Representation of your
    problem – Sudoku Game

    View Slide

  26. Data Literals on steroids:
    Clojure EDN (Extended Data Notation)
    The Tagged Elements of Clojure separate the
    literal representation from the memory
    representation:
    #tag "valueString"!
    Instant
    UUID

    View Slide

  27. Object defined by a thread of
    continuity and a particular
    identity
    Ex :
    The bank account n° 30003
    02367 000593287642 13,
    The credit card 8644 7023 0531 105
    The EventBrite ticket for
    SoftShake
    Domain-Driven Design
    Immutable object that
    represents a concept whose
    equality is based on all the
    attributes and has no
    identity
    Ex :
    The 42 EUR amount
    The UUID f4fbe22a-9d53-4359-
    adbb-57f7d21ed549
    The instant 21 october 2015
    at 16:29
    The distance 53 meters
    Value Object Entity

    View Slide

  28. Tagged Data Literals are perfect for
    Value Object
    For each Value Object, add a literal
    representation as a « String » with
    valueOf(String)and toString() !
    methods!

    View Slide

  29. What’s a state?
    State is « summarized » with a past participle that has a static and
    stable meaning (order paid, user connected, etc.)
    State influence the object behavior
    State implies that some Data Transformations occured during a
    state transition, triggered by an event
    State is associated with an identity, then together they form an
    entity, that represents the different states and transitions along the
    time
    A state is the relation between an entity and
    the values that she owns at the t instant;
    That state can change

    View Slide

  30. State, Identity and Function
    State
    Identity
    State 1 State 2 State 3
    Function Function
    Event triggers Event triggers
    •  value 1a
    •  value 1b
    •  value 1c
    •  value 1a
    •  value 1b
    •  value 1c
    •  value 2a
    •  value 2b
    •  value 1a
    •  value 1b
    •  value 1c
    •  value 2a
    •  value 2b
    •  value 3a
    Publish
    Internal Event
    Publish
    Internal Event
    ...
    Time

    View Slide

  31. Exanple with the Order entity
    Identity : Order N° 123456
    Checkout Paid Shipped
    pay ship
    Payment details
    from Payment
    Gateway received
    Carrier data
    q  Customer Data
    q  Product Lines
    (product + qty)
    q  Shipping Data
    States
    checkout
    q  Customer Data
    q  Product Lines
    (product + qty)
    q  Shipping Data
    q  Payment Data
    q  Customer Data
    q  Product Lines
    (product + qty)
    q  Shipping Data
    q  Payment Data
    q  Carrier Data
    Front website form
    submitted
    Order submitted
    event Order paid
    event Order shipped event
    Time

    View Slide

  32. Les Data Literals sont également
    parfait pour visualiser les
    transitions d’états d’une Entity
    Identity
    State 1 State 2 State 3
    Function Function
    •  value 1a
    •  value 1b
    •  value 1c
    •  value 1a
    •  value 1b
    •  value 1c
    •  value 2a
    •  value 2b
    •  value 1a
    •  value 1b
    •  value 1c
    •  value 2a
    •  value 2b
    •  value 3a
    ...
    Time
    States

    View Slide

  33. State transitions seen as
    functions
    Each « Snapshot » of the entity state at a
    particular instant can be displayed with a
    data literal
    A diff allows to visualize the differences
    between the prevState and the nextState
    provided by the function
    function(input): output!
    function(prevState, event): nextState!

    View Slide

  34. setShippingMethod!
    increaseItemQty!
    Easy Diff

    View Slide

  35. OOP done right with Clojure
    Identity State as values Function
    Method = function applied to an entity’s
    state, producing a new state (eventually)
    Entity = Identity + State
    Protocol = Interface = set of methods

    View Slide

  36. During Testing and
    Debugging
    Use Cases
    Of
    Data Literals

    View Slide

  37. TDD

    View Slide

  38. Testing BDD
    In BDD scenarios (Behavior-Driven
    Development) the data can be expressed in
    a tabular form following the Gherkin
    grammar
    Some framework allows to input a Data
    Literal directly: cf framework BDD Spexec
    See speakerdeck.com/jgrodziski/anatomy-of-a-bdd-execution-library-in-clojure

    View Slide

  39. Example of a BDD scenario
    with tabular data

    View Slide

  40. Debugging
    Tools.trace is a clojure library that
    instrumentalize functions or a whole
    namespace and log functions input and output
    Combined with
    aprint (Awesome
    Print) it does
    miracles…almost no
    need of a debugger

    View Slide

  41. Data Integration
    Use Cases
    Of
    Data Literals

    View Slide

  42. Data Flow

    View Slide

  43. Principles for Functions that fit
    Data Flows
    Pures
    Functions
    Immutable
    Data
    Structures
    seen as Data
    Literals
    Composables
    Steps (aka.
    Functions)
    Isolated Side
    Effects

    View Slide

  44. Pure Function
    Always return the same result given the same
    arguments whatever the number of
    invocation (idempotence)
    Lead to no observable side effect (safe)
    Referential Transparency: Time doesn’t affect
    the result of a function, hence the function is
    transparent regarding time
    Allows « memoization », composition increase
    the testability

    View Slide

  45. Immutable Data Structures
    The functional approach foster functions
    consuming values and returning new values
    The initial data structure is never modified
    Clojure provides an efficient implementation
    with Structural Sharing of Persistent Data
    Structures (values are shared between the old
    and new version of the structure)

    View Slide

  46. Composable Functions
    Function 1
    Function 2
    Function 3
    state
    t0
    State
    t2
    State
    t1
    State
    t3

    View Slide

  47. Isolated Side Effects
    Separate the data processing logic from
    the side effect (persistence, messaging, etc.)
    Place the non-pure functions at the
    beginning or end of the chain

    View Slide

  48. Pure Functions or not?
    Pure Functions without
    side effects (Safe and
    Idempotent)
    •  Data Transformations
    •  Data structure shape (ex :
    map -> list)
    •  Values (transcoding)
    •  Creation of new data (ex:
    from computation)
    •  Decision : authorisation,
    result
    Non-pure Functions
    Impures with side effects
    (Non Safe or Non
    Idempotent)
    •  Safe, Non Idempotent :
    Data enrichment from a
    « mutable » external data
    source (ex : read from a
    database)
    •  Non Safe, Non
    Idempotent : Persistence
    (insert, update),
    Messaging

    View Slide

  49. Data Pipeline
    Outside
    World
    Outside
    World
    Convert from External
    Repr. To Internal
    Enrich from
    external DataSource
    Validate
    Transform
    Transform
    Transform Operate
    (Persistence,
    Messaging)
    Pures Functions
    Get Data
    Non-pures
    Functions
    Non-pures
    Functions
    The non-pures functions
    are pushed to both ends
    of the pipeline

    View Slide

  50. Domain Specific
    Language (DSL)
    Use Cases
    Of
    Data Literals

    View Slide

  51. Domain-Specific Language
    A DSL with a data structure benefits from
    the powerful data manipulation functions of
    Clojure
    SQL
    HTML

    View Slide

  52. Logs
    Use Cases
    Of
    Data Literals

    View Slide

  53. Logs with Data Literals
    Store the contextual data of logs in a form
    that can be processed easily is a
    recommendation from the visualization
    tools (Splunk, Logstash)

    View Slide

  54. Logs HTTP Requests with Data
    Literals
    Ring is the idiomatic library of Clojure for
    the web: each request and response are
    represented by a map processed by a
    handler

    View Slide

  55. It’s all well and good that logs
    but…
    Can I replay a user interaction?

    View Slide

  56. Logs HTTP Requests with Data
    Literals and the method
    invocation
    A simple eval allows
    to replay the whole
    set of HTTP requests

    View Slide

  57. Clojure is an Homoiconic language
    Every Clojure clojure is a data structure of
    type List of the form: (verb a b)!
    Clojure is itself a Data Literal !
    Allows very powerful meta-programming
    « Code is Data, Data is Code »
    from homo - the same - and icon - representation -

    View Slide

  58. Code is Data
    Vector
    List
    Symbols
    List

    View Slide

  59. Food for Thoughts about Event
    Sourcing
    Event Sourcing means storing all the state
    transitions of an application like a series of
    events
    Replay is almost built-in … but is strongly
    dependant on the source code version that
    process the event at a particular momentnt t
    Can we store the source code which process an
    event besides the even itself?

    View Slide

  60. Food for Thoughts about Event
    Sourcing
    Storing data in a textual or literal form is
    the most lasting one
    It’s easily readable both by a human AND a
    machine

    View Slide

  61. And Feedback?
    A REPL or a notebook enables direct
    interaction with Data Literals
    And moreover enter « in the Zone », Be in
    the « Flow » where productivity is at its
    high
    cf Mihály Csíkszentmihályi

    View Slide

  62. Takeaways
    Make your data VISIBLE! Touch it!
    Data express your DOMAIN
    Your Data Literals are your clay, your
    functions are your hands that shape it
    Make your functions composables
    Isolate Side-effects

    View Slide

  63. « An algorithm must be seen to be
    believed »
    Donal Knuth
    « Source Code is static, Behavior is
    dynamic:
    Good programs make it easy to reason
    about their Behavior »
    Barbara Liskov

    View Slide

  64. « Give me exemples »
    « Give me something concrete
    it’s my job to abstract it! »
    « Premature Abstraction is the
    Root of all Evil »
    « Give me a REPL! »

    View Slide

  65. Thank you!
    Questions? Comments?
    @jgrodziski
    www.grodziski.com
    Get the slides:
    speakerdeck.com/jgrodziski/why-data-literals-matters-en

    View Slide

  66. Bowling Game

    View Slide

  67. TDD Bowling Game
    Domain : Bowling
    The game consists of 10 frames as shown above. In each frame the player has
    two opportunities to knock down 10 pins. The score for the frame is the total
    number of pins knocked down, plus bonuses for strikes and spares.
    A spare is when the player knocks down all 10 pins in two tries. The bonus for
    that frame is the number of pins knocked down by the next roll. ‘/’ denote a spare in
    the score sheet.
    A strike is when the player knocks down all 10 pins on his first try. The bonus
    for that frame is the value of the next two balls rolled. ‘X’ denote a striker in the
    score sheet.
    In the tenth frame a player who rolls a spare or strike is allowed to roll the extra
    balls to complete the frame (so 3 balls can be rolled in tenth frame).

    View Slide

  68. Code Kata : Number
    theory 2, the return
    Now, let’s get back to the « number theory » stories for which we will
    rewrite but in a TDD manner.
    Then we’ll compare the two solution.

    View Slide