Why Data Literals Matters? English version

Why Data Literals Matters? English version

English version


Jérémie Grodziski

September 15, 2017


  1. Why Data Literals Matters? Jérémie Grodziski - @jgrodziski

  2. At the Beginning of every Project I look for Two

  3. Some Examples of Concepts from the Domain How can I

    obtain Immediate Feedback?
  4. Some Examples of Concepts from the Domain Concrete and Real

    Visible and Understandable With Data !
  5. How can I obtain Immediate Feedback? From the Software I

    build? From my Assumptions?
  6. Why the Success of JSON? Concrete and Real Visible and

    Understandable Data! WYSIWYG JS is ubiquituous with browser
  7. Usable directly or with a simple JSON.parse()

  8. Where do you usually see your Data? Stdout? Debugger? Databases?

    Text Files (CSV, XML, JSON)? When you write code? rarely...
  9. Data Literal « A notation for representing a fixed value

    in source code » Available in some languages for « composite » types Array, associative structure (map, vector, object, etc.) Available in every languages for « scalar » types Integer, Float, Boolean, String, Character
  10. Some « Scalars » String Boolean Numbers 1 2 3

    4.5 Keyword :key1 :key2 Symbol this that λ
  11. Some Data Structures List Vector Map Set

  12. …and lot of functions Alan J. Perlis, developed Algol language,

    1st recipient of the Turing Award For other « Perlism »: http://www.cs.yale.edu/quotes.html « It is better to have 100 functions operate on 1 data structure than 10 functions on 10 data structures »
  13. Lot of functions Etc. Excerpt from clojure.org/cheatsheet

  14. Tree Representation Two functions : Does this node has some

    child nodes? ⇒  returns a boolean How to retrieve the data structure of the child nodes? => Returns child nodes data structure
  15. Tree Literal in JSON with a map and arrays Ex:

    JSON output of the tree command on my mac
  16. Tree Literal in Clojure with map and arrays

  17. Each node is an array [node-root child1 child2 …] More

    simple, a Tree Literal with an array of arrays tree-seq function in Clojure structure that tree and allows “walking” it with clojure.walk Viz done with github.com/ztellman/rhizome
  18. Tree literal with more complex objects

  19. Graph Representation Given a node Retrieve the list of identifiers

    of the adjacent nodes Two functions Retrieve the list of identifiers of the graph nodes
  20. None
  21. A graphical representation of the graph is often necessary to

    « touch » it Viz done with github.com/ztellman/rhizome
  22. Use Cases Of Data Literals At Design-time

  23. Visual Representation of your problem – Bowling Game

  24. Visual Representation of your problem – Chess Game

  25. Visual Representation of your problem – Sudoku Game

  26. Data Literals on steroids: Clojure EDN (Extended Data Notation) The

    Tagged Elements of Clojure separate the literal representation from the memory representation: #tag "valueString"! Instant UUID
  27. Object defined by a thread of continuity and a particular

    identity Ex : The bank account n° 30003 02367 000593287642 13, The credit card 8644 7023 0531 105 The EventBrite ticket for SoftShake Domain-Driven Design Immutable object that represents a concept whose equality is based on all the attributes and has no identity Ex : The 42 EUR amount The UUID f4fbe22a-9d53-4359- adbb-57f7d21ed549 The instant 21 october 2015 at 16:29 The distance 53 meters Value Object Entity
  28. Tagged Data Literals are perfect for Value Object For each

    Value Object, add a literal representation as a « String » with valueOf(String)and toString() ! methods!
  29. What’s a state? State is « summarized » with a

    past participle that has a static and stable meaning (order paid, user connected, etc.) State influence the object behavior State implies that some Data Transformations occured during a state transition, triggered by an event State is associated with an identity, then together they form an entity, that represents the different states and transitions along the time A state is the relation between an entity and the values that she owns at the t instant; That state can change
  30. State, Identity and Function State Identity State 1 State 2

    State 3 Function Function Event triggers Event triggers •  value 1a •  value 1b •  value 1c •  value 1a •  value 1b •  value 1c •  value 2a •  value 2b •  value 1a •  value 1b •  value 1c •  value 2a •  value 2b •  value 3a Publish Internal Event Publish Internal Event ... Time
  31. Exanple with the Order entity Identity : Order N° 123456

    Checkout Paid Shipped pay ship Payment details from Payment Gateway received Carrier data q  Customer Data q  Product Lines (product + qty) q  Shipping Data States checkout q  Customer Data q  Product Lines (product + qty) q  Shipping Data q  Payment Data q  Customer Data q  Product Lines (product + qty) q  Shipping Data q  Payment Data q  Carrier Data Front website form submitted Order submitted event Order paid event Order shipped event Time
  32. Les Data Literals sont également parfait pour visualiser les transitions

    d’états d’une Entity Identity State 1 State 2 State 3 Function Function •  value 1a •  value 1b •  value 1c •  value 1a •  value 1b •  value 1c •  value 2a •  value 2b •  value 1a •  value 1b •  value 1c •  value 2a •  value 2b •  value 3a ... Time States
  33. State transitions seen as functions Each « Snapshot » of

    the entity state at a particular instant can be displayed with a data literal A diff allows to visualize the differences between the prevState and the nextState provided by the function function(input): output! function(prevState, event): nextState!
  34. setShippingMethod! increaseItemQty! Easy Diff

  35. OOP done right with Clojure Identity State as values Function

    Method = function applied to an entity’s state, producing a new state (eventually) Entity = Identity + State Protocol = Interface = set of methods
  36. During Testing and Debugging Use Cases Of Data Literals

  37. TDD

  38. Testing BDD In BDD scenarios (Behavior-Driven Development) the data can

    be expressed in a tabular form following the Gherkin grammar Some framework allows to input a Data Literal directly: cf framework BDD Spexec See speakerdeck.com/jgrodziski/anatomy-of-a-bdd-execution-library-in-clojure
  39. Example of a BDD scenario with tabular data

  40. Debugging Tools.trace is a clojure library that instrumentalize functions or

    a whole namespace and log functions input and output Combined with aprint (Awesome Print) it does miracles…almost no need of a debugger
  41. Data Integration Use Cases Of Data Literals

  42. Data Flow

  43. Principles for Functions that fit Data Flows Pures Functions Immutable

    Data Structures seen as Data Literals Composables Steps (aka. Functions) Isolated Side Effects
  44. Pure Function Always return the same result given the same

    arguments whatever the number of invocation (idempotence) Lead to no observable side effect (safe) Referential Transparency: Time doesn’t affect the result of a function, hence the function is transparent regarding time Allows « memoization », composition increase the testability
  45. Immutable Data Structures The functional approach foster functions consuming values

    and returning new values The initial data structure is never modified Clojure provides an efficient implementation with Structural Sharing of Persistent Data Structures (values are shared between the old and new version of the structure)
  46. Composable Functions Function 1 Function 2 Function 3 state t0

    State t2 State t1 State t3
  47. Isolated Side Effects Separate the data processing logic from the

    side effect (persistence, messaging, etc.) Place the non-pure functions at the beginning or end of the chain
  48. Pure Functions or not? Pure Functions without side effects (Safe

    and Idempotent) •  Data Transformations •  Data structure shape (ex : map -> list) •  Values (transcoding) •  Creation of new data (ex: from computation) •  Decision : authorisation, result Non-pure Functions Impures with side effects (Non Safe or Non Idempotent) •  Safe, Non Idempotent : Data enrichment from a « mutable » external data source (ex : read from a database) •  Non Safe, Non Idempotent : Persistence (insert, update), Messaging
  49. Data Pipeline Outside World Outside World Convert from External Repr.

    To Internal Enrich from external DataSource Validate Transform Transform Transform Operate (Persistence, Messaging) Pures Functions Get Data Non-pures Functions Non-pures Functions The non-pures functions are pushed to both ends of the pipeline
  50. Domain Specific Language (DSL) Use Cases Of Data Literals

  51. Domain-Specific Language A DSL with a data structure benefits from

    the powerful data manipulation functions of Clojure SQL HTML
  52. Logs Use Cases Of Data Literals

  53. Logs with Data Literals Store the contextual data of logs

    in a form that can be processed easily is a recommendation from the visualization tools (Splunk, Logstash)
  54. Logs HTTP Requests with Data Literals Ring is the idiomatic

    library of Clojure for the web: each request and response are represented by a map processed by a handler
  55. It’s all well and good that logs but… Can I

    replay a user interaction?
  56. Logs HTTP Requests with Data Literals and the method invocation

    A simple eval allows to replay the whole set of HTTP requests
  57. Clojure is an Homoiconic language Every Clojure clojure is a

    data structure of type List of the form: (verb a b)! Clojure is itself a Data Literal ! Allows very powerful meta-programming « Code is Data, Data is Code » from homo - the same - and icon - representation -
  58. Code is Data Vector List Symbols List

  59. Food for Thoughts about Event Sourcing Event Sourcing means storing

    all the state transitions of an application like a series of events Replay is almost built-in … but is strongly dependant on the source code version that process the event at a particular momentnt t Can we store the source code which process an event besides the even itself?
  60. Food for Thoughts about Event Sourcing Storing data in a

    textual or literal form is the most lasting one It’s easily readable both by a human AND a machine
  61. And Feedback? A REPL or a notebook enables direct interaction

    with Data Literals And moreover enter « in the Zone », Be in the « Flow » where productivity is at its high cf Mihály Csíkszentmihályi
  62. Takeaways Make your data VISIBLE! Touch it! Data express your

    DOMAIN Your Data Literals are your clay, your functions are your hands that shape it Make your functions composables Isolate Side-effects
  63. « An algorithm must be seen to be believed »

    Donal Knuth « Source Code is static, Behavior is dynamic: Good programs make it easy to reason about their Behavior » Barbara Liskov
  64. « Give me exemples » « Give me something concrete

    it’s my job to abstract it! » « Premature Abstraction is the Root of all Evil » « Give me a REPL! »
  65. Thank you! Questions? Comments? @jgrodziski www.grodziski.com Get the slides: speakerdeck.com/jgrodziski/why-data-literals-matters-en

  66. Bowling Game

  67. TDD Bowling Game Domain : Bowling The game consists of

    10 frames as shown above. In each frame the player has two opportunities to knock down 10 pins. The score for the frame is the total number of pins knocked down, plus bonuses for strikes and spares. A spare is when the player knocks down all 10 pins in two tries. The bonus for that frame is the number of pins knocked down by the next roll. ‘/’ denote a spare in the score sheet. A strike is when the player knocks down all 10 pins on his first try. The bonus for that frame is the value of the next two balls rolled. ‘X’ denote a striker in the score sheet. In the tenth frame a player who rolls a spare or strike is allowed to roll the extra balls to complete the frame (so 3 balls can be rolled in tenth frame).
  68. Code Kata : Number theory 2, the return Now, let’s

    get back to the « number theory » stories for which we will rewrite but in a TDD manner. Then we’ll compare the two solution.