Why Data Literals Matters?

Slides for a talk about Data Literals in programming languages with a focus on Clojure and the benefits
English Version

Transcript

1. Why Data Literals Matters? Jérémie Grodziski - @jgrodziski
2. At the Beginning of every Project I look for Two Things
3. Some Examples of Concepts from the Domain How can I obtain Immediate Feedback?
4. Some Examples of Concepts from the Domain Concrete and Real Visible and Understandable With Data !
5. How can I obtain Immediate Feedback? From the Software I build? From my Assumptions?
6. Why the Success of JSON? Concrete and Real Visible and Understandable Data! WYSIWYG JS is ubiquituous with browser
7. Usable directly or with a simple JSON.parse()
8. Where do you usually see your Data? Stdout? Debugger? Databases? Text Files (CSV, XML, JSON)? When you write code? rarely...
9. Data Literal « A notation for representing a fixed value in source code » Available in some languages for « composite » types Array, associative structure (map, vector, object, etc.) Available in every languages for « scalar » types Integer, Float, Boolean, String, Character
10. Some « Scalars » String Boolean Numbers 1 2 3 4.5 Keyword :key1 :key2 Symbol this that λ
11. Some Data Structures List Vector Map Set
12. …and lot of functions Alan J. Perlis, developed Algol language, 1st recipient of the Turing Award For other « Perlism »: http://www.cs.yale.edu/quotes.html « It is better to have 100 functions operate on 1 data structure than 10 functions on 10 data structures »
13. Lot of functions Etc. Excerpt from clojure.org/cheatsheet
14. Tree Representation Two functions : Does this node has some child nodes? ⇒  returns a boolean How to retrieve the data structure of the child nodes? => Returns child nodes data structure
15. Tree Literal in JSON with a map and arrays Ex: JSON output of the tree command on my mac
16. Tree Literal in Clojure with map and arrays
17. Each node is an array [node-root child1 child2 …] More simple, a Tree Literal with an array of arrays tree-seq function in Clojure structure that tree and allows “walking” it with clojure.walk Viz done with github.com/ztellman/rhizome
18. Tree literal with more complex objects
19. Graph Representation Given a node Retrieve the list of identifiers of the adjacent nodes Two functions Retrieve the list of identifiers of the graph nodes
21. A graphical representation of the graph is often necessary to « touch » it Viz done with github.com/ztellman/rhizome
22. Use Cases Of Data Literals At Design-time
23. Visual Representation of your problem – Bowling Game
24. Visual Representation of your problem – Chess Game
25. Visual Representation of your problem – Sudoku Game
26. Data Literals on steroids: Clojure EDN (Extended Data Notation) The Tagged Elements of Clojure separate the literal representation from the memory representation: #tag "valueString"! Instant UUID
27. Object defined by a thread of continuity and a particular identity Ex : The bank account n° 30003 02367 000593287642 13, The credit card 8644 7023 0531 105 The EventBrite ticket for SoftShake Domain-Driven Design Immutable object that represents a concept whose equality is based on all the attributes and has no identity Ex : The 42 EUR amount The UUID f4fbe22a-9d53-4359- adbb-57f7d21ed549 The instant 21 october 2015 at 16:29 The distance 53 meters Value Object Entity
28. Tagged Data Literals are perfect for Value Object For each Value Object, add a literal representation as a « String » with valueOf(String)and toString() ! methods!
29. What’s a state? State is « summarized » with a past participle that has a static and stable meaning (order paid, user connected, etc.) State influence the object behavior State implies that some Data Transformations occured during a state transition, triggered by an event State is associated with an identity, then together they form an entity, that represents the different states and transitions along the time A state is the relation between an entity and the values that she owns at the t instant; That state can change
30. State, Identity and Function State Identity State 1 State 2 State 3 Function Function Event triggers Event triggers •  value 1a •  value 1b •  value 1c •  value 1a •  value 1b •  value 1c •  value 2a •  value 2b •  value 1a •  value 1b •  value 1c •  value 2a •  value 2b •  value 3a Publish Internal Event Publish Internal Event ... Time
31. Exanple with the Order entity Identity : Order N° 123456 Checkout Paid Shipped pay ship Payment details from Payment Gateway received Carrier data q  Customer Data q  Product Lines (product + qty) q  Shipping Data States checkout q  Customer Data q  Product Lines (product + qty) q  Shipping Data q  Payment Data q  Customer Data q  Product Lines (product + qty) q  Shipping Data q  Payment Data q  Carrier Data Front website form submitted Order submitted event Order paid event Order shipped event Time
32. Les Data Literals sont également parfait pour visualiser les transitions d’états d’une Entity Identity State 1 State 2 State 3 Function Function •  value 1a •  value 1b •  value 1c •  value 1a •  value 1b •  value 1c •  value 2a •  value 2b •  value 1a •  value 1b •  value 1c •  value 2a •  value 2b •  value 3a ... Time States
33. State transitions seen as functions Each « Snapshot » of the entity state at a particular instant can be displayed with a data literal A diff allows to visualize the differences between the prevState and the nextState provided by the function function(input): output! function(prevState, event): nextState!
34. setShippingMethod! increaseItemQty! Easy Diff
35. OOP done right with Clojure Identity State as values Function Method = function applied to an entity’s state, producing a new state (eventually) Entity = Identity + State Protocol = Interface = set of methods
36. During Testing and Debugging Use Cases Of Data Literals
37. TDD
38. Testing BDD In BDD scenarios (Behavior-Driven Development) the data can be expressed in a tabular form following the Gherkin grammar Some framework allows to input a Data Literal directly: cf framework BDD Spexec See speakerdeck.com/jgrodziski/anatomy-of-a-bdd-execution-library-in-clojure
39. Example of a BDD scenario with tabular data
40. Debugging Tools.trace is a clojure library that instrumentalize functions or a whole namespace and log functions input and output Combined with aprint (Awesome Print) it does miracles…almost no need of a debugger
41. Data Integration Use Cases Of Data Literals
42. Data Flow
43. Principles for Functions that fit Data Flows Pures Functions Immutable Data Structures seen as Data Literals Composables Steps (aka. Functions) Isolated Side Effects
44. Pure Function Always return the same result given the same arguments whatever the number of invocation (idempotence) Lead to no observable side effect (safe) Referential Transparency: Time doesn’t affect the result of a function, hence the function is transparent regarding time Allows « memoization », composition increase the testability
45. Immutable Data Structures The functional approach foster functions consuming values and returning new values The initial data structure is never modified Clojure provides an efficient implementation with Structural Sharing of Persistent Data Structures (values are shared between the old and new version of the structure)
46. Composable Functions Function 1 Function 2 Function 3 state t0 State t2 State t1 State t3
47. Isolated Side Effects Separate the data processing logic from the side effect (persistence, messaging, etc.) Place the non-pure functions at the beginning or end of the chain
48. Pure Functions or not? Pure Functions without side effects (Safe and Idempotent) •  Data Transformations •  Data structure shape (ex : map -> list) •  Values (transcoding) •  Creation of new data (ex: from computation) •  Decision : authorisation, result Non-pure Functions Impures with side effects (Non Safe or Non Idempotent) •  Safe, Non Idempotent : Data enrichment from a « mutable » external data source (ex : read from a database) •  Non Safe, Non Idempotent : Persistence (insert, update), Messaging
49. Data Pipeline Outside World Outside World Convert from External Repr. To Internal Enrich from external DataSource Validate Transform Transform Transform Operate (Persistence, Messaging) Pures Functions Get Data Non-pures Functions Non-pures Functions The non-pures functions are pushed to both ends of the pipeline
50. Domain Specific Language (DSL) Use Cases Of Data Literals
51. Domain-Specific Language A DSL with a data structure benefits from the powerful data manipulation functions of Clojure SQL HTML
52. Logs Use Cases Of Data Literals
53. Logs with Data Literals Store the contextual data of logs in a form that can be processed easily is a recommendation from the visualization tools (Splunk, Logstash)
54. Logs HTTP Requests with Data Literals Ring is the idiomatic library of Clojure for the web: each request and response are represented by a map processed by a handler
55. It’s all well and good that logs but… Can I replay a user interaction?
56. Logs HTTP Requests with Data Literals and the method invocation A simple eval allows to replay the whole set of HTTP requests
57. Clojure is an Homoiconic language Every Clojure clojure is a data structure of type List of the form: (verb a b)! Clojure is itself a Data Literal ! Allows very powerful meta-programming « Code is Data, Data is Code » from homo - the same - and icon - representation -
58. Code is Data Vector List Symbols List
59. Food for Thoughts about Event Sourcing Event Sourcing means storing all the state transitions of an application like a series of events Replay is almost built-in … but is strongly dependant on the source code version that process the event at a particular momentnt t Can we store the source code which process an event besides the even itself?
60. Food for Thoughts about Event Sourcing Storing data in a textual or literal form is the most lasting one It’s easily readable both by a human AND a machine
61. And Feedback? A REPL or a notebook enables direct interaction with Data Literals And moreover enter « in the Zone », Be in the « Flow » where productivity is at its high cf Mihály Csíkszentmihályi
62. Takeaways Make your data VISIBLE! Touch it! Data express your DOMAIN Your Data Literals are your clay, your functions are your hands that shape it Make your functions composables Isolate Side-effects
63. « An algorithm must be seen to be believed » Donal Knuth « Source Code is static, Behavior is dynamic: Good programs make it easy to reason about their Behavior » Barbara Liskov
64. « Donnez moi des exemples » « Donnez moi du concret je me charge de l’abstraire ! » « Premature Abstraction is the Root of all Evil » « Donnez moi une REPL ! »
65. Merci ! Des questions ? Des remarques ? @jgrodziski www.grodziski.com Get the slides: speakerdeck.com/jgrodziski/why-data-literals-matters
66. Bowling Game
67. TDD Bowling Game Domain : Bowling The game consists of 10 frames as shown above. In each frame the player has two opportunities to knock down 10 pins. The score for the frame is the total number of pins knocked down, plus bonuses for strikes and spares. A spare is when the player knocks down all 10 pins in two tries. The bonus for that frame is the number of pins knocked down by the next roll. ‘/’ denote a spare in the score sheet. A strike is when the player knocks down all 10 pins on his first try. The bonus for that frame is the value of the next two balls rolled. ‘X’ denote a striker in the score sheet. In the tenth frame a player who rolls a spare or strike is allowed to roll the extra balls to complete the frame (so 3 balls can be rolled in tenth frame).
68. Code Kata : Number theory 2, the return Now, let’s get back to the « number theory » stories for which we will rewrite but in a TDD manner. Then we’ll compare the two solution.