Why Data Literals Matters? Jérémie Grodziski - @jgrodziski 

At the Beginning of every Project I look for Two Things 

Some Examples of Concepts from the Domain How can I obtain Immediate Feedback? 

Some Examples of Concepts from the Domain Concrete and Real Visible and Understandable With Data ! 

How can I obtain Immediate Feedback? From the Software I build? From my Assumptions? 

Why the Success of JSON? Concrete and Real Visible and Understandable Data! WYSIWYG JS is ubiquituous with browser 

Usable directly or with a simple JSON.parse() 

Where do you usually see your Data? Stdout? Debugger? Databases? Text Files (CSV, XML, JSON)? When you write code? rarely... 

Data Literal « A notation for representing a fixed value in source code » Available in some languages for « composite » types Array, associative structure (map, vector, object, etc.) Available in every languages for « scalar » types Integer, Float, Boolean, String, Character 

Some « Scalars » String Boolean Numbers 1 2 3 4.5 Keyword :key1 :key2 Symbol this that λ 

Some Data Structures List Vector Map Set 

…and lot of functions Alan J. Perlis, developed Algol language, 1st recipient of the Turing Award For other « Perlism »: http://www.cs.yale.edu/quotes.html « It is better to have 100 functions operate on 1 data structure than 10 functions on 10 data structures » 

Lot of functions Etc. Excerpt from clojure.org/cheatsheet 

Tree Representation Two functions : Does this node has some child nodes? ⇒  returns a boolean How to retrieve the data structure of the child nodes? => Returns child nodes data structure 

Tree Literal in JSON with a map and arrays Ex: JSON output of the tree command on my mac 

Tree Literal in Clojure with map and arrays 

Each node is an array [node-root child1 child2 …] More simple, a Tree Literal with an array of arrays tree-seq function in Clojure structure that tree and allows “walking” it with clojure.walk Viz done with github.com/ztellman/rhizome 

Tree literal with more complex objects 

Graph Representation Given a node Retrieve the list of identifiers of the adjacent nodes Two functions Retrieve the list of identifiers of the graph nodes 

No content

A graphical representation of the graph is often necessary to « touch » it Viz done with github.com/ztellman/rhizome 

Use Cases Of Data Literals At Design-time 

Visual Representation of your problem – Bowling Game 

Visual Representation of your problem – Chess Game 

Visual Representation of your problem – Sudoku Game 

Data Literals on steroids: Clojure EDN (Extended Data Notation) The Tagged Elements of Clojure separate the literal representation from the memory representation: #tag "valueString"! Instant UUID 

Object defined by a thread of continuity and a particular identity Ex : The bank account n° 30003 02367 000593287642 13, The credit card 8644 7023 0531 105 The EventBrite ticket for SoftShake Domain-Driven Design Immutable object that represents a concept whose equality is based on all the attributes and has no identity Ex : The 42 EUR amount The UUID f4fbe22a-9d53-4359- adbb-57f7d21ed549 The instant 21 october 2015 at 16:29 The distance 53 meters Value Object Entity 

Tagged Data Literals are perfect for Value Object For each Value Object, add a literal representation as a « String » with valueOf(String)and toString() ! methods! 

What’s a state? State is « summarized » with a past participle that has a static and stable meaning (order paid, user connected, etc.) State influence the object behavior State implies that some Data Transformations occured during a state transition, triggered by an event State is associated with an identity, then together they form an entity, that represents the different states and transitions along the time A state is the relation between an entity and the values that she owns at the t instant; That state can change 

State, Identity and Function State Identity State 1 State 2 State 3 Function Function Event triggers Event triggers •  value 1a •  value 1b •  value 1c •  value 1a •  value 1b •  value 1c •  value 2a •  value 2b •  value 1a •  value 1b •  value 1c •  value 2a •  value 2b •  value 3a Publish Internal Event Publish Internal Event ... Time 

Exanple with the Order entity Identity : Order N° 123456 Checkout Paid Shipped pay ship Payment details from Payment Gateway received Carrier data q  Customer Data q  Product Lines (product + qty) q  Shipping Data States checkout q  Customer Data q  Product Lines (product + qty) q  Shipping Data q  Payment Data q  Customer Data q  Product Lines (product + qty) q  Shipping Data q  Payment Data q  Carrier Data Front website form submitted Order submitted event Order paid event Order shipped event Time 

Les Data Literals sont également parfait pour visualiser les transitions d’états d’une Entity Identity State 1 State 2 State 3 Function Function •  value 1a •  value 1b •  value 1c •  value 1a •  value 1b •  value 1c •  value 2a •  value 2b •  value 1a •  value 1b •  value 1c •  value 2a •  value 2b •  value 3a ... Time States 

State transitions seen as functions Each « Snapshot » of the entity state at a particular instant can be displayed with a data literal A diff allows to visualize the differences between the prevState and the nextState provided by the function function(input): output! function(prevState, event): nextState! 

setShippingMethod! increaseItemQty! Easy Diff 

OOP done right with Clojure Identity State as values Function Method = function applied to an entity’s state, producing a new state (eventually) Entity = Identity + State Protocol = Interface = set of methods 

During Testing and Debugging Use Cases Of Data Literals 

TDD 

Testing BDD In BDD scenarios (Behavior-Driven Development) the data can be expressed in a tabular form following the Gherkin grammar Some framework allows to input a Data Literal directly: cf framework BDD Spexec See speakerdeck.com/jgrodziski/anatomy-of-a-bdd-execution-library-in-clojure 

Example of a BDD scenario with tabular data 

Debugging Tools.trace is a clojure library that instrumentalize functions or a whole namespace and log functions input and output Combined with aprint (Awesome Print) it does miracles…almost no need of a debugger 

Data Integration Use Cases Of Data Literals 

Data Flow 

Principles for Functions that fit Data Flows Pures Functions Immutable Data Structures seen as Data Literals Composables Steps (aka. Functions) Isolated Side Effects 

Pure Function Always return the same result given the same arguments whatever the number of invocation (idempotence) Lead to no observable side effect (safe) Referential Transparency: Time doesn’t affect the result of a function, hence the function is transparent regarding time Allows « memoization », composition increase the testability 

Immutable Data Structures The functional approach foster functions consuming values and returning new values The initial data structure is never modified Clojure provides an efficient implementation with Structural Sharing of Persistent Data Structures (values are shared between the old and new version of the structure) 

Composable Functions Function 1 Function 2 Function 3 state t0 State t2 State t1 State t3 

Isolated Side Effects Separate the data processing logic from the side effect (persistence, messaging, etc.) Place the non-pure functions at the beginning or end of the chain 

Pure Functions or not? Pure Functions without side effects (Safe and Idempotent) •  Data Transformations •  Data structure shape (ex : map -> list) •  Values (transcoding) •  Creation of new data (ex: from computation) •  Decision : authorisation, result Non-pure Functions Impures with side effects (Non Safe or Non Idempotent) •  Safe, Non Idempotent : Data enrichment from a « mutable » external data source (ex : read from a database) •  Non Safe, Non Idempotent : Persistence (insert, update), Messaging 

Data Pipeline Outside World Outside World Convert from External Repr. To Internal Enrich from external DataSource Validate Transform Transform Transform Operate (Persistence, Messaging) Pures Functions Get Data Non-pures Functions Non-pures Functions The non-pures functions are pushed to both ends of the pipeline 

Domain Specific Language (DSL) Use Cases Of Data Literals 

Domain-Specific Language A DSL with a data structure benefits from the powerful data manipulation functions of Clojure SQL HTML 

Logs Use Cases Of Data Literals 

Logs with Data Literals Store the contextual data of logs in a form that can be processed easily is a recommendation from the visualization tools (Splunk, Logstash) 

Logs HTTP Requests with Data Literals Ring is the idiomatic library of Clojure for the web: each request and response are represented by a map processed by a handler 

It’s all well and good that logs but… Can I replay a user interaction? 

Logs HTTP Requests with Data Literals and the method invocation A simple eval allows to replay the whole set of HTTP requests 

Clojure is an Homoiconic language Every Clojure clojure is a data structure of type List of the form: (verb a b)! Clojure is itself a Data Literal ! Allows very powerful meta-programming « Code is Data, Data is Code » from homo - the same - and icon - representation - 

Code is Data Vector List Symbols List 

Food for Thoughts about Event Sourcing Event Sourcing means storing all the state transitions of an application like a series of events Replay is almost built-in … but is strongly dependant on the source code version that process the event at a particular momentnt t Can we store the source code which process an event besides the even itself? 

Food for Thoughts about Event Sourcing Storing data in a textual or literal form is the most lasting one It’s easily readable both by a human AND a machine 

And Feedback? A REPL or a notebook enables direct interaction with Data Literals And moreover enter « in the Zone », Be in the « Flow » where productivity is at its high cf Mihály Csíkszentmihályi 

Takeaways Make your data VISIBLE! Touch it! Data express your DOMAIN Your Data Literals are your clay, your functions are your hands that shape it Make your functions composables Isolate Side-effects 

« An algorithm must be seen to be believed » Donal Knuth « Source Code is static, Behavior is dynamic: Good programs make it easy to reason about their Behavior » Barbara Liskov 

« Give me exemples » « Give me something concrete it’s my job to abstract it! » « Premature Abstraction is the Root of all Evil » « Give me a REPL! » 

Thank you! Questions? Comments? @jgrodziski www.grodziski.com Get the slides: speakerdeck.com/jgrodziski/why-data-literals-matters-en 

Bowling Game 

TDD Bowling Game Domain : Bowling The game consists of 10 frames as shown above. In each frame the player has two opportunities to knock down 10 pins. The score for the frame is the total number of pins knocked down, plus bonuses for strikes and spares. A spare is when the player knocks down all 10 pins in two tries. The bonus for that frame is the number of pins knocked down by the next roll. ‘/’ denote a spare in the score sheet. A strike is when the player knocks down all 10 pins on his first try. The bonus for that frame is the value of the next two balls rolled. ‘X’ denote a striker in the score sheet. In the tenth frame a player who rolls a spare or strike is allowed to roll the extra balls to complete the frame (so 3 balls can be rolled in tenth frame). 

Code Kata : Number theory 2, the return Now, let’s get back to the « number theory » stories for which we will rewrite but in a TDD manner. Then we’ll compare the two solution.