Datomic Spotlight

1. DATOMIC @amitayh THE FUNCTIONAL DATABASE
2. FULL DISCLOSURE ⚠
3. ABOUT DATOMIC • Developed by Cognitect • Initial release 2012 • Designed by Rich Hickey (author of the Clojure programming language) • Used by Facebook, Netflix, more…
4. FACTS ✅
5. FACTS • “Abraham Lincoln is the president of the United States”
6. FACTS • “Abraham Lincoln is the president of the United States” • “Abraham Lincoln was elected president of the United States on March 4, 1861”
7. FACTS • Abraham Lincoln • Position • President • March 4, 1861 Entity Attribute Value Timestamp
8. FACT = DATOM Entity Attribute Value Tx Operation
9. FACT = DATOM Entity Attribute Value Tx Operation 1033 :first-name "Abraham" 260 add 1033 :last-name "Lincoln" 260 add 1033 :position "Lawyer" 277 add
10. FACT = DATOM Entity Attribute Value Tx Operation 1033 :first-name "Abraham" 260 add 1033 :last-name "Lincoln" 260 add 1033 :position "Lawyer" 277 add 1033 :position "President" 522 add
11. FACT = DATOM Entity Attribute Value Tx Operation 1033 :first-name "Abraham" 260 add 1033 :last-name "Lincoln" 260 add 1033 :position "Lawyer" 277 add 1033 :position "President" 522 add 1033 :position "President" 881 retract
12. FACT BASED MODEL Facts Facts Facts Facts Facts Facts Facts Facts Time Facts Facts Time
13. GIT Facts Facts Facts Facts Objects Facts Facts Objects Time Facts Objects Time
14. A DATABASE • A collection of datoms • At a specific point in time • An immutable value
15. DATABASE AS A VALUE • Same as 42 is a value • Safe to share, easy to reason about • Functions that take a database as an argument, or return a database
16. TIME BUILT IN • Get the database value as of, or since, a point in time • See how the database would have looked like as if certain transactions took place • Reactive transaction reports
17. QUERIES
18. DATALOG • Equivalent to relational model + recursion • Declarative, expressive and powerful • Pattern matching style • No more string concatenation!
19. EXAMPLE DATABASE Entity Attribute Value 42 :email john@doe.com 43 :email jane@doe.com 42 :orders 107 42 :orders 141
20. DATA PATTERN Constrains the results returned, binds variables: [?customer :email ?email] Attribute Value Entity
21. DATA PATTERN Constrains the results returned, binds variables: [?customer :email ?email] Constant Variable Variable
22. Entity Attribute Value 42 :email john@doe.com 43 :email jane@doe.com 42 :orders 107 42 :orders 141 “Find all customers with emails” [?customer :email ?email]
23. Entity Attribute Value 42 :email john@doe.com 43 :email jane@doe.com 42 :orders 107 42 :orders 141 “Find a particular customer’s email” [42 :email ?email]
24. Entity Attribute Value 42 :email john@doe.com 43 :email jane@doe.com 42 :orders 107 42 :orders 141 “What attributes does 42 have?” [42 ?attribute]
25. Entity Attribute Value 42 :email john@doe.com 43 :email jane@doe.com 42 :orders 107 42 :orders 141 “What attributes and values does 42 have?” [42 ?attribute ?value]
26. WHERE CLAUSE [:find ?customer
 :where [?customer :email]] Data pattern
27. FIND CLAUSE [:find ?customer
 :where [?customer :email]]
28. IMPLICIT JOIN [:find ?customer
 :where [?customer :email] [?customer :orders]] “Find all the customers who have placed orders”
29. PREDICATES [:find ?item
 :where [?item :item/price ?price] [(< 50 ?price)]] “Find the expensive items”
30. CALLING FUNCTIONS [:find ?customer ?product
 :where [?customer :ship-address ?addr] [?adde :zip ?zip] [?product :product/weight ?weight] [?product :product/price ?price] [(Shipping/estimate ?zip ?weight) ?ship-cost] [(<= ?price ?ship-cost)]] “Find me the customer/product combinations where the shipping cost dominates the product cost”
31. ARCHITECTURE
32. DATABASE ROLES • Queries • Transactions • Consistency • Storage
33. BREAK DOWN
34. DATOMIC COMPONENTS Peer library Transactor Storage
35. PEER LIBRARY Transactor Storage Peer library Your app • Embedded in your app • Executed queries locally
36. Peer library Your app cache PEER LIBRARY Transactor Storage • Reads data from storage • Caches locally
37. Peer library Your app cache Peer library Your app cache Peer library Your app cache SCALE HORIZONTALLY Transactor Storage
38. TRANSACTOR Transactor Storage Peer library Your app • Standalone service • Scales vertically
39. Transactor TRANSACTOR Transactor Storage Peer library Your app • Standalone service • Scales vertically • Hot standby for failover
40. Transactor TRANSACTOR Storage Peer library Your app • Coordinates writes • Guarantees ACID transactions (isolation level “serializable”)
41. Storage Transactor TRANSACTOR Peer library Your app • Writes transaction log to storage • Generates indices
42. Peer library Your app cache Peer library Your app cache Peer library Your app cache Storage Transactor TRANSACTOR • Broadcasts live updates
43. STORAGE Transactor Storage Peer library Your app • Provided as a service • Many different backends
44. LOCAL STORAGE Transactor Storage Peer library Your app • Memory • Filesystem • Great for testing!
45. NEARBY STORAGE Transactor Storage Peer library Your app • SQL database (any JDBC)
46. DISTRIBUTED STORAGE ☁ Transactor Storage Peer library Your app • DynamoDB • Riak • CouchBase • Cassandra • …
47. Peer library Your app cache Peer library Your app cache Peer library Your app cache SHARED MEMCACHED Transactor Storage Memcached
48. EVENT SOURCING? • Datomic is lower level - facts vs events • Annotate transactions! • Querying already built in • Listen on transaction queue to build reactive systems
49. Q&A
50. RESOURCES • http://www.datomic.com/ • Intro to Datomic by Rich Hickey -
 http://wix.to/b8CQABU • The Value of Values by Rich Hickey -
 http://wix.to/D8CRABU • Datomic Datalog - http://wix.to/cMCQABU