DataScript for Web Development

Slide 1

Slide 1 text

Slide 2

Slide 2 text

Nikita Prokopov tonsky.me Novosibirsk, Russia

Slide 3

Slide 3 text

No content

Slide 4

Slide 4 text

Server-side traditionally benefits from relational storage + queries Rationale

Slide 5

Slide 5 text

Why web developers should limit themselves?

Slide 6

Slide 6 text

Datomic model feels right and was within my reach

Slide 7

Slide 7 text

DataScript is a post-modern DB: someone else’s good idea redone on a more primitive level

Slide 8

Slide 8 text

triple store What’s DataScript?

Slide 9

Slide 9 text

Database as an immutable value

Slide 10

Slide 10 text

Completely in-memory

Slide 11

Slide 11 text

Datalog queries over it

Slide 12

Slide 12 text

Tries to mimic Datomic API

Slide 13

Slide 13 text

Written in ClojureScript, JS bindings available

Slide 14

Slide 14 text

(def conn (d/create-conn {:aka {:db/cardinality :db.cardinality/many }})] (d/transact! conn [ {:db/id 1 :name “Ivan” :aka [“The Terrible”, “Durak”]}]) (d/q ‘[:find ?name :where [?e :aka “Durak”] [?e :name ?name]] @conn)

Slide 15

Slide 15 text

(defn create-conn [schema] (atom (empty-db schema) :meta { :listeners (atom {}) })) How it works?

Slide 16

Slide 16 text

(defrecord DB [ schema eavt aevt avet max-eid max-tx ])

Slide 17

Slide 17 text

(defn with [db datoms] (→ db (update-in [:eavt] into datoms) (update-in [:aevt] into datoms)

Slide 18

Slide 18 text

(defn transact [conn datoms] (swap! conn with datom))

Slide 19

Slide 19 text

In-house implementation of B+ tree (sorted set) BTSet

Slide 20

Slide 20 text

Performance comparable with sorted-set: ~ 50% slower conj ~200% faster iterate

Slide 21

Slide 21 text

Binary search lookups, fast range scans, reverse iteration, fast first-time creation

Slide 22

Slide 22 text

ClojureScript, but heavy use of JS arrays and APIs

Slide 23

Slide 23 text

700 loc btset 550 loc query engine 1700 loc total 1200 loc tests DataScript is lightweight

Slide 24

Slide 24 text

Unfortunate to be associated with the word “database”: no networking, query over memory,

Slide 25

Slide 25 text

no log, no history & compound indexes, no external segments, constant space operation

Slide 26

Slide 26 text

DataScript is more of a persistent data structure. It’s just three sorted sets, literally.

Slide 27

Slide 27 text

Every app has an ad-hoc state Put everything in a database How to use it

Slide 28

Slide 28 text

Non-trivial SPA has complex state KV stores do not cut it

Slide 29

Slide 29 text

No inherent hierarchy, very natural for any data: sparse, irregular, hierarchical, graph

Slide 30

Slide 30 text

Faster data retrieval from big datasets: indexes + hash joins

Slide 31

Slide 31 text

Uniform solution for storage encourages decoupling and reusable components: server sync, undo/redo, local caching, audit

Slide 32

Slide 32 text

Datoms, DB and transactions have standard way to be serialized No need to invent format for delta exchange

Slide 33

Slide 33 text

Immutability + db as a value + transactions = always consistent render

Slide 34

Slide 34 text

Encourages to write decoupled, pure renders Good fit for React and Flux

Slide 35

Slide 35 text

Example architecture: CatChat github/tonsky/datascript-chat

Slide 36

Slide 36 text

No content

Slide 37

Slide 37 text

No content

Slide 38

Slide 38 text

Raw react render, pure, decoupled

Slide 39

Slide 39 text

Re-render triggered by database mutations, always top-down, full re-render

Slide 40

Slide 40 text

“Server” sync, db cleanup all decoupled from render code

Slide 41

Slide 41 text

Operates in a bound space via cleanup

Slide 42

Slide 42 text

Second DB used to emulate server

Slide 43

Slide 43 text

Example architecture: Acha-Acha acha-acha.co

Slide 44

Slide 44 text

No content

Slide 45

Slide 45 text

Whole DB prefetch: no server fetch on any navigation, all queries and aggregations hap- pen on a client

Slide 46

Slide 46 text

Always up-to-date server sync: initial dump + deltas via websock- et (tricky to do correctly)

Slide 47

Slide 47 text

Non-hierarchical layout: user page, repo page, main page

Slide 48

Slide 48 text

14K datoms in 100..150ms

Slide 49

Slide 49 text

Example architecture: Menu [WIP] github/tonsky/datascript-menu

Slide 50

Slide 50 text

No content

Slide 51

Slide 51 text

Component can listen for a specific [e a v] pattern

Slide 52

Slide 52 text

Optimized for huge amount of listeners, e.g. every person component ever can subscribe to [(:db/id person) :person/name _]

Slide 53

Slide 53 text

Similar to Om ref-cursors Have to figure out how to play nicely with top-down rendering

Slide 54

Slide 54 text

Considered alpha stage Project status

Slide 55

Slide 55 text

No docs, must know/learn Datomic Extensive acceptance tests suite helps

Slide 56

Slide 56 text

Almost no validation, silent fails/ unexpected results instead

Slide 57

Slide 57 text

Occasional breaking changes (not significant though) If it’s in Datomic and DataScript already, it probably won’t change

Slide 58

Slide 58 text

Cover all Datomic APIs components lookup refs find specifications pull API Future plans