Cognician's new architecture with Clojure and Datomic

Our new architecture with Clojure and Datomic Robert Stuttaford Notes
for this talk: http://goo.gl/9dDnq Friday 19 April 13

Notes: http://goo.gl/9dDnq Before we begin, a disclaimer... • Launched the
rebuild mid Jan 2013. • Not running at scale, yet; 1000 users. • You’re about to see our strategy for scale. We think it’s a good one :-) • I’d like to do a follow-up at ScaleConf 2014 and share how things are going! Friday 19 April 13

Notes: http://goo.gl/9dDnq About our startup • Started Cognician with Barry
and Patrick Kayton in early 2010 • Proved the concept over 18 months with good old LAMP and Adobe Flex/AIR • Started V2 app rebuild with Google Closure Javascript late 2011 Friday 19 April 13

Notes: http://goo.gl/9dDnq So, why Clojure? • In April 2012, after
several months and 11,000 lines of (soul-crushingly verbose) Javascript development, discovered Clojure • Rewrote the model portion of this new app in two weeks and 1500 lines of Clojure • Evaluated it for back end, and decided to go all-in! Friday 19 April 13

Notes: http://goo.gl/9dDnq What is Cognician? Friday 19 April 13

Notes: http://goo.gl/9dDnq Friday 19 April 13

Notes: http://goo.gl/9dDnq Who uses Cognician? Friday 19 April 13

Notes: http://goo.gl/9dDnq So. Clojure. Friday 19 April 13

Notes: http://goo.gl/9dDnq Clojure •Modern Lisp dialect •General purpose functional language
•Immutable data structures •JVM, Javascript, CLI, and more Friday 19 April 13

Notes: http://goo.gl/9dDnq Hello ScaleConf! (defn handler [request] {:status 200 :headers
{"Content-Type" "text/html"} :body (html [:div [:h1 "Hello ScaleConf! You said:" [:pre (:body request)]]])}) (run-jetty handler {:port 3000}) Friday 19 April 13

Notes: http://goo.gl/9dDnq Our Architecture Friday 19 April 13

Notes: http://goo.gl/9dDnq Cognician app SERVER SIDE Web nodes (2) AWS
ELB AWS S3 Datomic Transactor Datomic Storage (Postgres) Worker nodes (1) Our code Self-hosted Services Memcached RabbitMQ Managed Services CLIENT SIDE Cognician’s Architecture Friday 19 April 13

Notes: http://goo.gl/9dDnq Datomic Friday 19 April 13

Notes: http://goo.gl/9dDnq Datomic is a database of flexible, time-based facts,
supporting queries and joins, with elastic scalability, and ACID transactions. Friday 19 April 13

Notes: http://goo.gl/9dDnq How Datomic models data • Entity-attribute-value tuples, or
‘datoms’ • Entities are really just IDs • Any mix of attributes can be put on any entity • Schema defines what attributes can do • Entity relations are just a type of attribute Friday 19 April 13

Notes: http://goo.gl/9dDnq Schema definition {:db/doc "User email address" :db/ident :user/email
:db/valueType :db.type/string :db/cardinality :db.cardinality/one :db/unique :db.unique/value :db/index true} {:db/doc "User group references" :db/ident :user/groups :db/valueType :db.type/ref :db/cardinality :db.cardinality/many} Friday 19 April 13

Notes: http://goo.gl/9dDnq Robert likes beer Friday 19 April 13

Notes: http://goo.gl/9dDnq Robert likes beer today Friday 19 April 13

Notes: http://goo.gl/9dDnq How Datomic models time • Novelty enters as
transactions, of • Assertions and retractions of datoms • Every datom linked to its transaction • Every transaction is timestamped • So, it’s actually EAVT, a.k.a. facts! Friday 19 April 13

Notes: http://goo.gl/9dDnq Query with Datalog ; find all entity ids
that have an email address [:find ?entity :where [?entity :user/email]] Friday 19 April 13

that have an email address [:find ?entity :where [?entity :user/email]] ; find a user id and password for a given email address [:find ?user ?password :where [?user :user/email "[email protected]"] [?user :user/password ?password]] Friday 19 April 13

that have an email address [:find ?entity :where [?entity :user/email]] ; find a user id and password for a given email address [:find ?user ?password :where [?user :user/email "[email protected]"] [?user :user/password ?password]] ; find all the users in all the groups I belong to [:find ?user :in $ ?me [?me :user/groups ?group] [?user :user/groups ?group]] Friday 19 April 13

Notes: http://goo.gl/9dDnq Datomic’s Architecture Friday 19 April 13

Notes: http://goo.gl/9dDnq Cognician app SERVER SIDE Web nodes (2) AWS
ELB AWS S3 Datomic Transactor Datomic Storage (Postgres) Worker nodes (1) Our code Self-hosted Services Memcached RabbitMQ Managed Services CLIENT SIDE Cognician’s Architecture Friday 19 April 13

Notes: http://goo.gl/9dDnq In Summary • Event-sourcing and immutable data are
a great match • Clojure code is concise and easier to reason about (and has a lot of parenthesis) • Datomic cuts out a lot of the hassle in building scalable, distributed systems Friday 19 April 13

Notes: http://goo.gl/9dDnq One more thing! Friday 19 April 13

Notes: http://goo.gl/9dDnq Cape Town Clojure User Group •Next meet: 18
May 2013 •Venue: Codebridge in Claremont •Intro to Clojure session •Talks on Datomic, ClojureScript, etc •Details in the notes: http://goo.gl/9dDnq Friday 19 April 13

Notes: http://goo.gl/9dDnq Thank you! Friday 19 April 13

Cognician's new architecture with Clojure and D...

Cognician's new architecture with Clojure and Datomic

More Decks by Robert Stuttaford

Other Decks in Programming

Featured

Transcript