How we migrated a complex JavaScript application to ClojureScript step-by-step

Slide 1

Slide 1 text

Migrating a complex JS app to ClojureScript Alex King

Slide 2

Slide 2 text

Alexander James King • github.com/alexanderjamesking • Clojure dev at Exoscale • Clojure(script) / Scala / Java / JS • Previous: JUXT / BBC / HMRC

Slide 3

Slide 3 text

No content

Slide 4

Slide 4 text

• European cloud provider • Data centres in Switzerland, Germany and Austria • CPU, GPU, Object Storage, Elastic IPs, DNS, Run Status • Data Security, GDPR Compliance, Data Privacy

Slide 5

Slide 5 text

No content

Slide 6

Slide 6 text

No content

Slide 7

Slide 7 text

Product Problems Going fast Implementation Lessons learned Conclusion

Slide 8

Slide 8 text

No content

Slide 9

Slide 9 text

No content

Slide 10

Slide 10 text

No content

Slide 11

Slide 11 text

Browser Python API Architecture JS Platform API Gateway

Slide 12

Slide 12 text

Product Problems Going fast Implementation Lessons learned Conclusion

Slide 13

Slide 13 text

#1 Feature Slowdown Features Time

Slide 14

Slide 14 text

#2 Tests UI Integration Unit Manual

Slide 15

Slide 15 text

#3 Tooling

Slide 16

Slide 16 text

No content

Slide 17

Slide 17 text

No content

Slide 18

Slide 18 text

No content

Slide 19

Slide 19 text

No content

Slide 20

Slide 20 text

No content

Slide 21

Slide 21 text

Problems • Velocity • Lack of tests • Tooling • Build tools • Dependencies • Speed of change

Slide 22

Slide 22 text

Product Problems Going fast Implementation Lessons learned Conclusion

Slide 23

Slide 23 text

–Robert Cecil Martin “The higher the quality, the faster you go. The only way to go fast is to go well.”

Slide 24

Slide 24 text

Improving the Quality • Tests at all levels of the pyramid • Fewer dependencies • Small, focused libraries • Better tooling

Slide 25

Slide 25 text

ClojureScript • Functional programming • Immutability • Data Driven • Small, focused libraries • Browser REPL

Slide 26

Slide 26 text

ClojureScript Reitit Integrant re-frame More Dependencies! Reagent

Slide 27

Slide 27 text

“Sometimes, things have to get worse before they can get better”

Slide 28

Slide 28 text

Product Problems Going fast Implementation Lessons learned Conclusion

Slide 29

Slide 29 text

The grand redesign in the sky This time we will get it right!

Slide 30

Slide 30 text

No content

Slide 31

Slide 31 text

No content

Slide 32

Slide 32 text

No content

Slide 33

Slide 33 text

Greenﬁeld vs Brownﬁeld Build a Replacement Not very agile ∴ High risk Moving target ∴ High risk Migrate the existing app Agile ∴ Low risk Complex ∴ Medium risk

Slide 34

Slide 34 text

Brownﬁeld: Migrate the existing app

Slide 35

Slide 35 text

How can we migrate? URL: Server Page reloads (e.g nginx) URL: Browser HTML5 History API Component Lots of JS Interop 1 2 3

Slide 36

Slide 36 text

Clojure Python :option-1 URL: Server JS CLJS

Slide 37

Slide 37 text

Clojure Python :option-1 URL: Server JS CLJS

Slide 38

Slide 38 text

Clojure Python :option-1 URL: Server JS CLJS + Clean codebase - Page reloads required - Multiple apps to manage

Slide 39

Slide 39 text

Python :option-2 URL: Browser JS CLJS A2 Router A1 Router History API

Slide 40

Slide 40 text

Python :option-2 URL: Browser JS CLJS A2 Router Router A1 History API

Slide 41

Slide 41 text

Python :option-2 URL: Browser JS CLJS A2 Router A1 +Single app +No page reloads - Challenging Interop? History API

Slide 42

Slide 42 text

:option-3 Component ClojureScript JS

Slide 43

Slide 43 text

:option-3 Component ClojureScript +Single app +No page reloads +Agile - Challenging Interop - High complexity

Slide 44

Slide 44 text

(and :option-2 :option-3) ClojureScript JS

Slide 45

Slide 45 text

…

Slide 46

Slide 46 text

One URL at a time • Bootstrap both JS and CLJS apps • Routing: History API • Minimal JS <-> CLJS interop • Write UI tests against the JS app

Slide 47

Slide 47 text

Browser Python API Architecture JS CLJS Platform API Gateway

Slide 48

Slide 48 text

UI Tests Browser Python API JS CLJS Mock API Scope

Slide 49

Slide 49 text

UI Tests • Selenium and Chrome Headless • Clojure mock of the API • Independent of implementation • Write tests against JS • Swap JS with CLJS

Slide 50

Slide 50 text

No content

Slide 51

Slide 51 text

Python API CLJS Architecture Controller Router Data Instance IP IP Addresses View Model

Slide 52

Slide 52 text

Python API Initial Release Controller Router Data Instance IP IP Addresses View Model JS API CLJS API Poller

Slide 53

Slide 53 text

Python API Additional Modules Controller Router Instances Data Instance Firewall IP View Model IP Addresses View Model JS API CLJS API Poller

Slide 54

Slide 54 text

Libraries & Frameworks

Slide 55

Slide 55 text

reagent-project / reagent • Deﬁne React components with data (Hiccup-like syntax) • reagent.core/atom for component state • reagent.core/render

Slide 56

Slide 56 text

;; basic component (defn title-bar [] [:div.title-bar [:h1.page-title.ellipsis "New Instance”]]) ;; with local state (defn confirm-delete [callback] (let [confirming? (reagent/atom false) toggle #(swap! confirming? not)] (fn [delete-fn] (if @confirming? [:li [:button {:on-click callback} "Delete"] [:button {:on-click toggle} "Cancel"]] [:li [:button {:on-click toggle} "Delete"]]))))

Slide 57

Slide 57 text

Day8 / re-frame • State management (e.g. Flux/Redux) • Unidirectional data ﬂow DB Subscriptions View Events

Slide 58

Slide 58 text

(ns portal.compute.ssh-keys.model) (rf/reg-event-db ::quick-filter (fn [db [_ value]] (assoc db ::quick-filter value))) (rf/reg-sub ::quick-filter (fn [db _] (not-empty (::quick-filter db)))) (ns portal.compute.ssh-keys.view) (defn quick-filter-names [] [:input {:on-change #(rf/dispatch [::model/quick-filter (.. % -target -value)]) :value @(rf/subscribe [::model/quick-filter]) :type "text"}])

Slide 59

Slide 59 text

weavejester / integrant • Application structure (e.g. Component/Mount) • System as data • Why? • Dependency Injection • Multiple conﬁgurations

Slide 60

Slide 60 text

(ns portal.system) {::router/init {:routes routes/app-routes :route-data route-data} ::notification/init {:callback js/window.jsAPI.displayNotification} ::http/init {:api-base-url api-base-url :csrf-token csrf-token} ::controller/init {:target-id target-id}} (ns portal.notification) (defmethod ig/init-key ::init [_ {:keys [callback]}] (rf/reg-fx ::info (partial display-notification callback “info")))

Slide 61

Slide 61 text

Primary Content Only ClojureScript JS

Slide 62

Slide 62 text

Primary Content + Top Bar ClojureScript JS

Slide 63

Slide 63 text

Everything in CLJS :) ClojureScript

Slide 64

Slide 64 text

Mock JS Interop ClojureScript Mock

Slide 65

Slide 65 text

Subset of JS Interop ClojureScript Mock

Slide 66

Slide 66 text

metosin/reitit • Bi-directional • Data-driven • Attach data to routes ! • Deﬁne in CLJC, add data in CLJ or CLJS

Slide 67

Slide 67 text

(ns portal.routes) ;; CLJC (def app-routes ["" ["/u/:workspace" ["/compute" ["/ip" ["" :compute-ip] ["/:zone" :compute-ip-zone]] ["/instances" ["" :compute-instances] ["/add" :compute-instance-add] ["/:id" ["" :compute-instance] ["/move" :compute-instance-move] ["/scale" :compute-instance-scale]]]]]])

Slide 68

Slide 68 text

;; reitit route data (ns portal.system) ;; CLJS {:compute-instance {:view instance-view/view :mount ::instance-model/mount :unmount ::instance-model/unmount :query->ttl {::instance/list 10 ::volume/list 10 ::private-network/list 10 ::snapshot/list 10}}} ;; route-data at construction (reitit/router routes {:conflicts nil :expand (expand route-data)})

Slide 69

Slide 69 text

Product Problems Going fast Implementation Lessons learned Conclusion

Slide 70

Slide 70 text

– Alan J. Perlis “It is better to have 100 functions operate on one data structure than 10 functions on 10 data structures.”

Slide 71

Slide 71 text

Small functions • Performance • Re-frame subscriptions • Reagent components • React Developer Tools

Slide 72

Slide 72 text

::namespaced/keywords • Namespace owns the data • Avoid deeply nested DB • Avoid circular dependencies • (::ip/list not :portal.data.ip/list) • Separate transient and persistent state

Slide 73

Slide 73 text

::namespaced/keywords in re-frame db cljs.user> (-> @re-frame.db/app-db keys sort) (:portal.compute.instance.model/editing-displayname? :portal.compute.instance.model/input-displayname :portal.data/last-fetch :portal.data/subscriptions :portal.data.account/account :portal.data.firewall/list :portal.data.instance/list :portal.data.private-network/list :portal.data.service-offering/list :portal.data.ssh-key/list :portal.data.volume/list :portal.data.zone/list :portal.router/route :portal.router/workspace)

Slide 74

Slide 74 text

::namespaced/keywords in re-frame event handler (ns portal.data.ip) (rf/reg-event-fx ::allocate-success (fn [_ _] {::notification/info "Successfully allocated IP” ::js-api/fetch [::list] ::data/fetch-n [[::list] [::instance/list]]}))

Slide 75

Slide 75 text

Encapsulation

Slide 76

Slide 76 text

;; Java 101 public class Person { private String name; public String getName() { return name; } public void setName(String name) { this.name = name; } }

Slide 77

Slide 77 text

(ns portal.router) (defn route-handler [db] ;; encapsulation (get-in db [::route :data :name])) (rf/reg-sub ::route-handler route-handler) (ns portal.compute.firewalling.model) (when (= :compute-firewalling (router/route-handler db)) ;; ...)

Slide 78

Slide 78 text

Embrace JS Interop • Interop by URL alone is not enough • Outgoing: JS API • Incoming: CLJS API

Slide 79

Slide 79 text

Slide 80

Slide 80 text

(ns portal.cljs-api) (defn ^:export match-route [route] (rf/dispatch-sync [::router/match-route route])) (defn ^:export fetch-ips [] (rf/dispatch [::data/fetch [::ip/list]])) (defn ^:export fetch-instances [] (rf/dispatch [::data/fetch [::instance/list]])) ...

Slide 81

Slide 81 text

;; Calling JS from CLJS (js/window.jsAPI.fetchInstances) (js/window.jsAPI.displayNotification "info" "VM Created") ;; Calling CLJS from JS portal.cljs_api.fetch_instances(); portal.cljs_api.match_route(action.payload); ;; Converting data structures cljs.user> (clj->js {:hello "Dutch Clojure Days"}) #js {:hello "Dutch Clojure Days"} cljs.user> (js->clj #js {:hello "Dutch Clojure Days"} :keywordize-keys true)

Slide 82

Slide 82 text

Model data as maps • Even lists! • Easier data access • Optimistic updates • get-in, assoc-in, update-in • keys, vals

Slide 83

Slide 83 text

;; vector cljs.user> (->> [{:id "a1" :ipaddress "1.2.3.4"} {:id "a2" :ipaddress "1.2.3.3"}] (filter #(= "a2" (:id %))) first :ipaddress) “1.2.3.3” ;; map cljs.user> (-> {"a1" {:id "a1" :ipaddress "1.2.3.4"} "a2" {:id "a2" :ipaddress "1.2.3.3"}} (get-in ["a2" :ipaddress])) “1.2.3.3"

Slide 84

Slide 84 text

Don’t re-frame all the things!

Slide 85

Slide 85 text

Product Problems Going fast Implementation Lessons learned Conclusion

Slide 86

Slide 86 text

What went well? • UI Tests • One URL at a time • Data layer • JS Interop

Slide 87

Slide 87 text

Things we can improve • More JS Interop • Smaller initial scope • Integration tests • Reusable components

Slide 88

Slide 88 text

If we could change one thing…

Slide 89

Slide 89 text

Python API Initial Release Controller Router Data Instance IP IP Addresses View Model JS API CLJS API Poller

Slide 90

Slide 90 text

Share the data layer Controller Router Data IP Addresses View Model JS API CLJS API

Slide 91

Slide 91 text

Thank you for listening! www.exoscale.com alexanderjamesking.com github.com/alexanderjamesking