Frontend Archaeology - Speaker Deck

Slide 1

Slide 1 text

Frontend Archaeology Digging through TweetDeck for lessons at scale @tgvashworth 1

Slide 2

Slide 2 text

Tom Ashworth @tgvashworth Tech Lead, TweetDeck @tgvashworth 2

Slide 3

Slide 3 text

@tgvashworth 3

Slide 4

Slide 4 text

@tgvashworth 4

Slide 5

Slide 5 text

TweetDeck A few million users. Six billion API requests every day. Maybe eighty thousand lines of JavaScript. Six years of git log. @tgvashworth 5

Slide 6

Slide 6 text

Topics Architecture & refactoring Complexity & communication Operations @tgvashworth 6

Slide 7

Slide 7 text

As frontend technology matures, so must our practice. @tgvashworth 7

Slide 8

Slide 8 text

Architecture & refactoring @tgvashworth 8

Slide 9

Slide 9 text

Flight.js Event-driven JavaScript framework Used on TweetDeck & Twitter.com As a company, we're moving to React @tgvashworth 9

Slide 10

Slide 10 text

Slide 11

Slide 11 text

"Attaching" a component to the DOM: Example.attachTo("#example", { conﬁg: "data" }); @tgvashworth 11

Slide 12

Slide 12 text

Using "advice": this.onClick = function () { ... }; this.before("onClick", this.doAThing); this.after("onClick", this.doSomethingElse); @tgvashworth 12

Slide 13

Slide 13 text

Flight had some problems: » Couldn't nest components » No standard state management » Events for data flow @tgvashworth 13

Slide 14

Slide 14 text

Frameworks we like: » React (user interface library) » Elm (language for functional & reactive UI) » Cycle (functional & reactive UI in JS) @tgvashworth 14

Slide 15

Slide 15 text

Ideas we like: » Component nesting & composition » Easy, predictable state management » Normal functions for data manipulation @tgvashworth 15

Slide 16

Slide 16 text

In Flight, nesting components was icky: » Tightly coupled » Not reusable » Hard to debug (because events) @tgvashworth 16

Slide 17

Slide 17 text

// Parent this.trigger("parentTearingDown"); // Child this.on("parentTearingDown", this.teardown); @tgvashworth 17

Slide 18

Slide 18 text

@tgvashworth 18

Slide 19

Slide 19 text

@tgvashworth 19

Slide 20

Slide 20 text

@tgvashworth 20

Slide 21

Slide 21 text

It's this easy now: this.attachChild( Child, this.select('childNode') ); @tgvashworth 21

Slide 22

Slide 22 text

State management was ad-hoc: this.active = true; this.$node.attr("disabled", false); this.attr.tooltip = "..."; this.update(); @tgvashworth 22

Slide 23

Slide 23 text

We created a mixin called withState: this.mergeState({ active: true, tooltip: "Hello!" }); this.after("stateChanged", this.render); @tgvashworth 23

Slide 24

Slide 24 text

this.on("usersResponse", (e, response) => { if (response.requestId === this.id) { response.users.map(user => { this.trigger("tweetsRequest", { requestId: this.id, userId: user.id }); }); } }); this.on("tweetsResponse", (e, response) => { /* ... */ }); this.trigger("usersRequest", { requestId: this.id }); @tgvashworth 24

Slide 25

Slide 25 text

Users.getAll() .then(users => Promise.all( users.map(user => Tweets.getByUser(user.id)) )); @tgvashworth 25

Slide 26

Slide 26 text

Ideas we like: » Functional programming » Promises » Observables @tgvashworth 26

Slide 27

Slide 27 text

Observables with RxJS: » Observable: represents a collection of future values » Observer: a collection of callbacks » Operators: pure functions like map, ﬁlter, concat, ﬂatMap... @tgvashworth 27

Slide 28

Slide 28 text

// Poll the force-refresh endpoint this.observe(this.getTimer()) .ﬂatMap(this.getVersion) .map(this.processVersion) .do(this.sendMetrics) .ﬁlter(this.shouldRefresh) .subscribe(this.refresh); @tgvashworth 28

Slide 29

Slide 29 text

Distill good tech into great ideas. You can apply ideas anywhere. @tgvashworth 29

Slide 30

Slide 30 text

Refactoring @tgvashworth 30

Slide 31

Slide 31 text

Way back in 2015... » require.js & AMD modules » Monoglobal (TD) » Bower dependency management @tgvashworth 31

Slide 32

Slide 32 text

We had... deﬁne(["lodash"], function (_) { // . . . }); and we had... deﬁne(function (require) { var _ = require("lodash"); // . . . }); @tgvashworth 32

Slide 33

Slide 33 text

But we wanted... var _ = require("lodash"); and eventually... import _ from "lodash"; @tgvashworth 33

Slide 34

Slide 34 text

codemod noun A large-scale codebase refactor, often mechanical or repetitive. js-codeshift noun Toolkit for running automated codemods. @tgvashworth 34

Slide 35

Slide 35 text

module.exports = function (fileInfo, api) { return api .jscodeshift(fileInfo.source) .findVariableDeclarators('foo') .renameTo('bar') .toSource(); }; @tgvashworth 35

Slide 36

Slide 36 text

@@ -3,10 +3,9 @@ * * Manages & removing tweets from collections */ -define([ - 'flight/lib/component', - 'data/with_client' -], function (defineComponent, withClient) { +define(function (require) { + var defineComponent = require('flight').component; + var withClient = require('data/with_client'); return defineComponent(customTimelines, withClient); @tgvashworth 36

Slide 37

Slide 37 text

commit 3f617af133cb251ca10a0fadf223a49e71be2440 Author: Tom Ashworth Date: Mon Nov 16 15:47:18 2015 +0000 . . . 70 ﬁles changed, 3508 insertions(+), 3617 deletions(-) @tgvashworth 37

Slide 38

Slide 38 text

Three step refactor recipe: 1. Find all the patterns 2. Choose the two most similar 3. Unify with a codemod Repeat. @tgvashworth 38

Slide 39

Slide 39 text

Evolving Complex Systems Incrementally by Christoph Pojer facebook/jscodeshift reactjs/react-codemod cpojer/js-codemod benjamn/recast @tgvashworth 39

Slide 40

Slide 40 text

@tgvashworth 40

Slide 41

Slide 41 text

Complexity & communication @tgvashworth 41

Slide 42

Slide 42 text

Over time we had acquired... » Lots of complexity » Dependency on people, not systems » Word-of-mouth sharing @tgvashworth 42

Slide 43

Slide 43 text

Complexity @tgvashworth 43

Slide 44

Slide 44 text

Complexity: intrinsic or incidental? @tgvashworth 44

Slide 45

Slide 45 text

Intrinsic complexity » Fundamental to the problem » Evident in design » Bounded @tgvashworth 45

Slide 46

Slide 46 text

Incidental complexity » Unrelated to a specific problem » Emergent over time » Unbounded @tgvashworth 46

Slide 47

Slide 47 text

Code is entropic; it naturally tends to disorder. @tgvashworth 47

Slide 48

Slide 48 text

Some factors in increasing entropy and incidental complexity: » Changing requirements (it did X, now it must do Y too) » Bugs (ugh, it broke, fix it quick) » Intrinsic complexity (complexity replicates) @tgvashworth 48

Slide 49

Slide 49 text

Take an active stance against debt & decay. @tgvashworth 49

Slide 50

Slide 50 text

How we've addressed it... » Portfolio approach » Feature and Platform balance » Advocate for the codebase @tgvashworth 50

Slide 51

Slide 51 text

PM & Design: user advocates. You: codebase advocate. @tgvashworth 51

Slide 52

Slide 52 text

Sharing knowledge @tgvashworth 52

Slide 53

Slide 53 text

Avoid complexity & decay by sharing knowledge: » Pairing » Onboarding » Code review @tgvashworth 53

Slide 54

Slide 54 text

Take time to design your onboarding experience. @tgvashworth 54

Slide 55

Slide 55 text

A great onboarding has side-effects: » Documentation! » More outside contributions » Faster incident management @tgvashworth 55

Slide 56

Slide 56 text

@tgvashworth 56

Slide 57

Slide 57 text

Code review! We all do it... right? @tgvashworth 57

Slide 58

Slide 58 text

Code review is the single most effective practice you can introduce to improve your team's work. @tgvashworth 58

Slide 59

Slide 59 text

Every pull request gets a +1 from someone else. @tgvashworth 59

Slide 60

Slide 60 text

Over time, we learned some things about code review: » Don't review for more than hourDunsmore 2000 » Keep reviews smaller than ~400 linesCohen 2006 » Code review your own code firstCohen 2006 Yes, this slide features science. Cohen 2006 Cohen, Jason. 2006. Best Kept Secrets of Peer Code Review. Proceedings of the 22nd ICSE 2000: 467-476. Dunsmore 2000 Dunsmore et al. 2000. Object-Oriented Inspection in the Face of Delocalisation. Beverly, MA: SmartBear Software. @tgvashworth 60

Slide 61

Slide 61 text

Making Software What Really Works, and Why We Believe It @tgvashworth 61

Slide 62

Slide 62 text

@tgvashworth 62

Slide 63

Slide 63 text

Operations @tgvashworth 63

Slide 64

Slide 64 text

Way-er back in 2014: » Scary deploys (a nervous day in person-hours) » Built from a laptop » Manual testing checklist » Big commits (hard to code review) @tgvashworth 64

Slide 65

Slide 65 text

Scary deploys: » Manual » Infrequent » Lots of code » Slow feedback loop (what broke?) @tgvashworth 65

Slide 66

Slide 66 text

Optimise for confidence by tightening feedback loops. @tgvashworth 66

Slide 67

Slide 67 text

A tight feedback loop lends agility and confidence leading to momentum. @tgvashworth 67

Slide 68

Slide 68 text

Practical ways to optimise for confidence: » Work on master (branches mean big merges) » Use feature flags (no-sweat releases) » Deploy as often as you can » Use alerts to spot trouble @tgvashworth 68

Slide 69

Slide 69 text

Feature flags? function enabled(id, ﬂag, threshold) { return hash(id + ﬂag) > threshold; } const showSuperCoolFeature = enabled(user.id, "super_cool_feature", 50); @tgvashworth 69

Slide 70

Slide 70 text

Track everything with fire-and-forget metrics. @tgvashworth 70

Slide 71

Slide 71 text

Why can't it be this easy? count("login/forgot-password/click"); distribution("api/request/time", 100); @tgvashworth 71

Slide 72

Slide 72 text

In summary » As frontend technology matures, so must our practice. » Find and reapply good ideas. » Take an active stance against debt & decay. » Optimise for confidence by tightening feedback loops. @tgvashworth 72

Slide 73

Slide 73 text

Thanks ❤ Tom Ashworth @tgvashworth [email protected] Special thanks to the team @TwitterUK, especially @passy, for feedback! @tgvashworth 73