Upgrade to Pro — share decks privately, control downloads, hide ads and more …
Sign up for free

Practicing Failure: Gamedays on the Obama Campaign

Avatar for Dylan Richard Dylan Richard
September 26, 2013
Technology
8
0

Practicing Failure: Gamedays on the Obama Campaign

Avatar for Dylan Richard

Dylan Richard

September 26, 2013

More Decks by Dylan Richard

See All by Dylan Richard
Monitorama EU 2013
Avatar for Dylan Richard dylanr
0
8

Other Decks in Technology

See All in Technology
20261017福岡devIO 「AIは"良いダッシュボード"を作れるか？
Avatar for 武田 jtakeda
0
170
その Valkey 使いこなしていますか？
Avatar for Takafumi ONAKA onk
PRO
5
1.6k
Webの常識が通じないHTMLメールのダークモード
Avatar for teamLab teamlab
PRO
0
130
今さら理解するウェブフォントver.2026
Avatar for takanorip takanorip
3
880
Rust で関数型ドメインモデリング
Avatar for ONDA, Takashi takonda
0
150
初海外で注意すべきこと12選！！！安全快適なre:invent生活にするために
Avatar for Yuuuuuuu yuuuuuuu168
1
180
Benchmarking Disaggregated Serving on vLLM + Dynamo with a Real Coding-Agent Workload
Avatar for NTT docomo Business nttcom
1
190
オントロジーって 結局なにもの？ AI エージェントに「意味」を渡す知識設計の話
Avatar for TomoyukiMurakami greenmidori
0
230
TinyGoでオブザーバビリティ？OpenTelemetryはTinyGoで使えるのか / Is OpenTelemetry available in TinyGo environment?
Avatar for ymotongpoo ymotongpoo
2
140
Kiro Crewをはじめよう
Avatar for t.kizawa kizawa2020
1
160
DGX Spark でどこまでハッキングができるのか？
Avatar for kash1064 kash1064
3
1.3k
JAWS-UGグッズ職人仕事の流儀
Avatar for Hiroo Katoh kentapapa
0
120

Featured

See All Featured
DBのスキルで生き残る技術 - AI時代におけるテーブル設計の勘所
Avatar for soudai sone soudai
PRO
68
58k
Visualization
Avatar for Eitan Lees eitanlees
153
17k
First, design no harm
Avatar for Per Axbom axbom
PRO
2
1.3k
XXLCSS - How to scale CSS and keep your sanity
Avatar for Zaharenia Atzitzikaki sugarenia
250
1.3M
Why Your Marketing Sucks and What You Can Do About It - Sophie Logan
Avatar for Sophie Logan marketingsoph
0
430
The AI Revolution Will Not Be Monopolized: How open-source beats economies of scale, even for LLMs
Avatar for Ines Montani inesmontani
PRO
3
3.8k
Impact Scores and Hybrid Strategies: The future of link building
Avatar for Tamara Novitovic tamaranovitovic
0
460
Noah Learner - AI + Me: how we built a GSC Bulk Export data pipeline
Avatar for Tech SEO Connect techseoconnect
PRO
0
460
Marketing Yourself as an Engineer | Alaka | Gurzu
Avatar for Gurzu gurzu
0
330
Done Done
Avatar for chrislema chrislema
187
17k
Gemini Prompt Engineering: Practical Techniques for Tangible AI Outcomes
Avatar for Mfonobong Umondia mfonobong
2
590
Getting science done with accelerated Python computing platforms
Avatar for Jacob Tomlinson jacobtomlinson
2
540

Transcript

  1. Practicing Failure Gamedays on the Obama Campaign t: @dylanr e:

    [email protected]

  2. t: @dylanr e: [email protected] what I’m gonna talk about: §campaign

    context §making apps resilient to failure §making teams resilient to failure

  3. Text t: @dylanr e: [email protected] me: Director of Engineering, OFA

    typical political operative...

  4. Text t: @dylanr e: [email protected] Crate&Barrel

  5. Text t: @dylanr e: [email protected] threadless

  6. Text t: @dylanr e: [email protected] what is ofa?

  7. Text t: @dylanr e: [email protected] rebuild

  8. Text t: @dylanr e: [email protected] from:

  9. Text t: @dylanr e: [email protected] to:

  10. Text t: @dylanr e: [email protected] while it’s in heavy use

  11. Text t: @dylanr e: [email protected] 18 months

  12. Text t: @dylanr e: [email protected] 7 days a week

  13. Text t: @dylanr e: [email protected] 18 hours a day

  14. Text t: @dylanr e: [email protected] ~40 engineers

  15. Text t: @dylanr e: [email protected] 7 teams

  16. Text t: @dylanr e: [email protected] ~300 repos

  17. Text t: @dylanr e: [email protected] ~200 deployed products

  18. Text t: @dylanr e: [email protected] ~3,000 servers

  19. Text t: @dylanr e: [email protected] some million views a day

  20. Text t: @dylanr e: [email protected] 1,000,000+ volunteers

  21. Text t: @dylanr e: [email protected] ~8,000 staff

  22. Text t: @dylanr e: [email protected] 1 metric that matters

  23. Text t: @dylanr e: [email protected] 270

  24. Text t: @dylanr e: [email protected] campaigns in 3 minutes

  25. t: @dylanr e: [email protected] a campaign: §plan §build §execute §GOTV

  26. Text t: @dylanr e: [email protected] PLAN!

  27. Text t: @dylanr e: [email protected] dreams

  28. Text t: @dylanr e: [email protected] not fast

  29. Text t: @dylanr e: [email protected] for tech: start building

  30. Text t: @dylanr e: [email protected] BUILD!

  31. Text t: @dylanr e: [email protected] staff/ teams

  32. Text t: @dylanr e: [email protected] volunteer base

  33. Text t: @dylanr e: [email protected] $

  34. Text t: @dylanr e: [email protected] technology

  35. Text t: @dylanr e: [email protected] for tech: keep building, make

    money

  36. Text t: @dylanr e: [email protected] EXECUTE!

  37. Text t: @dylanr e: [email protected] talk to voters

  38. Text t: @dylanr e: [email protected] get volunteers

  39. Text t: @dylanr e: [email protected] do ALL the things!

  40. Text t: @dylanr e: [email protected] for tech: keep building, with

    feedback!

  41. Text t: @dylanr e: [email protected] GOTV!

  42. Text t: @dylanr e: [email protected] get out the vote

  43. Text t: @dylanr e: [email protected] this is it

  44. Text t: @dylanr e: [email protected] for tech:

  45. Text t: @dylanr e: [email protected] scale.

  46. Text t: @dylanr e: [email protected] scale!

  47. Text t: @dylanr e: [email protected] SCALE!

  48. t: @dylanr e: [email protected]

  49. t: @dylanr e: [email protected]

  50. Text t: @dylanr e: [email protected] also...

  51. Text t: @dylanr e: [email protected] Text

  52. Text t: @dylanr e: [email protected] 1 shot

  53. Text t: @dylanr e: [email protected]

  54. Text t: @dylanr e: [email protected] make the apps resilient!

  55. Text t: @dylanr e: [email protected] good news!

  56. Text t: @dylanr e: [email protected] we’re smart!

  57. Text t: @dylanr e: [email protected] consider the apps

  58. Text t: @dylanr e: [email protected] what matters?

  59. Text t: @dylanr e: [email protected] commerce?

  60. Text t: @dylanr e: [email protected] $

  61. Text t: @dylanr e: [email protected] community?

  62. Text t: @dylanr e: [email protected] communication

  63. Text t: @dylanr e: [email protected] campaign volunteer tool?

  64. Text t: @dylanr e: [email protected] doing work

  65. Text t: @dylanr e: [email protected] talk to your stakeholders

  66. Text t: @dylanr e: [email protected] features are cool

  67. Text t: @dylanr e: [email protected] working is cooler

  68. Text t: @dylanr e: [email protected] *prepare for wall of text*

  69. Text t: @dylanr e: [email protected] no feature is cool enough

    to not have a working app. most features don’t matter when it matters.

  70. Text t: @dylanr e: [email protected] graceful degradation

  71. Text t: @dylanr e: [email protected] /me pats team and self

    on back

  72. Text t: @dylanr e: [email protected] bad news!

  73. Text t: @dylanr e: [email protected] tech fails

  74. Text t: @dylanr e: [email protected] two options

  75. Text t: @dylanr e: [email protected] make tech not fail

  76. Text t: @dylanr e: [email protected] deal with failure

  77. Text t: @dylanr e: [email protected]

  78. Text t: @dylanr e: [email protected] make the team resilient to

    failure

  79. Text t: @dylanr e: [email protected] practice

  80. Text t: @dylanr e: [email protected] practice!

  81. Text t: @dylanr e: [email protected] PRACTICE!

  82. Text t: @dylanr e: [email protected]

  83. Text t: @dylanr e: [email protected] 2 weeks to change code

  84. Text t: @dylanr e: [email protected] 2 days for final prep

  85. Text t: @dylanr e: [email protected] 1 hour for it to

    sink in

  86. Text t: @dylanr e: [email protected] dylan is lying

  87. Text t: @dylanr e: [email protected] real disasters aren’t scripted

  88. Text t: @dylanr e: [email protected] real response = real learning

  89. Text t: @dylanr e: [email protected] gameday

  90. Text t: @dylanr e: [email protected] gameday

  91. t: @dylanr e: [email protected] what did it look like? §me

    & part of devops vs engineers §organizing in campfire §maintaining updated google doc

  92. t: @dylanr e: [email protected]

  93. Text t: @dylanr e: [email protected] what did we learn?

  94. Text t: @dylanr e: [email protected] broken things we thought were

    fixed

  95. Text t: @dylanr e: [email protected] what failure really looks like

  96. Text t: @dylanr e: [email protected] how things fail

  97. Text t: @dylanr e: [email protected] how to fix it

  98. Text t: @dylanr e: [email protected] make your instruction manual

  99. Text t: @dylanr e: [email protected] ...the day after

  100. Text t: @dylanr e: [email protected]

  101. Text t: @dylanr e: [email protected] resilient apps:

  102. Text t: @dylanr e: [email protected] plan for failure

  103. Text t: @dylanr e: [email protected] define what matters

  104. Text t: @dylanr e: [email protected] make your plan clear

  105. Text t: @dylanr e: [email protected] fail to the things that

    matter

  106. Text t: @dylanr e: [email protected]

  107. Text t: @dylanr e: [email protected] resilient teams:

  108. Text t: @dylanr e: [email protected] practice failing

  109. Text t: @dylanr e: [email protected] learn from failure

  110. Text t: @dylanr e: [email protected] use your instruction manual

  111. Text t: @dylanr e: [email protected] thanks!

  112. t: @dylanr e: [email protected] hit me up! § twitter: @dylanr

    § email: [email protected]