Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Frontend Infrastructure at Etsy

Frontend Infrastructure at Etsy

Delivered at Smashing Conf NYC, June 17th 2015

Daniel Espeset

June 17, 2015
Tweet

More Decks by Daniel Espeset

Other Decks in Programming

Transcript

  1. Daniel Espeset @danielespeset Smashing Conf NYC, June 2015 at Etsy

    Frontend Infrastructure
  2. talks.desp.in/smashing Resources for this talk

  3. 3

  4. Daniel Espeset @danielespeset A front-end operations engineer is not a

    title you’ve likely come across, but hopefully one that you will [...] an expert at serving and hosting front-end resources. [finding] the best ways to piece together the parts of a Web application, and [being a pro] at versioning, caching and deployment. Alex Sexton @slexaxton "Front-End Ops", Smashing Magazine, July 2013 “ ” 4
  5. Daniel Espeset @danielespeset Frontend Infrastructure 5 Daniel Espeset Daniel Na

    Takashi Mizohata Etsy Seth Walker Sufian Rhazi Jonathan Lai
  6. Etsy Engineering

  7. Daniel Espeset @danielespeset Anybody can touch (mostly) anything 7

  8. Daniel Espeset @danielespeset Blameless postmortems 8

  9. Daniel Espeset @danielespeset Everyone deploys 9

  10. Continuous Deployment

  11. Daniel Espeset @danielespeset Push Button Deploys 11

  12. pages/search_results.php if (Feature::isEnabled('my_new_feature')) { // TODO build this part }

    else { // Existing code } 20 21 22 .. 100 12 config/production.php // Owner: despeset, frontend infrastructure server_config['my_new_feature'] = ( 'enabled' => 0 ); 890 891 892 893 Checking a flag Defining a flag
  13. 13 desp .join pushbot has changed topic to: "<prod> seth

    dna* gio* | desp" pushbot desp jklein : You're up desp .in pushbot has changed topic to: "desp jklein" pushbot has changed topic to: "desp* jklein" jklein .in pushbot has changed topic to: "desp* jklein*" pushbot desp jklein : everyone is ready Coordinating Deploys
  14. 14

  15. 15

  16. 16

  17. 17

  18. 18

  19. 19 25-50Pushes a day

  20. 20 // Owner: despeset, frontend infrastructure server_config['my_new_feature'] = ( 'enabled'

    => 0, 'group' => 9903 ); 890 891 892 893 894 config/production.php
  21. 21 config/production.php // Owner: despeset, frontend infrastructure server_config['my_new_feature'] = (

    'enabled' => 0, 'group' => 9903, 'admin' => 'on' ); 890 891 892 893 894 895
  22. 22 // Owner: despeset, frontend infrastructure server_config['my_new_feature'] = ( 'enabled'

    => 50, 'group' => 9903, 'admin' => 'on' ); 890 891 892 893 894 895 config/production.php
  23. Continuous Experimentation

  24. Daniel Espeset @danielespeset idea code release idea code release? A/B

    test idea prototy release A/B test validate A/B test refine 2005 2010 2012 Shamelessly stolen from Dan McKinley mcfunley.com/data-driven-products-now Evolution of Continuous Experimentation 24
  25. Daniel Espeset @danielespeset • Engineers are trusted and given agency

    • The production codebase is constantly changing • That codebase is filled with feature flags In a nutshell 25
  26. Daniel Espeset @danielespeset 26

  27. Daniel Espeset @danielespeset 27 Sass conversion

  28. Daniel Espeset @danielespeset Our frontend is made up of >300

     different “pages” 28
  29. Daniel Espeset @danielespeset At any given time we may be

    running dozens of tests per page 29
  30. Daniel Espeset @danielespeset 4,096 212= 30

  31. Daniel Espeset @danielespeset 212 300 x =1.2 M 31

  32. Daniel Espeset @danielespeset Reasoning about our frontend can be hard

    32
  33. Daniel Espeset @danielespeset Deleting code is often scary 33

  34. Daniel Espeset @danielespeset Verifying changes is difficult 34

  35. Frontend Infrastructure

  36. Daniel Espeset @danielespeset 36 Performance Allison McKnight Kristyn Reith Etsy

    Natalya Hoota Lara Hogan
  37. Daniel Espeset @danielespeset source code deploy build system 37

  38. Daniel Espeset @danielespeset source code production build 38 deploy build

    system
  39. Daniel Espeset @danielespeset Frontend Infrastructure 39

  40. Daniel Espeset @danielespeset 40 ...the bridge between an application's intent

    and an application's reality. Alex Sexton (JavaScript Laureate of Texas) "Front-End Ops", Smashing Magazine, July 2013 “ ”
  41. {% css files="base" %}{% /css %} Template Code <link href="css/base.20150607182402.css"

    ... /> Rendered HTML most recent last modified date of file and all dependencies.
  42. {% css files="base" %}{% /css %} Template Code <link href="css/base.20150607182402.css"

    ... /> Rendered HTML adds version number, always happens.
  43. {% css files="base" %}{% /css %} Template Code <link href="css/base.20150607182402.css"

    ... /> Rendered HTML if (rand() > 0.01) { log(template, asset); }
  44. Where are my things? (Ranger)

  45. Daniel Espeset @danielespeset + + JS graph Asset logs CSS

    graph Ranger combines 3 dependency graphs 45
  46. 46

  47. 47

  48. 48

  49. 49

  50. 50

  51. 51

  52. 52

  53. 53

  54. Daniel Espeset @danielespeset Ranger gave us something valuable The set

    of files that aren't used 54
  55. Profiling CSS (Shrinkray)

  56. Daniel Espeset @danielespeset 56 • Pick a stylesheet loaded on

    the page Shrinkray - CSS analysis
  57. Daniel Espeset @danielespeset 57 • Pick a stylesheet loaded on

    the page • Randomly pick 50 selectors via CSSOM Shrinkray - CSS analysis
  58. Daniel Espeset @danielespeset 58 • Pick a stylesheet loaded on

    the page • Randomly pick 50 selectors via CSSOM • Search for matches with document.querySelector Shrinkray - CSS analysis
  59. Daniel Espeset @danielespeset 59 • Pick a stylesheet loaded on

    the page • Randomly pick 50 selectors via CSSOM • Search for matches with document.querySelector • Send results to server Shrinkray - CSS analysis
  60. Daniel Espeset @danielespeset • Pick a stylesheet loaded on the

    page • Randomly pick 50 selectors via CSSOM • Search for matches with document.querySelector • Send results to server • Aggregate data with Map Reduce Shrinkray - CSS analysis 60
  61. 61

  62. Daniel Espeset @danielespeset • Low confidence on long-tail pages •

    Doesn't capture CSS appearing by user action Why doesn't it work yet? 62
  63. Daniel Espeset @danielespeset • Increase measurement volume if perf allows

    • Add time skew to measurements • Intelligently target selectors based on the data • Add a second pass using an more invasive but thorough technique Possible solutions 63
  64. 64 #container .button-cool ul { color: pink; float: left; }

    #container .button-cool ul:before { content: ''; background-image: url('selector-used?id=1'); } 890 891 892 893 894 895 896 899 some_page.css CSS Background Beacon
  65. Moving to Sass

  66. Daniel Espeset @danielespeset Existing preprocessor inlines @import, versions images 66

  67. Daniel Espeset @danielespeset Linted for syntax (sorta) 67

  68. 68 Every color defined in our CSS

  69. Daniel Espeset @danielespeset • @import statements needed changed • Converter

    fixes 171,244 Sass-lint errors • Had to resolve hundreds of invalid CSS rules & selectors Converter script 69
  70. Daniel Espeset @danielespeset Will fixing this break something? color: rgb(25555,

    -10, 0); 70
  71. Daniel Espeset @danielespeset What about this one? color: rgb(0, 0,

    0, 0); 71
  72. Daniel Espeset @danielespeset CSS Source Legacy CSS Build Legacy Build

    72
  73. Daniel Espeset @danielespeset CSS Source Legacy CSS Build Converter Sass

    Build New CSS Build 73 Legacy Build
  74. Daniel Espeset @danielespeset CSS Source Legacy CSS Build Converter Sass

    Build New CSS Build AST Differ Parse to AST Parse to AST 74 Legacy Build
  75. Daniel Espeset @danielespeset CSS Source Legacy CSS Build Converter Sass

    Build 100% New CSS Build 75 Legacy Build
  76. Daniel Espeset @danielespeset CSS Source Legacy CSS Build Converter Sass

    Build 100% Etsy New CSS Build 76 Legacy Build
  77. Daniel Espeset @danielespeset CSS Source Legacy CSS Build Converter Sass

    Build 50% 50% New CSS Build 77 Legacy Build
  78. Daniel Espeset @danielespeset CSS Source Legacy CSS Build New CSS

    Build Converter Sass Build 100% 78 Legacy Build
  79. Daniel Espeset @danielespeset Sass Source New CSS Build Sass Build

    100% 79
  80. Daniel Espeset @danielespeset • No @extends or functions, ever •

    @mixins only defined in one global file • Nesting level < 4 • Variable definitions should in a dedicated file Restricting Sass 80
  81. 81

  82. Daniel Espeset @danielespeset • Enforce globally unique variable names •

    Sass AST! More to do 82
  83. What have we learned?

  84. Daniel Espeset @danielespeset Disposability > Modularity 84

  85. Daniel Espeset @danielespeset The frontend is getting more complex everywhere.

    85
  86. Daniel Espeset @danielespeset We need new strategies to manage and

    understand it. 86
  87. Daniel Espeset @danielespeset It helps to have a team dedicated

    to this work. (or even just a person) 87
  88. Daniel Espeset @danielespeset The browser is part of your distributed

    system, not just a client you support. 88
  89. codeascraft.com etsy.com/careers Thank you. talks.desp.in/smashing