Frontend Infrastructure at Etsy

Frontend Infrastructure at Etsy

Delivered at Smashing Conf NYC, June 17th 2015

757c233176d3ad14136df4a547ac4234?s=128

Daniel Espeset

June 17, 2015
Tweet

Transcript

  1. Daniel Espeset @danielespeset Smashing Conf NYC, June 2015 at Etsy

    Frontend Infrastructure
  2. talks.desp.in/smashing Resources for this talk

  3. 3

  4. Daniel Espeset @danielespeset A front-end operations engineer is not a

    title you’ve likely come across, but hopefully one that you will [...] an expert at serving and hosting front-end resources. [finding] the best ways to piece together the parts of a Web application, and [being a pro] at versioning, caching and deployment. Alex Sexton @slexaxton "Front-End Ops", Smashing Magazine, July 2013 “ ” 4
  5. Daniel Espeset @danielespeset Frontend Infrastructure 5 Daniel Espeset Daniel Na

    Takashi Mizohata Etsy Seth Walker Sufian Rhazi Jonathan Lai
  6. Etsy Engineering

  7. Daniel Espeset @danielespeset Anybody can touch (mostly) anything 7

  8. Daniel Espeset @danielespeset Blameless postmortems 8

  9. Daniel Espeset @danielespeset Everyone deploys 9

  10. Continuous Deployment

  11. Daniel Espeset @danielespeset Push Button Deploys 11

  12. pages/search_results.php if (Feature::isEnabled('my_new_feature')) { // TODO build this part }

    else { // Existing code } 20 21 22 .. 100 12 config/production.php // Owner: despeset, frontend infrastructure server_config['my_new_feature'] = ( 'enabled' => 0 ); 890 891 892 893 Checking a flag Defining a flag
  13. 13 desp .join pushbot has changed topic to: "<prod> seth

    dna* gio* | desp" pushbot desp jklein : You're up desp .in pushbot has changed topic to: "desp jklein" pushbot has changed topic to: "desp* jklein" jklein .in pushbot has changed topic to: "desp* jklein*" pushbot desp jklein : everyone is ready Coordinating Deploys
  14. 14

  15. 15

  16. 16

  17. 17

  18. 18

  19. 19 25-50Pushes a day

  20. 20 // Owner: despeset, frontend infrastructure server_config['my_new_feature'] = ( 'enabled'

    => 0, 'group' => 9903 ); 890 891 892 893 894 config/production.php
  21. 21 config/production.php // Owner: despeset, frontend infrastructure server_config['my_new_feature'] = (

    'enabled' => 0, 'group' => 9903, 'admin' => 'on' ); 890 891 892 893 894 895
  22. 22 // Owner: despeset, frontend infrastructure server_config['my_new_feature'] = ( 'enabled'

    => 50, 'group' => 9903, 'admin' => 'on' ); 890 891 892 893 894 895 config/production.php
  23. Continuous Experimentation

  24. Daniel Espeset @danielespeset idea code release idea code release? A/B

    test idea prototy release A/B test validate A/B test refine 2005 2010 2012 Shamelessly stolen from Dan McKinley mcfunley.com/data-driven-products-now Evolution of Continuous Experimentation 24
  25. Daniel Espeset @danielespeset • Engineers are trusted and given agency

    • The production codebase is constantly changing • That codebase is filled with feature flags In a nutshell 25
  26. Daniel Espeset @danielespeset 26

  27. Daniel Espeset @danielespeset 27 Sass conversion

  28. Daniel Espeset @danielespeset Our frontend is made up of >300

     different “pages” 28
  29. Daniel Espeset @danielespeset At any given time we may be

    running dozens of tests per page 29
  30. Daniel Espeset @danielespeset 4,096 212= 30

  31. Daniel Espeset @danielespeset 212 300 x =1.2 M 31

  32. Daniel Espeset @danielespeset Reasoning about our frontend can be hard

    32
  33. Daniel Espeset @danielespeset Deleting code is often scary 33

  34. Daniel Espeset @danielespeset Verifying changes is difficult 34

  35. Frontend Infrastructure

  36. Daniel Espeset @danielespeset 36 Performance Allison McKnight Kristyn Reith Etsy

    Natalya Hoota Lara Hogan
  37. Daniel Espeset @danielespeset source code deploy build system 37

  38. Daniel Espeset @danielespeset source code production build 38 deploy build

    system
  39. Daniel Espeset @danielespeset Frontend Infrastructure 39

  40. Daniel Espeset @danielespeset 40 ...the bridge between an application's intent

    and an application's reality. Alex Sexton (JavaScript Laureate of Texas) "Front-End Ops", Smashing Magazine, July 2013 “ ”
  41. {% css files="base" %}{% /css %} Template Code <link href="css/base.20150607182402.css"

    ... /> Rendered HTML most recent last modified date of file and all dependencies.
  42. {% css files="base" %}{% /css %} Template Code <link href="css/base.20150607182402.css"

    ... /> Rendered HTML adds version number, always happens.
  43. {% css files="base" %}{% /css %} Template Code <link href="css/base.20150607182402.css"

    ... /> Rendered HTML if (rand() > 0.01) { log(template, asset); }
  44. Where are my things? (Ranger)

  45. Daniel Espeset @danielespeset + + JS graph Asset logs CSS

    graph Ranger combines 3 dependency graphs 45
  46. 46

  47. 47

  48. 48

  49. 49

  50. 50

  51. 51

  52. 52

  53. 53

  54. Daniel Espeset @danielespeset Ranger gave us something valuable The set

    of files that aren't used 54
  55. Profiling CSS (Shrinkray)

  56. Daniel Espeset @danielespeset 56 • Pick a stylesheet loaded on

    the page Shrinkray - CSS analysis
  57. Daniel Espeset @danielespeset 57 • Pick a stylesheet loaded on

    the page • Randomly pick 50 selectors via CSSOM Shrinkray - CSS analysis
  58. Daniel Espeset @danielespeset 58 • Pick a stylesheet loaded on

    the page • Randomly pick 50 selectors via CSSOM • Search for matches with document.querySelector Shrinkray - CSS analysis
  59. Daniel Espeset @danielespeset 59 • Pick a stylesheet loaded on

    the page • Randomly pick 50 selectors via CSSOM • Search for matches with document.querySelector • Send results to server Shrinkray - CSS analysis
  60. Daniel Espeset @danielespeset • Pick a stylesheet loaded on the

    page • Randomly pick 50 selectors via CSSOM • Search for matches with document.querySelector • Send results to server • Aggregate data with Map Reduce Shrinkray - CSS analysis 60
  61. 61

  62. Daniel Espeset @danielespeset • Low confidence on long-tail pages •

    Doesn't capture CSS appearing by user action Why doesn't it work yet? 62
  63. Daniel Espeset @danielespeset • Increase measurement volume if perf allows

    • Add time skew to measurements • Intelligently target selectors based on the data • Add a second pass using an more invasive but thorough technique Possible solutions 63
  64. 64 #container .button-cool ul { color: pink; float: left; }

    #container .button-cool ul:before { content: ''; background-image: url('selector-used?id=1'); } 890 891 892 893 894 895 896 899 some_page.css CSS Background Beacon
  65. Moving to Sass

  66. Daniel Espeset @danielespeset Existing preprocessor inlines @import, versions images 66

  67. Daniel Espeset @danielespeset Linted for syntax (sorta) 67

  68. 68 Every color defined in our CSS

  69. Daniel Espeset @danielespeset • @import statements needed changed • Converter

    fixes 171,244 Sass-lint errors • Had to resolve hundreds of invalid CSS rules & selectors Converter script 69
  70. Daniel Espeset @danielespeset Will fixing this break something? color: rgb(25555,

    -10, 0); 70
  71. Daniel Espeset @danielespeset What about this one? color: rgb(0, 0,

    0, 0); 71
  72. Daniel Espeset @danielespeset CSS Source Legacy CSS Build Legacy Build

    72
  73. Daniel Espeset @danielespeset CSS Source Legacy CSS Build Converter Sass

    Build New CSS Build 73 Legacy Build
  74. Daniel Espeset @danielespeset CSS Source Legacy CSS Build Converter Sass

    Build New CSS Build AST Differ Parse to AST Parse to AST 74 Legacy Build
  75. Daniel Espeset @danielespeset CSS Source Legacy CSS Build Converter Sass

    Build 100% New CSS Build 75 Legacy Build
  76. Daniel Espeset @danielespeset CSS Source Legacy CSS Build Converter Sass

    Build 100% Etsy New CSS Build 76 Legacy Build
  77. Daniel Espeset @danielespeset CSS Source Legacy CSS Build Converter Sass

    Build 50% 50% New CSS Build 77 Legacy Build
  78. Daniel Espeset @danielespeset CSS Source Legacy CSS Build New CSS

    Build Converter Sass Build 100% 78 Legacy Build
  79. Daniel Espeset @danielespeset Sass Source New CSS Build Sass Build

    100% 79
  80. Daniel Espeset @danielespeset • No @extends or functions, ever •

    @mixins only defined in one global file • Nesting level < 4 • Variable definitions should in a dedicated file Restricting Sass 80
  81. 81

  82. Daniel Espeset @danielespeset • Enforce globally unique variable names •

    Sass AST! More to do 82
  83. What have we learned?

  84. Daniel Espeset @danielespeset Disposability > Modularity 84

  85. Daniel Espeset @danielespeset The frontend is getting more complex everywhere.

    85
  86. Daniel Espeset @danielespeset We need new strategies to manage and

    understand it. 86
  87. Daniel Espeset @danielespeset It helps to have a team dedicated

    to this work. (or even just a person) 87
  88. Daniel Espeset @danielespeset The browser is part of your distributed

    system, not just a client you support. 88
  89. codeascraft.com etsy.com/careers Thank you. talks.desp.in/smashing