Truly scale a modern engineering organization

Truly scale a modern engineering organization

July 10, 2013 as part of the Engineering @ Scale series at App Nexus


Michael Bryzek

July 10, 2013


  1. 1 Truly  scale  a  modern   engineering  organiza2on    

        Michael  Bryzek   Co-­‐founder  &  CTO,  Gilt   @mbryzek  
  2. Hypothesis! Decentraliza2on   is  the  best  way   to  scale

     today’s   engineering  orgs  
  3. Today’s Talk! •  Millennials  (briefly)   •  The  social  experiment

     at  gilt   •  Culture  and  policies   •  Tech  infrastructure   •  Q&A  
  4. Background! •  Large  orgs  challenged  by  process  and  bureaucracy  

    •  SoNware  engineers  increasingly  choosing  startups   •  Innova2on  con2nues  to  drive  majority  of  growth       Today’s  talk  is  mostly  a  reflec2on  on  growing   gilt  from  0  to  1000+  people    
  5. Millennials in the workplace! Millennial  =  born  in  early  80s

     to  ~2000   •  I  think  of  it  as  having  netscape  in  high  school   •  Only  29%  of  millenials  "expect  to  work  regular  office  hours”     •  Development  and  work/life  balance  are  more  important  today   •  Technology  inside  companies  has  fla_ened  communica2ons   •  Want  flexibility  -­‐  when  to  work,  where  to  work  from,  and  how  to   deliver  work   •  Greater  focus  on  results  (vs.  tenure  for  example)  
  6. The Social Experiment @ Gilt! •  Started  Jan  2012  

    •  Inspired  most  directly  by  Neclix  (culture)  and  amazon   (2  pizza  teams  and  fitness  func2ons)   •  Can  we  pivot  @  gilt  to  empower  small  teams  to   autonomously  innovate  and  deliver  value?      
  7. Traditional Orgs!   •  Somebody  has  an  idea.  In  fact

     there  are  lots  of  ideas   •  Organize  ideas  based  on  impact  –  big  back  logs   •  Assign  engineers  to  work  on  the  items  from  the  back  log      
  8. Traditional Orgs!   •  Works  great  at  first  –  startups

     essen2ally  have  one   team  working  on  one  idea   •  Over  2me,  end  up  feeling  like  you  can’t  get  anything   done   •  Unnamed  company:  “Current  backlog  is  18  months  to   get  an  idea  priori2zed”      
  9. Works great. Be cognizant of potential problems!   At  some

     point,  at  least  at  Gilt,  we  realized  we:   •  Spent  lots  of  2me  talking  about  ideas  that  never  got   done   •  Did  not  feel  great  about  the  ideas  we  were  building   •  Felt  slow  –  too  hard  to  gain  consensus  on  approach     These  were  the  problems  we  set  about  addressing.  It  hit   us  hard  in  our  4th  year  and  roughly  ~800  people  
  10. Ideas are Cheap! Innova2on  is  oNen  confused  with  just  having

     an  idea,  but   innova2on  is  of  course  having  the  idea  and  doing   everything  else  –  the  R&D,  the  tes2ng  and  re-­‐designing  to   develop  a  new  product/service/process,  and  then   successfully  gemng  it  out  to  the  market  or  pumng  it  into   use,  so  that  it’s  not  just  an  idea  but  an  applied  result.     h_p://original-­‐­‐are-­‐ cheap/    
  11. On Execution! “it’s  usually  the  execu2on  that  ma_ers”    

  12. Can we focus more on execution?! change  who   gets

     to  decide   what  is  executed  
  13. Scrum: Chickens and Pigs! •  Ques%on:  In  a  bacon-­‐and-­‐egg  breakfast,

     what's  the   difference  between  the  Chicken  and  the  Pig?   •  Answer:  The  Chicken  is  involved,  but  the  Pig  is   commi_ed!       “Pigs”  should  choose  what  we  build   h_ps://  
  14. Immediate Question! How  can  the  individual  teams  make  decisions  that

     forward   the  corporate  strategy?   •  Develop  culture  to  expect  mistakes  and  iterate  quickly   •  Key  Performance  Indicators  (KPIs)   h_ps://  
  15. Key Performance Indicators! •  They  seem  simple   •  They

     are  not   •  ONen  takes  months  of  debate  to  agree  on  a  KPI     Really  flushes  out  what  is  truly  important  
  16. KPI Example! •  Strategic  objec2ve  to  increase  users  through  organic

      search   •  KPIs:   •  #  of  users  acquired  in  past  30  days  via  non  branded   organic  search   •  Session  revenue  in  past  30  days  from  users  acquired   via  organic  search  
  17. KPI: The important Step! •  Once  the  KPIs  are  defined

     –  work  to  align  on  the   specific  KPIs  –  then  management  shiNs  to  ONLY   focusing  on  the  KPI  and  any  targets     •  This  is  really  the  cri2cal  step  
  18. KPI: Empowerment! •  Drive  by:  CEO  stops  by  and  says

     “I  think  we  need  to   make  the  register  bu_on  blink.”  Team  says:  “Great  idea   –  our  KPI  is  x  and  we  think  our  current  idea  is  going  to   move  our  KPI  towards  its  target  faster.”   •  Decision  making:  The  team  working  on  the  KPI   independently  selects  its  ideas  to  execute  on.  They  are   now  autonomous  and  measured  only  by  the  KPI  
  19. KPI: Lessons Learned! •  Really  important  that  the  set  of

     KPIs  rolls  up  to  the   overall  strategy   •  One  common  ques2on:  “Who  does  the  sh*t  work?”   •  As  much  as  possible,  choose  KPIs  that  are  independent;   where  two  KPIs  conflict,  ensure  the  teams  are   communica2ng  
  20. Tactics at Gilt – Real World Example! •  Ini2a2ves  

    •  KPIs   •  Teams   •  Ingredients   •  Management  
  21. Tactics: Initiatives! •  Corporate  Strategy  defined  as  a  set  of

     ini2a2ves   •  Ini2a2ve  is  a  project  we  intend  to  work  on  for  the   foreseeable  future   •  Future  defined  as  3-­‐12  months   •  Evaluated  once  every  3  months   •  Each  team  works  on  exactly  1  ini2a2ve  
  22. Tactics: KPIs! •  Each  ini2a2ve  defined  in  terms  of  1-­‐2

     “strategic”  KPIs   •  Goal  is  a  single  “strategic”  KPI  where  possible;   some2mes  need  a  second   •  Teams  may  create  their  own  “tac2cal”  KPIs  to  help   measure  progress  on  individual  projects  –  tend  to  be   short-­‐lived.  
  23. Tactics: Teams! •  4-­‐10  cross  func2onal  team  members   • 

    Team  members  select  the  ini2a2ves  they  work  on   •  Tend  to  outlive  ini2a2ves   •  Encourage  mobility  between  teams,  but  up  to   individuals    
  24. Tactics: Ingredients! •  Key  skills  iden2fied  as  ingredients  –  e.g.

     product   visionary,  coder,  presenter,  visual  designer,  business   thinker,  organizer,  mo2vator,  quality  manager,  coach,   analyst,  architect,  writer,  visual  designer  and  ux   specialist   •  Individuals  fulfill  1  or  more  ingredients   •  Ideal  team  has  every  ingredient  
  25. Tactics: Teams!

  26. Tactics: Management
 ! Mee2ng  once  every  2  weeks.  Same  agenda:

      •  Review  KPI  and  progress  towards  targets   •  Discuss  ongoing  mul2  variant  tests  and  outcomes   •  Story:  What’s  the  elevator  pitch  for  next  few  weeks     Works  great  if  the  the  KPIs  are  solid  and  responsive.  Hard   if  KPIs  are  not  great.       h_p://­‐agile-­‐at-­‐ gilt-­‐with-­‐teams-­‐ingredients   [from  John  Quinn,  ex  Digg  CTO  and  current  SVP  engineering  at  gilt]  
  27. Results after 18 months! Tradi2onal  metrics:     •  Reten2on

     is  very  high,  esp.  among  top  engineers   •  Just  about  all  the  charts  are  up  and  to  the  right   •  Pace  of  feature  delivery  increasing  every  month    
  28. Hypothesis! Decentraliza2on   is  the  best  way   to  scale

     today’s   engineering  orgs  
  29. Two basic ways to hire – both work! People  who

     will   take  orders   Smart  and   Autonomous   People   But  don’t  hire  from  one  group  and  expect  the  other  
  30. What do Smart and Autonomous People Want?! •  Authority  to

     make  project  decisions   •  To  build  something  that  ma_ers   •  To  impact  business   •  To  develop  a  strong  sense  of  well  being   •  To  work  with  great  people  (great  =  be_er  than  me)  
  31. What do Smart and Autonomous People Want?! Strong  and  Uncompromising

     Values:     •  Integrity  /  honesty   •  Humility   •  Respect   •  Transparency    
  32. What do Smart and Autonomous People Want?!      

      Ul2mately  seems  to  boil  down  to  Trust  
  33. Trust Defined! Confidence  in  or  reliance  on   some  person

     or  quality         h_ps://  
  34. Trust in Engineering! •  We  trust  our  engineers  to  write

     soNware   •  We  trust  our  engineers  to  manage  our  systems   •  We  trust  our  engineers  to  change  our  databases   •  But  do  we  really  trust  their  judgment?  
  35. Policies! •  Each  and  every  documented  policy  /  approval  is

     a   source  of  fric2on  and  erodes  trust   •  Let’s  look  at  some  common  examples  
  36. Travel & Entertainment! •  Should  be  “Use  your  judgment.  If

     unsure,  ask  in   advance”   •  Probably  is  “You  can  spent  up  to  $50  per  diem”  
  37. Who can post to the company blog?! •  Should  be

     “Anybody”   •  Probably  is  “Send  a  draN  of  your  post  to  xxx  for   approval”    
  38. Policy! We  have  a  public-­‐facing  Gilt  Technologie  blog:  h_p://

      It  is  up  to  YOU  to  fill  it  up  with  the  awesome  stuff  we're  working  on  and  doing   here.     What  you  post  there  is  auto  syndicated  to  our  gil_ech  twi_er  account,  which   you  can  post  to  directly.  Password=*****     Things  to  Consider   •  It's  oNen  a  good  idea  to  ask  for  a  peer  review  before  pos2ng  to  double   check  technical  accuracy,  grammar  etc.  We've  also  got  a  team  of   wordsmithers  (see  appendix)  that  have  signed  up  to  help  you  out.   •  If  you've  got  content  that  you're  not  sure  about  e.g.  talking  about  a  feature   that's  not  live  or  a  subject  that  you're  not  sure  is  suitable,  ask  your   manager.    
  39. Peer to Peer Budget Experiment! How  to  use  the  peer-­‐to-­‐peer

     budget                     •  Enter  your  proposed  (or  incurred)  expense  on  the  next  sheet.  Your  expense   should  be  related  to  team  building  e.g.  organizing  a  team  building   breakfast  or  a  dinner  to  celebrate  a  big  release.         •  If  the  total  remaining  is  now  nega2ve,  remove  your  expense.  You'll  need  to   wait  un2l  next  month  when  the  budget  is  refreshed.  (the  1st  of  the  month)     •  If  the  total  remaining  is  posi2ve,  you're  approved!         •  If  the  budget  is  exhausted,  please  add  a  $0  entry  for  the  thing  you  would   have  liked  to  do.         S2ll  early  –  but  really  what  happens  is  you  find  yourself   always  considering  if  decentraliza2on  would  be  be_er.  It   oNen  turns  out  to  be.  
  40. Policies are really hard to get right! Some  inspira2on:  

      •  Valve  employee  handbook  h_p:// Valve_Handbook_LowRes.pdf   •  Neclix  culture  h_p:// culture-­‐1798664     And  nobody  gets  it  100%  right.  
  41. Enabling Autonomy with Technology! •  LOSA  –  Lots  of  Small

     applica2ons   •  Ioncannon  –  Con2nuous  Delivery   •  Galac2ca  –  Placorm  as  a  Service  
  42. Lots of Small Applications (LOSA)! •  Goals:  Isola2on  and  Accountability

      •  create  isola2on  and  truly  simplify  accountability  /  ownership  
  43. Continuous Delivery! •  Goal:  Enable  safe,  simple,  automated  produc2on  deploys

      •  Emphasizes  automated  tests   •  Single  command  “sbt  release”  
  44. Galactica! •  Goal:  Enable  autonomous  hardware  provisioning   •  “Heroku”

     for  the  enterprise   •  Work  in  progress  at  gilt  –  automa2on  first,  then  self  service  APIs  
  45. Tech Infrastructure Lessons! •  Autonomy  requires  significant  investment  in  tech

      •  Focus  on  providing  con2nuous  feedback   •  “Lots  of  small  errors”     Ul2mately  reinforces  having  great  internal  tech  –  and  that  by  itself   leads  to  higher  produc2vity,  well  being  and  and  happiness    
  46. Before we wrap up… A word on Innovation! •  We

     didn’t  talk  about  innova2on  today  explicitly   •  But  implicitly  everything  here  is  about  enabling  innova2on   Nobody  has  figured  out  how  to  create  innova2on   But  we  do  know  what  kills  innova2on   Empowerment  and  autonomy  allow  for  innova2on  to  occur  
  47. Thank You! Decentraliza2on   is  the  best  way   to

     scale  today’s   engineering  orgs       Ques%ons?