Scaling R&D While Maintaining Quality - ILTeckTalks

Case study of how Wix supports rapid growth while maintaining quality by company structure, Games of Gangs - a game like activity that promotes knowledge sharing and more

Transcript

1. Scaling  R&D  Organiza/on   While  Maintaining  Quality Aviran Mordo Head of Back-End Engineering @ Wix @aviranm http://www.linkedin.com/in/aviran http://www.aviransplace.com
2. Wix  In  Numbers •  53M  users  from  190  countries   •  Sta5c  storage  is  >1PB  of  data   •  3  Data  centers  +  2  Clouds  (Google,  Amazon)   •  1.5B  HTTP  requests/day   •  ~800  people  work  at  Wix   •  Of  which  ~  300  in  R&D  
4. May 2014 Deployments  /produc5on  changes  per  month  
5. Structures  for  Scalability •  There  are  2  key  common  structures  in  the  industry   •  Func5onal  unit  model   •  Business  unit  model  
6. Func/onal  Model •  Disadvantages   •  Lack  of  product  ownership     •  Lack  of  product  level  exper5se     •  Hard  to  predict  and  plan  product   roadmap   •  Cross-­‐func5on  communica5on  is   hard   •  Less  focus  on  delivery  and  TTM  
7. Business  Unit  Model •  Disadvantages   •  Product  integra5on  is  hard   •  Resource  and  work  duplica5on   •  Architecture  alignment  is  hard   •  Technology  knowledge  sharing  is   hard   •  Limited  opportunity  for   professional  development  
8. •  There  is  no  perfect  model     •  It  depends  on  the  company’s  current  challenges,   life  cycle  phase  and  culture   •  Every  model  should  be  tuned  constantly  and  evolve   with  the  company Our  Assump/ons  
10. •  Guild  –  a  group  of  people   that  share  exper5se,   knowledge,  tools,  code,   and  prac5ces   •  Gang  –  a  group  of  people   that  work  in  a  related   products   •  Compose  of  all  required   resources  from  the   different  disciplines   (Guild)     Wix’s  Gangs  &  Guilds  Model Guild Gang Guild Gang Gang
11. Wix’s  Company  Model Guild Gang Guild Gang Gang Company •  Company  focus  on  large  segment   •  Has  all  the  resources  it  needs  to   be  independent   •  People  within  the  company  are   aligned  with  the  Guilds     CEO  
12. Wix’s  Gangs  &  Guilds  Model  –  Why? •  Technical  power  of  the  Guild   •  Independence  of  the  Gang   •  Healthy  balance  between  product  and  tech   •  Product  features  and  technical  equal  in  priority   Guild  Leader What How Guild  Leader How Team  member Team  member
13. Quality   Trust   Independence   Growth  
14. Dev-­‐Centric  Culture
15. Tradi/onal  Dev  Pipeline Product   Dev   QA   Opera5ons   15:57  
17. Product   Dev   QA   Opera5ons   15:57  
18. What  Is  The  Common  Denominator? •  Product  manager   •  Project  manager   •  QA   •  Opera5ons   •  DBA  
19. Dev  Centric  Culture  –  Involve  The  Developer  –  How? •  Product  defini5on  (with  product)     •  Development  (with  architect)   •  Tes5ng  (with  QA  developers)   •  Deployment  /  Rollback(with  opera5ons)     •  Monitoring  /  BI  (with  BI  team)   •  DevOps  –  to  enable  deployment  and   rollback,  fully  automated   Developer   Product   QA   Management   Opera5on   BI  
20. esponsibility   wnership   uality   haring   Dev  Centric  Culture  –  Why?
22. Project  Rota/on  –  How? •  Team  usually  has  more  than  one  project   •  Team  members  rotate  tasks  between  projects   •  Developers  can  switch  teams  and  Gangs  
23. Project  Rota/on  –  Why? •  There  is  no  single  person  with  a  knowledge  on  a   specific  component   •  Ongoing  review   •  Keep  it  interes5ng   •  It  is  everyone's  responsibility   •  Everybody  owns  everything  
25. Guild  Day •  Engineer  works  4  days  with  his  company   •  Thursday  is  Guild  day.   •  Developers  conduct  quality  enhancing  ac5vi5es   with  the  Guild.        
26. Guild  Day  Goals •  Builds  cross-­‐team  rela5onships   •  Shares  knowledge     •  Assimilate  the  culture   •  Lesson  learned   •  Con5nuous  improvement   •  Promotes  innova5on   •  Professional  development  
27. Produc/on  Bug  Hunt •  Service  owner  picks  a  service  running  on  produc5on   •  Present  the  service  to  all  Guild  members   •  What  the  service  does   •  Excep5ons  thrown  in  produc5on   •  Performance  matrix   •  Build  warnings   •  Get  a  list  of  AI  from  group  feedback  (some5mes  for   dependent  services)      
28.  Produc/on  Bug  Hunt  -­‐  Why? •  Improve  service  stability   •  Teach  developers  about  services  they  don’t  own   •  Understand  how  services  behave  on  produc5on   •  Open  discussion  and  ideas  from  group  members   •  Discover  unexpected  use  pajerns   •  Find  bugs  on  produc5on   •  Lessons  learned  from  other  teams  experience  
29. Retrospec/ve  –  How  ?   •  Whenever  developers  encounter  issues  /  dilemmas,   they  are  encouraged  to  post  them  on  a  board  (pulse)   for  public  discussion   •  Topics  posted  on  the  board  during  the  week   cons5tute  the  agenda  of  the  retrospec5ve  
31. Retrospec/ve  –  Why  ? •  Main  tool  for  con5nuous  improvement     •  Share  lesson  learned.   •  Solve  problems     •  Sugges5ons  on  how  to  improve:   •  Our  team   •  Quality  of  products   •  Process     •  Effec5veness  of  the  R&D  organiza5on.    
33. Tech  Talks  –  How? •  Developers  give  a  tech  talks   •  People  from  other  departments  in  the  company   •  Guest  speakers   •  Open  to  anyone  from  Wix   •  Using  Meetup  hjp://www.meetup.com/at-­‐wix/  to   invite  outsiders  to  our  internal  talks  (if  appropriate)   •  Talks  videos  hjp://goo.gl/IDqXTi  
34. Tech  Talks  –  Why  ? •  Training  employees   •  Knowledge  sharing   •  Educa5ng  about  other  ac5vi5es  at  Wix   •  Professional  development    
35. Thursday  Tasks
36. Games  of  Gangs •  Build  tools  that  help  developers   •  Enhance  plalorm  /  framework   •  Pay  technical  (legacy)  debt  (refactor,  etc’)   •  Improve  tests   •  Research  something  new  for  your  “company”   •  Work  on  a  task  for  a  different  “company”   •  Open  source  a  project   •  1  on  1  mentoring  
38. Games  of  Gangs  –  Why? •  Improve  code  quality   •  Improve  developer  produc5vity   •  Find  repea5ng  pajerns  across  projects  -­‐  generalize  a   solu5on  or  improve  the  infrastructure   •  Learn  other  projects  (easier  to  step  in  if  necessary)   •  Unbiased  review  of  other  projects   •  Learn  about  problems  and  solu5ons  other  teams   faced  and  solved  that  you  may  also  encounter.   •  It  fun  and  breaks  the  day  to  day  rou5ne    
40. Test  Driven  Development  –  How? •  No  new  code  is  pushed  to  Git  without  being  fully  tested   •  We  currently  have  around  10,000  automated  tests   •  Before  fixing  a  bug  first  write  a  test  to  reproduce  the  bug   •  Cover  legacy  (untested)  systems  with  Integra5on  tests   15:57  
41. Test  Driven  Development  –  Why? •  Developers  are  responsible  for  their  own  code   •  Less  bugs   •  Easier  to  enter  someone  else’s  project  
43. Do  Not  Compromise  On  Hiring •  Hire  only  good  people   •  Fit  the  culture   •  Excellent  technically   •  Candidates  can  be  dropped   •  By  anyone   •  At  any  5me   •  If  there  is  any  doubt,  then   there  is  no  doubt  
44. Quality   Trust   Independence   Growth  
46. Dedicated  Resources  =  Organiza/on  Structure
47. Minimize  Architectural  Dependencies •  Service  oriented  architecture  (SOA)   •  Separa5on  of  client  (UI)  and  server  projects   •  Dedicated  data  stores  for  each  service   •  Stateless  services   Lesson  learned:  System  architecture  that  decouples  concerns  
48. Communica/on  Channels •  To  company  wide  ac5vi5es   •  To  knowledge  centers   •  To  key  personnel  
49. Quality   Trust   Independence   Growth  
50. Lessons  Learned:  Growing  Teams
51. Seeds  Of  Ambassadors •  Train  a  group  of  ambassadors  that  prac5ce  dev-­‐ centric  culture  from  the  Guild   •  Build  new  teams  with  at  least  one  dev-­‐centric   ambassador  to  assimilate  new  teams  into  the   culture.   •  Beware  of  hiring  more  people  than  you  can  train  /   assimilate  successfully  
53. CFO  asks  CEO:  “What  happens  if  we   invest  in  developing  our  people  and  they   leave  us?”     CEO:  “What  happened  if  we  don’t  and   they  stay?”  
54. Yes!  We  Are  Hiring  –  jobs@wix.com  
55. Aviran Mordo Head of Back-End Engineering @ Wix @aviranm linkedin.com/in/aviran aviransplace.com Q&A Read  some  more  about  maintaining  quality  R&D:  hjp://goo.gl/c3WLsz