Upgrade to Pro — share decks privately, control downloads, hide ads and more …

The Architecture of Stack Overflow - Merge Lisbon 2014

3fd9e5b2c59170ec3d74dde30d233fa4?s=47 Marco Cecconi
September 26, 2014

The Architecture of Stack Overflow - Merge Lisbon 2014

The Architecture of Stack Overflow slides from Merge Lisbon 2014

3fd9e5b2c59170ec3d74dde30d233fa4?s=128

Marco Cecconi

September 26, 2014
Tweet

Transcript

  1. The  Architecture  Of   Marco Cecconi @sklivvz http://sklivvz.com

  2. Sizing  up  the  problem  

  3. None
  4. None
  5. None
  6. *source: Quantcast, Alexa #50-­‐ish  network  for  traffic*   (between imdb.com

    and t.co)
  7. #50-­‐ish  network  for  traffic*   …and  #17  in  Portugal!?  

      *source: Quantcast, Alexa   much successful very traffic obrigado, yo!
  8. 543,255,552  pageviews  in  the  last  30  days*   *source: Quantcast

  9. Our  datacenter?  

  10. None
  11. web servers load balancers redis search database http(s) http rest

    http protobuf sql sql protobuf tag engine
  12. None
  13. We  are  sNll  scaling  up!  

  14. Our  development  cycle  

  15. BAT!CAVE Code-build-test cycle running on home machine feature requests

  16. BAT!CAVE DEV.SO Test on the real servers git push

  17. BAT!CAVE DEV.SO META.SE 1-click deploy Users test on meta.stackexchange.com (“baking”)

    git push
  18. BAT!CAVE DEV.SO META.SE 1-click deploy HALP! git revert git push

  19. BAT!CAVE DEV.SO NETWORK META.SE 1-click deploy git push 1-click deploy

    It’s live! Tons of users use it…
  20. BAT!CAVE DEV.SO NETWORK META.SE 1-click deploy git push 1-click deploy

    …and provide new feature requests
  21. Move  fast  and  break  things*   * Not the home

    page or question page :-)
  22. Move  fast  and  break  things*   * Not the home

    page or question page :-)
  23. Abusing  caching  

  24. Network Level Caches (CDN, etc.) Server Level Cache (HttpRuntime.Cache) Site

    Level Cache (Redis) SQL Server Database Cache (384 gigs of RAM!) Solid State Disk
  25. None
  26. None
  27. None
  28. RisoQo  &  SpagheS  

  29. None
  30. None
  31. None
  32. None
  33. None
  34. Throwing away risotto… one grain at a time!

  35. Abuse  caching  for  GC   performance  

  36. IRepository<Order>  orderRepository  =            container.Resolve<IRepository<Order>>();  

      Order  order  =  orderRepository.Get(35); This is what you think you are doing…
  37. …but if you think about it a bit more…

  38. ...this is what you are actually doing! IRepository<Order>  repository  =

         new  ValidatingOrderRepository  (          new  SecurityRepository<Order>  (              new  LoggingRepository<Order>  (                  new  CachingRepository<Order>  (                      new  NHibernateRepository<Order>  ()                  )              )          )      );         Order  order  =  repository.Get(35);
  39. Throwing away spaghetti… one strand at a time!

  40. Dependency  InjecNon  is  our   version  of  spagheS  code  

  41. Our  source  code  

  42. Few projects :-)

  43. Few projects :-) Few lines of code :-)

  44. Few projects :-) Few lines of code :-) Eeek! very

    few tests :-S
  45. Few projects :-) Few lines of code :-) Awesome community

    to help :-D Eeek! very few tests :-S
  46. YAGNI*   * You Ain’t Gonna Need It!

  47. Libraries  and  open  source  

  48. None
  49. None
  50. None
  51. Wrap  code  in  libraries     and  open  source  it

     
  52. None
  53. None
  54. None
  55. “SUPERHERO” CODING

  56. HIRE “NINJA” CODERS

  57. TOOLS LEVEL: “NINJA”

  58. •     Performance  is  a  feature   •     Always.

     Be.  Shipping.   •     Use  your  circumstances.   •     Open  source  your  libraries   •     3  obscenely  big  monitors.     KEY TAKEAWAYS
  59. None
  60. Marco Cecconi @sklivvz http://sklivvz.com we are hiring new ninjas from

    anywhere in the world!