Upgrade to Pro — share decks privately, control downloads, hide ads and more …

The Architecture of StackOverflow - QCon China 2014

The Architecture of StackOverflow - QCon China 2014

The Architecture of Stack OVerflow - Slides updated for QCon China 2014 including data for April 2014, new slides on garbage collection and China-specific content.

3fd9e5b2c59170ec3d74dde30d233fa4?s=128

Marco Cecconi

April 27, 2014
Tweet

Transcript

  1. The Architecture Of Marco Cecconi @sklivvz sklivvz@stackoverflow.com

  2. None
  3. None
  4. #49 network for traffic* *source: Quantcast, Alexa

  5. #49 network for traffic* …and #7 in China! *source: Quantcast,

    Alexa much successful very traffic 谢谢 wow
  6. 561,027,840 pageviews in the last 30 days* (~100% growth year

    over year) *source: Quantcast
  7. 561,027,840 pageviews in the last 30 days* (~100% growth year

    over year) *source: Quantcast
  8. None
  9. web servers load balancers redis search database http(s) http rest

    http protobuf sql sql protobuf tag engine
  10. None
  11. We are still scaling up, yo!

  12. BATCAVE

  13. BATCAVE DEV.SO

  14. BATCAVE DEV.SO META.SO

  15. BATCAVE DEV.SO META.SO

  16. BATCAVE DEV.SO META.SO NETWORK

  17. Move fast and break things

  18. Move fast and break things* * Not the home page

    or question page :-)
  19. Move fast and break things* * Not the home page

    or question page :-)
  20. Network Level Caches (CDN, etc.) Server Level Cache (HttpRuntime.Cache) Site

    Level Cache (Redis) SQL Server Database Cache (384 gigs of RAM!) Solid State Disk
  21. None
  22. None
  23. None
  24. None
  25. Too Many Allocations This is really the most basic thing

    that can go wrong. Too Many Pointers If you create a data structure that is a large mesh of pointers you'll have two problems. First, there will be a lot of object writes […] and, secondly, when it comes time to collect that data structure, you will make the garbage collector follow all those pointers and if necessary change them all as things move around. […] But if you create such a structure on a transitory basis, […], then you will pay the cost much more often. http://msdn.microsoft.com/en-us/library/ms973837.aspx#dotnetgcbasics_topic2
  26. Too Many Allocations This is really the most basic thing

    that can go wrong. Too Many Pointers If you create a data structure that is a large mesh of pointers you'll have two problems. First, there will be a lot of object writes […] and, secondly, when it comes time to collect that data structure, you will make the garbage collector follow all those pointers and if necessary change them all as things move around. […] But if you create such a structure on a transitory basis, […], then you will pay the cost much more often. http://msdn.microsoft.com/en-us/library/ms973837.aspx#dotnetgcbasics_topic2
  27. None
  28. None
  29. None
  30. None
  31. Too Many Allocations This is really the most basic thing

    that can go wrong. Too Many Pointers If you create a data structure that is a large mesh of pointers you'll have two problems. First, there will be a lot of object writes […] and, secondly, when it comes time to collect that data structure, you will make the garbage collector follow all those pointers and if necessary change them all as things move around. […] But if you create such a structure on a transitory basis, […], then you will pay the cost much more often. http://msdn.microsoft.com/en-us/library/ms973837.aspx#dotnetgcbasics_topic2
  32. IRepository<Order> orderRepository = container.Resolve<IRepository<Order>>(); Order order = orderRepository.Get(35); This is

    what you think you are doing…
  33. …but if you think about it a bit more…

  34. ...this is what you are actually doing! IRepository<Order> repository =

    new ValidatingOrderRepository ( new SecurityRepository<Order> ( new LoggingRepository<Order> ( new CachingRepository<Order> ( new NHibernateRepository<Order> () ) ) ) ); Order order = repository.Get(35);
  35. None
  36. None
  37. None
  38. 馄饨!

  39. Few projects :-)

  40. Few projects :-) Few lines of code :-)

  41. Few projects :-) Few lines of code :-) Eeek! very

    few tests :-S
  42. Few projects :-) Few lines of code :-) Awesome community

    to help :-D Eeek! very few tests :-S
  43. YAGNI*. It works. * You Ain’t Gonna Need It!

  44. * Source http://bit.ly/1eSLr8Z

  45. None
  46. None
  47. None
  48. Write libraries & open them to the world

  49. None
  50. None
  51. None
  52. DEEP DIVES

  53. OPINIONATED HIRING

  54. REMOTE WORK

  55. • Performance is a feature • Always. Be. Shipping. •

    Use your circumstances. • Open source your libraries • 3 obscenely big monitors. CONCLUSION
  56. $_='@mk=uf=radimdp1Z--&ewxuhhl';tr/=1m-za-l@&Z/ !a-zP@\n/&print; Marco Cecconi @sklivvz sklivvz@stackoverflow.com