$30 off During Our Annual Pro Sale. View Details »

The Architecture of StackOverflow - QCon China 2014

The Architecture of StackOverflow - QCon China 2014

The Architecture of Stack OVerflow - Slides updated for QCon China 2014 including data for April 2014, new slides on garbage collection and China-specific content.

Marco Cecconi

April 27, 2014
Tweet

More Decks by Marco Cecconi

Other Decks in Programming

Transcript

  1. The Architecture Of
    Marco Cecconi
    @sklivvz
    [email protected]

    View Slide

  2. View Slide

  3. View Slide

  4. #49 network for traffic*
    *source: Quantcast, Alexa

    View Slide

  5. #49 network for traffic*
    …and #7 in China!
    *source: Quantcast, Alexa
    much successful
    very traffic
    谢谢
    wow

    View Slide

  6. 561,027,840 pageviews in the last 30 days*
    (~100% growth year over year)
    *source: Quantcast

    View Slide

  7. 561,027,840 pageviews in the last 30 days*
    (~100% growth year over year)
    *source: Quantcast

    View Slide

  8. View Slide

  9. web servers
    load balancers
    redis
    search
    database
    http(s)
    http
    rest
    http
    protobuf
    sql
    sql
    protobuf
    tag engine

    View Slide

  10. View Slide

  11. We are still scaling up, yo!

    View Slide

  12. BATCAVE

    View Slide

  13. BATCAVE DEV.SO

    View Slide

  14. BATCAVE DEV.SO
    META.SO

    View Slide

  15. BATCAVE DEV.SO
    META.SO

    View Slide

  16. BATCAVE DEV.SO
    META.SO
    NETWORK

    View Slide

  17. Move fast and break things

    View Slide

  18. Move fast and break things*
    * Not the home page or question page :-)

    View Slide

  19. Move fast and break things*
    * Not the home page or question page :-)

    View Slide

  20. Network Level Caches (CDN, etc.)
    Server Level Cache (HttpRuntime.Cache)
    Site Level Cache (Redis)
    SQL Server Database Cache (384 gigs of RAM!)
    Solid State Disk

    View Slide

  21. View Slide

  22. View Slide

  23. View Slide

  24. View Slide

  25. Too Many Allocations
    This is really the most basic thing that can go wrong.
    Too Many Pointers
    If you create a data structure that is a large mesh of pointers
    you'll have two problems. First, there will be a lot of object
    writes […] and, secondly, when it comes time to collect that
    data structure, you will make the garbage collector follow all
    those pointers and if necessary change them all as things
    move around. […] But if you create such a structure on a
    transitory basis, […], then you will pay the cost much more
    often.
    http://msdn.microsoft.com/en-us/library/ms973837.aspx#dotnetgcbasics_topic2

    View Slide

  26. Too Many Allocations
    This is really the most basic thing that can go wrong.
    Too Many Pointers
    If you create a data structure that is a large mesh of pointers
    you'll have two problems. First, there will be a lot of object
    writes […] and, secondly, when it comes time to collect that
    data structure, you will make the garbage collector follow all
    those pointers and if necessary change them all as things
    move around. […] But if you create such a structure on a
    transitory basis, […], then you will pay the cost much more
    often.
    http://msdn.microsoft.com/en-us/library/ms973837.aspx#dotnetgcbasics_topic2

    View Slide

  27. View Slide

  28. View Slide

  29. View Slide

  30. View Slide

  31. Too Many Allocations
    This is really the most basic thing that can go wrong.
    Too Many Pointers
    If you create a data structure that is a large mesh of pointers
    you'll have two problems. First, there will be a lot of object
    writes […] and, secondly, when it comes time to collect that
    data structure, you will make the garbage collector follow all
    those pointers and if necessary change them all as things
    move around. […] But if you create such a structure on a
    transitory basis, […], then you will pay the cost much more
    often.
    http://msdn.microsoft.com/en-us/library/ms973837.aspx#dotnetgcbasics_topic2

    View Slide

  32. IRepository orderRepository =
    container.Resolve>();
    Order order = orderRepository.Get(35);
    This is what you think you are doing…

    View Slide

  33. …but if you think about it a bit more…

    View Slide

  34. ...this is what you are actually doing!
    IRepository repository =
    new ValidatingOrderRepository (
    new SecurityRepository (
    new LoggingRepository (
    new CachingRepository (
    new NHibernateRepository ()
    )
    )
    )
    );
    Order order = repository.Get(35);

    View Slide

  35. View Slide

  36. View Slide

  37. View Slide

  38. 馄饨!

    View Slide

  39. Few projects :-)

    View Slide

  40. Few projects :-) Few lines of code :-)

    View Slide

  41. Few projects :-) Few lines of code :-)
    Eeek! very few tests :-S

    View Slide

  42. Few projects :-) Few lines of code :-)
    Awesome community to help :-D
    Eeek! very few tests :-S

    View Slide

  43. YAGNI*.
    It works.
    * You Ain’t Gonna Need It!

    View Slide

  44. * Source http://bit.ly/1eSLr8Z

    View Slide

  45. View Slide

  46. View Slide

  47. View Slide

  48. Write libraries &
    open them to the world

    View Slide

  49. View Slide

  50. View Slide

  51. View Slide

  52. DEEP DIVES

    View Slide

  53. OPINIONATED
    HIRING

    View Slide

  54. REMOTE WORK

    View Slide

  55. • Performance is a feature
    • Always. Be. Shipping.
    • Use your circumstances.
    • Open source your libraries
    • 3 obscenely big monitors.
    CONCLUSION

    View Slide

  56. $_='@mk=uf=radimdp1Z--&ewxuhhl';tr/=1m-za-l@&Z/ !a-zP@\n/&print;
    Marco Cecconi
    @sklivvz
    [email protected]

    View Slide