Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Behind the scenes of a grown-up web application

Behind the scenes of a grown-up web application

Behind the scenes of a grown-up web application - YAPC::EU 2014, Sofia, Bulgaria

Kerstin Puschke

August 24, 2014
Tweet

More Decks by Kerstin Puschke

Other Decks in Technology

Transcript

  1. Behind the
    scenes of a
    grown-up web
    application
    Kerstin Puschke
    YAPC::EU 2014

    View Slide

  2. Who am I
    Contact
    •  http://www.kpuschke.eu
    •  twitter: @titanoboa42
    •  https://www.xing.com/profile/
    Kerstin_Puschke
    software engineer
    at XING Hamburg
    •  xing.com
    social network for business
    professionals
    •  about 14M users
    •  debian, perl, javascript, ruby on
    rails, mysql, redis, riak, . . .
    We’re hiring!
    http://corporate.xing.com/
    english/company/careers-at-xing/
    August 14 // Behind the scenes of a grown-up web application // Kerstin Puschke // YAPC::EU 2014 2

    View Slide

  3. Outline
    August 14 // Behind the scenes of a grown-up web application // Kerstin Puschke // YAPC::EU 2014 3
    •  Technology & Architecture
    •  programming languages, databases, storage,...
    •  applications, APIs & asynchronous communication
    •  Development Process and Tools
    •  development and test environments
    •  version control
    •  continous integration, deployment

    View Slide

  4. How it all began
    10 years ago - Perl & MySQL
    August 14 // Behind the scenes of a grown-up web application // Kerstin Puschke // YAPC::EU 2014 4

    View Slide

  5. The camel has been growing
    Today - Perl & MySQL & more
    August 14 // Behind the scenes of a grown-up web application // Kerstin Puschke // YAPC::EU 2014 5

    View Slide

  6. xing.com
    August 14 // Behind the scenes of a grown-up web application // Kerstin Puschke // YAPC::EU 2014 6
    perl
    rails
    rails
    rails
    mobile
    ...

    View Slide

  7. Backend
    •  Perl
    •  Ruby on Rails
    •  Scala
    •  Hadoop
    •  Elastic Search
    •  Insect / Arachnid
    August 14 // Behind the scenes of a grown-up web application // Kerstin Puschke // YAPC::EU 2014 7

    View Slide

  8. Frontend
    •  HTML
    •  Javascript
    •  jquery
    •  backbone
    August 14 // Behind the scenes of a grown-up web application // Kerstin Puschke // YAPC::EU 2014 8

    View Slide

  9. Storage, Databases & Cache
    •  MySQL
    •  Redis
    •  Memcache
    •  Swift / Open Stack
    •  Riak
    August 14 // Behind the scenes of a grown-up web application // Kerstin Puschke // YAPC::EU 2014 9

    View Slide

  10. Perl „core“
    August 14 // Behind the scenes of a grown-up web application // Kerstin Puschke // YAPC::EU 2014 10
    perl
    „core“
    rails
    rails
    rails
    mobile
    ...

    View Slide

  11. Rails
    August 14 // Behind the scenes of a grown-up web application // Kerstin Puschke // YAPC::EU 2014 11
    perl
    „core“
    rails
    jobs
    rails
    ...
    rails
    events
    mobile
    rails
    ...
    rails
    ~ 20 apps

    View Slide

  12. Communication
    •  REST API
    •  RabbitMQ AMQP
    •  plus Beetle
    August 14 // Behind the scenes of a grown-up web application // Kerstin Puschke // YAPC::EU 2014 12
    perl
    „core“
    rails
    jobs
    rails
    public API
    rails
    events
    mobile
    rails
    ...

    View Slide

  13. The camel has been growing...
    August 14 // Behind the scenes of a grown-up web application // Kerstin Puschke // YAPC::EU 2014 13

    View Slide

  14. Dissolving Profile
    August 14 // Behind the scenes of a grown-up web application // Kerstin Puschke // YAPC::EU 2014 14
    perl
    „core“
    rails
    rails
    (public API)
    rails
    mobile
    rails profile
    frontend

    View Slide

  15. Shadow calls
    August 14 // Behind the scenes of a grown-up web application // Kerstin Puschke // YAPC::EU 2014 15
    old app
    new app user request
    worker amqp message
    shadow call

    View Slide

  16. Still at work
    August 14 // Behind the scenes of a grown-up web application // Kerstin Puschke // YAPC::EU 2014 16

    View Slide

  17. Perl App
    August 14 // Behind the scenes of a grown-up web application // Kerstin Puschke // YAPC::EU 2014 17
    •  ~600k lines of code, ~300k lines of perl code

    View Slide

  18. Perl App
    August 14 // Behind the scenes of a grown-up web application // Kerstin Puschke // YAPC::EU 2014 18
    •  ~30M external requests / day

    View Slide

  19. Perl App
    August 14 // Behind the scenes of a grown-up web application // Kerstin Puschke // YAPC::EU 2014 19
    •  ~360M internal REST requests / day, 6k / second

    View Slide

  20. Perl App
    August 14 // Behind the scenes of a grown-up web application // Kerstin Puschke // YAPC::EU 2014 20
    DC1
    DC2
    2 x 6
    application server
    2 x 18
    API Server
    2 x 4
    worker server

    View Slide

  21. Taking care
    August 14 // Behind the scenes of a grown-up web application // Kerstin Puschke // YAPC::EU 2014 21

    View Slide

  22. Office Locations
    August 14 // Behind the scenes of a grown-up web application // Kerstin Puschke // YAPC::EU 2014 22
    Hamburg, Munich, Vienna, Barcelona
    ~520 people from ~28 countries, ~80 engineers

    View Slide

  23. Provisioning Development VMs
    August 14 // Behind the scenes of a grown-up web application // Kerstin Puschke // YAPC::EU 2014 23
    •  Vagrant & Chef

    View Slide

  24. Version Control
    August 14 // Behind the scenes of a grown-up web application // Kerstin Puschke // YAPC::EU 2014 24
    •  Github >1800 repositories

    View Slide

  25. Continuous Integration & Deployment
    August 14 // Behind the scenes of a grown-up web application // Kerstin Puschke // YAPC::EU 2014 25
    •  Jenkins CI ~30 projects, ~250 build jobs
    •  weekly to continuous deployment

    View Slide

  26. Monitoring
    August 14 // Behind the scenes of a grown-up web application // Kerstin Puschke // YAPC::EU 2014 26
    •  logjam

    View Slide

  27. Monitoring
    August 14 // Behind the scenes of a grown-up web application // Kerstin Puschke // YAPC::EU 2014 27
    •  graphite

    View Slide

  28. Monitoring
    August 14 // Behind the scenes of a grown-up web application // Kerstin Puschke // YAPC::EU 2014 28
    •  cube

    View Slide

  29. famous last words
    die "WTF?"; # should not be possible
    August 14 // Behind the scenes of a grown-up web application // Kerstin Puschke // YAPC::EU 2014 29

    View Slide

  30. The professional network
    www.xing.com
    Thank you
    for your
    attention!

    View Slide

  31. Who am I
    Contact
    •  http://www.kpuschke.eu
    •  twitter: @titanoboa42
    •  https://www.xing.com/profile/
    Kerstin_Puschke
    software engineer
    at XING Hamburg
    •  xing.com
    social network for business
    professionals
    •  about 14M users
    •  debian, perl, javascript, ruby on
    rails, mysql, redis, riak, . . .
    We’re hiring!
    http://corporate.xing.com/
    english/company/careers-at-xing/
    August 14 // Behind the scenes of a grown-up web application // Kerstin Puschke // YAPC::EU 2014 31

    View Slide

  32. Credits
    Map
    •  The map on slide 21 was created as a screenshot from
    http://umap.openstreetmap.fr
    •  Map background credits
    Outdoors (OSM) Tiles © Gravitystorm / map data OpenStreetMap
    August 14 // Behind the scenes of a grown-up web application // Kerstin Puschke // YAPC::EU 2014 32

    View Slide