Using Ruby & Rails in the wild - Gogobot use case

B7d890bed68fa564c18ff00dfd8207cd?s=47 Avi Tzurel
September 10, 2012

Using Ruby & Rails in the wild - Gogobot use case

Talk I gave at an Israeli meetup about Ruby and Rails vs other languages/frameworks.

B7d890bed68fa564c18ff00dfd8207cd?s=128

Avi Tzurel

September 10, 2012
Tweet

Transcript

  1. ݆ૉ Ruby & Rails In the wild Gogobot use-case Avi

    Tzurel Gogobot Monday, September 10, 12
  2. Who Am I? Avi Tzurel http://avi.io http://twitter.com/kensodev http://kensodev.com http://github.com/kensodev Monday,

    September 10, 12
  3. What is Gogobot? Monday, September 10, 12

  4. Social Recommendation Engine For Travel Monday, September 10, 12

  5. Personalized Recommendations From Friends, not strangers Monday, September 10, 12

  6. The Gogobot Architecture Monday, September 10, 12

  7. How we make shit work? Monday, September 10, 12

  8. CDN + Reverse Proxy Monday, September 10, 12

  9. CDN • Edge servers across the world (hundreds) • Caches

    static assets • CSS, JS, Images • Caches full pages (with smart expire API) Monday, September 10, 12
  10. Reverse Proxy • Super fast connection from edges means users

    get a local experience, with minimal latency • Cache miss? get from load balancer • Logged out traffic almost never hits or affects logged in traffic • scale differently, control differently Monday, September 10, 12
  11. CDN + Reverse Proxy Monday, September 10, 12

  12. CDN + Reverse Proxy Front End Back End Monday, September

    10, 12
  13. Front End • Serves user facing content • Communicates with

    cache layer and the DB • No heavy lifting, respond to the user as fast as possible Monday, September 10, 12
  14. Back End • Serves realtime services (Facebook, Twitter) • Heavy

    lifting thrown at it from the Front End Machines • Hosts workers for the queue service Monday, September 10, 12
  15. CDN + Reverse Proxy Front End Back End Monday, September

    10, 12
  16. Front End Back End Cache Monday, September 10, 12

  17. Cache • Memcached cluster • Memcached 1.4+ • 6+ Machines

    • 4.5 - 15K operations per second • Hosted on Amazon ElastiCache • Solved tons of problems with memcached dying Monday, September 10, 12
  18. Front End Back End Cache Monday, September 10, 12

  19. Cache MySql MongoDB Redis SOLR Monday, September 10, 12

  20. MySql • Master + 2 Slaves • 64G memory for

    each machine with 400G storage • Hourly backups • Used as the main persistence layer for the site • Reads are from slaves, writes are from master • Logged out traffic will never have access the master Monday, September 10, 12
  21. MySql • EBS snapshots are used as backups • Multi

    region support for Amazon (1a, 1b, 1c) Monday, September 10, 12
  22. MongoDB • 6 Shard • 3 Replica sets in each

    shards • 16HD (100G) raid on each machine • 64G memory for each machine to provide out of memory index for all queries • Used for the Graph Engine + Scoring system Monday, September 10, 12
  23. Monday, September 10, 12

  24. Redis • Used for key+value store • Cache tagging solution

    on top of memcached • Queue services is hosted on Redis • Master / Slave replica • different redis cluster for different things Monday, September 10, 12
  25. Redis • Different Redis clusters for each need • Indexer

    • Cache tagging • Realtime push with Node.js to the client • When one down, others behave normally Monday, September 10, 12
  26. SOLR • Search index • NoSQL (Schema Less) • Master/Slave

    • Slave on each app machine, single master • Eventual consistency Monday, September 10, 12
  27. Numbers • 400m+ graph users • 10K triggers per user

    in the scoring system • Grew ~170X last 18 months • Announced 1m registered users a month ago • hit 2m registered users 2 weeks agter Monday, September 10, 12
  28. Numbers • HUGE growth in a relatively short time Monday,

    September 10, 12
  29. 99.9% uptime Monday, September 10, 12

  30. ~200ms avg server response time Monday, September 10, 12

  31. Logged out user gets a page in 30ms Monday, September

    10, 12
  32. How can you manage all of this? Monday, September 10,

    12
  33. Not too technical Monday, September 10, 12

  34. Full Ruby & Rails shop Monday, September 10, 12

  35. Ruby will increase developer happiness (That’s what it was designed

    for) Monday, September 10, 12
  36. Community Drives you towards Best Practices Monday, September 10, 12

  37. TDD Monday, September 10, 12

  38. “Sometimes I want to do Ruby because it means I’m

    more likely to be able to TDD.” Greg Malcolm Monday, September 10, 12
  39. Continuos Integration Monday, September 10, 12

  40. Continuous Deployment Monday, September 10, 12

  41. Monday, September 10, 12

  42. Machine Configuration using Chef Monday, September 10, 12

  43. SOA Application with Rails Engines Monday, September 10, 12

  44. EVERYTHING IS OPEN SOURCE! Monday, September 10, 12

  45. Laptop Monday, September 10, 12

  46. Perform Later Monday, September 10, 12

  47. Rails fixes & features Monday, September 10, 12

  48. Resque Monday, September 10, 12

  49. Methodologies and sharing ideas Monday, September 10, 12

  50. CI Configuration Monday, September 10, 12

  51. Full pages crawler Monday, September 10, 12

  52. and more... http://github.com/gogobot http://github.com/kensodev Monday, September 10, 12

  53. Super small team Monday, September 10, 12

  54. Gets a lot done Monday, September 10, 12

  55. No QA! Monday, September 10, 12

  56. Rails is awesome Monday, September 10, 12

  57. “Ruby on Rails is astounding. Using it is like watching

    a kung-fu movie, where a dozen bad-ass frameworks prepare to beat up the little newcomer only to be handed their asses in a variety of imaginative ways.” Nathan Tarkington, O'Reilly Publishing Monday, September 10, 12
  58. No other language / framework is able to do such

    things Monday, September 10, 12
  59. Best community in the world Monday, September 10, 12

  60. Tons of open source projects Monday, September 10, 12

  61. Github Monday, September 10, 12

  62. Heroku EngineYard Amazon Monday, September 10, 12

  63. Thank you! Monday, September 10, 12

  64. Questions? Monday, September 10, 12