Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Setting Up New Relic's Infrastructure

B5b0f56c5edce4c54f1fea72e31d8378?s=47 Nic Benders
October 25, 2013

Setting Up New Relic's Infrastructure

Benefit from our mistakes and hindsight! There's a lot that goes into running a SaaS company - from continuous deployment to DNS to who gets to play with production. We've been running New Relic for over five years now, but if we had to do it over again there are a few things we'd do differently. But there's also a couple things we'd repeat.


Nic Benders

October 25, 2013


  1. Setting up New Relic's Infrastructure: A bunch of things we

    did wrong and a few we did right. DAVE PETERSON & NIC BENDERS 2013-10-25
  2. David Peterson (dave@newrelic.com) Nic Benders (nic@newrelic.com)

  3. [“star” graph of code base at launch]

  4. None
  5. 100,000req/sec 2Gbps 200TB

  6. Who Does What?

  7. Site Operations != IT

  8. You are not Batman Do your work during the day

  9. Lowest hour is still 60% of peak hour!

  10. Put your servers on the moon

  11. Never open the box

  12. Domain names are cheap

  13. 2 4 8 16 32 64 128 256 512 1024 2048 4096 8192 16384 32768 65536 Private IPs are FREE
  14. Hardware

  15. System Management

  16. X6550 E5-2450 E5-2643 E5-2690 Average response time 137 ms 167

    ms 158 ms 189 ms
  17. 25% performance improvement 2x 6-Core X6550 @ 2.67 GHz 2x

    8-Core E5-2690 @ 2.90 GHz
  18. Storage √

  19. Databases

  20. 32-bits isn't enough class CreateTribbles < ActiveRecord::Migration def change create_table

    :tribbles do |t| t.boolean :hungry t.string :color t.references :parent, index: true end end end
  21. 32-bits isn't enough `id` int(11) NOT NULL AUTO_INCREMENT `id` bigint(20)

  22. Your database is not a queue

  23. Long running queries will kill you

  24. And now, a thanks to the people who are out

    there every day fixing the mistakes that we made:
  25. Jonathan Owens Jonathan Thurman Aaron Bento Sean Kane Paul Showalter

    Karl Matthias