Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Lessons learned defying Joel Spolsky with Django

Lessons learned defying Joel Spolsky with Django

Talk given at the Copenhagen Django meet up about our experiences porting Iconfinder from a LAMP stack to a de facto Python web stack.

9d198f37cdf56e90b09b5efd39b36d83?s=128

Nick Bruun

April 24, 2013
Tweet

Transcript

  1. Stayin sane while defyin Joel Spolsky with Djan o

  2. Hola, me llamo Nick nickbruun @nickbruun http://bruun.co/

  3. I’m the CTO of Iconfinder

  4. “… the sin le worst strate ic mistake that any

    software company can make: They decided to rewrite the code from scratch.” That’s Joel!
  5. September 2012 LAMP stack Linux, Apache, MySQL, PHP and memcached

    (dv) (-ish) 16 cores, 16 GiB RAM, death star
  6. Scientific chart required processin power Not so far future Sep

    2012 May 2007 0 limit of (dv)
  7. Restructure Rebuild

  8. Fast and flexible Two simple oals: (aka code base nirvana)

  9. None
  10. None
  11. None
  12. April 2013 3 M monthly visitors 3 M requests per

    day 30 GiB data in Post reSQL
  13. April 2013 09 PM Mon 22 03 AM 06 AM

    09 AM 12 PM 03 PM 0 ms 200 ms 400 ms 600 ms 800 ms 83ms avera e response time 329ms 99th percentile response time
  14. We survived! Lon story short… - and learned a few

    lessons in the process
  15. 1. ORMs are stupid.

  16. Watch it like a boss # In postgresql.conf: log_destination =

    'csvlog' log_directory = 'pg_log' logging_collector = on log_filename = 'postgres-%Y-%m-%d_%H%M%S' log_rotation_age = 1d log_rotation_size = 1GB log_min_duration_statement = 0ms log_checkpoints = on log_connections = on log_disconnections = on log_lock_waits = on log_temp_files = 0 dump every statement and its execution time
  17. Watch it like a boss $ tail -f "/usr/local/var/postgres/pg_log/ $(ls

    -1t /usr/local/var/postgres/pg_log/ | head -1)" Latest log file Follow the tail of the log file
  18. Did I say stupid? SomeModel.objects.filter( other = None ) SomeModel.objects.filter(

    other_id = None )
  19. Did I say stupid? SomeModel.objects.filter( other = None ) SomeModel.objects.filter(

    other_id = None ) same exact thing!
  20. Did I say stupid? SomeModel.objects.filter( other = None ) SomeModel.objects.filter(

    other_id = None ) SELECT "myapp_somemodel"."id", "myapp_somemodel"."arbitrary", "myapp_somemodel"."other_id" FROM "myapp_somemodel" LEFT OUTER JOIN "myapp_othermodel" ON ( "myapp_somemodel"."other_id" = "myapp_othermodel"."id" ) WHERE "myapp_othermodel"."id" IS NULL SELECT "myapp_somemodel"."id", "myapp_somemodel"."arbitrary", "myapp_somemodel"."other_id" FROM "myapp_somemodel" WHERE "myapp_somemodel"."other_id" IS NULL
  21. None
  22. ORMs are leaky abstractions count_by_name = {m.name: m.other_set.count() for m

    in SomeModel.objects.all()} count_by_names = {m['name']: m['num_other'] for m in SomeModel.objects.all().annotate( num_other=Count('other') ).values('name', 'num_other')} If we don’t want to waste resources like this: The essence of SQL leaks throu h:
  23. I, for one, welcome our SQL overlords

  24. 2. Monitor everythin .

  25. Graphite Sentry

  26. No, your lo file probably isn’t ood enou h.

  27. build stuff Side effect: awesome deployment workflow

  28. build stuff deploy Side effect: awesome deployment workflow

  29. build stuff deploy watch exceptions Side effect: awesome deployment workflow

  30. build stuff deploy watch exceptions Side effect: awesome deployment workflow

  31. 3. Don’t assume. Profile.

  32. Hot startup optimization strate y

  33. Hot startup optimization strate y 1. Database indexes

  34. Hot startup optimization strate y 1. Database indexes 2. Pa

    e cachin
  35. Hot startup optimization strate y 1. Database indexes 2. Pa

    e cachin 3. Database relationship denormalization
  36. Hot startup optimization strate y 1. Database indexes 2. Pa

    e cachin 3. Database relationship denormalization 4. Fine rained cachin
  37. Hot startup optimization strate y 1. Database indexes 2. Pa

    e cachin 3. Database relationship denormalization 4. Fine rained cachin 5. Throw money at it
  38. Hot startup optimization strate y 1. Database indexes 2. Pa

    e cachin 3. Database relationship denormalization 4. Fine rained cachin 5. Throw money at it 6. Go to step 5
  39. None
  40. 91ms ori inal response time

  41. “… the sin le worst strate ic mistake that any

    software company can make: They decided to rewrite the code from scratch.” 87.91 % of time spent renderin
  42. Japanese temple

  43. 29ms response time with Jinja2 91ms ori inal response time

  44. “… the sin le worst strate ic mistake that any

    software company can make: They decided to rewrite the code from scratch.” 29.16 % of time spent escapin HTML
  45. 20ms response time with Jinja2 without auto-escapin 29ms response time

    with Jinja2 91ms ori inal response time
  46. 4. Everythin prepacka ed isn’t ood, and everythin ood isn’t

    prepacka ed.
  47. Watch your transactions Transactions

  48. Watch your transactions Transactions Cachin +

  49. Watch your transactions Transactions Back round tasks + Cachin +

  50. Watch your transactions Transactions = ANGER!!1 Back round tasks +

    Cachin +
  51. Enqueuin tasks in transactions Request Back round worker

  52. Enqueuin tasks in transactions Request Back round worker Be in

    transaction
  53. Enqueuin tasks in transactions Request Back round worker Be in

    transaction Create m odel
  54. Enqueuin tasks in transactions Request Back round worker Be in

    transaction Create m odel Add task to queue
  55. Dequeue task from queue Enqueuin tasks in transactions Request Back

    round worker Be in transaction Create m odel Add task to queue
  56. Dequeue task from queue Enqueuin tasks in transactions Request Back

    round worker Be in transaction Create m odel Add task to queue Get m odel
  57. Dequeue task from queue Enqueuin tasks in transactions Request Back

    round worker Be in transaction Create m odel Add task to queue Get m odel raises M odel.DoesNotExist
  58. Dequeue task from queue Enqueuin tasks in transactions Request Back

    round worker Be in transaction Create m odel Add task to queue Com m it transaction Get m odel raises M odel.DoesNotExist
  59. With djan o-celery-transactions Request Back round worker

  60. With djan o-celery-transactions Request Back round worker Be in transaction

  61. With djan o-celery-transactions Request Back round worker Be in transaction

    Create m odel
  62. With djan o-celery-transactions Request Back round worker Be in transaction

    Create m odel Locally add task to queue
  63. With djan o-celery-transactions Request Back round worker Be in transaction

    Create m odel Locally add task to queue Com m it transaction
  64. With djan o-celery-transactions Request Back round worker Be in transaction

    Create m odel Locally add task to queue Com m it transaction Actually add task to queue
  65. Dequeue task from queue With djan o-celery-transactions Request Back round

    worker Be in transaction Create m odel Locally add task to queue Com m it transaction Actually add task to queue
  66. Dequeue task from queue With djan o-celery-transactions Request Back round

    worker Be in transaction Create m odel Locally add task to queue Com m it transaction Get m odel Actually add task to queue
  67. Dequeue task from queue With djan o-celery-transactions Request Back round

    worker Be in transaction Create m odel Locally add task to queue Com m it transaction Get m odel Actually add task to queue Finish task
  68. is awesome

  69. is a trap!

  70. are a trap! Frameworks

  71. Thanks! It’s been awesome listenin to me.