Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Memory Leaks, Tweaks, and Techniques

Memory Leaks, Tweaks, and Techniques

Richard Schneeman

November 07, 2015
Tweet

More Decks by Richard Schneeman

Other Decks in Programming

Transcript



  1. Memory
    Leaks
    2015
    @schneems

    View Slide

  2. Leaks are bad

    View Slide

  3. View Slide

  4. View Slide

  5. View Slide

  6. View Slide

  7. Obviously
    memory leaks
    are bad

    View Slide

  8. Why?

    View Slide

  9. First…

    View Slide

  10. They
    Call me
    @Schneems

    View Slide

  11. View Slide

  12. View Slide

  13. “Favor
    composition
    over
    inheritance

    View Slide

  14. Golang
    rulz
    Elixr
    4 lyfe

    View Slide

  15. Rails
    Commit
    Bit

    View Slide

  16. View Slide

  17. View Slide

  18. View Slide

  19. View Slide

  20. I like
    working
    with
    wood

    View Slide

  21. View Slide

  22. Plan
    for
    seasonal
    wood
    movement

    View Slide

  23. Why is
    memory
    important?

    View Slide

  24. RAM is a
    limited
    resource

    View Slide

  25. When you run
    out, the OS
    starts using
    the HD

    View Slide

  26. Hard Drive



    View Slide

  27. RAM

    View Slide

  28. swap

    View Slide

  29. When you run
    out of room…

    View Slide

  30. View Slide

  31. View Slide

  32. Performance
    Profiling

    View Slide

  33. Start with a
    known

    View Slide

  34. Repro the
    performance
    problem

    View Slide

  35. #include
    void f(void)
    {
    int* x = malloc(10 *sizeof(int));
    x[10] = 0;
    }
    int main(void) {
    f();
    return 0;
    }

    View Slide

  36. #include
    void f(void)
    {
    int* x = malloc(10 *sizeof(int));
    x[10] = 0;
    }
    int main(void) {
    f();
    return 0;
    }

    View Slide

  37. Find the right
    tools

    View Slide

  38. $ valgrind --tool=memcheck ./memoryleak
    ==31747== Memcheck, a memory error detector
    ==31747== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
    ==31747== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info
    ==31747== Command: ./memoryleak
    ==31747==
    ==31747== Invalid write of size 4
    ==31747== at 0x100000F4C: f (memoryleak.c:6)
    ==31747== by 0x100000F73: main (memoryleak.c:10)
    ==31747== Address 0x100ae34a8 is 0 bytes after a block of size 40 alloc'd
    ==31747== at 0x100008EBB: malloc (in /usr/local/Cellar/valgrind/3.11.0/lib/
    valgrind/vgpreload_memcheck-amd64-darwin.so)
    ==31747== by 0x100000F43: f (memoryleak.c:5)
    ==31747== by 0x100000F73: main (memoryleak.c:10)
    ==31747==
    ==31747==
    ==31747== HEAP SUMMARY:
    ==31747== in use at exit: 35,304 bytes in 431 blocks
    ==31747== total heap usage: 503 allocs, 72 frees, 41,272 bytes allocated
    ==31747==
    ==31747== LEAK SUMMARY:
    ==31747== definitely lost: 40 bytes in 1 blocks
    ==31747== indirectly lost: 0 bytes in 0 blocks
    ==31747== possibly lost: 0 bytes in 0 blocks
    ==31747== still reachable: 0 bytes in 0 blocks
    ==31747== suppressed: 35,264 bytes in 430 blocks
    ==31747== Rerun with --leak-check=full to see details of leaked memory
    ==31747==
    ==31747== For counts of detected and suppressed errors, rerun with: -v
    ==31747== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)

    View Slide

  39. $ valgrind --tool=memcheck ./memoryleak
    ==31747== Memcheck, a memory error detector
    ==31747== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
    ==31747== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info
    ==31747== Command: ./memoryleak
    ==31747==
    ==31747== Invalid write of size 4
    ==31747== at 0x100000F4C: f (memoryleak.c:6)
    ==31747== by 0x100000F73: main (memoryleak.c:10)
    ==31747== Address 0x100ae34a8 is 0 bytes after a block of size 40 alloc'd
    ==31747== at 0x100008EBB: malloc (in /usr/local/Cellar/valgrind/3.11.0/lib/
    valgrind/vgpreload_memcheck-amd64-darwin.so)
    ==31747== by 0x100000F43: f (memoryleak.c:5)
    ==31747== by 0x100000F73: main (memoryleak.c:10)
    ==31747==
    ==31747==
    ==31747== HEAP SUMMARY:
    ==31747== in use at exit: 35,304 bytes in 431 blocks
    ==31747== total heap usage: 503 allocs, 72 frees, 41,272 bytes allocated
    ==31747==
    ==31747== LEAK SUMMARY:
    ==31747== definitely lost: 40 bytes in 1 blocks
    ==31747== indirectly lost: 0 bytes in 0 blocks
    ==31747== possibly lost: 0 bytes in 0 blocks
    ==31747== still reachable: 0 bytes in 0 blocks
    ==31747== suppressed: 35,264 bytes in 430 blocks
    ==31747== Rerun with --leak-check=full to see details of leaked memory
    ==31747==
    ==31747== For counts of detected and suppressed errors, rerun with: -v
    ==31747== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)

    View Slide

  40. $ valgrind --tool=memcheck ./memoryleak
    ==31747== Memcheck, a memory error detector
    ==31747== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
    ==31747== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info
    ==31747== Command: ./memoryleak
    ==31747==
    ==31747== Invalid write of size 4
    ==31747== at 0x100000F4C: f (memoryleak.c:6)
    ==31747== by 0x100000F73: main (memoryleak.c:10)
    ==31747== Address 0x100ae34a8 is 0 bytes after a block of size 40 alloc'd
    ==31747== at 0x100008EBB: malloc (in /usr/local/Cellar/valgrind/3.11.0/lib/
    valgrind/vgpreload_memcheck-amd64-darwin.so)
    ==31747== by 0x100000F43: f (memoryleak.c:5)
    ==31747== by 0x100000F73: main (memoryleak.c:10)
    ==31747==
    ==31747==
    ==31747== HEAP SUMMARY:
    ==31747== in use at exit: 35,304 bytes in 431 blocks
    ==31747== total heap usage: 503 allocs, 72 frees, 41,272 bytes allocated
    ==31747==
    ==31747== LEAK SUMMARY:
    ==31747== definitely lost: 40 bytes in 1 blocks
    ==31747== indirectly lost: 0 bytes in 0 blocks
    ==31747== possibly lost: 0 bytes in 0 blocks
    ==31747== still reachable: 0 bytes in 0 blocks
    ==31747== suppressed: 35,264 bytes in 430 blocks
    ==31747== Rerun with --leak-check=full to see details of leaked memory
    ==31747==
    ==31747== For counts of detected and suppressed errors, rerun with: -v
    ==31747== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)

    View Slide

  41. Apply the
    tools to your
    App

    View Slide

  42. How
    does
    Ruby
    Memory?

    View Slide

  43. Ruby is a
    “managed”
    Language

    View Slide

  44. Don’t manually
    allocate, use
    Garbage
    Collector
    instead

    View Slide

  45. 2 causes for
    increased
    memory use

    View Slide

  46. Retained
    Objects
    1)

    View Slide

  47. RETAINED = []
    1.upto(100_000_000).each do |i|
    RETAINED << “#{i}”
    end

    View Slide

  48. 7.32 GB

    View Slide

  49. WHY?

    View Slide

  50. Memory

    View Slide

  51. Memory
    Object slots

    View Slide

  52. Memory
    “1” “2” “3”“4” “5” “6” “7”
    RETAINED = []
    1.upto(100_000_000).each do |i|
    RETAINED << “#{i}”
    end

    View Slide

  53. Out of slots!
    Allocate more
    memory

    View Slide

  54. Memory
    “1” “2” “3”“4” “5” “6” “7”
    “10”
    “8” “9” “11” “12” “13” “14”

    View Slide

  55. Allocated
    Objects
    2)

    View Slide

  56. 1.upto(100_000_000).each do |i|
    puts "#{i}"
    end

    View Slide

  57. 21.8 MB

    View Slide

  58. Memory
    “1” “2” “3”“4” “5” “6” “7”

    View Slide

  59. Out of slots!
    Run GC!

    View Slide

  60. Memory
    “2” “3”“4” “5” “6” “7”
    1.upto(100_000_000).each do |i|
    puts "#{i}"
    end
    “1”

    View Slide

  61. Memory
    “2” “3”“4” “5” “6” “7”
    1.upto(100_000_000).each do |i|
    puts "#{i}"
    end

    View Slide

  62. Memory
    “8” “3”“4” “5” “6” “7”
    1.upto(100_000_000).each do |i|
    puts "#{i}"
    end
    “2”

    View Slide

  63. GC will save
    us!

    View Slide

  64. def make_objects
    array = []
    1.upto(100_000_000).each do |i|
    array << "#{i}"
    end
    return nil
    end
    make_objects
    GC.start(full_mark: true,
    immediate_sweep: true)

    View Slide

  65. def make_objects
    array = []
    1.upto(100_000_000).each do |i|
    array << "#{i}"
    end
    return nil
    end
    make_objects
    GC.start(full_mark: true,
    immediate_sweep: true)

    View Slide

  66. def make_objects
    array = []
    1.upto(100_000_000).each do |i|
    array << "#{i}"
    end
    return nil
    end
    make_objects
    GC.start(full_mark: true,
    immediate_sweep: true)

    View Slide

  67. Result?

    View Slide

  68. 7.32 GB

    View Slide

  69. WAT

    View Slide

  70. GC.start(full_mark: true, immediate_sweep: true)
    GC.disable
    # Code goes here
    before = GC.stat[:total_freed_objects]
    GC.enable
    GC.start(full_mark: true, immediate_sweep: true)
    after = GC.stat[:total_freed_objects]
    # => “Objects cleared by GC: 100_010_687”

    View Slide

  71. Cannot open
    slots while
    reference
    exists

    View Slide

  72. After
    program runs

    View Slide

  73. Memory
    “1” “2” “3”“4” “5” “6” “7”
    “10”
    “8” “9” “11” “12” “13” “14”

    View Slide

  74. Run
    GC

    View Slide

  75. Memory
    “1” “2” “3”“4” “5” “6” “7”
    “10”
    “8” “9” “11” “12” “13” “14”

    View Slide

  76. Ruby
    releases
    memory
    SLOWLY

    View Slide

  77. Memory

    View Slide

  78. Test
    Everything
    (even these
    slides)

    View Slide

  79. How does this
    affect my
    Rails app?

    View Slide

  80. Retained
    App

    View Slide

  81. Retained
    Allocated
    App

    View Slide

  82. Mem
    Usage
    App

    View Slide

  83. Request
    Starts

    View Slide

  84. Request Full
    .

    View Slide

  85. Request Full
    Full
    ..

    View Slide

  86. Request Full
    Full
    Full
    ...

    View Slide

  87. Request Full
    Full
    Full
    Full
    ....

    View Slide

  88. Request Full
    Full
    Full
    Full
    Full
    ......

    View Slide

  89. Out of
    memory
    Full
    Full
    Full
    Full
    Full

    View Slide

  90. Run GC,
    All objs
    needed
    0_o
    Full
    Full
    Full
    Full
    Full

    View Slide

  91. Allocate
    mem
    Full
    Full
    Full
    Full
    Full

    View Slide

  92. Request
    Done
    Full
    Full
    Full
    Full
    Full
    ........

    View Slide

  93. Eventually

    View Slide

  94. GC
    Objs
    not used
    Full
    Full
    Full
    Full
    Full

    View Slide

  95. GC
    Objs
    not used

    View Slide

  96. Memory
    increases even
    without
    object retention

    View Slide

  97. Object creation/
    deletion
    takes
    RAM

    View Slide

  98. Object creation/
    deletion
    takes
    Time

    View Slide

  99. Further
    Reading:

    View Slide

  100. Generational
    GC

    View Slide

  101. What is a
    memory leak?

    View Slide

  102. What is a
    memory leak?

    View Slide

  103. #include
    void f(void)
    {
    int* x = malloc(10 *sizeof(int));
    x[10] = 0;
    }
    int main(void) {
    f();
    return 0;
    }

    View Slide

  104. We lost the
    reference

    View Slide

  105. We can never
    free() that
    memory

    View Slide

  106. RETAINED = []
    1.upto(100_000_000).each do |i|
    RETAINED << “#{i}”
    end

    View Slide

  107. We retained
    the reference

    View Slide

  108. We can never
    GC that
    memory

    View Slide

  109. Apply the
    tools to your
    App

    View Slide

  110. Do you have a
    memory leak?

    View Slide

  111. View Slide

  112. View Slide

  113. Ruby memory
    should
    plateau

    View Slide

  114. Do you have a
    memory leak?

    View Slide

  115. No

    View Slide

  116. Memory Bloat

    View Slide

  117. Real life is
    messy

    View Slide

  118. Easiest way
    to stop a
    memory bloat
    ???

    View Slide

  119. View Slide

  120. Puma
    Worker
    Killer

    View Slide

  121. Rolling
    Restarts

    View Slide

  122. Memory
    Triggered
    Restarts

    View Slide

  123. Doesn’t work
    with
    containers

    View Slide

  124. Puma
    Worker
    Killer

    View Slide

  125. Concurrency
    via
    processes

    View Slide

  126. Reduce
    “worker”
    count to
    reduce RAM

    View Slide

  127. Hardest way
    to stop
    memory bloat
    ???

    View Slide

  128. Debugging
    and
    Introspection

    View Slide

  129. Who has
    heard of
    derailed?

    View Slide

  130. derailed_benchmarks

    View Slide

  131. Memory
    at
    Boot

    View Slide

  132. $ bundle exec derailed bundle:mem
    TOP: 54.1836 MiB
    mail: 18.9688 MiB
    mime/types: 17.4453 MiB
    mail/field: 0.4023 MiB
    mail/message: 0.3906 MiB
    action_view/view_paths: 0.4453 MiB
    action_view/base: 0.4336 MiB

    View Slide

  133. Memory
    at
    Runtime

    View Slide

  134. $ bundle exec derailed exec perf:objects
    # ...
    ## allocated memory by location
    ---------------------------------------
    1935548 /Users/richardschneeman/.gem/ruby/2.2.3/gems/actionpack-4.2.3/lib/action_dispa
    896100 /Users/richardschneeman/.gem/ruby/2.2.3/gems/pg-0.16.0/lib/pg/result.rb:10
    741488 /Users/richardschneeman/.gem/ruby/2.2.3/gems/activerecord-4.2.3/lib/active_rec
    689299 /Users/richardschneeman/.gem/ruby/2.2.3/gems/activesupport-4.2.3/lib/active_su
    660672 /Users/richardschneeman/.gem/ruby/2.2.3/gems/actionpack-4.2.3/lib/action_dispa
    606384 /Users/richardschneeman/Documents/projects/codetriage/app/views/repos/\_repo.h
    579384 /Users/richardschneeman/.gem/ruby/2.2.3/gems/activesupport-4.2.3/lib/active\_s
    532800 /Users/richardschneeman/.gem/ruby/2.2.3/gems/actionpack-4.2.3/lib/action_dispa
    391392 /Users/richardschneeman/.gem/ruby/2.2.3/gems/activerecord-4.2.3/lib/active_rec
    385920 /Users/richardschneeman/.gem/ruby/2.2.3/gems/temple-0.7.5/lib/temple/utils.rb:

    View Slide

  135. Case Study

    View Slide

  136. View Slide

  137. View Slide

  138. View Slide

  139. $ heroku pg:pull

    View Slide

  140. $ PATH_TO_HIT=rails/rails bundle exec derailed exec perf:objects
    Booting: production
    Running 1 times
    Total allocated: 21495556 bytes (207735 objects)
    Total retained: 15041423 bytes (133220 objects)
    allocated memory by gem
    -----------------------------------
    19427636 activerecord-4.2.3
    1471380 activesupport-4.2.3
    214048 actionview-4.2.3
    151066 actionpack-4.2.3
    121983 codetriage/app
    24308 will_paginate-3.0.7
    23193 rack-1.6.4
    21105 arel-6.0.3
    13397 ruby-2.2.3/lib
    5560 warden-1.2.3
    5536 rack-timeout-0.3.2
    3912 activemodel-4.2.3
    3000 hashie-3.4.2
    2154 devise-3.5.2
    1573 omniauth-1.2.2
    1536 i18n-0.7.0
    1495 railties-4.2.3
    787 temple-0.7.6

    View Slide

  141. 19427636
    bytes…

    View Slide

  142. 19.4
    MB

    View Slide

  143. Per

    View Slide

  144. Request

    View Slide

  145. allocated memory by location
    -----------------------------------
    5131624 /Users/richardschneeman/.gem/ruby/2.2.3/gems/activerec
    4616480 /Users/richardschneeman/.gem/ruby/2.2.3/gems/activerec
    1336408 /Users/richardschneeman/.gem/ruby/2.2.3/gems/activerec
    1336088 /Users/richardschneeman/.gem/ruby/2.2.3/gems/activerec
    1330904 /Users/richardschneeman/.gem/ruby/2.2.3/gems/activerec
    1325648 /Users/richardschneeman/.gem/ruby/2.2.3/gems/activerec
    921807 /Users/richardschneeman/.gem/ruby/2.2.3/gems/activesup
    737376 /Users/richardschneeman/.gem/ruby/2.2.3/gems/activerec
    596288 /Users/richardschneeman/.gem/ruby/2.2.3/gems/activerec
    460720 /Users/richardschneeman/.gem/ruby/2.2.3/gems/activerec

    View Slide

  146. ???

    View Slide

  147. allocated objects by class
    -----------------------------------
    108939 String
    35456 Hash
    21605 Array
    5890 Proc
    5886 RubyVM::Env
    5859 ActiveRecord::Attribute::FromDatabase
    5759 ActiveRecord::AttributeSet
    5759 ActiveRecord::LazyAttributeHash
    5731 ActiveRecord::Associations::BelongsToAssociation
    5731 Issue
    217 ActiveSupport::SafeBuffer
    173 MatchData
    112 ActionView::OutputBuffer
    59 Time
    43 ActiveRecord::Relation
    42 ActiveRecord::ConnectionAdapters::PostgreSQL::Name
    27 ActiveRecord::AssociationRelation
    27 User
    }

    View Slide

  148. allocated objects by class
    -----------------------------------
    108939 String
    35456 Hash
    21605 Array
    5890 Proc
    5886 RubyVM::Env
    5859 ActiveRecord::Attribute::FromDatabase
    5759 ActiveRecord::AttributeSet
    5759 ActiveRecord::LazyAttributeHash
    5731 ActiveRecord::Associations::BelongsToAssociation
    5731 Issue
    217 ActiveSupport::SafeBuffer
    173 MatchData
    112 ActionView::OutputBuffer
    59 Time
    43 ActiveRecord::Relation
    42 ActiveRecord::ConnectionAdapters::PostgreSQL::Name
    27 ActiveRecord::AssociationRelation
    27 User
    }

    View Slide

  149. allocated objects by class
    -----------------------------------
    108939 String
    35456 Hash
    21605 Array
    5890 Proc
    5886 RubyVM::Env
    5859 ActiveRecord::Attribute::FromDatabase
    5759 ActiveRecord::AttributeSet
    5759 ActiveRecord::LazyAttributeHash
    5731 ActiveRecord::Associations::BelongsToAssociation
    5731 Issue
    217 ActiveSupport::SafeBuffer
    173 MatchData
    112 ActionView::OutputBuffer
    59 Time
    43 ActiveRecord::Relation
    42 ActiveRecord::ConnectionAdapters::PostgreSQL::Name
    27 ActiveRecord::AssociationRelation
    27 User

    View Slide

  150. 27 Users
    }

    View Slide

  151. allocated objects by class
    -----------------------------------
    108939 String
    35456 Hash
    21605 Array
    5890 Proc
    5886 RubyVM::Env
    5859 ActiveRecord::Attribute::FromDatabase
    5759 ActiveRecord::AttributeSet
    5759 ActiveRecord::LazyAttributeHash
    5731 ActiveRecord::Associations::BelongsToAssociation
    5731 Issue
    217 ActiveSupport::SafeBuffer
    173 MatchData
    112 ActionView::OutputBuffer
    59 Time
    43 ActiveRecord::Relation
    42 ActiveRecord::ConnectionAdapters::PostgreSQL::Name
    27 ActiveRecord::AssociationRelation
    27 User

    View Slide

  152. NOT 5731 Issues
    }

    View Slide

  153. View Slide

  154. Started GET "/rails/rails" for ::1 at 2015-11-04 14:54:55 -0600
    ActiveRecord::SchemaMigration Load (19.2ms) SELECT "schema_migrations".* FROM "schema_migration
    Processing by ReposController#show as HTML
    Parameters: {"full_name"=>"rails/rails"}
    Repo Load (21.2ms) SELECT "repos".* FROM "repos" WHERE "repos"."full_name" = $1 LIMIT 1 [["fu
    source=rack-timeout id=22a6fea79c2f050a9d9999b007298d12 timeout=30000ms service=1006ms state=activ
    Issue Load (1091.1ms) SELECT "issues".* FROM "issues" WHERE "issues"."repo_id" IN (36)
    source=rack-timeout id=22a6fea79c2f050a9d9999b007298d12 timeout=30000ms service=2093ms state=activ
    source=rack-timeout id=22a6fea79c2f050a9d9999b007298d12 timeout=30000ms service=3101ms state=activ
    source=rack-timeout id=22a6fea79c2f050a9d9999b007298d12 timeout=30000ms service=4027ms state=activ
    (48.1ms) SELECT COUNT(*) FROM "users" INNER JOIN "repo_subscriptions" ON "users"."id" = "repo_
    User Load (53.4ms) SELECT "users".* FROM "users" INNER JOIN "repo_subscriptions" ON "users"."i
    Rendered subscribers/_avatars.html.slim (138.9ms)
    CACHE (0.0ms) SELECT COUNT(*) FROM "users" INNER JOIN "repo_subscriptions" ON "users"."id" = "r
    (0.6ms) SELECT COUNT(count_column) FROM (SELECT 1 AS count_column FROM "users" INNER JOIN "re
    36]]
    Issue Load (41.8ms) SELECT "issues".* FROM "issues" WHERE "issues"."repo_id" = $1 AND "issues"
    (44.2ms) SELECT COUNT(*) FROM "issues" WHERE "issues"."repo_id" = $1 AND "issues"."state" = $2
    Rendered repos/show.html.slim within layouts/application (496.5ms)
    source=rack-timeout id=22a6fea79c2f050a9d9999b007298d12 timeout=30000ms service=5005ms state=activ
    Rendered application/_head.html.slim (1360.4ms)
    Rendered application/_flashes.html.slim (15.9ms)
    Rendered application/_logo.html.slim (21.5ms)
    Rendered application/_nav.html.slim (73.0ms)
    Rendered application/_thoughtbot.html.slim (11.5ms)
    Rendered application/_footer.html.slim (36.4ms)
    Completed 200 OK in 5760ms (Views: 1889.1ms | ActiveRecord: 1323.2ms)
    source=rack-timeout id=22a6fea79c2f050a9d9999b007298d12 timeout=30000ms service=6004ms state=activ

    View Slide

  155. Started GET "/rails/rails" for ::1 at 2015-11-04 14:54:55 -0600
    ActiveRecord::SchemaMigration Load (19.2ms) SELECT "schema_migrations".* FROM "schema_migration
    Processing by ReposController#show as HTML
    Parameters: {"full_name"=>"rails/rails"}
    Repo Load (21.2ms) SELECT "repos".* FROM "repos" WHERE "repos"."full_name" = $1 LIMIT 1 [["fu
    source=rack-timeout id=22a6fea79c2f050a9d9999b007298d12 timeout=30000ms service=1006ms state=activ
    Issue Load (1091.1ms) SELECT "issues".* FROM "issues" WHERE "issues"."repo_id" IN (36)
    source=rack-timeout id=22a6fea79c2f050a9d9999b007298d12 timeout=30000ms service=2093ms state=activ
    source=rack-timeout id=22a6fea79c2f050a9d9999b007298d12 timeout=30000ms service=3101ms state=activ
    source=rack-timeout id=22a6fea79c2f050a9d9999b007298d12 timeout=30000ms service=4027ms state=activ
    (48.1ms) SELECT COUNT(*) FROM "users" INNER JOIN "repo_subscriptions" ON "users"."id" = "repo_
    User Load (53.4ms) SELECT "users".* FROM "users" INNER JOIN "repo_subscriptions" ON "users"."i
    Rendered subscribers/_avatars.html.slim (138.9ms)
    CACHE (0.0ms) SELECT COUNT(*) FROM "users" INNER JOIN "repo_subscriptions" ON "users"."id" = "r
    (0.6ms) SELECT COUNT(count_column) FROM (SELECT 1 AS count_column FROM "users" INNER JOIN "re
    36]]
    Issue Load (41.8ms) SELECT "issues".* FROM "issues" WHERE "issues"."repo_id" = $1 AND "issues"
    (44.2ms) SELECT COUNT(*) FROM "issues" WHERE "issues"."repo_id" = $1 AND "issues"."state" = $2
    Rendered repos/show.html.slim within layouts/application (496.5ms)
    source=rack-timeout id=22a6fea79c2f050a9d9999b007298d12 timeout=30000ms service=5005ms state=activ
    Rendered application/_head.html.slim (1360.4ms)
    Rendered application/_flashes.html.slim (15.9ms)
    Rendered application/_logo.html.slim (21.5ms)
    Rendered application/_nav.html.slim (73.0ms)
    Rendered application/_thoughtbot.html.slim (11.5ms)
    Rendered application/_footer.html.slim (36.4ms)
    Completed 200 OK in 5760ms (Views: 1889.1ms | ActiveRecord: 1323.2ms)
    source=rack-timeout id=22a6fea79c2f050a9d9999b007298d12 timeout=30000ms service=6004ms state=activ

    View Slide

  156. Loading EVERY
    issue for
    rails/rails
    repo

    View Slide

  157. View Slide

  158. def self.find_by_full_name(full_name)
    Repo.includes(:issues).find_by!(full_name: full_name)
    end

    View Slide

  159. def self.find_by_full_name(full_name)
    Repo.includes(:issues).find_by!(full_name: full_name)
    end

    View Slide

  160. Screenshot 2015-10-22 09.39.20

    View Slide

  161. Not a memory
    leak

    View Slide

  162. Objects were
    getting
    Collected via
    GC

    View Slide

  163. Puma
    Threads

    View Slide

  164. Idle

    View Slide

  165. Now Total

    View Slide

  166. Requests

    View Slide

  167. rails/rail

    View Slide

  168. Now Total

    View Slide

  169. saltstack/salt

    View Slide

  170. Now Total

    View Slide

  171. idle

    View Slide

  172. Now Total

    View Slide

  173. rustlang/rust

    View Slide

  174. golang/go

    View Slide

  175. isacs/npm

    View Slide

  176. Now Total

    View Slide

  177. Idle

    View Slide

  178. Now Total

    View Slide

  179. Was that
    necessary?

    View Slide

  180. View Slide

  181. /rails/rails
    Unused Eager Loading detected
    Repo => [:issues]
    Remove from your finder: :includes => [:issues]

    View Slide

  182. No

    View Slide

  183. What if there
    was another
    cause?

    View Slide

  184. Proof that
    memory tooling
    provides
    results

    View Slide

  185. My

    Secret

    View Slide

  186. https://www.dropbox.com/s/vyd7teot0pg349q/Screenshot%202015-11-05%2011.34.04.png?dl=0

    View Slide

  187. $ bundle gem derailed_benchmarks

    View Slide

  188. mime-types
    boot

    View Slide

  189. https://www.dropbox.com/s/wdabatbn0crbgw2/Screenshot%202015-11-05%2011.54.12.png?dl=0

    View Slide

  190. Thanks
    @jeremyevans

    View Slide

  191. gem “mime-types”,
    “~> 2.6.1”,
    require: “mime/types/columnar”

    View Slide

  192. https://www.dropbox.com/s/56bw1mewpm7rc65/Screenshot%202015-11-05%2011.38.15.png?dl=0
    Omniauth
    28% faster

    View Slide

  193. https://www.dropbox.com/s/0n72bg81s0rsfdh/Screenshot%202015-11-05%2011.45.58.png?dl=0
    36% smaller
    require
    memory

    View Slide

  194. gem “mail”, “~> 2.6.2”

    View Slide

  195. https://www.dropbox.com/s/7j0m6zxl27yl7e7/Screenshot%202015-11-05%2011.47.07.png?dl=0
    10% faster
    requests in
    Rails

    View Slide

  196. 1+ years
    of building
    and using
    tools

    View Slide

  197. I made results
    however

    View Slide

  198. All these
    changes

    View Slide

  199. View Slide

  200. Now I can
    reproduce the a
    basic memory
    problem at will

    View Slide

  201. What did I
    learn from 1+
    years of PRs?

    View Slide

  202. Our community has
    almost
    zero
    knowledge of
    how our programs
    use memory

    View Slide

  203. Most apps are
    slow due to
    app code, not
    framework

    View Slide

  204. Your Apps

    View Slide

  205. Monitor
    Production
    Memory

    View Slide

  206. https://www.dropbox.com/s/tzxhepl8o8gizw0/Screenshot%202015-11-05%2014.46.06.png?dl=0

    View Slide

  207. Benchmark!

    View Slide

  208. $ gem install derailed

    View Slide

  209. Puma
    Worker
    Killer

    View Slide

  210. $ gem install bullet

    View Slide

  211. The Future

    View Slide

  212. I want a
    memory tool
    as easy to use
    as Ruby

    View Slide

  213. Using Heap
    Dumps to take
    Production
    snapshots

    View Slide

  214. Better heap
    analysis

    View Slide

  215. Getting this
    to work with
    Heroku

    View Slide

  216. Can we make
    this better?

    View Slide

  217. allocated memory by location
    -----------------------------------
    5131624 active_record/result.rb:116
    1336408 active_record/attribute_set/builder.rb:30
    1336088 active_record/persistence.rb:69
    1330904 active_record/core.rb:547
    1325648 active_record/attribute_methods.rb:359
    921807 active_support/callbacks.rb:81
    737376 active_record/core.rb:114
    596288 active_record/associations.rb:162
    460720 active_record/attribute_set/builder.rb:16

    View Slide

  218. Stack
    information in
    heap dumps
    ???

    View Slide

  219. What do other
    languages do
    ???

    View Slide

  220. https://www.dropbox.com/s/rzs0owrqq3rja96/Screenshot%202015-11-05%2014.32.52.png?dl=0

    View Slide

  221. https://www.dropbox.com/s/k0j31exhh2ir9oj/Screenshot%202015-11-05%2015.12.49.png?dl=0

    View Slide

  222. Can we
    leverage
    existing
    tooling for C?

    View Slide

  223. Valgrind
    ?

    View Slide

  224. GDB
    ?

    View Slide

  225. Visual
    debuggers?

    View Slide

  226. Further
    Reading

    View Slide

  227. View Slide

  228. View Slide

  229. View Slide

  230. One last thing

    View Slide

  231. View Slide

  232. PSA:
    Sprockets is
    now owned
    by Rails

    View Slide

  233. Refactors on
    the Horizon

    View Slide

  234. 21 commits
    makes me #4
    contributor

    View Slide

  235. Let’s save
    sprockets

    View Slide

  236. View Slide

  237. View Slide

  238. View Slide