Using Skylight to Solve Real-World Performance Problems

Transcript

1. None
2. None

3. OSSATHON Eating our own dogfood.

4. None

5. An Open Source Event Manager tailored to
 Free and Open

   Source Software conferences. openSUSE/osem
6. None
7. None
8. None

9. A B C D E F G H I J

   K L M N O P Q R S T U V W X Y Z AA BB CC DD

10. A B C D E F G H I J

    K L M N O P Q R S T U V W X Y Z AA BB CC DD Not a real response time distribution.

11. A B C D E F G H I J

    K L M N O P Q R S T U V W X Y Z AA BB CC DD Average! Not a real response time distribution.

12. A B C D E F G H I J

    K L M N O P Q R S T U V W X Y Z AA BB CC DD 95% requests Not a real response time distribution.

13. A B C D E F G H I J

    K L M N O P Q R S T U V W X Y Z AA BB CC DD

14. A B C D E F G H I J

    K L M N O P Q R S T U V W X Y Z AA BB CC DD Average…?

15. A B C D E F G H I J

    K L M N O P Q R S T U V W X Y Z AA BB CC DD 50% requests

16. A B C D E F G H I J

    K L M N O P Q R S T U V W X Y Z AA BB CC DD 95% requests
17. None
18. None

19. class PostsController < ApplicationController def index @posts = Post.limit(10) end

    end

20. class Post < ActiveRecord::Base belongs_to :author end

21. <% @posts.each do |post| %> <!-- ... --> <h2>By <%=

    post.author.name %></h2> <!-- ... --> <% end %>

22. <% @posts.each do |post| %> <!-- ... --> <h2>By <%=

    post.author.name %></h2> <!-- ... --> <% end %> @posts post.author SELECT * FROM authors WHERE id = 13; SELECT * FROM authors WHERE id = 15; SELECT * FROM authors WHERE id = 21; SELECT * FROM authors WHERE id = 33; SELECT * FROM authors WHERE id = 37; SELECT * FROM posts LIMIT 10;

23. class PostsController < ApplicationController def index @posts = Post.limit(10) end

    end @posts = Post.limit(10).includes(:author) SELECT * FROM posts LIMIT 10; SELECT * FROM authors WHERE id IN (13, 15, 21, 33, 37, 42, 46, 47, 53, 55);
24. None

25. An open source community and curriculum for learning web development.

    theodinproject/theodinproject
26. None

27. PERCENTAGE = COURSES COMPLETED ⨉ 100% LESSONS AVAILABLE LESSONS COMPLETED

28. PERCENTAGE = COURSES COMPLETED ⨉ 100% LESSONS AVAILABLE LESSONS COMPLETED

    SELECT * FROM lessons WHERE course_id = 1;

29. PERCENTAGE = COURSES COMPLETED ⨉ 100% LESSONS AVAILABLE LESSONS COMPLETED

    SELECT * FROM lessons WHERE course_id = 1; SELECT * FROM lesson_completions WHERE student_id = 13;

30. SELECT * FROM lesson_completions WHERE student_id = 13; create_table "lesson_completions"

    do |t| t.integer "lesson_id" t.integer "student_id" t.t.timestamps t.index ["lesson_id", "student_id"], unique: true, using: :btree end PERCENTAGE = COURSES COMPLETED ⨉ 100% LESSONS AVAILABLE LESSONS COMPLETED SELECT * FROM lessons WHERE course_id = 1;

31. SELECT * FROM lesson_completions WHERE student_id = 13; create_table "lesson_completions"

    do |t| t.integer "lesson_id" t.integer "student_id" t.t.timestamps t.index ["lesson_id", "student_id"], unique: true, using: :btree end

32. NOPE.

33. NOPE. Sometimes?

34. LessonCompletion Load (57.2ms) SELECT "lesson_completions".* FROM ... => EXPLAIN for:

    SELECT "lesson_completions".* FROM "lesson_completions" ... ----------------------------------------------------------------------- Seq Scan on lesson_completions (cost=0.00..4063.50 rows=49 width=28) Filter: (student_id = 3074) (2 rows)

35. LessonCompletion Load (57.2ms) SELECT "lesson_completions".* FROM ... => EXPLAIN for:

    SELECT "lesson_completions".* FROM "lesson_completions" ... ----------------------------------------------------------------------- Seq Scan on lesson_completions (cost=0.00..4063.50 rows=49 width=28) Filter: (student_id = 3074) (2 rows) 57.2ms Seq Scan on lesson_completions

36. LessonCompletion Load (9.8ms) SELECT "lesson_completions".* FROM ... => EXPLAIN for:

    SELECT "lesson_completions".* FROM "lesson_completions" ... ----------------------------------------------------------------------- Index Scan using index_lesson_completions_on_lesson_id_and_student_id ... Filter: (student_id = 3074) (2 rows) 9.8ms 
 Index Scan using index_lesson_completions_on_lesson_id_and_student_id


37. LessonCompletion Load (1.6ms) SELECT "lesson_completions".* FROM ... => EXPLAIN for:

    SELECT "lesson_completions".* FROM "lesson_completions" ... ----------------------------------------------------------------------- Index Scan using index_lesson_completions_on_student_id on ... Filter: (student_id = 3074) (2 rows) 1.6ms Index Scan using index_lesson_completions_on_student_id

38. SELECT * FROM lesson_completions WHERE ... lesson_id = ? student_id

    = ? lesson_id = ? AND
 student_id = ? student_id = ? AND
 lesson_id = ? Dedicated Indexes (lesson_id) (student_id) Compound Index (lesson_id, student_id) Compound Index (student_id, lesson_id) ! ! ! ! ! ! ! " " ! ! !
39. None
40. None
41. None

42. Help out your favorite open source projects and become a

    better developer while doing it. codetriage/codetriage
43. None
44. None
45. None
46. None
47. None
48. None
49. None

50. <% @repos.each do |repo| %> <% cache repo do %>

    <!-- ... --> <% end %> <% end %>

51. <% @repos.each do |repo| %> <% cache repo do %>

    <!-- ... --> <% end %> <% end %> <% cache repo do %> READ views/repos/1894-20180413141703167092/b20 READ views/repos/1245-20180413141214948480/b20 READ views/repos/3353-20180413142449772302/b20 READ views/repos/987-20180413140921805080/b201 READ views/repos/1240-20180413141121053193/b20 READ views/repos/2193-20180413141853545411/b20 READ views/repos/1055-20180413141006040871/b20 READ views/repos/1332-20180413141232839666/b20

52. <%= render partial: 'repo', collection: @repos, cached: true %> READ

    MULTI views/repos/1894-2018041 views/repos/1245-2018041 views/repos/3353-2018041 views/repos/987-20180413 views/repos/1240-2018041 views/repos/2193-2018041 views/repos/1055-2018041 views/repos/1332-2018041 views/repos/1258-2018041
53. None

54. Cache write: views/repos/1901-20180413141634501842/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/863-20180413140900779427/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/386-20180413140542430556/b201d971de1a77b9f19a62934e479a96 Cache

    write: views/repos/1996-20180413141654477686/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/528-20180413140609506027/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/3072-20180413142336550445/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/1421-20180413141154176342/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/2216-20180413141847819637/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/3916-20180413142706315602/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/3793-20180413142632521852/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/696-20180413140702557180/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/1521-20180413141315869506/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/781-20180413140752378480/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/297-20180413140459380343/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/764-20180413140728504164/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/2188-20180413141834504728/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/1224-20180413141032106598/b201d971de1a77b9f19a62934e479a96 Rendered collection of repos/_repo.html.slim [23 / 50 cache hits] (135.1ms) Rendered pages/_repos_with_pagination.html.slim (148.7ms) Rendered pages/index.html.slim within layouts/application (166.9ms) Rendered application/_head.html.slim (1.0ms) Rendered application/_flashes.html.slim (0.2ms) Rendered application/_nav.html.slim (1.0ms) Rendered application/_footer.html.slim (1.0ms) Rendered layouts/application.html.slim (172.2ms) Completed 200 OK in 183ms (Views: 166.3ms | ActiveRecord: 7.9ms) Rendered collection of repos/_repo.html.slim [23 / 50 cache hits] (135.1ms)

55. 100 MB

56. 100 MB

57. 100 MB

58. STAT items:0:chunk_size 64 STAT items:0:evicted 0 STAT items:0:evicted_nonzero 0 STAT

    items:0:number 0 STAT items:0:outofmemory 0 STAT items:1:chunk_size 128 STAT items:1:evicted 94 STAT items:1:evicted_nonzero 4 STAT items:1:number 18 STAT items:1:outofmemory 5 STAT items:2:chunk_size 256 STAT items:2:evicted 106659 STAT items:2:evicted_nonzero 0 STAT items:2:number 9040 STAT items:2:outofmemory 0 STAT items:3:chunk_size 512 STAT items:3:evicted 5356 STAT items:3:evicted_nonzero 981 STAT items:3:number 2 STAT items:3:outofmemory 57855 STAT items:4:chunk_size 1024 STAT items:4:evicted 1880488 STAT items:4:evicted_nonzero 0 STAT items:4:number 4982 STAT items:4:outofmemory 85445 STAT items:5:chunk_size 2048

59. STAT items:1:number 18 STAT items:1:outofmemory 5 STAT items:2:chunk_size 256 STAT

    items:2:evicted 106659 STAT items:2:evicted_nonzero 0 STAT items:2:number 9040 STAT items:2:outofmemory 0 STAT items:3:chunk_size 512 STAT items:3:evicted 5356 STAT items:3:evicted_nonzero 981 STAT items:3:number 2 STAT items:3:outofmemory 57855 STAT items:4:chunk_size 1024 STAT items:4:evicted 1880488 STAT items:4:evicted_nonzero 0 STAT items:4:number 4982 STAT items:4:outofmemory 85445 STAT items:5:chunk_size 2048 STAT items:5:evicted 67687 STAT items:5:evicted_nonzero 0 STAT items:5:number 5605 STAT items:5:outofmemory 0 STAT items:6:chunk_size 4096 STAT items:6:evicted 135319 STAT items:6:evicted_nonzero 0 STAT items:6:number 11126 STAT items:6:outofmemory 0 STAT items:4:chunk_size 1024 STAT items:4:evicted 1880488 STAT items:4:evicted_nonzero 0 STAT items:4:number 4982 STAT items:4:outofmemory 85445 1 KB 4982 items

60. STAT items:1:number 18 STAT items:1:outofmemory 5 STAT items:2:chunk_size 256 STAT

    items:2:evicted 106659 STAT items:2:evicted_nonzero 0 STAT items:2:number 9040 STAT items:2:outofmemory 0 STAT items:3:chunk_size 512 STAT items:3:evicted 5356 STAT items:3:evicted_nonzero 981 STAT items:3:number 2 STAT items:3:outofmemory 57855 STAT items:4:chunk_size 1024 STAT items:4:evicted 1880488 STAT items:4:evicted_nonzero 0 STAT items:4:number 4982 STAT items:4:outofmemory 85445 STAT items:5:chunk_size 2048 STAT items:5:evicted 67687 STAT items:5:evicted_nonzero 0 STAT items:5:number 5605 STAT items:5:outofmemory 0 STAT items:6:chunk_size 4096 STAT items:6:evicted 135319 STAT items:6:evicted_nonzero 0 STAT items:6:number 11126 STAT items:6:outofmemory 0 STAT items:4:chunk_size 1024 STAT items:4:evicted 1880488 STAT items:4:evicted_nonzero 0 STAT items:4:number 4982 STAT items:4:outofmemory 85445 5 MB

61. 5 MB

62. 5 MB 50 cards

63. STAT items:1:number 18 STAT items:1:outofmemory 5 STAT items:2:chunk_size 256 STAT

    items:2:evicted 106659 STAT items:2:evicted_nonzero 0 STAT items:2:number 9040 STAT items:2:outofmemory 0 STAT items:3:chunk_size 512 STAT items:3:evicted 5356 STAT items:3:evicted_nonzero 981 STAT items:3:number 2 STAT items:3:outofmemory 57855 STAT items:4:chunk_size 1024 STAT items:4:evicted 1880488 STAT items:4:evicted_nonzero 0 STAT items:4:number 4982 STAT items:4:outofmemory 85445 STAT items:5:chunk_size 2048 STAT items:5:evicted 67687 STAT items:5:evicted_nonzero 0 STAT items:5:number 5605 STAT items:5:outofmemory 0 STAT items:6:chunk_size 4096 STAT items:6:evicted 135319 STAT items:6:evicted_nonzero 0 STAT items:6:number 11126 STAT items:6:outofmemory 0 STAT items:4:chunk_size 1024 STAT items:4:evicted 1880488 STAT items:4:evicted_nonzero 0 STAT items:4:number 4982 STAT items:4:outofmemory 85445 !!!

64. 5 MB

65. None

66. Rendered application/_down.html.slim (0.1ms) Cache read: views/language_list/05660542d890bbeab574f26e0cbbd974 ({:expires_in=>1 hour}) Repo Load

    (1.7ms) SELECT "repos"."id", "repos"."updated_at", "repos"."issues_count", " (2.7ms) SELECT COUNT(*) FROM "repos" WHERE (issues_count > 0) AND ("repos"."id" NOT IN Cache read_multi: ["views/repos/1894-20180413141703167092/b201d971de1a77b9f19a62934e479a96 Cache read: views/repos/3353-20180413142449772302/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/3353-20180413142449772302/b201d971de1a77b9f19a62934e479a96 Cache read: views/repos/1055-20180413141006040871/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/1055-20180413141006040871/b201d971de1a77b9f19a62934e479a96 Cache read: views/repos/1332-20180413141232839666/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/1332-20180413141232839666/b201d971de1a77b9f19a62934e479a96 Cache read: views/repos/2151-20180413141907871737/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/2151-20180413141907871737/b201d971de1a77b9f19a62934e479a96 Cache read: views/repos/1082-20180413140958427040/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/1082-20180413140958427040/b201d971de1a77b9f19a62934e479a96 Cache read: views/repos/1524-20180413141412726203/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/1524-20180413141412726203/b201d971de1a77b9f19a62934e479a96 Cache read: views/repos/1916-20180413141546875915/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/1916-20180413141546875915/b201d971de1a77b9f19a62934e479a96 Cache read: views/repos/1348-20180413141215739127/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/1348-20180413141215739127/b201d971de1a77b9f19a62934e479a96 Cache read: views/repos/177-20180413140322053324/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/177-20180413140322053324/b201d971de1a77b9f19a62934e479a96 Cache read: views/repos/1824-20180413141520630598/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/1824-20180413141520630598/b201d971de1a77b9f19a62934e479a96 Cache read: views/repos/1901-20180413141634501842/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/1901-20180413141634501842/b201d971de1a77b9f19a62934e479a96 Cache read_multi: ["views/repos/1894-20180413141703167092/b201d971de1a77b9f19a62934e479a96

67. Rendered application/_down.html.slim (0.1ms) Cache read: views/language_list/05660542d890bbeab574f26e0cbbd974 ({:expires_in=>1 hour}) Repo Load

    (1.7ms) SELECT "repos"."id", "repos"."updated_at", "repos"."issues_count", " (2.7ms) SELECT COUNT(*) FROM "repos" WHERE (issues_count > 0) AND ("repos"."id" NOT IN Cache read_multi: ["views/repos/1894-20180413141703167092/b201d971de1a77b9f19a62934e479a96 Cache read: views/repos/3353-20180413142449772302/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/3353-20180413142449772302/b201d971de1a77b9f19a62934e479a96 Cache read: views/repos/1055-20180413141006040871/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/1055-20180413141006040871/b201d971de1a77b9f19a62934e479a96 Cache read: views/repos/1332-20180413141232839666/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/1332-20180413141232839666/b201d971de1a77b9f19a62934e479a96 Cache read: views/repos/2151-20180413141907871737/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/2151-20180413141907871737/b201d971de1a77b9f19a62934e479a96 Cache read: views/repos/1082-20180413140958427040/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/1082-20180413140958427040/b201d971de1a77b9f19a62934e479a96 Cache read: views/repos/1524-20180413141412726203/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/1524-20180413141412726203/b201d971de1a77b9f19a62934e479a96 Cache read: views/repos/1916-20180413141546875915/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/1916-20180413141546875915/b201d971de1a77b9f19a62934e479a96 Cache read: views/repos/1348-20180413141215739127/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/1348-20180413141215739127/b201d971de1a77b9f19a62934e479a96 Cache read: views/repos/177-20180413140322053324/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/177-20180413140322053324/b201d971de1a77b9f19a62934e479a96 Cache read: views/repos/1824-20180413141520630598/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/1824-20180413141520630598/b201d971de1a77b9f19a62934e479a96 Cache read: views/repos/1901-20180413141634501842/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/1901-20180413141634501842/b201d971de1a77b9f19a62934e479a96 Cache read: views/repos/3353-20180413142449772302/b201d971de1a77b9f19a62934e479a96 Cache read: views/repos/1055-20180413141006040871/b201d971de1a77b9f19a62934e479a96 Cache read: views/repos/1332-20180413141232839666/b201d971de1a77b9f19a62934e479a96 Cache read: views/repos/2151-20180413141907871737/b201d971de1a77b9f19a62934e479a96 Cache read: views/repos/1082-20180413140958427040/b201d971de1a77b9f19a62934e479a96 Cache read: views/repos/1524-20180413141412726203/b201d971de1a77b9f19a62934e479a96 Cache read: views/repos/1916-20180413141546875915/b201d971de1a77b9f19a62934e479a96 Cache read: views/repos/1348-20180413141215739127/b201d971de1a77b9f19a62934e479a96 Cache read: views/repos/177-20180413140322053324/b201d971de1a77b9f19a62934e479a96 Cache read: views/repos/1824-20180413141520630598/b201d971de1a77b9f19a62934e479a96 Cache read: views/repos/1901-20180413141634501842/b201d971de1a77b9f19a62934e479a96
68. None

69. Cache write: views/repos/696-20180413140702557180/b201d971de1a77b9f19a62934e479a96 Cache read: views/repos/1521-20180413141315869506/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/1521-20180413141315869506/b201d971de1a77b9f19a62934e479a96 Cache

    read: views/repos/781-20180413140752378480/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/781-20180413140752378480/b201d971de1a77b9f19a62934e479a96 Cache read: views/repos/297-20180413140459380343/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/297-20180413140459380343/b201d971de1a77b9f19a62934e479a96 Cache read: views/repos/764-20180413140728504164/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/764-20180413140728504164/b201d971de1a77b9f19a62934e479a96 Cache read: views/repos/2188-20180413141834504728/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/2188-20180413141834504728/b201d971de1a77b9f19a62934e479a96 Cache read: views/repos/1224-20180413141032106598/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/1224-20180413141032106598/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/3353-20180413142449772302/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/1055-20180413141006040871/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/1332-20180413141232839666/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/2151-20180413141907871737/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/1082-20180413140958427040/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/1524-20180413141412726203/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/1916-20180413141546875915/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/1348-20180413141215739127/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/177-20180413140322053324/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/1824-20180413141520630598/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/1901-20180413141634501842/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/863-20180413140900779427/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/386-20180413140542430556/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/1996-20180413141654477686/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/3353-20180413142449772302/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/1055-20180413141006040871/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/1332-20180413141232839666/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/2151-20180413141907871737/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/1082-20180413140958427040/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/1524-20180413141412726203/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/1916-20180413141546875915/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/1348-20180413141215739127/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/177-20180413140322053324/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/1824-20180413141520630598/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/1901-20180413141634501842/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/863-20180413140900779427/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/386-20180413140542430556/b201d971de1a77b9f19a62934e479a96 Cache write: views/repos/1996-20180413141654477686/b201d971de1a77b9f19a62934e479a96
70. None
71. None

72. DO THE ANSWER DANCE. Don’t struggle to learn why your

    app is slow. Get answers with Skylight. SKYLIGHT