Microarchitectural Implications of Event-driven Server-side Web Applications

Transcript

1. 1 MICRO 2015 Microarchitectural Implications of Event-driven Server-side Web Applications

   Yuhao Zhu UT Austin with Daniel Richins, Matthew Halpern, Vijay Janapa Reddi

2. 2 Instruction Supply is a Critical Aspect of Microarchitecture Design

3. Exploit Instruction Locality a.k.a., Common Case Design 3 Instruction Supply

   Cache Branch Predictor TLB

4. Exploit Instruction Locality a.k.a., Common Case Design 3 Instruction Supply

   Cache Branch Predictor TLB Hot instructions

5. Exploit Instruction Locality a.k.a., Common Case Design 3 Instruction Supply

   Cache Branch Predictor TLB Hot instructions Hot branch history patterns

6. Exploit Instruction Locality a.k.a., Common Case Design 3 Instruction Supply

   Cache Branch Predictor TLB Hot instructions Hot branch history patterns Hot pages

7. Exploit Instruction Locality a.k.a., Common Case Design 3 SPEC CPU

   (mostly) Instruction Supply Cache Branch Predictor TLB Hot instructions Hot branch history patterns Hot pages

8. Exploit Instruction Locality a.k.a., Common Case Design 3 SPEC CPU

   (mostly) Instruction Supply Cache Branch Predictor TLB Event-driven Applications Hot instructions Hot branch history patterns Hot pages

9. Locality Lost 4 SPEC CPU (mostly) Event-driven Applications Hot instructions

   Hot branch history pattern Hot pages Instruction Supply Cache Branch Predictor TLB

10. Locality Lost 5 SPEC CPU (mostly) Event-driven Applications Tight loops

    Hot pages Little branch aliasing Instruction Supply Cache Branch Predictor TLB

11. Locality Lost 5 SPEC CPU (mostly) Event-driven Applications Tight loops

    Hot pages Little branch aliasing Instruction Supply Cache Branch Predictor TLB

12. Event-driven Execution Model 6 Event Queue Head Tail

13. Event-driven Execution Model 6 Event Queue Client Request Head Tail

14. Event-driven Execution Model 6 Event Queue Single-threaded Event Loop Client

    Request Head Tail

15. Event-driven Execution Model 6 Event Queue Single-threaded Event Loop Client

    Request Head Tail

16. Event-driven Execution Model 6 Event Queue Single-threaded Event Loop DB

    Access File I/O Network Client Request Head Tail

17. Event-driven Execution Model 6 Event Queue Single-threaded Event Loop DB

    Access File I/O Network Client Request Head Tail

18. Event-driven Execution Model 6 Event Queue Single-threaded Event Loop DB

    Access File I/O Network Client Request Head Tail

19. Event-driven Execution Model 6 Event Queue Single-threaded Event Loop DB

    Access File I/O Network Client Request Head Tail

20. Event-driven Execution Model 6 Event Queue Single-threaded Event Loop DB

    Access File I/O Network Client Request Head Tail Our Focus

21. Applications 7 Application Domain Etherpad Lite Document Collaboration Let’s Chat

    Messaging Lighter Content Management Mud Gaming Todo Task Management Word Finder API Services https://github.com/nodebenchmark/

22. Locality Lost 8

23. Locality Lost 8 lbm leslie3d libquantum astar hmmer mcf bzip2

    namd gromacs zeusmp calculix GemsFDTD soplex sphinx3 bwaves milc dealII wrf h264ref gamess tonto xalancbmk povray perlbench sjeng gcc gobmk omnetpp cactusADM L1 I-Cache MPKI 0 30 60 90 120 Word Finder Todo Mud Etherpad Let's Chat Lighter SPEC CPU 2006 NodeJS I-Cache Parameters 32 KB, 64 B cache line, 8-way

24. Locality Lost 9 lbm leslie3d libquantum astar hmmer mcf bzip2

    namd gromacs zeusmp calculix GemsFDTD soplex sphinx3 bwaves milc dealII wrf h264ref gamess tonto xalancbmk povray perlbench sjeng gcc gobmk omnetpp cactusADM L1 I-Cache MPKI 0 30 60 90 120 Word Finder Todo Mud Etherpad Let's Chat Lighter SPEC CPU 2006 NodeJS I-Cache Parameters 32 KB, 64 B cache line, 8-way

25. Locality Lost 9 lbm leslie3d libquantum astar hmmer mcf bzip2

    namd gromacs zeusmp calculix GemsFDTD soplex sphinx3 bwaves milc dealII wrf h264ref gamess tonto xalancbmk povray perlbench sjeng gcc gobmk omnetpp cactusADM L1 I-Cache MPKI 0 30 60 90 120 Word Finder Todo Mud Etherpad Let's Chat Lighter SPEC CPU 2006 NodeJS I-Cache Parameters 32 KB, 64 B cache line, 8-way SPEC CPU 2006 Average

26. Locality Lost 10 lbm leslie3d libquantum astar hmmer mcf bzip2

    namd gromacs zeusmp calculix GemsFDTD soplex sphinx3 bwaves milc dealII wrf h264ref gamess tonto xalancbmk povray perlbench sjeng gcc gobmk omnetpp cactusADM L1 I-Cache MPKI 0 30 60 90 120 Word Finder Todo Mud Etherpad Let's Chat Lighter SPEC CPU 2006 NodeJS I-Cache Parameters 32 KB, 64 B cache line, 8-way SPEC CPU 2006 Average

27. Locality Lost 10 lbm leslie3d libquantum astar hmmer mcf bzip2

    namd gromacs zeusmp calculix GemsFDTD soplex sphinx3 bwaves milc dealII wrf h264ref gamess tonto xalancbmk povray perlbench sjeng gcc gobmk omnetpp cactusADM L1 I-Cache MPKI 0 30 60 90 120 Word Finder Todo Mud Etherpad Let's Chat Lighter SPEC CPU 2006 NodeJS I-Cache Parameters 32 KB, 64 B cache line, 8-way SPEC CPU 2006 Average Node.js has 4.2 X higher MPKI than SPEC CPU.

28. Root Cause Analysis 11

29. Root Cause Analysis 11 High I-$ Miss Ratio

30. Root Cause Analysis 11 High I-$ Miss Ratio Large Instruction

    Reuse-distance

31. Root Cause Analysis 11 100 80 60 40 20 0

    Dynamic Instructions (%) 20 24 28 212 216 Reuse Distance (log) High I-$ Miss Ratio Large Instruction Reuse-distance

32. Root Cause Analysis 11 100 80 60 40 20 0

    Dynamic Instructions (%) 20 24 28 212 216 Reuse Distance (log) 100 80 60 40 20 0 Dynamic Instructions (%) 20 24 28 212 216 Reuse Distance (log) omnetpp (worst) lbm (best) High I-$ Miss Ratio Large Instruction Reuse-distance

33. Root Cause Analysis 11 100 80 60 40 20 0

    Dynamic Instructions (%) 20 24 28 212 216 Reuse Distance (log) 100 80 60 40 20 0 Dynamic Instructions (%) 20 24 28 212 216 Reuse Distance (log) 100 80 60 40 20 0 Dynamic Instructions (%) 20 24 28 212 216 Reuse Distance (log) omnetpp (worst) lbm (best) Let’s Chat High I-$ Miss Ratio Large Instruction Reuse-distance

34. Root Cause Analysis 11 100 80 60 40 20 0

    Dynamic Instructions (%) 20 24 28 212 216 Reuse Distance (log) 100 80 60 40 20 0 Dynamic Instructions (%) 20 24 28 212 216 Reuse Distance (log) 100 80 60 40 20 0 Dynamic Instructions (%) 20 24 28 212 216 Reuse Distance (log) 100 80 60 40 20 0 Dynamic Instructions (%) 20 24 28 212 216 Reuse Distance (log) High I-$ Miss Ratio Large Instruction Reuse-distance

35. Root Cause Analysis 12 High I-$ Miss Ratio Large Instruction

    Reuse-distance

36. Root Cause Analysis 12 … … Instruction
 Stream High I-$

    Miss Ratio Large Instruction Reuse-distance

37. Root Cause Analysis 12 … … Instruction
 Stream High I-$

    Miss Ratio Large Instruction Reuse-distance

38. Event 2 Event 1 Event 3 Root Cause Analysis 12

    … … Instruction
 Stream High I-$ Miss Ratio Inter-event Code Reuse Large Instruction Reuse-distance

39. Event 2 Event 1 Event 3 Root Cause Analysis 12

    … … Instruction
 Stream High I-$ Miss Ratio Inter-event Code Reuse Large Instruction Reuse-distance Large Event Footprint

40. Root Cause Analysis 13 100 80 60 40 20 0

    Instructions (%) 32 64 96 128 160 192 224 256 > 256 # of reuses Intra-event Inter-event High I-$ Miss Ratio Inter-event Code Reuse Large Event Footprint Large Instruction Reuse-distance

41. Root Cause Analysis 13 100 80 60 40 20 0

    Instructions (%) 32 64 96 128 160 192 224 256 > 256 # of reuses Intra-event Inter-event High I-$ Miss Ratio Inter-event Code Reuse Large Event Footprint Large Instruction Reuse-distance 100 80 60 40 20 0 Static Instructions (%) 32 64 96 128 160 192 224 256 > 256 # of reuses

42. Root Cause Analysis 13 100 80 60 40 20 0

    Instructions (%) 32 64 96 128 160 192 224 256 > 256 # of reuses Intra-event Inter-event High I-$ Miss Ratio Inter-event Code Reuse Large Event Footprint Large Instruction Reuse-distance 100 80 60 40 20 0 Static Instructions (%) 32 64 96 128 160 192 224 256 > 256 # of reuses 100 80 60 40 20 0 Static Instructions (%) 32 64 96 128 160 192 224 256 > 256 # of reuses

43. Root Cause Analysis 13 100 80 60 40 20 0

    Instructions (%) 32 64 96 128 160 192 224 256 > 256 # of reuses Intra-event Inter-event High I-$ Miss Ratio Inter-event Code Reuse Large Event Footprint Large Instruction Reuse-distance 100 80 60 40 20 0 Static Instructions (%) 32 64 96 128 160 192 224 256 > 256 # of reuses 100 80 60 40 20 0 Static Instructions (%) 32 64 96 128 160 192 224 256 > 256 # of reuses 100 80 60 40 20 0 Static Instructions (%) 32 64 96 128 160 192 224 256 > 256 # of reuses

44. Root Cause Analysis 13 100 80 60 40 20 0

    Instructions (%) 32 64 96 128 160 192 224 256 > 256 # of reuses Intra-event Inter-event High I-$ Miss Ratio Inter-event Code Reuse Large Event Footprint Large Instruction Reuse-distance 100 80 60 40 20 0 Static Instructions (%) 32 64 96 128 160 192 224 256 > 256 # of reuses 100 80 60 40 20 0 Static Instructions (%) 32 64 96 128 160 192 224 256 > 256 # of reuses 100 80 60 40 20 0 Static Instructions (%) 32 64 96 128 160 192 224 256 > 256 # of reuses

45. Root Cause Analysis 13 100 80 60 40 20 0

    Instructions (%) 32 64 96 128 160 192 224 256 > 256 # of reuses Intra-event Inter-event High I-$ Miss Ratio Inter-event Code Reuse Large Event Footprint Large Instruction Reuse-distance 100 80 60 40 20 0 Static Instructions (%) 32 64 96 128 160 192 224 256 > 256 # of reuses 100 80 60 40 20 0 Static Instructions (%) 32 64 96 128 160 192 224 256 > 256 # of reuses 100 80 60 40 20 0 Static Instructions (%) 32 64 96 128 160 192 224 256 > 256 # of reuses Most instruction reuses are inter-event.

46. Root Cause Analysis 14 High I-$ Miss Ratio Inter-event Code

    Reuse Large Event Footprint Large Instruction Reuse-distance

47. Root Cause Analysis 14 213 215 217 219 221 Event

    Footprint (Bytes) 100 80 60 40 20 0 Events (%) High I-$ Miss Ratio Inter-event Code Reuse Large Event Footprint Large Instruction Reuse-distance

48. Root Cause Analysis 14 213 215 217 219 221 Event

    Footprint (Bytes) 100 80 60 40 20 0 Events (%) L1-I$ Size 32KB High I-$ Miss Ratio Inter-event Code Reuse Large Event Footprint Large Instruction Reuse-distance

49. Root Cause Analysis 14 213 215 217 219 221 Event

    Footprint (Bytes) 100 80 60 40 20 0 Events (%) 213 215 217 219 221 Event Footprint (Bytes) 100 80 60 40 20 0 Events (%) 13 215 217 219 221 Event Footprint (Bytes) L1 Cache Size 32 KB Word Finder Todo Mud Etherpad Let's Chat Lighter High I-$ Miss Ratio Inter-event Code Reuse Large Event Footprint Large Instruction Reuse-distance

50. Root Cause Analysis 14 213 215 217 219 221 Event

    Footprint (Bytes) 100 80 60 40 20 0 Events (%) 213 215 217 219 221 Event Footprint (Bytes) 100 80 60 40 20 0 Events (%) 13 215 217 219 221 Event Footprint (Bytes) L1 Cache Size 32 KB Word Finder Todo Mud Etherpad Let's Chat Lighter Only 12% Events Fit in a 32 KB I-$ High I-$ Miss Ratio Inter-event Code Reuse Large Event Footprint Large Instruction Reuse-distance

51. Root Cause Analysis 15 213 215 217 219 221 Event

    Footprint (Bytes) 100 80 60 40 20 0 Events (%) High I-$ Miss Ratio Inter-event Code Reuse Large Event Footprint Large Instruction Reuse-distance

52. Root Cause Analysis 15 213 215 217 219 221 Event

    Footprint (Bytes) 100 80 60 40 20 0 Events (%) 213 215 217 219 221 Event Footprint (Bytes) 100 80 60 40 20 0 Events (%) High I-$ Miss Ratio Inter-event Code Reuse Large Event Footprint Large Instruction Reuse-distance

53. Root Cause Analysis 15 213 215 217 219 221 Event

    Footprint (Bytes) 100 80 60 40 20 0 Events (%) 213 215 217 219 221 Event Footprint (Bytes) 100 80 60 40 20 0 Events (%) Most events’ footprints do not fit in a typical I-$. High I-$ Miss Ratio Inter-event Code Reuse Large Event Footprint Large Instruction Reuse-distance

54. Root Cause Analysis 16 High I-$ Miss Ratio Inter-event Code

    Reuse Large Event Footprint Few Event Types Large Instruction Reuse-distance

55. Root Cause Analysis 16 High I-$ Miss Ratio Inter-event Code

    Reuse Large Event Footprint Few Event Types Minimal Tight Loops Large Instruction Reuse-distance

56. Root Cause Analysis 16 High I-$ Miss Ratio Inter-event Code

    Reuse Large Event Footprint Few Event Types Minimal Tight Loops Large Instruction Reuse-distance Event-driven Applications

57. Microarchitecture Behaviors Root Cause Analysis 16 High I-$ Miss Ratio

    Inter-event Code Reuse Large Event Footprint Few Event Types Minimal Tight Loops Large Instruction Reuse-distance Event-driven Applications

58. Application Characteristics Microarchitecture Behaviors Root Cause Analysis 16 High I-$

    Miss Ratio Inter-event Code Reuse Large Event Footprint Few Event Types Minimal Tight Loops Large Instruction Reuse-distance Event-driven Applications

59. Application Characteristics Microarchitecture Behaviors Root Cause Analysis 16 High I-$

    Miss Ratio Inter-event Code Reuse Large Event Footprint Few Event Types Minimal Tight Loops Large Instruction Reuse-distance Event-driven Applications

60. 17 Can we better capture instruction locality to improve instruction

    supply efficiency?

61. Scale Up Hardware Resources? 18 2.0 1.8 1.6 1.4 1.2

    1.0 Norm. Time 3.2 2.4 1.6 Let's Chat Word Finder Todo Mud Etherpad Lighter

62. Scale Up Hardware Resources? 18 120 90 60 30 0

    I-Cache MPKI 16 64 256 1024 I-Cache size (KB) 2.0 1.8 1.6 1.4 1.2 1.0 Norm. Time 3.2 2.4 1.6 Let's Chat Word Finder Todo Mud Etherpad Lighter

63. Scale Up Hardware Resources? 18 120 90 60 30 0

    I-Cache MPKI 16 64 256 1024 I-Cache size (KB) SPEC CPU 2006 Avarage @ 32KB 2.0 1.8 1.6 1.4 1.2 1.0 Norm. Time 3.2 2.4 1.6 Let's Chat Word Finder Todo Mud Etherpad Lighter

64. Scale Up Hardware Resources? 18 120 90 60 30 0

    I-Cache MPKI 16 64 256 1024 I-Cache size (KB) 120 90 60 30 0 I-Cache MPKI 16 64 256 1024 I-Cache size (KB) 2.0 1.8 1.6 1.4 1.2 1.0 Norm. Time 3.2 2.4 1.6 Let's Chat Word Finder Todo Mud Etherpad Lighter

65. Scale Up Hardware Resources? 18 120 90 60 30 0

    I-Cache MPKI 16 64 256 1024 I-Cache size (KB) 120 90 60 30 0 I-Cache MPKI 16 64 256 1024 I-Cache size (KB) 2.0 1.8 1.6 1.4 1.2 1.0 Norm. Time 3.2 2.4 1.6 Let's Chat Word Finder Todo Mud Etherpad Lighter

66. Scale Up Hardware Resources? 18 120 90 60 30 0

    I-Cache MPKI 16 64 256 1024 I-Cache size (KB) 120 90 60 30 0 I-Cache MPKI 16 64 256 1024 I-Cache size (KB) 120 90 60 30 0 I-Cache MPKI 16 64 256 1024 I-Cache size (KB) 2.0 1.8 1.6 1.4 1.2 1.0 Norm. Time 3.2 2.4 1.6 Let's Chat Word Finder Todo Mud Etherpad Lighter

67. Scale Up Hardware Resources? 18 120 90 60 30 0

    I-Cache MPKI 16 64 256 1024 I-Cache size (KB) 120 90 60 30 0 I-Cache MPKI 16 64 256 1024 I-Cache size (KB) 120 90 60 30 0 I-Cache MPKI 16 64 256 1024 I-Cache size (KB) 2.0 1.8 1.6 1.4 1.2 1.0 Norm. Time 3.2 2.4 1.6 Let's Chat Word Finder Todo Mud Etherpad Lighter ~256 KB Needed!

68. Scale Up Hardware Resources? 18 120 90 60 30 0

    I-Cache MPKI 16 64 256 1024 I-Cache size (KB) 120 90 60 30 0 I-Cache MPKI 16 64 256 1024 I-Cache size (KB) 120 90 60 30 0 I-Cache MPKI 16 64 256 1024 I-Cache size (KB) 2.0 1.8 1.6 1.4 1.2 1.0 Norm. Time 3.2 2.4 1.6 Let's Chat Word Finder Todo Mud Etherpad Lighter ~256 KB Needed!

69. Exploit Inter-Event Locality 19

70. Exploit Inter-Event Locality 19 … … Event 1 Instruction
 Stream

    Event 2 Event 3

71. Exploit Inter-Event Locality 19 1. Retain the reused portion of

    an event’s footprint in the cache … … Event 1 Instruction
 Stream Event 2 Event 3

72. Exploit Inter-Event Locality 19 1. Retain the reused portion of

    an event’s footprint in the cache … … Event 1 Instruction
 Stream Event 2 Event 3

73. Exploit Inter-Event Locality 19 1. Retain the reused portion of

    an event’s footprint in the cache 2. Prefetch the unretained part … … Event 1 Instruction
 Stream Event 2 Event 3

74. 2. Prefetch the unretained part 1. Retain the reused portion

    of an event’s footprint in the cache 20 PRINCIPLES Exploit Inter-Event Locality

75. 2. Prefetch the unretained part 1. Retain the reused portion

    of an event’s footprint in the cache 20 PRINCIPLES PRACTICES Exploit Inter-Event Locality

76. 2. Prefetch the unretained part 1. Retain the reused portion

    of an event’s footprint in the cache 20 PRINCIPLES PRACTICES 1. Retain the reused portion of an event’s footprint in the cache Exploit Inter-Event Locality

77. 2. Prefetch the unretained part 1. Retain the reused portion

    of an event’s footprint in the cache ▸ LRU Cache Insertion Policy (LIP) (Qureshi et al., [ISCA’07]) ▹ Insert incoming line into LRU position, not MRU position 20 PRINCIPLES PRACTICES 1. Retain the reused portion of an event’s footprint in the cache Exploit Inter-Event Locality

78. 2. Prefetch the unretained part 1. Retain the reused portion

    of an event’s footprint in the cache ▸ LRU Cache Insertion Policy (LIP) (Qureshi et al., [ISCA’07]) ▹ Insert incoming line into LRU position, not MRU position 20 a b c d e f g h i PRINCIPLES PRACTICES 1. Retain the reused portion of an event’s footprint in the cache MRU LRU Exploit Inter-Event Locality

79. 2. Prefetch the unretained part 1. Retain the reused portion

    of an event’s footprint in the cache ▸ LRU Cache Insertion Policy (LIP) (Qureshi et al., [ISCA’07]) ▹ Insert incoming line into LRU position, not MRU position 20 a b c d e f g h i PRINCIPLES PRACTICES 1. Retain the reused portion of an event’s footprint in the cache MRU LRU j Exploit Inter-Event Locality

80. 2. Prefetch the unretained part 1. Retain the reused portion

    of an event’s footprint in the cache ▸ LRU Cache Insertion Policy (LIP) (Qureshi et al., [ISCA’07]) ▹ Insert incoming line into LRU position, not MRU position 20 a b c d e f g h i PRINCIPLES PRACTICES 1. Retain the reused portion of an event’s footprint in the cache MRU LRU j Exploit Inter-Event Locality

81. 2. Prefetch the unretained part 1. Retain the reused portion

    of an event’s footprint in the cache ▸ LRU Cache Insertion Policy (LIP) (Qureshi et al., [ISCA’07]) ▹ Insert incoming line into LRU position, not MRU position 20 a b c d e f g h i PRINCIPLES PRACTICES 1. Retain the reused portion of an event’s footprint in the cache MRU LRU j k Exploit Inter-Event Locality

82. 2. Prefetch the unretained part 1. Retain the reused portion

    of an event’s footprint in the cache ▸ LRU Cache Insertion Policy (LIP) (Qureshi et al., [ISCA’07]) ▹ Insert incoming line into LRU position, not MRU position 20 a b c d e f g h i PRINCIPLES PRACTICES 1. Retain the reused portion of an event’s footprint in the cache MRU LRU j k Exploit Inter-Event Locality

83. 2. Prefetch the unretained part 1. Retain the reused portion

    of an event’s footprint in the cache ▸ LRU Cache Insertion Policy (LIP) (Qureshi et al., [ISCA’07]) ▹ Insert incoming line into LRU position, not MRU position 20 a b c d e f g h i PRINCIPLES PRACTICES 1. Retain the reused portion of an event’s footprint in the cache MRU LRU j k … … … … … … … k j Exploit Inter-Event Locality

84. 2. Prefetch the unretained part 1. Retain the reused portion

    of an event’s footprint in the cache ▸ LRU Cache Insertion Policy (LIP) (Qureshi et al., [ISCA’07]) ▹ Insert incoming line into LRU position, not MRU position 20 a b c d e f g h i PRINCIPLES PRACTICES 1. Retain the reused portion of an event’s footprint in the cache MRU LRU j k … … … … … … … k j a Exploit Inter-Event Locality

85. 2. Prefetch the unretained part 1. Retain the reused portion

    of an event’s footprint in the cache ▸ LRU Cache Insertion Policy (LIP) (Qureshi et al., [ISCA’07]) ▹ Insert incoming line into LRU position, not MRU position 20 a b c d e f g h i PRINCIPLES PRACTICES 1. Retain the reused portion of an event’s footprint in the cache MRU LRU j k … … … … … … … k j a MISS Exploit Inter-Event Locality

86. 2. Prefetch the unretained part 1. Retain the reused portion

    of an event’s footprint in the cache ▸ LRU Cache Insertion Policy (LIP) (Qureshi et al., [ISCA’07]) ▹ Insert incoming line into LRU position, not MRU position 20 a b c d e f g h i PRINCIPLES PRACTICES 1. Retain the reused portion of an event’s footprint in the cache MRU LRU j k Inter-event Locality Lost! … … … … … … … k j a MISS Exploit Inter-Event Locality

87. Exploit Inter-Event Locality 2. Prefetch the unretained part ▸ LRU

    Cache Insertion Policy (LIP) (Qureshi et al., [ISCA’07]) ▹ Insert incoming line into LRU position, not MRU position 21 PRINCIPLES PRACTICES 1. Retain the reused portion of an event’s footprint in the cache a b c d e f g h i MRU LRU

88. Exploit Inter-Event Locality 2. Prefetch the unretained part ▸ LRU

    Cache Insertion Policy (LIP) (Qureshi et al., [ISCA’07]) ▹ Insert incoming line into LRU position, not MRU position 21 PRINCIPLES PRACTICES 1. Retain the reused portion of an event’s footprint in the cache a b c d e f g h i MRU LRU j

89. Exploit Inter-Event Locality 2. Prefetch the unretained part ▸ LRU

    Cache Insertion Policy (LIP) (Qureshi et al., [ISCA’07]) ▹ Insert incoming line into LRU position, not MRU position 21 PRINCIPLES PRACTICES 1. Retain the reused portion of an event’s footprint in the cache a b c d e f g h i MRU LRU j

90. Exploit Inter-Event Locality 2. Prefetch the unretained part ▸ LRU

    Cache Insertion Policy (LIP) (Qureshi et al., [ISCA’07]) ▹ Insert incoming line into LRU position, not MRU position 21 PRINCIPLES PRACTICES 1. Retain the reused portion of an event’s footprint in the cache a b c d e f g h i MRU LRU k j

91. Exploit Inter-Event Locality 2. Prefetch the unretained part ▸ LRU

    Cache Insertion Policy (LIP) (Qureshi et al., [ISCA’07]) ▹ Insert incoming line into LRU position, not MRU position 21 PRINCIPLES PRACTICES 1. Retain the reused portion of an event’s footprint in the cache a b c d e f g h i MRU LRU j k

92. Exploit Inter-Event Locality 2. Prefetch the unretained part ▸ LRU

    Cache Insertion Policy (LIP) (Qureshi et al., [ISCA’07]) ▹ Insert incoming line into LRU position, not MRU position 21 PRINCIPLES PRACTICES 1. Retain the reused portion of an event’s footprint in the cache a b c d e f g h i MRU LRU j k a

93. Exploit Inter-Event Locality 2. Prefetch the unretained part ▸ LRU

    Cache Insertion Policy (LIP) (Qureshi et al., [ISCA’07]) ▹ Insert incoming line into LRU position, not MRU position 21 PRINCIPLES PRACTICES 1. Retain the reused portion of an event’s footprint in the cache a b c d e f g h i MRU LRU j k a HIT

94. Exploit Inter-Event Locality 2. Prefetch the unretained part ▸ LRU

    Cache Insertion Policy (LIP) (Qureshi et al., [ISCA’07]) ▹ Insert incoming line into LRU position, not MRU position 21 PRINCIPLES PRACTICES 1. Retain the reused portion of an event’s footprint in the cache a b c d e f g h i MRU LRU j k Reused Portion Retained! a HIT

95. 22 I-cache MPKI 0 25 50 75 100 W ord

    Finder Todo M ud Etherpad Let's C hat Lighter

96. 22 I-cache MPKI 0 25 50 75 100 W ord

    Finder Todo M ud Etherpad Let's C hat Lighter SPEC CPU 2006 Average @ 32KB

97. 23 I-cache MPKI 0 25 50 75 100 W ord

    Finder Todo M ud Etherpad Let's C hat Lighter Baseline

98. 24 I-cache MPKI 0 25 50 75 100 W ord

    Finder Todo M ud Etherpad Let's C hat Lighter Baseline LIP

99. Exploit Inter-Event Locality 2. Prefetch the unretained part 25 PRINCIPLES

    PRACTICES 1. Retain the reused portion of an event’s footprint in the cache

100. Exploit Inter-Event Locality 2. Prefetch the unretained part 25 PRINCIPLES

     PRACTICES 1. Retain the reused portion of an event’s footprint in the cache ▸ Temporal Instruction Fetch Streaming (TIFS) (Ferdman et al., [MICRO’08]) ▹ Find patterns in miss sequences

101. Exploit Inter-Event Locality 2. Prefetch the unretained part 25 PRINCIPLES

     PRACTICES 1. Retain the reused portion of an event’s footprint in the cache ▸ Temporal Instruction Fetch Streaming (TIFS) (Ferdman et al., [MICRO’08]) ▹ Find patterns in miss sequences … …

102. Exploit Inter-Event Locality 2. Prefetch the unretained part 25 PRINCIPLES

     PRACTICES 1. Retain the reused portion of an event’s footprint in the cache ▸ Temporal Instruction Fetch Streaming (TIFS) (Ferdman et al., [MICRO’08]) ▹ Find patterns in miss sequences … … x y z

103. Exploit Inter-Event Locality 2. Prefetch the unretained part 25 PRINCIPLES

     PRACTICES 1. Retain the reused portion of an event’s footprint in the cache ▸ Temporal Instruction Fetch Streaming (TIFS) (Ferdman et al., [MICRO’08]) ▹ Find patterns in miss sequences … … x y z

104. Exploit Inter-Event Locality 2. Prefetch the unretained part 25 PRINCIPLES

     PRACTICES 1. Retain the reused portion of an event’s footprint in the cache ▸ Temporal Instruction Fetch Streaming (TIFS) (Ferdman et al., [MICRO’08]) ▹ Find patterns in miss sequences … … x y z x

105. Exploit Inter-Event Locality 2. Prefetch the unretained part 25 PRINCIPLES

     PRACTICES 1. Retain the reused portion of an event’s footprint in the cache ▸ Temporal Instruction Fetch Streaming (TIFS) (Ferdman et al., [MICRO’08]) ▹ Find patterns in miss sequences … … x y z x

106. Exploit Inter-Event Locality 2. Prefetch the unretained part 25 PRINCIPLES

     PRACTICES 1. Retain the reused portion of an event’s footprint in the cache ▸ Temporal Instruction Fetch Streaming (TIFS) (Ferdman et al., [MICRO’08]) ▹ Find patterns in miss sequences … … x y z x y z

107. Exploit Inter-Event Locality 2. Prefetch the unretained part 25 PRINCIPLES

     PRACTICES 1. Retain the reused portion of an event’s footprint in the cache ▸ Temporal Instruction Fetch Streaming (TIFS) (Ferdman et al., [MICRO’08]) ▹ Find patterns in miss sequences … … x y z x y z y

108. Exploit Inter-Event Locality 2. Prefetch the unretained part 25 PRINCIPLES

     PRACTICES 1. Retain the reused portion of an event’s footprint in the cache ▸ Temporal Instruction Fetch Streaming (TIFS) (Ferdman et al., [MICRO’08]) ▹ Find patterns in miss sequences … … x y z x y z y

109. Exploit Inter-Event Locality 2. Prefetch the unretained part 25 PRINCIPLES

     PRACTICES 1. Retain the reused portion of an event’s footprint in the cache ▸ Temporal Instruction Fetch Streaming (TIFS) (Ferdman et al., [MICRO’08]) ▹ Find patterns in miss sequences … … x y z x y z y z …

110. 26 I-cache MPKI 0 25 50 75 100 W ord

     Finder Todo M ud Etherpad Let's C hat Lighter Baseline LIP

111. 27 I-cache MPKI 0 25 50 75 100 W ord

     Finder Todo M ud Etherpad Let's C hat Lighter Baseline LIP LIP+TIFS

112. 27 I-cache MPKI 0 25 50 75 100 W ord

     Finder Todo M ud Etherpad Let's C hat Lighter Baseline LIP LIP+TIFS 88% Average MPKI Reduction

113. Exploit Instruction Locality a.k.a., Common Case Design 28 SPEC CPU

     (mostly) Instruction Supply Cache Branch Predictor TLB Event-driven Applications Hot instructions Hot branch history patterns Hot pages Cache

114. Exploit Instruction Locality a.k.a., Common Case Design 28 SPEC CPU

     (mostly) Instruction Supply Cache Branch Predictor TLB Event-driven Applications Hot instructions Hot branch history patterns Hot pages Cache Branch Predictor TLB

115. Beyond Instruction Cache — Branch Predictor 29 milc GemsFDTD lbm

     libquantum calculix zeusmp wrf perlbench leslie3d cactusADM hmmer xalancbmk tonto gamess dealII sphinx3 h264ref omnetpp soplex bwaves namd povray mcf gromacs gcc bzip2 sjeng astar gobmk Misprediction (%) 0 5 10 15 20 Word Finder Todo Mud Etherpad Let's Chat Lighter SPEC CPU 2006 NodeJS Tournament Predictor 12-bit history register 256 local branch histories

116. Beyond Instruction Cache — Branch Predictor 29 milc GemsFDTD lbm

     libquantum calculix zeusmp wrf perlbench leslie3d cactusADM hmmer xalancbmk tonto gamess dealII sphinx3 h264ref omnetpp soplex bwaves namd povray mcf gromacs gcc bzip2 sjeng astar gobmk Misprediction (%) 0 5 10 15 20 Word Finder Todo Mud Etherpad Let's Chat Lighter SPEC CPU 2006 NodeJS Tournament Predictor 12-bit history register 256 local branch histories 2.4 X Higher!

117. Beyond Instruction Cache — TLB 30 libquantum lbm hmmer astar

     milc bzip2 mcf namd sjeng bwaves zeusmp cactusADM leslie3d GemsFDTD wrf gromacs sphinx3 gamess calculix h264ref gobmk soplex tonto perlbench dealII omnetpp povray gcc xalancbmk L1 I-TLB MPKI 0 1 2 3 4 Word Finder Todo Mud Etherpad Let's Chat Lighter SPEC CPU 2006 NodeJS I-TLB Parameters 64 KB, 4-way 4 KB page size

118. Beyond Instruction Cache — TLB 30 libquantum lbm hmmer astar

     milc bzip2 mcf namd sjeng bwaves zeusmp cactusADM leslie3d GemsFDTD wrf gromacs sphinx3 gamess calculix h264ref gobmk soplex tonto perlbench dealII omnetpp povray gcc xalancbmk L1 I-TLB MPKI 0 1 2 3 4 Word Finder Todo Mud Etherpad Let's Chat Lighter SPEC CPU 2006 NodeJS 72 X Higher! I-TLB Parameters 64 KB, 4-way 4 KB page size

119. 31

120. 31

121. 31

122. 31 Event-based processing is a fundamental computation pattern.

123. 31 Event-based processing is a fundamental computation pattern. Web Mobile

     Internet-of-Things Sensor networks Cloud

124. 32 MICRO 2015 Microarchitectural Implications of Event-driven Server-side Web Applications

     Yuhao Zhu UT Austin with Daniel Richins, Matthew Halpern, Vijay Janapa Reddi