Slide 1

Slide 1 text

No content

Slide 2

Slide 2 text

BEYOND SIMPLE BENCHMARKS A PRACTICAL GUIDE TO OPTIMIZING CODE WITH BENCHMARK.NET  | ✉ |  danielmarbach [email protected] Daniel Marbach

Slide 3

Slide 3 text

"SIMPLE"

Slide 4

Slide 4 text

No content

Slide 5

Slide 5 text

Microsoft Teams’ Infrastructure and Azure Communication Services’ Journey to .NET 6 "We were able to see Azure Compute cost reduction of up to 50% per month, on average we observed 24% monthly cost reduction after migrating to .NET 6. The reduction in cores reduced Azure spend by 24%."

Slide 6

Slide 6 text

PERFORMANCE AWARE

Slide 7

Slide 7 text

BEAR AWARE

Slide 8

Slide 8 text

BE CURIOUS.... UNDERSTAND THE CONTEXT How is this code going to be executed at scale, and what would the memory characteristics be (gut feeling) Are there simple low-hanging fruits I can apply to accelerate this code? Are there things I can move away from the hot path by simply restructuring a bit my code? What part is under my control and what isn't really? What optimizations can I apply, and when should I stop?

Slide 9

Slide 9 text

THE PERFORMANCE LOOP Profile at least CPU and memory using a profiling harness Improve parts of the hot path Benchmark and compare Profile improvements again with the harness and make adjustments where necessary Ship and focus your attention to other parts

Slide 10

Slide 10 text

Queue Code NSERVICEBUS go.particular.net/dotnetday-2023-quickstart

Slide 11

Slide 11 text

Queue Message Pump Behaviors Code ... NSERVICEBUS PIPELINE

Slide 12

Slide 12 text

ASP.NET CORE MIDDLEWARE public class RequestCultureMiddleware { _next = next; public async Task InvokeAsync(HttpContext context) { await _next(context); 1 private readonly RequestDelegate _next; 2 3 public RequestCultureMiddleware(RequestDelegate next) { 4 5 } 6 7 8 // Do work that does something before 9 10 // Do work that does something after 11 } 12 } 13

Slide 13

Slide 13 text

public class Behavior : Behavior { public override Task Invoke(IIncomingLogicalMessageContext context, Func next) { await next(); 1 2 3 // Do work that does something before 4 5 // Do work that does something after 6 } 7 } 8 BEHAVIORS

Slide 14

Slide 14 text

PROFILING THE PIPELINE

Slide 15

Slide 15 text

THE HARNESS Compiled and executed in Release mode Runs a few seconds and keeps overhead minimal Disabled Tiered JIT false Emits full symbols pdbonlytrue var endpointConfiguration = new EndpointConfiguration("PublishSample"); endpointConfiguration.UseSerialization(); var transport = endpointConfiguration.UseTransport(); transport.Routing().RegisterPublisher(typeof(MyEvent), "PublishSample"); endpointConfiguration.UsePersistence(); endpointConfiguration.EnableInstallers(); endpointConfiguration.SendFailedMessagesTo("error"); var endpointInstance = await Endpoint.Start(endpointConfiguration); Console.WriteLine("Attach the profiler and hit ."); Console.ReadLine(); 1 2 3 4 5 6 7 8 9 10 11 12 13 var tasks = new List(1000); 14 for (int i = 0; i < 1000; i++) 15 { 16 tasks.Add(endpointInstance.Publish(new MyEvent())); 17 } 18 await Task.WhenAll(tasks); 19 20 Console.WriteLine("Publish 1000 done. Get a snapshot"); 21 Console.ReadLine(); 22 var transport = endpointConfiguration.UseTransport(); var endpointConfiguration = new EndpointConfiguration("PublishSample"); 1 endpointConfiguration.UseSerialization(); 2 3 transport.Routing().RegisterPublisher(typeof(MyEvent), "PublishSample"); 4 endpointConfiguration.UsePersistence(); 5 endpointConfiguration.EnableInstallers(); 6 endpointConfiguration.SendFailedMessagesTo("error"); 7 8 var endpointInstance = await Endpoint.Start(endpointConfiguration); 9 10 Console.WriteLine("Attach the profiler and hit ."); 11 Console.ReadLine(); 12 13 var tasks = new List(1000); 14 for (int i = 0; i < 1000; i++) 15 { 16 tasks.Add(endpointInstance.Publish(new MyEvent())); 17 } 18 await Task.WhenAll(tasks); 19 20 Console.WriteLine("Publish 1000 done. Get a snapshot"); 21 Console.ReadLine(); 22 endpointConfiguration.UseSerialization(); var endpointConfiguration = new EndpointConfiguration("PublishSample"); 1 2 var transport = endpointConfiguration.UseTransport(); 3 transport.Routing().RegisterPublisher(typeof(MyEvent), "PublishSample"); 4 endpointConfiguration.UsePersistence(); 5 endpointConfiguration.EnableInstallers(); 6 endpointConfiguration.SendFailedMessagesTo("error"); 7 8 var endpointInstance = await Endpoint.Start(endpointConfiguration); 9 10 Console.WriteLine("Attach the profiler and hit ."); 11 Console.ReadLine(); 12 13 var tasks = new List(1000); 14 for (int i = 0; i < 1000; i++) 15 { 16 tasks.Add(endpointInstance.Publish(new MyEvent())); 17 } 18 await Task.WhenAll(tasks); 19 20 Console.WriteLine("Publish 1000 done. Get a snapshot"); 21 Console.ReadLine(); 22 endpointConfiguration.UsePersistence(); var endpointConfiguration = new EndpointConfiguration("PublishSample"); 1 endpointConfiguration.UseSerialization(); 2 var transport = endpointConfiguration.UseTransport(); 3 transport.Routing().RegisterPublisher(typeof(MyEvent), "PublishSample"); 4 5 endpointConfiguration.EnableInstallers(); 6 endpointConfiguration.SendFailedMessagesTo("error"); 7 8 var endpointInstance = await Endpoint.Start(endpointConfiguration); 9 10 Console.WriteLine("Attach the profiler and hit ."); 11 Console.ReadLine(); 12 13 var tasks = new List(1000); 14 for (int i = 0; i < 1000; i++) 15 { 16 tasks.Add(endpointInstance.Publish(new MyEvent())); 17 } 18 await Task.WhenAll(tasks); 19 20 Console.WriteLine("Publish 1000 done. Get a snapshot"); 21 Console.ReadLine(); 22 var tasks = new List(1000); for (int i = 0; i < 1000; i++) { tasks.Add(endpointInstance.Publish(new MyEvent())); } await Task.WhenAll(tasks); var endpointConfiguration = new EndpointConfiguration("PublishSample"); 1 endpointConfiguration.UseSerialization(); 2 var transport = endpointConfiguration.UseTransport(); 3 transport.Routing().RegisterPublisher(typeof(MyEvent), "PublishSample"); 4 endpointConfiguration.UsePersistence(); 5 endpointConfiguration.EnableInstallers(); 6 endpointConfiguration.SendFailedMessagesTo("error"); 7 8 var endpointInstance = await Endpoint.Start(endpointConfiguration); 9 10 Console.WriteLine("Attach the profiler and hit ."); 11 Console.ReadLine(); 12 13 14 15 16 17 18 19 20 Console.WriteLine("Publish 1000 done. Get a snapshot"); 21 Console.ReadLine(); 22 var endpointConfiguration = new EndpointConfiguration("PublishSample"); endpointConfiguration.UseSerialization(); var transport = endpointConfiguration.UseTransport(); transport.Routing().RegisterPublisher(typeof(MyEvent), "PublishSample"); endpointConfiguration.UsePersistence(); endpointConfiguration.EnableInstallers(); endpointConfiguration.SendFailedMessagesTo("error"); var endpointInstance = await Endpoint.Start(endpointConfiguration); Console.WriteLine("Attach the profiler and hit ."); Console.ReadLine(); var tasks = new List(1000); for (int i = 0; i < 1000; i++) { tasks.Add(endpointInstance.Publish(new MyEvent())); } await Task.WhenAll(tasks); Console.WriteLine("Publish 1000 done. Get a snapshot"); Console.ReadLine(); 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 public class MyEventHandler : IHandleMessages { public Task Handle(MyEvent message, IMessageHandlerContext context) { Console.WriteLine("Event received"); return Task.CompletedTask; } }  Profiling the pipeline

Slide 16

Slide 16 text

PUBLISH MEMORY CHARACTERISTICS  Profiling the pipeline

Slide 17

Slide 17 text

RECEIVE MEMORY CHARACTERISTICS  Profiling the pipeline

Slide 18

Slide 18 text

BEHAVIORCHAIN MEMORY CHARACTERISTICS  Profiling the pipeline

Slide 19

Slide 19 text

CONTEXT MATTERS MEMORY CHARACTERISTICS  Profiling the pipeline

Slide 20

Slide 20 text

MEMORY CHARACTERISTICS  Profiling the pipeline

Slide 21

Slide 21 text

MEMORY CHARACTERISTICS  Profiling the pipeline

Slide 22

Slide 22 text

CPU CHARACTERISTICS  Profiling the pipeline

Slide 23

Slide 23 text

CPU CHARACTERISTICS  Profiling the pipeline

Slide 24

Slide 24 text

CPU CHARACTERISTICS PUBLISH  Profiling the pipeline

Slide 25

Slide 25 text

CPU CHARACTERISTICS RECEIVE  Profiling the pipeline

Slide 26

Slide 26 text

CPU CHARACTERISTICS  Profiling the pipeline

Slide 27

Slide 27 text

TESTING

Slide 28

Slide 28 text

 10X faster execution with compiled expression trees  How we achieved 5X faster pipeline execution by removing closure allocations IMPROVING go.particular.net/dotnetday-2023-pipeline

Slide 29

Slide 29 text

BENCHMARKING THE PIPELINE

Slide 30

Slide 30 text

 Benchmarking the pipeline Copy and paste relevant code Adjust it to the bare essentials to create a controllable environment EXTRACT CODE

Slide 31

Slide 31 text

Trim down to relevant behaviors Replaced dependency injection container with creating relevant classes Replaced IO-operations with completed tasks EXTRACT CODE  Benchmarking the pipeline

Slide 32

Slide 32 text

Get started with small steps Culture change takes time Make changes gradually PERFORMANCE CULTURE

Slide 33

Slide 33 text

[ShortRunJob] [MemoryDiagnoser] public class PipelineExecution { [Params(10, 20, 40)] public int PipelineDepth { get; set; } [GlobalSetup] public void SetUp() { behaviorContext = new BehaviorContext(); pipelineModificationsBeforeOptimizations = new PipelineModifications(); for (int i = 0; i < PipelineDepth; i++) { pipelineModificationsBeforeOptimizations.Additions.Add(RegisterStep.Create(i.ToString(), typeof(BaseLineBehavior), i.ToString(), b => new BaseLineBehavior())); } pipelineModificationsAfterOptimizations = new PipelineModifications(); for (int i = 0; i < PipelineDepth; i++) { pipelineModificationsAfterOptimizations.Additions.Add(RegisterStep.Create(i.ToString(), typeof(BehaviorOptimization), i.ToString(), b => new BehaviorOptimization())); } pipelineBeforeOptimizations = new BaseLinePipeline(null, new SettingsHolder(), pipelineModificationsBeforeOptimizations); pipelineAfterOptimizations = new PipelineOptimization(null, new SettingsHolder(), pipelineModificationsAfterOptimizations); } [Benchmark(Baseline = true)] public async Task Before() { await pipelineBeforeOptimizations.Invoke(behaviorContext); } [Benchmark] public async Task After() { await pipelineAfterOptimizations.Invoke(behaviorContext); } } 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42  Benchmarking the pipeline

Slide 34

Slide 34 text

[GlobalSetup] public void SetUp() { behaviorContext = new BehaviorContext(); pipelineModificationsBeforeOptimizations = new PipelineModifications(); for (int i = 0; i < PipelineDepth; i++) { pipelineModificationsBeforeOptimizations.Additions.Add(RegisterStep.Create(i.ToString(), typeof(BaseLineBehavior), i.ToString(), b => new BaseLineBehavior())); } pipelineModificationsAfterOptimizations = new PipelineModifications(); for (int i = 0; i < PipelineDepth; i++) { pipelineModificationsAfterOptimizations.Additions.Add(RegisterStep.Create(i.ToString(), typeof(BehaviorOptimization), i.ToString(), b => new BehaviorOptimization())); } pipelineBeforeOptimizations = new BaseLinePipeline(null, new SettingsHolder(), pipelineModificationsBeforeOptimizations); pipelineAfterOptimizations = new PipelineOptimization(null, new SettingsHolder(), pipelineModificationsAfterOptimizations); } [ShortRunJob] 1 [MemoryDiagnoser] 2 public class PipelineExecution { 3 4 [Params(10, 20, 40)] 5 public int PipelineDepth { get; set; } 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 [Benchmark(Baseline = true)] 33 public async Task Before() { 34 await pipelineBeforeOptimizations.Invoke(behaviorContext); 35 } 36 37 [Benchmark] 38 public async Task After() { 39 await pipelineAfterOptimizations.Invoke(behaviorContext); 40 } 41 } 42  Benchmarking the pipeline

Slide 35

Slide 35 text

[Params(10, 20, 40)] public int PipelineDepth { get; set; } for (int i = 0; i < PipelineDepth; i++) for (int i = 0; i < PipelineDepth; i++) [ShortRunJob] 1 [MemoryDiagnoser] 2 public class PipelineExecution { 3 4 5 6 7 8 [GlobalSetup] 9 public void SetUp() { 10 behaviorContext = new BehaviorContext(); 11 12 pipelineModificationsBeforeOptimizations = new PipelineModifications(); 13 14 { 15 pipelineModificationsBeforeOptimizations.Additions.Add(RegisterStep.Create(i.ToString(), 16 typeof(BaseLineBehavior), i.ToString(), b => new BaseLineBehavior())); 17 } 18 19 pipelineModificationsAfterOptimizations = new PipelineModifications(); 20 21 { 22 pipelineModificationsAfterOptimizations.Additions.Add(RegisterStep.Create(i.ToString(), 23 typeof(BehaviorOptimization), i.ToString(), b => new BehaviorOptimization())); 24 } 25 26 pipelineBeforeOptimizations = new BaseLinePipeline(null, new SettingsHolder(), 27 pipelineModificationsBeforeOptimizations); 28 pipelineAfterOptimizations = new PipelineOptimization(null, new SettingsHolder(), 29 pipelineModificationsAfterOptimizations); 30 } 31 32 [Benchmark(Baseline = true)] 33 public async Task Before() { 34 await pipelineBeforeOptimizations.Invoke(behaviorContext); 35 } 36 37 [Benchmark] 38 public async Task After() { 39 await pipelineAfterOptimizations.Invoke(behaviorContext); 40 } 41 } 42 [ShortRunJob] [MemoryDiagnoser] 1 2 public class PipelineExecution { 3 4 [Params(10, 20, 40)] 5 public int PipelineDepth { get; set; } 6 7 8 [GlobalSetup] 9 public void SetUp() { 10 behaviorContext = new BehaviorContext(); 11 12 pipelineModificationsBeforeOptimizations = new PipelineModifications(); 13 for (int i = 0; i < PipelineDepth; i++) 14 { 15 pipelineModificationsBeforeOptimizations.Additions.Add(RegisterStep.Create(i.ToString(), 16 typeof(BaseLineBehavior), i.ToString(), b => new BaseLineBehavior())); 17 } 18 19 pipelineModificationsAfterOptimizations = new PipelineModifications(); 20 for (int i = 0; i < PipelineDepth; i++) 21 { 22 pipelineModificationsAfterOptimizations.Additions.Add(RegisterStep.Create(i.ToString(), 23 typeof(BehaviorOptimization), i.ToString(), b => new BehaviorOptimization())); 24 } 25 26 pipelineBeforeOptimizations = new BaseLinePipeline(null, new SettingsHolder(), 27 pipelineModificationsBeforeOptimizations); 28 pipelineAfterOptimizations = new PipelineOptimization(null, new SettingsHolder(), 29 pipelineModificationsAfterOptimizations); 30 } 31 32 [Benchmark(Baseline = true)] 33 public async Task Before() { 34 await pipelineBeforeOptimizations.Invoke(behaviorContext); 35 } 36 37 [Benchmark] 38 public async Task After() { 39 await pipelineAfterOptimizations.Invoke(behaviorContext); 40 } 41 } 42 [Benchmark(Baseline = true)] public async Task Before() { await pipelineBeforeOptimizations.Invoke(behaviorContext); } [Benchmark] public async Task After() { await pipelineAfterOptimizations.Invoke(behaviorContext); } [ShortRunJob] 1 [MemoryDiagnoser] 2 public class PipelineExecution { 3 4 [Params(10, 20, 40)] 5 public int PipelineDepth { get; set; } 6 7 8 [GlobalSetup] 9 public void SetUp() { 10 behaviorContext = new BehaviorContext(); 11 12 pipelineModificationsBeforeOptimizations = new PipelineModifications(); 13 for (int i = 0; i < PipelineDepth; i++) 14 { 15 pipelineModificationsBeforeOptimizations.Additions.Add(RegisterStep.Create(i.ToString(), 16 typeof(BaseLineBehavior), i.ToString(), b => new BaseLineBehavior())); 17 } 18 19 pipelineModificationsAfterOptimizations = new PipelineModifications(); 20 for (int i = 0; i < PipelineDepth; i++) 21 { 22 pipelineModificationsAfterOptimizations.Additions.Add(RegisterStep.Create(i.ToString(), 23 typeof(BehaviorOptimization), i.ToString(), b => new BehaviorOptimization())); 24 } 25 26 pipelineBeforeOptimizations = new BaseLinePipeline(null, new SettingsHolder(), 27 pipelineModificationsBeforeOptimizations); 28 pipelineAfterOptimizations = new PipelineOptimization(null, new SettingsHolder(), 29 pipelineModificationsAfterOptimizations); 30 } 31 32 33 34 35 36 37 38 39 40 41 } 42 [ShortRunJob] [MemoryDiagnoser] public class PipelineExecution { [Params(10, 20, 40)] public int PipelineDepth { get; set; } [GlobalSetup] public void SetUp() { behaviorContext = new BehaviorContext(); pipelineModificationsBeforeOptimizations = new PipelineModifications(); for (int i = 0; i < PipelineDepth; i++) { pipelineModificationsBeforeOptimizations.Additions.Add(RegisterStep.Create(i.ToString(), typeof(BaseLineBehavior), i.ToString(), b => new BaseLineBehavior())); } pipelineModificationsAfterOptimizations = new PipelineModifications(); for (int i = 0; i < PipelineDepth; i++) { pipelineModificationsAfterOptimizations.Additions.Add(RegisterStep.Create(i.ToString(), typeof(BehaviorOptimization), i.ToString(), b => new BehaviorOptimization())); } pipelineBeforeOptimizations = new BaseLinePipeline(null, new SettingsHolder(), pipelineModificationsBeforeOptimizations); pipelineAfterOptimizations = new PipelineOptimization(null, new SettingsHolder(), pipelineModificationsAfterOptimizations); } [Benchmark(Baseline = true)] public async Task Before() { await pipelineBeforeOptimizations.Invoke(behaviorContext); } [Benchmark] public async Task After() { await pipelineAfterOptimizations.Invoke(behaviorContext); } } 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42  Benchmarking the pipeline

Slide 36

Slide 36 text

Single Responsibility Principle No side effects Prevents dead code elimination Delegates heavy lifting to the framework Is explicit No implicit casting No var Avoid running any other resource- heavy processes while benchmarking PRACTICES  Benchmarking the pipeline

Slide 37

Slide 37 text

Benchmarking is really hard BenchmarkDotNet will protect you from the common pitfalls because it does all the dirty work for you

Slide 38

Slide 38 text

[ShortRunJob] [MemoryDiagnoser] public class Step1_PipelineWarmup { // rest almost the same [Benchmark(Baseline = true)] public BaseLinePipeline Before() { var pipelineBeforeOptimizations = new BaseLinePipeline(null, new SettingsHolder(), pipelineModificationsBeforeOptimizations); return pipelineBeforeOptimizations; } [Benchmark] public PipelineOptimization After() { var pipelineAfterOptimizations = new PipelineOptimization(null, new SettingsHolder(), pipelineModificationsAfterOptimizations); return pipelineAfterOptimizations; } } 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19  Benchmarking the pipeline

Slide 39

Slide 39 text

[ShortRunJob] [MemoryDiagnoser] public class Step2_PipelineException { [GlobalSetup] public void SetUp() { ... var stepdId = PipelineDepth + 1; pipelineModificationsBeforeOptimizations.Additions.Add(RegisterStep.Create(stepdId.ToString(), typeof(Throwing), "1", b => new Throwing())); ... pipelineModificationsAfterOptimizations.Additions.Add(RegisterStep.Create(stepdId.ToString(), typeof(Throwing), "1", b => new Throwing())); pipelineBeforeOptimizations = new Step1.PipelineOptimization(null, new SettingsHolder(), pipelineModificationsBeforeOptimizations); pipelineAfterOptimizations = new PipelineOptimization(null, new SettingsHolder(), pipelineModificationsAfterOptimizations); } [Benchmark(Baseline = true)] public async Task Before() { try { await pipelineBeforeOptimizations.Invoke(behaviorContext).ConfigureAwait(false); } catch (InvalidOperationException) { } } [Benchmark] public async Task After() { try { await pipelineAfterOptimizations.Invoke(behaviorContext).ConfigureAwait(false); } catch (InvalidOperationException) { } } class Throwing : Behavior { public override Task Invoke(IBehaviorContext context, Func next) { throw new InvalidOperationException(); } } } 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47  Benchmarking the pipeline

Slide 40

Slide 40 text

[ShortRunJob] [MemoryDiagnoser] public class Step2_PipelineException { [GlobalSetup] public void SetUp() { ... var stepdId = PipelineDepth + 1; pipelineModificationsBeforeOptimizations.Additions.Add(RegisterStep.Create(stepd Id.ToString(), typeof(Throwing), "1", b => new Throwing())); ... pipelineModificationsAfterOptimizations.Additions.Add(RegisterStep.Create(stepdI d.ToString(), typeof(Throwing), "1", b => new Throwing())); pipelineBeforeOptimizations = new Step1.PipelineOptimization(null, new SettingsHolder(), pipelineModificationsBeforeOptimizations); pipelineAfterOptimizations = new PipelineOptimization (null, new SettingsHolder(), pipelineModificationsAfterOptimizations); } } 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 pipelineModificationsBeforeOptimizations.Additions.Add(RegisterStep.Create(stepd Id.ToString(), typeof(Throwing), "1", b => new Throwing())); pipelineModificationsAfterOptimizations.Additions.Add(RegisterStep.Create(stepdI d.ToString(), typeof(Throwing), "1", b => new Throwing())); [ShortRunJob] 1 [MemoryDiagnoser] 2 public class Step2_PipelineException { 3 [GlobalSetup] 4 public void SetUp() { 5 ... 6 var stepdId = PipelineDepth + 1; 7 8 9 ... 10 11 12 pipelineBeforeOptimizations = new Step1.PipelineOptimization(null, new SettingsHolder(), 13 pipelineModificationsBeforeOptimizations); 14 pipelineAfterOptimizations = new PipelineOptimization (null, new SettingsHolder(), 15 pipelineModificationsAfterOptimizations); 16 } 17 } 18  Benchmarking the pipeline

Slide 41

Slide 41 text

[ShortRunJob] [MemoryDiagnoser] public class Step2_PipelineException { [GlobalSetup] public void SetUp() { ... } [Benchmark(Baseline = true)] public async Task Before() { try { await pipelineBeforeOptimizations.Invoke(behaviorContext); } catch (InvalidOperationException) { } } [Benchmark] public async Task After() { try { await pipelineAfterOptimizations.Invoke(behaviorContext); } catch (InvalidOperationException) { } } ... } 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31  Benchmarking the pipeline

Slide 42

Slide 42 text

 Benchmarking the pipeline

Slide 43

Slide 43 text

PROFILING THE PIPELINE (AGAIN)

Slide 44

Slide 44 text

 Profiling the pipeline (again) PUBLISH MEMORY CHARACTERISTICS

Slide 45

Slide 45 text

RECEIVE MEMORY CHARACTERISTICS  Profiling the pipeline (again)

Slide 46

Slide 46 text

RECEIVE MEMORY CHARACTERISTICS  Profiling the pipeline (again)

Slide 47

Slide 47 text

MEMORY CHARACTERISTICS  Profiling the pipeline (again)

Slide 48

Slide 48 text

CPU CHARACTERISTICS PUBLISH  Profiling the pipeline (again)

Slide 49

Slide 49 text

CPU CHARACTERISTICS RECEIVE  Profiling the pipeline (again)

Slide 50

Slide 50 text

PREVENTING REGRESSIONS C:\Projects\performance\src\tools\ResultsComparer> dotnet run --base "C:\results\before" --diff "C:\results\after" --threshold 2% 1 2 Guidance Tool Preventing Regressions ResultComparer C:\Projects\performance\src\benchmarks\micro> dotnet run -c Release -f net8.0 \ --artifacts "C:\results\before" 1 2 C:\Projects\performance\src\benchmarks\micro> dotnet run -c Release -f net8.0 \ --artifacts "C:\results\after" 1 2

Slide 51

Slide 51 text

"Two subsequent builds on the same revision can have ranges of 1.5..2 seconds and 12..36 seconds. CPU-bound benchmarks are much more stable than Memory/Disk-bound benchmarks, but the “average” performance levels still can be up to three times different across builds." Andrey Akinshin - Performance stability of GitHub Actions

Slide 52

Slide 52 text

BEYOND SIMPLE BENCHMARKS A PRACTICAL GUIDE TO OPTIMIZING CODE WITH BENCHMARK.NET  | ✉ |  danielmarbach [email protected] Daniel Marbach github.com/danielmarbach/BeyondSimpleBenchmarks Use the performance loop to improve your code where it matters Combine it with profiling to observe how the small changes add up Optimize until you hit a diminishing point of return You'll learn a ton about potential improvements for a new design