.NET Day 2023: Beyond simple benchmarks—A practical guide to optimizing code with BenchmarkDotNet

It is vital for code executed at scale to perform well. It is crucial to ensure performance optimizations actually make the code faster. Luckily, we have powerful tools which help—BenchmarkDotNet is a .NET library for benchmarking optimizations, with plenty of simple examples to help get started. In most systems, the code we need to optimize is rarely simple. It contains assumptions we need to discover before we even know what to improve. The code is hard to isolate. It has dependencies, which may or may not be relevant to optimization. And even when we've decided what to optimize, it's hard to reliably benchmark the before and after. Only measurement can tell us if our changes actually make things faster. Without them, we could even make things slower, without even realizing. Understanding how to create benchmarks is the tip of the iceberg. In this talk, you'll also learn how to identify what to change, how to isolate code for benchmarking, and more. You'll leave with a toolkit of succinct techniques and the confidence to go ahead and optimize your code.

Transcript

2. BEYOND SIMPLE BENCHMARKS A PRACTICAL GUIDE TO OPTIMIZING CODE WITH BENCHMARK.NET  | ✉ |  danielmarbach daniel.marbach@openplace.net Daniel Marbach
3. "SIMPLE"
5. Microsoft Teams’ Infrastructure and Azure Communication Services’ Journey to .NET 6 "We were able to see Azure Compute cost reduction of up to 50% per month, on average we observed 24% monthly cost reduction after migrating to .NET 6. The reduction in cores reduced Azure spend by 24%."
6. PERFORMANCE AWARE
7. BEAR AWARE
8. BE CURIOUS.... UNDERSTAND THE CONTEXT How is this code going to be executed at scale, and what would the memory characteristics be (gut feeling) Are there simple low-hanging fruits I can apply to accelerate this code? Are there things I can move away from the hot path by simply restructuring a bit my code? What part is under my control and what isn't really? What optimizations can I apply, and when should I stop?
9. THE PERFORMANCE LOOP Profile at least CPU and memory using a profiling harness Improve parts of the hot path Benchmark and compare Profile improvements again with the harness and make adjustments where necessary Ship and focus your attention to other parts
10. Queue Code NSERVICEBUS go.particular.net/dotnetday-2023-quickstart
11. Queue Message Pump Behaviors Code ... NSERVICEBUS PIPELINE
12. ASP.NET CORE MIDDLEWARE public class RequestCultureMiddleware { _next = next; public async Task InvokeAsync(HttpContext context) { await _next(context); 1 private readonly RequestDelegate _next; 2 3 public RequestCultureMiddleware(RequestDelegate next) { 4 5 } 6 7 8 // Do work that does something before 9 10 // Do work that does something after 11 } 12 } 13
13. public class Behavior : Behavior<IIncomingLogicalMessageContext> { public override Task Invoke(IIncomingLogicalMessageContext context, Func<Task> next) { await next(); 1 2 3 // Do work that does something before 4 5 // Do work that does something after 6 } 7 } 8 BEHAVIORS
14. PROFILING THE PIPELINE
15. THE HARNESS Compiled and executed in Release mode Runs a few seconds and keeps overhead minimal Disabled Tiered JIT <TieredCompilation>false</TieredCompilation> Emits full symbols <DebugType>pdbonly</DebugType <DebugSymbols>true</DebugSymbols> var endpointConfiguration = new EndpointConfiguration("PublishSample"); endpointConfiguration.UseSerialization<JsonSerializer>(); var transport = endpointConfiguration.UseTransport<MsmqTransport>(); transport.Routing().RegisterPublisher(typeof(MyEvent), "PublishSample"); endpointConfiguration.UsePersistence<InMemoryPersistence>(); endpointConfiguration.EnableInstallers(); endpointConfiguration.SendFailedMessagesTo("error"); var endpointInstance = await Endpoint.Start(endpointConfiguration); Console.WriteLine("Attach the profiler and hit <enter>."); Console.ReadLine(); 1 2 3 4 5 6 7 8 9 10 11 12 13 var tasks = new List<Task>(1000); 14 for (int i = 0; i < 1000; i++) 15 { 16 tasks.Add(endpointInstance.Publish(new MyEvent())); 17 } 18 await Task.WhenAll(tasks); 19 20 Console.WriteLine("Publish 1000 done. Get a snapshot"); 21 Console.ReadLine(); 22 var transport = endpointConfiguration.UseTransport<MsmqTransport>(); var endpointConfiguration = new EndpointConfiguration("PublishSample"); 1 endpointConfiguration.UseSerialization<JsonSerializer>(); 2 3 transport.Routing().RegisterPublisher(typeof(MyEvent), "PublishSample"); 4 endpointConfiguration.UsePersistence<InMemoryPersistence>(); 5 endpointConfiguration.EnableInstallers(); 6 endpointConfiguration.SendFailedMessagesTo("error"); 7 8 var endpointInstance = await Endpoint.Start(endpointConfiguration); 9 10 Console.WriteLine("Attach the profiler and hit <enter>."); 11 Console.ReadLine(); 12 13 var tasks = new List<Task>(1000); 14 for (int i = 0; i < 1000; i++) 15 { 16 tasks.Add(endpointInstance.Publish(new MyEvent())); 17 } 18 await Task.WhenAll(tasks); 19 20 Console.WriteLine("Publish 1000 done. Get a snapshot"); 21 Console.ReadLine(); 22 endpointConfiguration.UseSerialization<JsonSerializer>(); var endpointConfiguration = new EndpointConfiguration("PublishSample"); 1 2 var transport = endpointConfiguration.UseTransport<MsmqTransport>(); 3 transport.Routing().RegisterPublisher(typeof(MyEvent), "PublishSample"); 4 endpointConfiguration.UsePersistence<InMemoryPersistence>(); 5 endpointConfiguration.EnableInstallers(); 6 endpointConfiguration.SendFailedMessagesTo("error"); 7 8 var endpointInstance = await Endpoint.Start(endpointConfiguration); 9 10 Console.WriteLine("Attach the profiler and hit <enter>."); 11 Console.ReadLine(); 12 13 var tasks = new List<Task>(1000); 14 for (int i = 0; i < 1000; i++) 15 { 16 tasks.Add(endpointInstance.Publish(new MyEvent())); 17 } 18 await Task.WhenAll(tasks); 19 20 Console.WriteLine("Publish 1000 done. Get a snapshot"); 21 Console.ReadLine(); 22 endpointConfiguration.UsePersistence<InMemoryPersistence>(); var endpointConfiguration = new EndpointConfiguration("PublishSample"); 1 endpointConfiguration.UseSerialization<JsonSerializer>(); 2 var transport = endpointConfiguration.UseTransport<MsmqTransport>(); 3 transport.Routing().RegisterPublisher(typeof(MyEvent), "PublishSample"); 4 5 endpointConfiguration.EnableInstallers(); 6 endpointConfiguration.SendFailedMessagesTo("error"); 7 8 var endpointInstance = await Endpoint.Start(endpointConfiguration); 9 10 Console.WriteLine("Attach the profiler and hit <enter>."); 11 Console.ReadLine(); 12 13 var tasks = new List<Task>(1000); 14 for (int i = 0; i < 1000; i++) 15 { 16 tasks.Add(endpointInstance.Publish(new MyEvent())); 17 } 18 await Task.WhenAll(tasks); 19 20 Console.WriteLine("Publish 1000 done. Get a snapshot"); 21 Console.ReadLine(); 22 var tasks = new List<Task>(1000); for (int i = 0; i < 1000; i++) { tasks.Add(endpointInstance.Publish(new MyEvent())); } await Task.WhenAll(tasks); var endpointConfiguration = new EndpointConfiguration("PublishSample"); 1 endpointConfiguration.UseSerialization<JsonSerializer>(); 2 var transport = endpointConfiguration.UseTransport<MsmqTransport>(); 3 transport.Routing().RegisterPublisher(typeof(MyEvent), "PublishSample"); 4 endpointConfiguration.UsePersistence<InMemoryPersistence>(); 5 endpointConfiguration.EnableInstallers(); 6 endpointConfiguration.SendFailedMessagesTo("error"); 7 8 var endpointInstance = await Endpoint.Start(endpointConfiguration); 9 10 Console.WriteLine("Attach the profiler and hit <enter>."); 11 Console.ReadLine(); 12 13 14 15 16 17 18 19 20 Console.WriteLine("Publish 1000 done. Get a snapshot"); 21 Console.ReadLine(); 22 var endpointConfiguration = new EndpointConfiguration("PublishSample"); endpointConfiguration.UseSerialization<JsonSerializer>(); var transport = endpointConfiguration.UseTransport<MsmqTransport>(); transport.Routing().RegisterPublisher(typeof(MyEvent), "PublishSample"); endpointConfiguration.UsePersistence<InMemoryPersistence>(); endpointConfiguration.EnableInstallers(); endpointConfiguration.SendFailedMessagesTo("error"); var endpointInstance = await Endpoint.Start(endpointConfiguration); Console.WriteLine("Attach the profiler and hit <enter>."); Console.ReadLine(); var tasks = new List<Task>(1000); for (int i = 0; i < 1000; i++) { tasks.Add(endpointInstance.Publish(new MyEvent())); } await Task.WhenAll(tasks); Console.WriteLine("Publish 1000 done. Get a snapshot"); Console.ReadLine(); 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 public class MyEventHandler : IHandleMessages<MyEvent> { public Task Handle(MyEvent message, IMessageHandlerContext context) { Console.WriteLine("Event received"); return Task.CompletedTask; } }  Profiling the pipeline
16. PUBLISH MEMORY CHARACTERISTICS  Profiling the pipeline
17. RECEIVE MEMORY CHARACTERISTICS  Profiling the pipeline
18. BEHAVIORCHAIN MEMORY CHARACTERISTICS  Profiling the pipeline
19. CONTEXT MATTERS MEMORY CHARACTERISTICS  Profiling the pipeline
20. MEMORY CHARACTERISTICS  Profiling the pipeline
21. MEMORY CHARACTERISTICS  Profiling the pipeline
22. CPU CHARACTERISTICS  Profiling the pipeline
23. CPU CHARACTERISTICS  Profiling the pipeline
24. CPU CHARACTERISTICS PUBLISH  Profiling the pipeline
25. CPU CHARACTERISTICS RECEIVE  Profiling the pipeline
26. CPU CHARACTERISTICS  Profiling the pipeline
27. TESTING
28.  10X faster execution with compiled expression trees  How we achieved 5X faster pipeline execution by removing closure allocations IMPROVING go.particular.net/dotnetday-2023-pipeline
29. BENCHMARKING THE PIPELINE
30.  Benchmarking the pipeline Copy and paste relevant code Adjust it to the bare essentials to create a controllable environment EXTRACT CODE
31. Trim down to relevant behaviors Replaced dependency injection container with creating relevant classes Replaced IO-operations with completed tasks EXTRACT CODE  Benchmarking the pipeline
32. Get started with small steps Culture change takes time Make changes gradually PERFORMANCE CULTURE
33. [ShortRunJob] [MemoryDiagnoser] public class PipelineExecution { [Params(10, 20, 40)] public int PipelineDepth { get; set; } [GlobalSetup] public void SetUp() { behaviorContext = new BehaviorContext(); pipelineModificationsBeforeOptimizations = new PipelineModifications(); for (int i = 0; i < PipelineDepth; i++) { pipelineModificationsBeforeOptimizations.Additions.Add(RegisterStep.Create(i.ToString(), typeof(BaseLineBehavior), i.ToString(), b => new BaseLineBehavior())); } pipelineModificationsAfterOptimizations = new PipelineModifications(); for (int i = 0; i < PipelineDepth; i++) { pipelineModificationsAfterOptimizations.Additions.Add(RegisterStep.Create(i.ToString(), typeof(BehaviorOptimization), i.ToString(), b => new BehaviorOptimization())); } pipelineBeforeOptimizations = new BaseLinePipeline<IBehaviorContext>(null, new SettingsHolder(), pipelineModificationsBeforeOptimizations); pipelineAfterOptimizations = new PipelineOptimization<IBehaviorContext>(null, new SettingsHolder(), pipelineModificationsAfterOptimizations); } [Benchmark(Baseline = true)] public async Task Before() { await pipelineBeforeOptimizations.Invoke(behaviorContext); } [Benchmark] public async Task After() { await pipelineAfterOptimizations.Invoke(behaviorContext); } } 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42  Benchmarking the pipeline
34. [GlobalSetup] public void SetUp() { behaviorContext = new BehaviorContext(); pipelineModificationsBeforeOptimizations = new PipelineModifications(); for (int i = 0; i < PipelineDepth; i++) { pipelineModificationsBeforeOptimizations.Additions.Add(RegisterStep.Create(i.ToString(), typeof(BaseLineBehavior), i.ToString(), b => new BaseLineBehavior())); } pipelineModificationsAfterOptimizations = new PipelineModifications(); for (int i = 0; i < PipelineDepth; i++) { pipelineModificationsAfterOptimizations.Additions.Add(RegisterStep.Create(i.ToString(), typeof(BehaviorOptimization), i.ToString(), b => new BehaviorOptimization())); } pipelineBeforeOptimizations = new BaseLinePipeline<IBehaviorContext>(null, new SettingsHolder(), pipelineModificationsBeforeOptimizations); pipelineAfterOptimizations = new PipelineOptimization<IBehaviorContext>(null, new SettingsHolder(), pipelineModificationsAfterOptimizations); } [ShortRunJob] 1 [MemoryDiagnoser] 2 public class PipelineExecution { 3 4 [Params(10, 20, 40)] 5 public int PipelineDepth { get; set; } 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 [Benchmark(Baseline = true)] 33 public async Task Before() { 34 await pipelineBeforeOptimizations.Invoke(behaviorContext); 35 } 36 37 [Benchmark] 38 public async Task After() { 39 await pipelineAfterOptimizations.Invoke(behaviorContext); 40 } 41 } 42  Benchmarking the pipeline
35. [Params(10, 20, 40)] public int PipelineDepth { get; set; } for (int i = 0; i < PipelineDepth; i++) for (int i = 0; i < PipelineDepth; i++) [ShortRunJob] 1 [MemoryDiagnoser] 2 public class PipelineExecution { 3 4 5 6 7 8 [GlobalSetup] 9 public void SetUp() { 10 behaviorContext = new BehaviorContext(); 11 12 pipelineModificationsBeforeOptimizations = new PipelineModifications(); 13 14 { 15 pipelineModificationsBeforeOptimizations.Additions.Add(RegisterStep.Create(i.ToString(), 16 typeof(BaseLineBehavior), i.ToString(), b => new BaseLineBehavior())); 17 } 18 19 pipelineModificationsAfterOptimizations = new PipelineModifications(); 20 21 { 22 pipelineModificationsAfterOptimizations.Additions.Add(RegisterStep.Create(i.ToString(), 23 typeof(BehaviorOptimization), i.ToString(), b => new BehaviorOptimization())); 24 } 25 26 pipelineBeforeOptimizations = new BaseLinePipeline<IBehaviorContext>(null, new SettingsHolder(), 27 pipelineModificationsBeforeOptimizations); 28 pipelineAfterOptimizations = new PipelineOptimization<IBehaviorContext>(null, new SettingsHolder(), 29 pipelineModificationsAfterOptimizations); 30 } 31 32 [Benchmark(Baseline = true)] 33 public async Task Before() { 34 await pipelineBeforeOptimizations.Invoke(behaviorContext); 35 } 36 37 [Benchmark] 38 public async Task After() { 39 await pipelineAfterOptimizations.Invoke(behaviorContext); 40 } 41 } 42 [ShortRunJob] [MemoryDiagnoser] 1 2 public class PipelineExecution { 3 4 [Params(10, 20, 40)] 5 public int PipelineDepth { get; set; } 6 7 8 [GlobalSetup] 9 public void SetUp() { 10 behaviorContext = new BehaviorContext(); 11 12 pipelineModificationsBeforeOptimizations = new PipelineModifications(); 13 for (int i = 0; i < PipelineDepth; i++) 14 { 15 pipelineModificationsBeforeOptimizations.Additions.Add(RegisterStep.Create(i.ToString(), 16 typeof(BaseLineBehavior), i.ToString(), b => new BaseLineBehavior())); 17 } 18 19 pipelineModificationsAfterOptimizations = new PipelineModifications(); 20 for (int i = 0; i < PipelineDepth; i++) 21 { 22 pipelineModificationsAfterOptimizations.Additions.Add(RegisterStep.Create(i.ToString(), 23 typeof(BehaviorOptimization), i.ToString(), b => new BehaviorOptimization())); 24 } 25 26 pipelineBeforeOptimizations = new BaseLinePipeline<IBehaviorContext>(null, new SettingsHolder(), 27 pipelineModificationsBeforeOptimizations); 28 pipelineAfterOptimizations = new PipelineOptimization<IBehaviorContext>(null, new SettingsHolder(), 29 pipelineModificationsAfterOptimizations); 30 } 31 32 [Benchmark(Baseline = true)] 33 public async Task Before() { 34 await pipelineBeforeOptimizations.Invoke(behaviorContext); 35 } 36 37 [Benchmark] 38 public async Task After() { 39 await pipelineAfterOptimizations.Invoke(behaviorContext); 40 } 41 } 42 [Benchmark(Baseline = true)] public async Task Before() { await pipelineBeforeOptimizations.Invoke(behaviorContext); } [Benchmark] public async Task After() { await pipelineAfterOptimizations.Invoke(behaviorContext); } [ShortRunJob] 1 [MemoryDiagnoser] 2 public class PipelineExecution { 3 4 [Params(10, 20, 40)] 5 public int PipelineDepth { get; set; } 6 7 8 [GlobalSetup] 9 public void SetUp() { 10 behaviorContext = new BehaviorContext(); 11 12 pipelineModificationsBeforeOptimizations = new PipelineModifications(); 13 for (int i = 0; i < PipelineDepth; i++) 14 { 15 pipelineModificationsBeforeOptimizations.Additions.Add(RegisterStep.Create(i.ToString(), 16 typeof(BaseLineBehavior), i.ToString(), b => new BaseLineBehavior())); 17 } 18 19 pipelineModificationsAfterOptimizations = new PipelineModifications(); 20 for (int i = 0; i < PipelineDepth; i++) 21 { 22 pipelineModificationsAfterOptimizations.Additions.Add(RegisterStep.Create(i.ToString(), 23 typeof(BehaviorOptimization), i.ToString(), b => new BehaviorOptimization())); 24 } 25 26 pipelineBeforeOptimizations = new BaseLinePipeline<IBehaviorContext>(null, new SettingsHolder(), 27 pipelineModificationsBeforeOptimizations); 28 pipelineAfterOptimizations = new PipelineOptimization<IBehaviorContext>(null, new SettingsHolder(), 29 pipelineModificationsAfterOptimizations); 30 } 31 32 33 34 35 36 37 38 39 40 41 } 42 [ShortRunJob] [MemoryDiagnoser] public class PipelineExecution { [Params(10, 20, 40)] public int PipelineDepth { get; set; } [GlobalSetup] public void SetUp() { behaviorContext = new BehaviorContext(); pipelineModificationsBeforeOptimizations = new PipelineModifications(); for (int i = 0; i < PipelineDepth; i++) { pipelineModificationsBeforeOptimizations.Additions.Add(RegisterStep.Create(i.ToString(), typeof(BaseLineBehavior), i.ToString(), b => new BaseLineBehavior())); } pipelineModificationsAfterOptimizations = new PipelineModifications(); for (int i = 0; i < PipelineDepth; i++) { pipelineModificationsAfterOptimizations.Additions.Add(RegisterStep.Create(i.ToString(), typeof(BehaviorOptimization), i.ToString(), b => new BehaviorOptimization())); } pipelineBeforeOptimizations = new BaseLinePipeline<IBehaviorContext>(null, new SettingsHolder(), pipelineModificationsBeforeOptimizations); pipelineAfterOptimizations = new PipelineOptimization<IBehaviorContext>(null, new SettingsHolder(), pipelineModificationsAfterOptimizations); } [Benchmark(Baseline = true)] public async Task Before() { await pipelineBeforeOptimizations.Invoke(behaviorContext); } [Benchmark] public async Task After() { await pipelineAfterOptimizations.Invoke(behaviorContext); } } 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42  Benchmarking the pipeline
36. Single Responsibility Principle No side effects Prevents dead code elimination Delegates heavy lifting to the framework Is explicit No implicit casting No var Avoid running any other resource- heavy processes while benchmarking PRACTICES  Benchmarking the pipeline
37. Benchmarking is really hard BenchmarkDotNet will protect you from the common pitfalls because it does all the dirty work for you
38. [ShortRunJob] [MemoryDiagnoser] public class Step1_PipelineWarmup { // rest almost the same [Benchmark(Baseline = true)] public BaseLinePipeline<IBehaviorContext> Before() { var pipelineBeforeOptimizations = new BaseLinePipeline<IBehaviorContext>(null, new SettingsHolder(), pipelineModificationsBeforeOptimizations); return pipelineBeforeOptimizations; } [Benchmark] public PipelineOptimization<IBehaviorContext> After() { var pipelineAfterOptimizations = new PipelineOptimization<IBehaviorContext>(null, new SettingsHolder(), pipelineModificationsAfterOptimizations); return pipelineAfterOptimizations; } } 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19  Benchmarking the pipeline
39. [ShortRunJob] [MemoryDiagnoser] public class Step2_PipelineException { [GlobalSetup] public void SetUp() { ... var stepdId = PipelineDepth + 1; pipelineModificationsBeforeOptimizations.Additions.Add(RegisterStep.Create(stepdId.ToString(), typeof(Throwing), "1", b => new Throwing())); ... pipelineModificationsAfterOptimizations.Additions.Add(RegisterStep.Create(stepdId.ToString(), typeof(Throwing), "1", b => new Throwing())); pipelineBeforeOptimizations = new Step1.PipelineOptimization<IBehaviorContext>(null, new SettingsHolder(), pipelineModificationsBeforeOptimizations); pipelineAfterOptimizations = new PipelineOptimization<IBehaviorContext>(null, new SettingsHolder(), pipelineModificationsAfterOptimizations); } [Benchmark(Baseline = true)] public async Task Before() { try { await pipelineBeforeOptimizations.Invoke(behaviorContext).ConfigureAwait(false); } catch (InvalidOperationException) { } } [Benchmark] public async Task After() { try { await pipelineAfterOptimizations.Invoke(behaviorContext).ConfigureAwait(false); } catch (InvalidOperationException) { } } class Throwing : Behavior<IBehaviorContext> { public override Task Invoke(IBehaviorContext context, Func<Task> next) { throw new InvalidOperationException(); } } } 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47  Benchmarking the pipeline
40. [ShortRunJob] [MemoryDiagnoser] public class Step2_PipelineException { [GlobalSetup] public void SetUp() { ... var stepdId = PipelineDepth + 1; pipelineModificationsBeforeOptimizations.Additions.Add(RegisterStep.Create(stepd Id.ToString(), typeof(Throwing), "1", b => new Throwing())); ... pipelineModificationsAfterOptimizations.Additions.Add(RegisterStep.Create(stepdI d.ToString(), typeof(Throwing), "1", b => new Throwing())); pipelineBeforeOptimizations = new Step1.PipelineOptimization<IBehaviorContext>(null, new SettingsHolder(), pipelineModificationsBeforeOptimizations); pipelineAfterOptimizations = new PipelineOptimization<IBehaviorContext> (null, new SettingsHolder(), pipelineModificationsAfterOptimizations); } } 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 pipelineModificationsBeforeOptimizations.Additions.Add(RegisterStep.Create(stepd Id.ToString(), typeof(Throwing), "1", b => new Throwing())); pipelineModificationsAfterOptimizations.Additions.Add(RegisterStep.Create(stepdI d.ToString(), typeof(Throwing), "1", b => new Throwing())); [ShortRunJob] 1 [MemoryDiagnoser] 2 public class Step2_PipelineException { 3 [GlobalSetup] 4 public void SetUp() { 5 ... 6 var stepdId = PipelineDepth + 1; 7 8 9 ... 10 11 12 pipelineBeforeOptimizations = new Step1.PipelineOptimization<IBehaviorContext>(null, new SettingsHolder(), 13 pipelineModificationsBeforeOptimizations); 14 pipelineAfterOptimizations = new PipelineOptimization<IBehaviorContext> (null, new SettingsHolder(), 15 pipelineModificationsAfterOptimizations); 16 } 17 } 18  Benchmarking the pipeline
41. [ShortRunJob] [MemoryDiagnoser] public class Step2_PipelineException { [GlobalSetup] public void SetUp() { ... } [Benchmark(Baseline = true)] public async Task Before() { try { await pipelineBeforeOptimizations.Invoke(behaviorContext); } catch (InvalidOperationException) { } } [Benchmark] public async Task After() { try { await pipelineAfterOptimizations.Invoke(behaviorContext); } catch (InvalidOperationException) { } } ... } 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31  Benchmarking the pipeline
42.  Benchmarking the pipeline
43. PROFILING THE PIPELINE (AGAIN)
44.  Profiling the pipeline (again) PUBLISH MEMORY CHARACTERISTICS
45. RECEIVE MEMORY CHARACTERISTICS  Profiling the pipeline (again)
46. RECEIVE MEMORY CHARACTERISTICS  Profiling the pipeline (again)
47. MEMORY CHARACTERISTICS  Profiling the pipeline (again)
48. CPU CHARACTERISTICS PUBLISH  Profiling the pipeline (again)
49. CPU CHARACTERISTICS RECEIVE  Profiling the pipeline (again)
50. PREVENTING REGRESSIONS C:\Projects\performance\src\tools\ResultsComparer> dotnet run --base "C:\results\before" --diff "C:\results\after" --threshold 2% 1 2 Guidance Tool Preventing Regressions ResultComparer C:\Projects\performance\src\benchmarks\micro> dotnet run -c Release -f net8.0 \ --artifacts "C:\results\before" 1 2 C:\Projects\performance\src\benchmarks\micro> dotnet run -c Release -f net8.0 \ --artifacts "C:\results\after" 1 2
51. "Two subsequent builds on the same revision can have ranges of 1.5..2 seconds and 12..36 seconds. CPU-bound benchmarks are much more stable than Memory/Disk-bound benchmarks, but the “average” performance levels still can be up to three times different across builds." Andrey Akinshin - Performance stability of GitHub Actions
52. BEYOND SIMPLE BENCHMARKS A PRACTICAL GUIDE TO OPTIMIZING CODE WITH BENCHMARK.NET  | ✉ |  danielmarbach daniel.marbach@openplace.net Daniel Marbach github.com/danielmarbach/BeyondSimpleBenchmarks Use the performance loop to improve your code where it matters Combine it with profiling to observe how the small changes add up Optimize until you hit a diminishing point of return You'll learn a ton about potential improvements for a new design