goal

The goal of this post is to present a very basic, naive benchmarking paradigm for C#. Benchmarking is hard so we'll leave further optimizations in efficiency and correctness for future posts.

benchmarking

First of all, what is benchmarking? Benchmarking is essentially running a variety of tests/processes across a variety of distinct implementations in order to better understand the performance of each implementation in specific scenarios.

Here's a benchmark between Rust and C gcc: https://benchmarksgame-team.pages.debian.net/benchmarksgame/faster/rust.html

So in order to benchmark, really you only need:

1. At least 1 method of performing a specific task that have the same inputs / outputs. We prefer to have more than one method of performing a specific task so we can use the results to help make a decision on what to actually implement however there are cases where alternative implementations may be non-obvious or non-trivial to implement in which case benchmarking the currently-implemented solution could serve to inform whether it's worthwhile to continue optimizing or to move on to more important tasks.

2. A common data set to input into each distinct method. We prefer a common data set over a different data set for each as the data sets themselves could mask internal optimizations if they differ too much and intuiting when they differ "too much" is hard.

naive benchmarking in C#

// main func

// create data set

var methodOneStartTime = DateTimeOffset.UtcNow;
// execute method one...
var methodOneEndTime = DateTimeOffset.UtcNow;

var methodTwoStartTime = DateTimeOffset.UtcNow;
// execute method two...
var methodTwoStartTime = DateTimeOffset.UtcNow;

// ... do the same for other methods

// print out results