I would like to benchmark my rustcode with criterion, but the information it presents to me do not match up. Here is some of the output I get
Benchmarking big/big decoding: Warming up for 1.0000 s
Warning: Unable to complete 10 samples in 5.0s. You may wish to increase target time to 306.4s.
big/big decoding time: [632.40 ns 724.40 ns 899.80 ns]
change: [+0.7649% +16.645% +46.889%] (p = 0.04 < 0.05)
Change within noise threshold.
Found 2 outliers among 10 measurements (20.00%)
1 (10.00%) high mild
1 (10.00%) high severe
If it took a criterion only 724.40 ns to run the code, why couldn’t it run the code 10 times in 5 seconds.
I looked at the outliers in the browser and the one outlier took 1.5 microseconds to run so this could not be the problem.