When you are optimizing benchmarks, the typical process involves running the same benchmark N
times, and picking an arbitrary run of the benchmark (called a run
) from these N runs to get the representative run. Another option is to average these N runs (creating a new run N') and pick that one as the representative run. In
, we have discussed automatically averaging a bunch of runs. Performance data can be of two types
- Numerical Data (Throughput, Response time, etc)
- Textual Data (OS Patch level, syslog messages, etc.)
Averaging numerical data is very easy. Averaging textual data is not possible, or desired. However, since we are creating a new run N'
, we need to select textual data to be part of this new run. Which run do we pick it from? We are trying to solve this via the
project. If you have any thoughts or suggestions regarding this, please feel free to contact us.