Towards a framework for source code instrumentation measurement validation

Ignoring monitoring overhead and not validating measurements are two common mistakes in benchmarking. We extend, apply and evaluate for a practical FFT library our own methodologies for overhead compensation and over-all validation. The overhead and error sources we address include the source instrumentation and separate activities unrelated to the phenomena under study. We are able to quantify the differences with simpler compared with more portable probe technology and relate them to compiler optimization effects. Finally we formulate our framework for validating performance measurement using software accessible counters within existing framework for general software measurements.