Glossary
Every term the pipeline chapters define, in one place.
The chapters are written for engineers rather than statisticians. Where a plain description works as well as the textbook name, the description is what the prose uses — but the textbook name is recorded here too, because it is what you need if you want to read further about a method.
Terminology is deliberately narrow here: dispersion is the engine-reported precision attached to one point, while scatter is the between-commit variation of a reconstructed series. The terms are not aliases.
| Term | What it means | Also called | Introduced in |
|---|---|---|---|
| agreement share | The fraction of before-and-after pairs that agree the level moved in the same direction. | probability of superiority | Noise gates |
| blessing | A recorded decision to treat a change as accepted, so earlier measurements no longer influence analysis. | Reconstruction | |
| census | The report’s account of how many series it judged, and why it did not judge the rest. | Reporting | |
| chance level | How often pure chance alone would produce a pattern at least this strong. | p-value | Detection |
| change point | An estimated location near which a series stops holding one level and starts holding another. | Detection | |
| comparison-base lag | A branch comparison made against base data from several commits back. | Reporting | |
| confidence interval | A range the benchmark engine reports alongside a measurement to say how precisely it pinned it down. | Noise gates | |
| detector | The procedure that examines one series for one shape of change: a change point, a drift, or a branch comparison. | Detection | |
| dirty run | A measurement taken with uncommitted changes in the working tree. | Collection | |
| discriminant set | The engine, target triple, and machine key a run was measured with, which together decide what it may be compared against. | Shape of the data | |
| drift | A series that moves steadily in one direction rather than stepping between levels. | monotonic trend | Detection |
| false-discovery family | In history mode, every series that carried enough data to be tested: the group a statistical finding has to stand out from. | Multiplicity and coverage | |
| finding | A reported move. A history finding survived the group-wide correction; a branch finding exceeded the current base range and survived every applicable gate. | Reporting | |
| ghost | A benchmark that history remembers but the analyzed commit no longer measures. | Reconstruction | |
| group-wide correction | A stricter bar applied when many things are tested at once, so that only a small share of what is reported is expected to be wrong. | Benjamini-Hochberg false discovery rate control | Multiplicity and coverage |
| harvest | Reading whatever output the benchmark engines left behind after a run. | Collection | |
| level | The value a series sits at over a stretch of commits, ignoring run-to-run scatter. | Detection | |
| machine key | A fingerprint of the host hardware, used to keep incomparable results apart. | Collection | |
| median | The middle value of a sample, which a few extreme measurements cannot drag around. | Detection | |
| merge base | The newest commit a branch and its base still share. | Selection | |
| observed range | The lowest through highest values actually recorded in the current base regime. | Detection | |
| one-way-trend check | A test for whether a series mostly moves in one direction, which counts rises against falls rather than fitting a line. | Mann-Kendall test | Detection |
| outlier-resistant slope | A trend line fitted from the middle of all the pairwise slopes, so a few odd measurements cannot tilt it. | Theil-Sen estimator | Detection |
| partition | One discriminant set’s slice of the store. | Collection | |
| rank comparison | A test for whether two regimes differ that weighs each measurement by its rank among all of them rather than by its size, so a few extreme values cannot dominate. Tested two-sided. | Mann-Whitney U test | Detection |
| reference lane | The other half of the base observations, which the branch is compared against and which never helps choose that comparison. | Detection | |
| regime | A stretch of commits over which a series holds one level. | Detection | |
| scatter | Between-commit variation in a series when nothing has changed. | Detection | |
| selector lane | The half of the base observations allowed to decide where the current regime begins. | Detection | |
| series | One metric of one benchmark, tracked across commits. | Reconstruction | |
| split search | A scan for the single most likely place a series changed level. | Pettitt test | Detection |
| typical residual | How far a representative point sits from the history model fitted to the series: a level for a change point, a line for a drift. | median absolute residual | Noise gates |
A note on what these are not
The definitions above are working descriptions, precise enough to follow what this tool does. They are not textbook definitions, and several deliberately gloss over conditions that do not arise here — a rank comparison, for instance, is described as evidence of a difference rather than as the assumptions the test makes about the data.
Where the distinction matters to how the tool behaves, the chapter says so at that point.