---
title: "A detector that fires on nothing"
description: "An event trigger set at 2.5 sigma produced 166 events on a device lying still and 151 while something was genuinely happening. Fixing it takes three conditions, not one."
date: 2026-08-18
lang: en
canonical: https://goingstellar.ro/research/detector-that-fires-on-nothing/
author: Going Stellar
publisher: GOING STELLAR S.R.L.
topic: Detection engineering
license: CC BY 4.0
alternate_ro: https://goingstellar.ro/ro/cercetare/detectorul-care-se-declanseaza-pe-nimic/
---

# A detector that fires on nothing

*An event trigger set at 2.5 sigma produced 166 events on a device lying still and 151 while something was genuinely happening. Fixing it takes three conditions, not one.*

> Detection engineering · 2026-08-18 · https://goingstellar.ro/research/detector-that-fires-on-nothing/

## Key points

- A z-score is scale-free, so a device at rest still produces deviations that look unusual relative to its own tiny noise.
- An event detector needs an absolute floor in physical units as well as a statistical threshold, and both conditions must be required on the same channel.
- A detection threshold should be derived from a false-alarm budget and the sampling rate, not guessed.
- Requiring several consecutive qualifying samples guards against the heavy tails created by dividing by an estimated spread.
- A flaky test on a detector is the detector telling you its threshold is arbitrary.

## What we measured

We built an event trigger on a z-score with a threshold of 2.5 sigma. On a simulated device lying still and untouched, it produced 166 events in five minutes. When something was genuinely happening, it produced 151. The rest condition scored higher than the real one.

The cause is not a tuning error. A z-score is **scale-free**. It divides the residual by the spread it has just measured. A device at rest has minuscule deviations, but they are still 2.5 sigma relative to themselves. The sensor was talking to itself.

This matters beyond one sensor, because the same shape appears wherever an alert is defined as "unusual compared to recent history". Unusual is not the same as large, and a detector that only knows the first will happily report the second when there is nothing there. What follows is the three defects we found, all of them real and all of them present at once.

## An absolute floor, in the units of the measurement

The first defect is that a statistical threshold was the only threshold. A z-score answers how unusual a reading is relative to recent noise. It does not answer whether the reading is *large*. A detector of "something happened" needs both answers, and the second one has to be expressed in physical units.

So pair the statistic with a floor: the residual must also exceed a real-world magnitude, well above the hardware's own quantisation step. Below that step you are not measuring the world at all, you are measuring the granularity of the conversion, and the statistics of that granularity are perfectly capable of clearing any scale-free threshold you set.

The subtle part is where the two conditions are evaluated. **Both must be required on the same channel.** If one channel supplies the statistics and another supplies the magnitude, two non-events combine into an event. This is easy to write by accident, and it defeats the floor without looking like it does.

## A threshold derived from a false-alarm budget

The second defect is that 2.5 was a guess. It sounds strict. It is not, once you count how many opportunities the noise gets. At 100 samples a second, thirty minutes is 180,000 opportunities. A z-score above 2.5 occurs about 1.2% of the time, which is roughly 2,200 crossings in that window of nothing at all.

This is a multiple-comparisons problem, and it has an arithmetic answer. Decide how often you are willing to be wrong, then solve for the threshold: **erfc(z / sqrt(2)) = budget / (rate × 3600)**. At one false alarm every ten hours, that gives 5.14 sigma, not 2.5.

Note what the formula contains. The required z depends on the sampling rate. A per-sample probability is not a per-hour probability, and the detector's user experience is measured in hours.

## Persistence, because the spread is estimated too

The third defect survives the first two. The spread in the denominator is itself estimated, on a sliding window, so it fluctuates. A residual divided by a fluctuating estimate has heavier tails than a normal distribution, which blows the nominal budget even after you have derived z correctly from it.

You can model those tails. The cheaper fix, and the one that does not require getting the model right, is **persistence**: require several consecutive qualifying samples before emitting an event. This is physically motivated rather than statistical bookkeeping. A real object holds the reading deviated for hundreds of milliseconds, which is tens of samples. A noise spike holds it for one.

Together, the three conditions — floor, derived threshold, persistence — are what "something happened" actually means in code.

## Where we were wrong: the flaky test

We got the test wrong before we got the detector right. The first version of it passed and failed alternately within the same run, because it used too few seeds.

**A flaky test on a detector is not a flaky test.** It is the detector telling you the threshold is arbitrary: the outcome depended on which noise realisation the run happened to draw.

The correction was more seeds. With enough of them, 166 events at rest against 151 during real activity stopped being a surprise and became the expected consequence of dividing by a spread that shrinks with the signal.

## Running the same check

Verify the effect, not the code. The analysis function was arithmetically correct throughout; what was wrong was what it did. So the check is behavioural: run a long simulated rest period and count the events it emits. A count that exceeds the active period is the same bug we had.

Vary the noise level and the seed, across several values of each. A rest-count that changes with the seed tells you the threshold is arbitrary.

Then test the three conditions separately. Confirm the floor sits above the quantisation step in the sensor's own units. Recompute the threshold from your sampling rate and your false-alarm budget rather than inheriting it. And feed a case where the statistics come from one channel and the magnitude from another: if the detector fires on that, the two conditions are not on the same channel.

## Limitations

The worked example is one sensor at one sampling rate. The arithmetic of the false-alarm budget is general; the specific thresholds are not. 5.14 sigma is the answer to one budget at one rate, not a constant to copy.

The rest condition was simulated. A simulated rest is a model of rest, not real hardware at rest.

Persistence assumes the event you care about lasts longer than the noise does. That held here, because a real object holds the reading deviated for hundreds of milliseconds against a spike's single sample. Where it does not hold — short events, or sampling too slow to see them — this fix does not apply and the tails have to be modelled instead.

## Questions and answers

### Why does a z-score detector fire when nothing is happening?

A z-score detector fires when nothing is happening because a z-score is scale-free: it divides the residual by the spread it has just measured. A device at rest has minuscule deviations, but those deviations still look unusual relative to the device's own noise. Unusual is not the same as large, so a detector that only measures unusualness will report events where there is nothing there.

### How do I set a z-score threshold for alerts?

Set a z-score threshold for alerts by deciding how often you are willing to be wrong and solving for the threshold, rather than guessing. The relationship is erfc(z / sqrt(2)) = budget / (rate × 3600). The required z depends on the sampling rate, because a per-sample probability is not a per-hour probability, and the user experience of a detector is measured in hours.

### Why is a 2.5 sigma threshold too low for a high-rate sensor?

A 2.5 sigma threshold is too low for a high-rate sensor because the noise gets many opportunities to cross it. At 100 samples a second, thirty minutes is 180,000 opportunities, and a z-score above 2.5 occurs about 1.2% of the time. That is roughly 2,200 crossings in a window where nothing happens. This is a multiple-comparisons problem with an arithmetic answer.

### What is an absolute floor in event detection?

An absolute floor in event detection is a minimum magnitude, in the physical units of the measurement, that a residual must exceed alongside the statistical threshold. The floor should sit well above the hardware's quantisation step, because below that step the detector measures the granularity of the conversion rather than the world. The floor and the statistic must be evaluated on the same channel.

### Why should an event detector require persistence?

An event detector should require persistence because the spread in the z-score denominator is estimated on a sliding window and fluctuates, giving heavier tails than a normal distribution. Requiring several consecutive qualifying samples is a cheap fix that does not depend on modelling those tails. A real object holds the reading deviated for tens of samples, while a noise spike holds it for one.

### How can I test whether an event detector has a false-alarm problem?

Test an event detector for a false-alarm problem by running a long simulated rest period and counting the events it emits. A rest count that exceeds the count during real activity indicates the bug. Vary the noise level and the seed; a rest count that changes with the seed means the threshold is arbitrary. Then test the floor, the derived threshold and same-channel evaluation separately.

## How to cite

**Cite this as:** Going Stellar (2026). *A detector that fires on nothing*. GOING STELLAR S.R.L.. https://goingstellar.ro/research/detector-that-fires-on-nothing/

```bibtex
@misc{goingstellar2026detector,
  title  = {A detector that fires on nothing},
  author = {{Going Stellar}},
  year   = {2026},
  url    = {https://goingstellar.ro/research/detector-that-fires-on-nothing/},
  note   = {2026-08-18}
}
```

Licence. Free to quote with attribution.

## Also available

- This page as HTML: https://goingstellar.ro/research/detector-that-fires-on-nothing/
- Same article in Romanian: https://goingstellar.ro/ro/cercetare/detectorul-care-se-declanseaza-pe-nimic/