We often talk about bias as if it appears somewhere inside the statistical analysis.
But sometimes the problem is much earlier:
What exactly did we measure?
Imagine a study investigating whether a particular exposure increases the risk of disease.
The exposure is measured using self-reported questionnaires.
The outcome is based on a diagnostic test that is not perfectly sensitive.
Both variables are imperfect.
The statistical model may be completely correct for the data it receives.
But what if the data do not accurately represent the underlying exposure and outcome?
This is where measurement bias becomes important.
📏 Measurement error can occur when an exposure, outcome, confounder, or other variable is measured inaccurately.
One important example is misclassification: participants are assigned to the wrong exposure or outcome category.
And the consequences depend on how that misclassification happens.
If the error is similar across groups, it may attenuate an association — but that should not be treated as a universal rule.
If measurement errors differ between groups, the distortion can be more complicated and may even move the estimate away from the null.
Consider recall.
Suppose researchers ask participants to remember past exposure after they have already developed a disease.
People with the disease may remember or report previous exposures differently from people without it.
The problem isn’t simply that the answers are “a little inaccurate.”
The measurement process itself may be related to the outcome.
That can create differential misclassification.
The same issue appears in many modern data sources:
📱 wearable devices
📝 self-reported behaviours
🧪 diagnostic tests
🏥 electronic health records
💊 medication databases
🌐 online questionnaires
More data does not necessarily mean better measurement.
A million observations of an inaccurately measured variable do not suddenly become a million accurate observations.
And statistical sophistication has limits.
You can adjust for measured confounders.
You can model uncertainty.
You can perform sensitivity analyses.
You can use validation data where available.
But you cannot simply “regress away” information that was never measured correctly.
This is why measurement should be part of study design, not an afterthought in the analysis section.
Before asking:
“Which model should I use?”
ask:
“How accurately does this variable represent the thing I actually want to study?”
Because a sophisticated analysis of poorly measured variables can still produce a very precise answer to the wrong question.
And precision is not the same thing as validity.