Foreword
Improving confidence in marketing measurement.
Measurement confidence can degrade in two ways: exposure data goes missing, or impressions get linked to the wrong person. When these issues occur, do they meaningfully distort multi-touch attribution and incrementality? And if so, can the distortions cause marketers to systematically undervalue advertising that is actually working?
New analysis with LiveRamp surfaces three truths CMOs need to hear: the pattern of data loss matters more than the volume, identity precision matters more than match rate, and RCTs hold up where other methods break.
Understanding which risk factor is present, and which measurement design is being used, is essential before acting on attribution, lift, or ROI signals.
Directional, based on synthetic-data simulations. Magnitudes need real-world validation.
What happens when measurement data is incomplete or inconsistently linked?
Not all data problems are created equal.
Overview
What’s in this report
Research thesis
When measurement data is incomplete or inconsistently linked, what happens to attribution and incrementality?Two measurement risk factors
Missing data and identity quality affect results differently.Measurement framework
Attribution and incrementality are related but distinct; failures affect each differently.Evidence
What the data shows across missingness and identity mismatch.Key takeaways
What decision makers need to know.Next steps
Three things you can do now.Odds ratio diagnostic
Assess missingness in your campaigns.
Research thesis
Measurement confidence can degrade in two ways: exposure data goes missing, or impressions get linked to the wrong person.
When these issues occur, do they meaningfully distort multi-touch attribution and incrementality?
Hypothesis 1: Missingness
Missing exposure data (ad impressions that were served but never recorded) can distort attribution models, shift channel rankings, and bias incrementality estimates. The pattern of loss matters more than the volume.
Hypothesis 2: Identity mismatch
Incorrect person-level matching (linking impressions to the wrong individual) can contaminate test-and-control designs, collapse measured lift, and cause profitable campaigns to appear unprofitable.
Can the distortions cause marketers to systematically undervalue advertising that is actually working?
- Ad served
- Identity resolution
- Missingness
- Mismatch
- Attribution Which publisher gets credit?
- Incrementality Did the ad cause conversions?
- Budget decision
Two measurement risk factors
Missing data and identity quality affect results differently
Why this happens in reality
- Missingness
- Safari ITP / cookie deprecation
- Ad blockers
- Cross-device journeys
- Walled garden log restrictions
- Privacy opt-outs / Apple Relay / VPNs
- Identity mismatch
- Probabilistic identity matching
- Shared household devices
- Stale identity links
- Cross-platform stitching errors
Missingness
Real ad exposures that never make it into your data. The ad was served and seen, but the measurement system never recorded it. The data has gaps that underrepresent what actually happened.
Random missingness
Real-world trigger: Impressions dropped uniformly across all users. Generic logging failures, intermittent pixel fires. This is what most stress tests simulate.
Frequency-dependent loss
Real-world trigger: Higher-frequency users are more likely to lose impressions. Users who see more ads hit cookie caps, rotate devices, or trigger anti-tracking sooner.
Audience-dependent loss
Real-world trigger: Loss concentrated among harder-to-match populations. Younger users move more, change devices and email addresses more often, and are less likely to participate in identity data co-ops. ID graphs match them at lower precision.
Overlap-dependent loss
Real-world trigger: Loss concentrated in multi-publisher journeys. Users exposed across several publishers are harder to stitch, so impressions from one publisher drop out while others remain.
Outcome-correlated loss
Real-world trigger: Loss concentrated among converters. High-value customers disproportionately use Safari or opt out at higher rates. The pattern of loss matters more than the volume: even small amounts of outcome-correlated loss can reshuffle channel rankings, while much larger volumes of random loss preserve them.
Hypothesis: the pattern of loss matters far more than the volume. Non-random missingness, even at small scale, can flip channel rankings and invert ROI signals.
Identity mismatch
Ad exposures linked to the wrong person. The system recorded the exposure but attributed it to someone else. The data looks complete; the errors are hidden inside it.
The precision vs. match rate distinction
Identity graphs optimize for recall (aka match rate), casting a wide net to reach as many prospects as possible. This is fine for targeting. But for measurement, precision is what matters: what proportion of matched IDs are actually correct?
High recall with low precision injects false positives into your treatment group. Every false positive dilutes measured lift. Worse, genuinely exposed users can be misclassified into the control group, inflating the baseline. Both biases push measured results downward.
How false positives and false negatives pull measured results down
Consider an RCT where the true conversion rate is 10% for the control group and 13% for the exposed group, a true lift of +3 percentage points (30% relative lift). True ROI is $1.50 per $1 spent. Now suppose identity mismatch contaminates both labels at 30%:
- False positives in the exposed group: 30% of labeled “exposed” were never actually exposed. They convert at the control rate (10%). Observed exposed = 70% × 13% + 30% × 10% = 12.1%.
- False negatives in the control group: 30% of labeled “control” were truly exposed. They convert at the exposed rate (13%). Observed control = 70% × 10% + 30% × 13% = 10.9%.
- Measured lift collapses from +3pp to +1.2pp (30% to 11% relative). Measured ROI drops from $1.50 to $0.60, an apparent loss of $0.40 per dollar spent. False positives dilute the treatment; false negatives inflate the baseline.
Exhibit 1
False positives dilute the treatment; false negatives inflate the baseline
People in an RCT’s labeled control and exposed groups, and the conversion rate, lift and ROI they measure
Source: MMA measurement integrity analysis
Measurement framework
Attribution and incrementality are often conflated, but they are two related, distinct analyses.
Identity resolution failures affect each one differently. In practice, measurement tools may do one or both. Understanding which category your current setup falls into determines how vulnerable your results are to data and identity issues.
- Attribution “Which ads led to a conversion?”
- Incrementality “How much did the ad cause conversions?”
Attribution only
Reports which ads appeared within the lookback window prior to a conversion event. Produces a time-stamped history of impressions without estimating causal impact.
A person sees an ad at home on Wi-Fi, then a second ad on cellular, then converts from a different cell tower. To most attribution systems, this looks like three different people, one of whom converted with no ad exposure. That is identity-driven missingness from journey fragmentation.
More often, the conversion on the cellular connection gets incorrectly matched to someone else who received an impression from the same tower in the lookback window. The result is over-attribution from cellular while the home connection is under-attributed, because the conversion from the cellular tower is matched to a different person.
Vulnerable to both missingness and identity mismatch.
Attribution + incrementality
Reports which ads appeared and estimates each touch’s contribution to conversion. This is where most MTA techniques live.
Fidelity levels within this category:
Gold: RCT / Design of Experiment
Randomized treatment vs. control with placebo or ghost ads, or Intent To Treat (ITT) designs. Enables saturation and creative analysis with time-stamped exposure by person or household. Resilient to many forms of missingness, but identity mismatch can destroy the incrementality signal.
Silver: Quasi-experimental design
Statistical or ML models (logistic regression, Shapley values) comparing exposed vs. unexposed. Supports saturation, frequency, and creative analysis where RCT does not, without complex embellishments to the design. Susceptible to both missingness and mismatch for exposure, plus the inherent challenge of weighting the unexposed sample to represent true exposure. Without the randomization of the gold-standard approach, that weighting carries judgment risk. Best suited for in-flight optimization rather than causal go/no-go ROI decisions.
Bronze: Fixed-formula attribution (“ascription”)
Last Touch, Even Credit, U-shape, Time Decay. Credit assigned by rule, not empirical evidence. The heuristics themselves create distortions; missingness and mismatch compound them further.
Incrementality only
Intent-to-treat (ITT) RCTs. Treatment and control populations are defined in advance. The marketer sends lists to the media owner, holding back the control. Does not require mapping impressions to individuals.
Most resilient to data missingness. Preserves correct per-capita ROI. However, ITT dilutes lift because it counts the entire population in the denominator (reached and unreached). Does not support saturation or creative analysis: no connection to individual impressions.
The weak link: most ITT methods rely on an identity crosswalk from the marketer’s CRM to the media owner’s delivery system. If that crosswalk introduces mismatch at the population-assignment stage, even ITT can be compromised.
ITT is generally augmented with time-stamped impressions for attribution analysis, which relies on a quasi-experimental comparison of exposed vs. matched unexposed for cross-channel optimization and frequency curves. The data-quality caveats above apply.
Most resilient to missingness, but not immune to identity mismatch at the assignment stage.
Evidence
What the data shows across missingness and identity mismatch for attribution & incrementality
| Attribution (which channels get credit) | Incrementality (did the campaign work) | |
|---|---|---|
| Missingness Real ad exposures missing from data | Non-random missingness leads to misallocationHigh risk Random missingness largely preserves channel rankings. But systematic, non-random missingness, even at low volumes, can cause rank reversals across publishers, leading to misallocated budgets. Standard model diagnostics (e.g., AUC) do not reliably flag this. | Some measurement designs understate ROI under missingnessDesign-dependent RCT ITT remained directionally correct across all missingness scenarios tested. Exposure-filtered RCTs and quasi-experimental/modeled designs were more sensitive to distortions and can understate or reverse true ROI, leading to premature cancellation. |
| Identity mismatch Ad exposures linked to the wrong person | Over-linking distorts channel signalsNot yet tested This paper did not simulate this scenario. It is identified as a next step. Hypothesis to be tested: Identity mismatch may contaminate attribution by assigning impressions to the wrong people. Over-linking could inflate some publishers while suppressing others, even when impression counts are complete. | Low precision collapses lift and ROICritical risk For exposure-based RCTs and modeled incrementality, incorrectly matched individuals contaminate test and control groups: lift collapses and ROI falls well below the true value. |
Methodology
The simulations use real impression logs (1.9M exposures, 147,941 users, 4 publishers) and transaction outcomes (12,956 transactions). The attribution model is a logistic regression trained on per-publisher impression counts. The incrementality simulations hold true campaign performance constant at 25% lift and $1.50 ROI and test how different measurement designs read out under each failure mode. Each cell above summarizes what the paper found for that combination of failure mode and measurement type.
- 1.9Mimpressions
- 147,941users
- 4publishers
- 12,956transactions
Chapter 1
The pattern of loss matters more than the volume
Which publisher gets credit?
The full report
Continue reading
4 chapters · 3 interactive exhibits
- 01 Missingness × attribution
- 02 Missingness × incrementality
- 03 Identity mismatch × incrementality
- 04 Conclusions
Not an MMA Member? Please fill out the form below to read this report and learn more about other useful resources.
MMA Members, log in to read now.
Thank you for your submission.
The Missing Piece 4 chapters · 3 interactive exhibits
Continue reading
