Common support: find comparisons the data cannot identify
A comparison model cannot recover a group’s untreated outcome if the data-generating rule never permits that group to be untreated. More flexible regression or a propensity-score model returning a finite number does not create the missing comparison. Common support concerns which intervention contrasts the evidence can identify, not merely whether software can produce a prediction.
The deliverable is a covariate-by-treatment support table tied to a defined target population. Distinguish structural absence from a chance empty sample cell, then state whether the response is new data, a narrower target, explicit extrapolation assumptions or bounds. Do not silently change the population while keeping the original effect label.
Define the target before examining overlap
Suppose a hypothetical study compares a standardized new research workflow, A=1, with an established workflow, A=0. Y is completion time capped at a prespecified 30 minutes, with noncompletion scored 30. Thus the outcome lies between 0 and 30 by design. Negative mean differences favor the new workflow on this particular time measure; quality and failure rates remain separate outcomes.
The intended company population is 70% small and 30% large. Size Z is assumed to be a sufficient baseline adjustment variable for this teaching example. Consistency, no relevant interference and conditional exchangeability given Z are stipulated, rather than inferred from the support table. Positivity is an additional condition: both workflows must be possible within each relevant target-population stratum.
Zivich, Cole and Westreich’s original positivity paper, “Definitions” and “Addressing Positivity Violations,” distinguishes population-level structural support from sample-level sparsity and discusses the added restrictions imposed by statistical models. Its treatment examples motivate the conditions; the company table below is an original illustration.
Expose the absent comparison in a simple table
Assume the following data are representative within their observed strata, and treat their means as population values solely to isolate the support problem from sampling error:
| Company size | Target share | New workflow count | Established count | Mean Y under new | Mean Y under established |
|---|---|---|---|---|---|
| Small | 0.70 | 70 | 70 | 8 | 10 |
| Large | 0.30 | 60 | 0 | 12 | unknown |
A verified operating rule requires every large company in this target setting to use the new workflow. Thus P(A=1|large)=1, not just an empirical 60/60 fraction that happened by chance. Small companies can use either workflow; their probability is 0.50 in this example.
The small-company contrast is 8−10=−2 minutes under the stipulated identification conditions. For the original target, the new-workflow mean is 0.7×8+0.3×12=9.2. The established-workflow mean is 0.7×10+0.3×m0_large=7+0.3m0_large. Therefore:
target average effect = 2.2−0.3m0_large.
The unknown large-company established mean has not disappeared. If it were 10, the target contrast would be −0.8; if it were 20, it would be −3.8. Both choices are compatible with the observed table. A model selecting one number has supplied an extra assumption, not recovered a comparison present in the data.
State the identification gap before choosing an estimator
Structurally, the example defines which workflows are possible in each size group. Identification then asks whether the full target contrast follows from the observed distribution under the specified causal assumptions. It does not follow nonparametrically because the 30% large stratum lacks the established-workflow counterfactual comparison.
Estimation is a later step: even the supported small-company means would have sampling uncertainty in a real finite dataset. Enlarging the large treated sample can make its mean 12 very precise while leaving m0_large unidentified. More observations of the same unsupported treatment pattern do not solve the missing comparison.
Inverse-probability weighting does not create untreated large companies. The relevant untreated probability is zero in that stratum. Replacing the fitted treatment probability 1 by 0.99 to avoid a numerical boundary does not change the actual rule or provide any untreated outcomes. Weight truncation can reduce instability for near-zero probabilities, but it is not a repair for structural zero support.
Restricting the population changes the estimand
One defensible response is to restrict the target to small companies where both workflows are possible. Renormalizing the original small share 0.70 to one gives the small-company contrast −2. Report it as the average contrast for the defined small-company population, under its identification conditions.
Do not retain the name “all-company average effect.” The 30% excluded share is substantively part of the original question, not an inconvenient outlier. A target-population coverage table should show who was removed and why, including changes in size, sector or operating-rule mix caused by restrictions.
Alternatively, external data might observe the established workflow among comparable large companies. Check whether those data use the same treatment versions, time horizon, outcome cap and confounder definitions. A historical period before the rule changed may add time-related differences; it is not automatically exchangeable with the current target.
Use bounds or explicit extrapolation instead of hiding the gap
Because capped Y lies in [0,30], m0_large also lies in that interval. With the observed supported means held fixed, the target effect must lie between 2.2−0.3×30=−6.8 and 2.2−0.3×0=2.2 minutes. These are identification bounds under the stated outcome range and other assumptions, not a confidence interval.
The range includes both signs. The table alone does not establish that the new workflow improves the full target’s mean time. A justified narrower range for the missing mean could tighten the bounds, but its provenance must be stated. A conjectured range chosen to obtain a preferred sign would not be new evidence.
A parametric extrapolation can also be useful as a conditional scenario. For example, assuming the untreated mean is constant across size groups would set m0_large10 and yield −0.8. Label that equality as the assumption carrying the unsupported comparison. A flexible AI model does not remove the need for such restrictions; its learned functional form and training support still determine how it fills the missing cell.
Distinguish a chance empty cell
For an independent boundary, suppose large companies actually have treatment probability 0.90 and the study samples 20 of them independently. The chance of seeing no established-workflow company is 0.9²⁰≈0.1216, about 12.16%. An empty observed cell is therefore possible even though both actions exist in the population.
This sample sparsity differs from the verified all-large-treatment rule. Collecting more relevant independent large-company observations can help the stochastic case; it cannot overturn the structural rule without changing the setting. With many continuous covariates, exact observed combinations are often sparse, so modeling and its form assumptions remain necessary even when population positivity is plausible.
An overlapping one-dimensional propensity-score histogram also does not prove support in every substantively important covariate combination. Inspect the covariates and the actual assignment mechanism, especially rare combinations. Conversely, a small number of rows alone is not proof of a structural impossibility.
Execute a support review with the target attached
| Check | Evidence to preserve | Action if unsupported |
|---|---|---|
| Define target | population weights, intervention and outcome | resolve an ambiguous effect label |
| Identify assignment | rules, exceptions and relevant dates | distinguish structural from sample zeros |
| Tabulate support | counts and outcome availability by baseline strata | mark missing counterfactual cells |
| Assess other assumptions | confounding, consistency and measurement | avoid treating positivity as sufficient |
| Choose response | recruitment, restriction, bounds or extrapolation | state the changed target or extra restriction |
| Report uncertainty | sampling precision and identification gap | do not merge them into one unexplained interval |
If source records conflict with an all-treated rule, investigate genuine exceptions, coding mistakes and period differences before classifying support. Missing workflow labels are unknown assignments, not automatically controls. Preserve the record-level evidence and the classification decision.
Update the judgment when valid comparator data appear, assignment rules change, a target restriction is accepted or external evidence supports the missing-cell assumptions. Keep the old support table and target weights. This makes clear whether the analysis gained information about the original question or merely moved to a question the existing data could answer.
Preserve what the observed mixture does identify
The support gap does not make every descriptive quantity unavailable. Under the table’s stipulated population interpretation, small companies use each workflow with probability 0.50, so their observed mean is 0.5×8+0.5×10=9. Combining size groups gives the natural-course observed mean 0.7×9+0.3×12=9.9.
That 9.9 describes the existing assignment mixture. It is neither the all-new mean 9.2 nor the unobserved all-established mean, and it is not an average treatment effect. Reporting it accurately can still help characterize the current workflow population. Keep descriptive and intervention columns separate so an identified observed average is not used to conceal an unidentified causal contrast. New data that improve precision of 9.9 may leave the original all-treated versus all-untreated question unresolved.
Good cross-validation among observed treatment patterns does not validate the missing untreated-large cell. A prediction can perform well on every held-out observed row while its unsupported counterfactual extrapolation remains untested. Match validation claims to the combinations actually represented in the validation data.
FAQ
Can extreme weights create a missing comparison group?
No. They reweight observed information. A structurally absent untreated stratum needs different evidence, a different target or explicit additional assumptions.
Does an empty sample cell always prove positivity is structurally violated?
No. Under treatment probability 0.90, twenty independent observations can all be treated by chance. Assignment-rule evidence distinguishes that case from impossibility.