Cutoff studies: audit the assignment rule before a causal claim
A rule that changes at a score threshold can create a useful comparison, but a difference between all companies below and above the threshold is not automatically a causal effect. The identification argument concerns behavior arbitrarily near the cutoff and the mechanism that changes treatment there. The score rule, actual receipt and outcome discontinuity need separate evidence.
The deliverable is a cutoff audit card: the exact assignment rule, local support, treatment jump, outcome jump, identification conditions and population to which the result applies. Estimation choices such as bandwidth and curve shape come after that design argument.
Reconstruct the rule before drawing a graph
Assume a hypothetical company-support program offers a standardized research subsidy when an eligibility score R is strictly below 50. Define Z=1 for R<50 and Z=0 otherwise. A company scored exactly 50 is on the ineligible side. Preserve the score version, calculation date, application cohort, appeals and exceptions rather than reconstructing eligibility from eventual payment.
Let D indicate actual subsidy receipt and Y a later research-quality outcome measured on a common scale and horizon. Assignment Z and receipt D are different variables. Some eligible companies may decline and some otherwise ineligible companies may receive support through exceptions.
World Bank DIME’s regression-discontinuity discussion, under the design appendix’s RD heading, connects the running variable, rule, local window and validation. Its general overview does not mean that adding the score to any regression automatically identifies an effect. The numerical example here is original.
Specify local limits rather than whole-side means
Suppose appropriately estimated outcome limits as R approaches 50 are 60 from below and 54 from above. Actual receipt-probability limits are 0.80 and 0.20. Use below minus above throughout:
| Quantity near the cutoff | Below 50 limit | Above 50 limit | Discontinuity |
|---|---|---|---|
| Eligibility Z | 1 | 0 | 1 |
| Receipt probability | 0.80 | 0.20 | 0.60 |
| Expected outcome | 60 | 54 | 6 |
These are hypothetical limits, not raw means for every score on each side. To see the difference, suppose the local expected-outcome functions are 60+(R−50) below and 54+(R−50) above. At scores 49 and 51 the expected outcomes are 59 and 55, a difference 4. The limiting jump is still 6. Even the nearest two score bins need not equal the discontinuity if there is a smooth score gradient.
The outcome jump 6 describes the local effect of crossing the eligibility boundary only under the relevant identification conditions. It should not immediately be called the effect of receiving a subsidy, because actual receipt changes by 0.60 rather than one.
Separate sharp and fuzzy designs
In a sharp alternative protocol, every below-cutoff company receives treatment and no above-cutoff company does. Then the receipt jump is one, and an identified local outcome discontinuity 6 corresponds to the sharp treatment contrast at the cutoff.
The main example is fuzzy: receipt jumps by 0.60. The ratio of outcome and receipt discontinuities is 6/0.60=10. Arithmetic alone does not make 10 a causal effect. It is the local Wald ratio whose interpretation requires a valid fuzzy-RD argument.
Cattaneo and Titiunik’s 2022 original review, section 2’s sharp/fuzzy discussions and the fuzzy-RD identification passages, distinguishes continuity and local-randomization frameworks. Under appropriate continuity, exclusion and monotonicity conditions, the fuzzy contrast can describe an average effect for local compliers at the cutoff. Those are companies whose receipt responds to eligibility, not every recipient or every company in the program.
State which framework is being used. A narrow score window does not automatically become a randomized experiment. A continuity-based limiting comparison and a justified local-randomization analysis use different assumptions; their labels should not be mixed merely because both focus near 50.
Audit the identification conditions
For the continuity argument, potential-outcome relationships need to behave smoothly through the cutoff in the absence of the assignment change. A separate policy that also starts at 50 can violate the proposed isolation of the subsidy mechanism. For fuzzy interpretation, crossing eligibility must not affect Y through a pathway other than the defined receipt treatment.
For example, an eligibility letter might itself provide training information that improves quality even among nonrecipients. Then Z can affect Y without D, threatening the exclusion restriction. Alternatively, the threshold might also trigger a tax benefit. The observed discontinuity may describe a policy bundle rather than the subsidy alone.
Monotonicity rules out local units whose receipt is reduced by being assigned eligibility in the relevant formulation. A positive average first stage does not prove that no such units exist. Treatment versions, interference and a nonzero receipt jump also need attention. A zero first-stage jump makes the simple ratio undefined; a weak estimated jump can make ratio inference unstable.
Do not attach an interval to 10 by independently dividing confidence endpoints from two unrelated tables. The estimated outcome and receipt jumps have joint sampling uncertainty, and weak-first-stage inference needs particular care. Retain the paired estimation inputs and the actual inference method.
Use falsification evidence without treating it as certification
Inspect the score distribution around 50, original scoring records and opportunities to sort or appeal. A mass just below the rule can indicate manipulation, administrative rounding or a real score feature; investigate the mechanism rather than choosing an automatic verdict.
The 2022 review’s section 4.2 explains that score-density continuity is neither necessary nor sufficient for RD validity. A nonsignificant density test does not prove that companies near the threshold are comparable. Check predetermined covariates, placebo outcomes and placebo cutoffs as complementary evidence, while recognizing their power and assumptions.
If scores take only widely separated values, collecting more companies at the same few values does not create observations arbitrarily close to 50. Extrapolating curves across a score gap needs additional restrictions. A local-randomization claim over discrete scores also needs its own justification; it is not supplied by the word “local.”
Keep estimation choices tied to the design
Record the bandwidth, separate-side functional form, weighting, uncertainty method and handling of scores exactly at the cutoff. Plot raw support and suitable summaries as well as fitted curves. Sensitivity across reasonable prespecified windows can expose dependence on remote observations or a restrictive curve.
Avoid fitting one high-order global curve just because it produces a visible jump. Removing a suspicious neighborhood around the cutoff likewise changes the available support and introduces a gap; it does not by itself repair manipulated assignment. Explain what comparison remains after exclusions.
| Audit item | Evidence to preserve | Response when unresolved |
|---|---|---|
| Rule | score formula, strict inequality, date and exceptions | do not infer assignment from receipt |
| Support | score values and counts on both sides | report gaps and discrete limits |
| First stage | actual receipt limits and jump | separate sharp, fuzzy and weak cases |
| Outcome | common definition, horizon and limiting jump | do not substitute whole-side averages |
| Identification | continuity, exclusion, compliance assumptions | withhold the unsupported causal label |
| Estimation | window, curves, uncertainty and exclusions | preserve sensitivity and protocol versions |
Missing scores should not be imputed from treatment status, since that would encode the desired first stage into the running variable. Missing outcomes require an observation-mechanism review; a clean assignment rule does not fix selective follow-up. Conflicting score records should be reconciled using the actual decision-time version.
Update the audit when appeals reveal assignment exceptions, another threshold policy is documented, scoring changes or valid local data become available. Preserve the original cutoff population and label any extrapolation beyond it. A credible local finding can be useful without becoming a claim about all companies or all score levels.
Make the local complier interpretation concrete
For an illustrative population at the cutoff, suppose 20% would receive the subsidy whether eligible or not, 60% receive only when eligible, and 20% never receive. There are no defiers in this constructed example. Receipt is then 80% under eligibility and 20% otherwise, matching the first-stage limits.
If eligibility affects outcomes only through receipt, appropriate local comparability holds, and the complier average receipt effect is ten points, the outcome contrast is 0.60×10=6. The always-receivers and never-receivers do not change receipt in that contrast. This reconstructs why dividing by 0.60 can recover ten under the assumptions; it does not reveal the receipt effects for the other 40%.
The observed 0.80 and 0.20 alone do not identify those behavioral categories without assumptions. A mixture containing defiers could yield the same positive average first stage. Keep the constructed breakdown in the explanation column, rather than presenting it as an empirical classification of individual companies. Extending the ten-point estimate to all recipients would require evidence about treatment-effect differences across these groups.
FAQ
Is the mean below the threshold minus the mean above it an RD effect?
Not automatically. The target concerns local limits or a separately justified local-randomization comparison, with design and identification conditions.
Does the ratio 6/0.60 prove a subsidy effect of 10?
No. It is a numerical Wald ratio. Its causal interpretation needs valid fuzzy-RD assumptions and applies to the corresponding local complier target.