Effect scales: reconstruct risk differences, ratios and odds
A probability ratio and an odds ratio can describe the same two groups while giving different numbers. Neither calculation is necessarily wrong. The error arises when a report changes the name of the quantity without changing its interpretation, such as describing an odds ratio of two as a doubling of probability.
The useful deliverable is a reproducible bridge from a binary outcome table to three separate measures: the absolute probability difference, the risk ratio and the odds ratio. Fix the event, comparison direction, follow-up period and sampling scheme before calculating any of them. These are initially descriptions of an association; arithmetic alone does not establish causation.
Reconstruct the original table
In this hypothetical comparison, two research workflows each process 100 independently sampled documents, with complete outcome ascertainment over the same prespecified period. The event is a material extraction error detected by a fixed review procedure. A has 40 event documents and B has 20. Each document is counted once, even if it contains several errors.
| Workflow | Event | No event | Total | Observed event proportion |
|---|---|---|---|---|
| A | 40 | 60 | 100 | 0.40 |
| B | 20 | 80 | 100 | 0.20 |
Keep all four cells. A spreadsheet containing only “40 versus 20” does not show whether the groups had the same size or whether the totals exclude missing reviews. An unreviewed document cannot be classified as no event. Save the eligible count, observed count and missing status separately so the population behind each proportion is visible.
Cochrane Handbook section 6.4.1 and Box 6.4.a distinguish the binary effect measures and the difference between risk and odds. Its treatment-study setting supplies the definitions; this document-workflow example is an original illustration. Here “risk” means probability of the specified event, not an investment risk score or a time-based event rate.
Calculate the three scales without relabeling them
Use A relative to B throughout. The risk difference is 0.40−0.20=0.20, an increase of 20 percentage points. It is not a 20% relative increase. Expressed per 100 comparable documents, the observed difference is 20 event documents.
The risk ratio is 0.40/0.20=2. A’s observed probability is twice B’s, or a 100% relative increase. The denominator in each group’s risk is its entire observed group: events plus nonevents.
The odds instead divide events by nonevents. A’s odds are 40/60=2/3; B’s are 20/80=1/4. Their odds ratio is:
OR = (40/60)/(20/80) = (40×80)/(60×20) = 8/3 ≈ 2.667.
Thus one table produces RD 0.20, RR 2 and OR 2.667. A conclusion saying “2.667 times the probability” would exaggerate the probability ratio. An odds value is also not restricted below one: it can exceed one when the event probability exceeds one half, whereas a probability cannot exceed one.
For a reported odds ratio theta and an appropriate comparator probability pB, the corresponding probability is obtained by solving the odds equation:
pA = theta pB / [1−pB+theta pB].
For theta 2 and pB0.20, this gives 0.40/1.20=1/3, not 0.40. The associated RR is 5/3, about 1.667, and the risk difference is about 0.1333. The baseline probability is an essential input; an odds ratio alone does not determine an absolute change.
Use a low-probability boundary carefully
For a separate comparison, let A have 20 events among 1000 documents and B have 10 among 1000. Risks are 0.02 and 0.01. The RD is 0.01, the RR is 2, and the OR is (20/980)/(10/990)=99/49≈2.0204. The ratio and odds ratio are close here because both probabilities are small.
This numerical proximity does not create a universal cutoff below which the measures become identical. State the actual probabilities and calculate the discrepancy for the decision at hand. A small absolute difference can coexist with a large relative ratio; a relative headline without the starting probability hides that distinction.
Conversely, simply transporting an absolute difference to a different baseline can produce impossible probabilities. A20-percentage-point reduction cannot be applied unchanged to a population whose starting event probability is 10%. Such a result is a warning that an effect-scale transport assumption has failed, not evidence for a negative event probability.
Reversing groups and reversing events are different operations
If B becomes the numerator group, the RD changes sign and both ratio measures become reciprocals. Save that comparison direction in every exported column.
Now keep A relative to B but redefine the event as no extraction error. The probabilities become 0.60 and 0.80. The new RD is −0.20, RR 0.75 and OR 3/8=0.375. The odds ratio is the reciprocal of 8/3, but the new RR 0.75 is not the reciprocal of the old RR 2.
That asymmetry matters when comparing reports that emphasize success rather than failure. “Half as many failures” and a statement about the ratio of successes cannot be interchanged mechanically. Reconstruct the cells and choose the event that matches the research question before combining or summarizing findings.
A sampling scheme can preserve the OR while changing raw risks
Return to the original table and create an illustrative outcome-based sample: retain every event document but only half the nonevent documents in each workflow. The sample table is now A40/30 and B 20/40 for event/nonevent counts.
Its group event proportions are 40/70=4/7 and 20/60=1/3. The sample RR is 12/7, about 1.714, and its RD is 5/21, about 0.2381. These are no longer the original workflow probabilities. However, its OR remains (40×40)/(30×20)=8/3, because the same outcome-specific sampling factor cancels across groups.
This exact constructed sampling example explains why a case-control-style table cannot generally supply population absolute risks directly. Preserving the OR requires the relevant selection assumptions; arbitrary differential sampling need not preserve it. External baseline information or known sampling fractions may be needed to recover probabilities. Do not treat the sampled group totals as a population denominator merely because a software function accepts them.
Execute a scale audit before interpreting the headline
| Check | Record to obtain | Response when unresolved |
|---|---|---|
| Define the event | error rule, unit and time horizon | withhold comparisons across incompatible outcomes |
| Identify sampling | cohort, experiment or outcome-based sample | do not interpret raw sample proportions as population risks |
| Rebuild cells | event/nonevent counts and missing reviews | retain unknown outcomes explicitly |
| Fix direction | numerator and comparator groups | recalculate reversed measures consistently |
| Calculate scales | RD, RR, odds and OR | attach the correct units and names |
| Interpret | baseline and intended decision | separate association from causal identification |
If the comparator has zero events, the simple RR denominator is zero; if a group’s event probability is one, its odds calculation has a zero nonevent denominator. Preserve the raw table and classify the undefined calculation rather than substituting an arbitrary finite value. Any continuity correction or interval procedure should be specified and reported separately; it is not an observed additional half-document.
Uncertainty also remains after the scale bridge. A point ratio of two does not establish its precision, and ordinary independent-document formulas are inappropriate for repeated documents or clustered reviews. A causal interpretation additionally needs a well-defined intervention, appropriate comparison and identification assumptions beyond these four cells.
Update the analysis when source counts, missing outcomes, sampling fractions, event definitions or the intended horizon change. Keep the original table and versioned transformations. This lets another analyst distinguish a real evidence update from a headline that merely switched between percentage points, relative probabilities and odds.
Resolve an ambiguous denominator before calculating more ratios
For a separate data-quality branch, suppose A’s 100 eligible documents actually comprise 40 verified events, 40 verified nonevents and 20 unreviewed documents. Its complete-case proportion is 40/80=0.50. Treating the unknown 20 as nonevents would instead produce 0.40, but that extra classification is unsupported.
For the full 100-document target, the event proportion can range from 0.40 to 0.60 depending on those missing outcomes. With B still completely observed at 0.20, the corresponding RD ranges from 0.20 to 0.40. These are logical missing-outcome bounds for the stated target, not sampling confidence intervals. The complete-case estimate 0.50 answers a selected-observation question unless the missingness mechanism justifies transport. Keep the source’s original denominator label and request the underlying review-status ledger when exports disagree. Adding decimal places to an OR cannot resolve a denominator whose membership is unknown.
FAQ
Does an odds ratio of two always mean probability doubles?
No. At comparator probability 0.20, OR 2 implies probability 1/3 in the other group, giving RR about 1.667.
Can a case-control sample directly provide population event probabilities?
Usually not from its raw group proportions. Outcome-based selection changes those denominators; recovering population probabilities requires additional information and assumptions.