Rank association: monotonic order is different from a linear slope
A perfect rank correlation means that two variables line up in order. It does not mean that a one-unit increase in one produces a constant increase in the other. Keep raw values and ranks together before choosing a correlation summary. Otherwise a useful ordering can become an unsupported claim about slope, marginal impact or causation.
The practical deliverable is a paired-row ledger, a tie-aware rank table and a statement of the relationship being measured. This can help compare task complexity with verification time or two independently produced research scores. It does not turn a high correlation between opinions into proof that either opinion is correct.
Define the two measurements and their pairing
Start with an observation identifier, the X measurement, the Y measurement, their dates and units. Both values must refer to the same intended object and window. Sorting each column independently before joining would manufacture agreement rather than measure it.
The SciPy Spearman documentation describes a measure of monotonic association. A Pearson correlation of the original values concerns linear association. Neither coefficient supplies a causal design, and neither is a universal test for every form of dependence. The numerical examples below are hypothetical demonstrations.
Decide whether higher values mean more difficulty, stronger evidence or faster completion. A speed score that rises when time falls reverses the direction compared with minutes. Also preserve missing measurements and exclusions: calculating the coefficient only on jointly observed rows changes the analyzed population when missingness is selective.
Reproduce increasing order without a straight line
Assume four invented tasks have X values one, two, three and four, and Y values one, four, nine and sixteen. These observations follow Y = X squared over the displayed positive range.
| Task | X | Y | Rank of X | Rank of Y |
|---|---|---|---|---|
| A | 1 | 1 | 1 | 1 |
| B | 2 | 4 | 2 | 2 |
| C | 3 | 9 | 3 | 3 |
| D | 4 | 16 | 4 | 4 |
The rank vectors are identical, giving Spearman correlation one. But successive Y increases are three, five and seven. The table does not support a constant increment per unit of X.
For an independent arithmetic check, center X at 2.5 and Y at 7.5. The sum of cross-products is twenty-five; the squared-deviation sums are five and 129. Pearson correlation is therefore 25 / square root of 645, approximately 0.984374. It is high, but not one.
The least-squares line for these four observations is Y-hat = −5 + 5X. Its fitted values are zero, five, ten and fifteen, leaving residuals one, minus one, minus one and one. That visible curvature remains even though both correlation summaries look strong. A rank coefficient of one cannot make the straight-line residuals disappear.
Add ties using an explicit rule
Now change X to one, two, two and four while leaving Y unchanged. The middle two X observations occupy rank positions two and three. Under average-rank handling, each receives 2.5.
| Task | X | Y | Average rank of X | Rank of Y |
|---|---|---|---|---|
| A | 1 | 1 | 1 | 1 |
| B | 2 | 4 | 2.5 | 2 |
| C | 2 | 9 | 2.5 | 3 |
| D | 4 | 16 | 4 | 4 |
The SciPy rankdata reference defines its default average treatment of ties and distinguishes it from minimum, maximum, dense and occurrence-order ranking. Record the selected rule, especially if the source score is rounded or categorical.
Compute the ordinary correlation of the two rank vectors. Both rank means are 2.5. Their cross-product sum is 4.5, and squared-deviation sums are 4.5 and five. The resulting coefficient is 4.5 / square root of 22.5, approximately 0.948683.
The shortcut one minus six times the sum of squared rank differences divided by n times n-squared-minus-one assumes no ties in its simple form. Applying it unchanged here would give 0.95. The small difference is a useful audit signal: a familiar formula can use the wrong assumptions even when its answer looks plausible.
Do not break ties by original row order just to obtain distinct ranks. That adds ordering information the measurement did not contain. If unrounded observations exist, decide whether those measurements are the intended inputs; switching precision after seeing which coefficient improves changes the analysis rule.
Trace a change caused only by measurement precision
For another concrete check, let X be 1.01, 1.11, 1.14 and 1.21 with an increasing Y rank vector one through four. Before rounding, both rank vectors agree and the coefficient is one. Rounding X to one decimal gives 1.0, 1.1, 1.1 and 1.2. Its ranks become one, 2.5, 2.5 and four, giving the same approximately 0.948683 coefficient as the tied example.
No task changed its underlying order. The reporting resolution changed, losing the distinction between the middle observations. Record that cause before interpreting the coefficient decrease as deteriorating research quality. Compare periods using a common justified precision, or report the break explicitly. Retaining only the rounded column makes it impossible to recover the original distinction reliably.
Test a different mechanism: a U-shaped relationship
Consider X values minus two, minus one, zero, one and two, with Y values four, one, zero, one and four. Here Y is again exactly X squared, but the observed range includes both sides of the minimum.
X ranks are one through five. Average Y ranks are 4.5, 2.5, one, 2.5 and 4.5. Their centered cross-products cancel, producing Spearman correlation zero. Nevertheless, knowing X determines Y exactly in this constructed example. Zero rank correlation has not established independence.
Inspect a paired plot or an equivalent ordered table before interpreting a small coefficient as no useful relationship. Nonmonotonic structure, subgroups and thresholds can be missed by a single monotonic summary. Conversely, do not fit a complicated curve solely to rescue a preferred thesis; specify the relevant relationship and seek additional observations.
This differs from the first example. The first has perfect increasing order but nonconstant increments. The second has deterministic dependence without one increasing or decreasing direction over the full range. They show why ordering, linearity and general dependence are separate questions.
Restrict the second example to X equal to zero, one and two and the rank coefficient becomes one. That is a statement about the restricted range, not a contradiction of the earlier zero. Save the sampling window and eligibility rule when comparing coefficients across datasets.
Keep descriptive coefficients apart from inference
Four or five rows can demonstrate arithmetic; they do not establish a stable population relationship. The spearmanr reference cautions about its approximate p-value for smaller samples and discusses permutation testing. That implementation guidance is not a universal sample-size certificate.
A permutation procedure also requires a justified exchangeability structure. Arbitrarily shuffling rows from repeated observations of the same issuer or operator can violate the sampling design. Preserve grouping and pairing, and distinguish a randomized task experiment from an observational association. Changing the test name does not repair dependence or selection.
Do not interpret a p-value as effect magnitude or a probability that the causal story is true. Report the coefficient, sample size, tie counts, missingness and uncertainty method together. If an input is constant, its rank variance is zero and the correlation is undefined, not evidence of no relationship.
Deliver a relationship-audit table
| Record | Required input | Check |
|---|---|---|
| Pairing | Object, date, X and Y | Were the values joined before sorting? |
| Meaning | Units and direction | Does higher mean the same kind of change? |
| Ranks | Tie rule and precision | Can average ranks be reconstructed? |
| Shape | Raw table or paired plot | Is the relationship monotonic or curved? |
| Inference | Sampling unit and method | Do independence or exchangeability assumptions hold? |
| Conclusion | Ordering, slope or another target | Does the wording match the coefficient? |
Recheck when rounding changes, new tasks extend the range or the population shifts. Save both original values and derived ranks so that a revised tie rule does not erase the previous calculation. The useful conclusion says what ordering was observed and what it leaves unresolved about distances, functional shape and cause.
FAQ
Does Spearman correlation one mean a perfect linear relationship?
No. The first example preserves order perfectly while following a curved relationship with changing increments.
Does zero rank correlation show the variables are independent?
No. A U-shaped deterministic relationship can have zero rank correlation over a symmetric range.