Test stock strategy code for look-ahead with truncated price data
A deterministic stock strategy should produce the same historical signals when observations after that history are removed or changed. Test this property by running the actual generator on prefixes, comparing common rows and then perturbing future values. The result is a row-level failure report rather than a reassuring search for suspicious function names.
This guide supplies three deliberately leaking functions and one causal comparator on synthetic closes. It does not prove that financial records were available historically, that model training data are clean or that a complete trading system is unbiased. CoTrading strategy testing remains planned or unverified; the executable example is an independent standard-library harness.
Define exactly what must remain invariant
Let a completed-close signal at row i be determined only by input rows zero through i. Run the strategy on the full dataset and again on the first k rows. Compare all first k outputs, including the last completed row of the prefix. This contract assumes that each row is complete when supplied; it does not describe an intraday candle that is still changing.
For a strategy that intentionally emits orders one session later, compare like with like. Either test decision signals before scheduling or consistently compare delayed targets. Do not exempt the last row merely because it exposes a bug. Document any output whose label truly matures later and exclude it according to an explicit output contract, not an after-the-fact exception.
Require deterministic inputs and reset state between runs. A clock call, mutable global portfolio or external model request can change results without directly reading a future price. Such differences still need investigation, but the failure report should describe observed behavior rather than immediately assign one cause. Freeze configuration, input order and security mapping alongside the tested source revision.
Start with a small causal rule
The reference function outputs one when the current close is strictly above the average of the current and previous two closes. The first two outputs are zero. It slices only up to the current row, so later prices cannot enter that average. It uses no random seed, external file or network resource.
Use closes [10, 10, 10, 12, 1000, 1, 500]. The exaggerated later values are intentional diagnostic inputs. They are not plausible market forecasts and should not be presented as historical observations. Their purpose is to make a hidden future dependence alter an earlier Boolean signal rather than disappear after thresholding.
The causal function passes every tested prefix and keeps its first four outputs when the final three closes are replaced with thirteen, nine and eight. A successful fixture checks one implementation under these inputs. It does not establish invariance for all possible branches; expand the test set around missing data, ties, warmup transitions and multiple securities when those behaviors exist.
Trigger three distinct leaking mechanisms
The first broken function compares each close with the following close. It resembles a negative shift applied to input data. At a prefix boundary, the next observation is absent, so the historical last signal changes when a future row is appended. The harness reports failures at cutoff three, row two; cutoff four, row three; and cutoff six, row five, using zero-based indices.
The second broken function compares every close with the full dataset mean. At cutoff four the mean uses only ten, ten, ten and twelve, so twelve qualifies. The full dataset mean includes one thousand and five hundred, so the same historical twelve no longer qualifies. The reported failure is cutoff four, row three. A global maximum or full-sample standard deviation can create the same information path even without a negative shift.
The third broken function uses a centered local window, including the next observation when present. At cutoff four, twelve exceeds the average of ten and twelve. After appending one thousand, its centered average changes and the signal flips. Its failure is also cutoff four, row three, but its cause differs from global normalization. Record the relevant input slice to distinguish those mechanisms.
Build a useful comparison report
The standard-library file docs/growth/research/round-5/examples/R574.py evaluates every prefix length from three to one less than the full length. Its central comparison is:
full = generate(closes)
for cutoff in range(3, len(closes)):
prefix = generate(closes[:cutoff])
changed = [i for i in range(cutoff) if prefix[i] != full[i]]Save generator identity, cutoff, affected row, prefix value, full-run value and the input window used by that calculation. For numeric scores, establish an appropriate tolerance before comparing; for these discrete outputs, exact equality is required. Also assert output length and timestamp alignment, because comparing two differently shifted arrays can otherwise manufacture a false pass.
Run python docs/growth/research/round-5/examples/R574.py. The assertions require no causal failures, at least one failure for each broken function and equal causal history after the future mutation. They also require the global-mean history to change after that mutation. Expected failures are positive test evidence here; silently removing a broken function from the harness would weaken coverage.
Repair the information path, then rerun
Replace a future comparison with an explicitly backward-looking decision rule, rather than shifting the already leaked result and hoping to cure it. Replace a full-sample transform with an expanding or trailing transform fitted only from permitted observations. Replace centered windows with trailing windows when the decision must be available at the current close. Each repair must preserve the intended economic rule or receive a new strategy specification.
After repair, rerun both the failing prefix and all other prefixes. Keep the original failing fixture so future refactors cannot reintroduce the issue. Python’s testing documentation describes the role of input fixtures and checks; this example uses direct assertions to keep the standalone artifact dependency-free. A production suite can aggregate the same cases with a test runner.
Understand a clean result’s limits
The ordinary sequence [10, 10, 10, 12, 13, 9, 8] does not make every broken function flip a Boolean. This is the boundary case: a biased algorithm can pass an inadequately chosen test dataset. The more extreme fixture detects failures because it crosses decision thresholds. Passing one prefix is therefore weaker than testing several boundaries and future perturbations.
Finally inspect the input pipeline. An availability violation in a prejoined earnings panel can remain identical across all prefixes; the strategy sees a contaminated historical value in both runs. Likewise a previously trained model may encode later information without reading appended rows. Accept the report as evidence about the exercised code path, then separately verify source availability, training periods and execution timing. Update tests whenever a new transform, external dependency or output convention is introduced.
Download the complete example. Save it as R574.py and run python R574.py with Python 3.12 or later. It uses only the standard library and synthetic data.
Verify the repaired functions as separate artifacts
The downloadable file also implements three repaired variants. The future-close comparison becomes a current-versus-previous-close rule; the global mean becomes an expanding mean ending at the current row; and the centered comparison becomes the causal three-observation trailing rule. These are explicitly changed decision definitions, so they require approval in the strategy specification rather than being disguised as harmless formatting fixes.
Every repaired variant is rerun over the same prefixes, with an empty failure list required for each one. The first four outputs must also remain equal after replacing the final three values. Keep both the broken and repaired functions so a later reviewer can establish that the harness detects the old failure and accepts the new permitted information path. If a repair merely returns zeros for every row, it may satisfy invariance while destroying strategy meaning. Compare ordinary-path expected outputs as well; temporal consistency and economic-rule fidelity are separate acceptance obligations.
List passing functions, intentionally detected failures and uncovered dependencies together, with input and source fingerprints, so the report remains interpretable on a later rerun.
FAQ
Is searching for negative shifts enough?
No. Full-sample statistics and centered windows can leak without a negative shift, and hidden dependencies can also alter outputs.
Does a passing test prove the backtest is unbiased?
No. It checks the specified generator and perturbations, not the historical validity of every upstream input.