Check generated stock strategies for hidden runtime inputs
A fixed-rule stock strategy should disclose every input that can change its output. Prices and configuration may look identical while the function reads a clock, global random generator, environment value, network endpoint or model. Audit and block those dependencies at the function boundary, then check repeated signals from the declared fixture.
This guide isolates a single synthetic signal function. It does not trace differences across an entire backtest’s data, configuration, fills and report layers. CoTrading dependency auditing remains planned or unverified. The example contacts no network or model and does not claim that repeatability proves historical data quality or profitability.
Inventory the actual lookup points
Read the strategy source and list external reads, module-level state, time calls, random draws and model wrappers. For each dependency record its lookup namespace, purpose, expected version and whether it belongs in a supposedly fixed historical rule. A function argument list is not a complete input inventory if the body reads global objects.
Distinguish code-writing assistance from runtime decision making. A model can help author a deterministic function without being called during each evaluation. If the strategy invokes a model for every date, that model response, prompt and context become runtime inputs. Do not describe that second architecture as a fixed price-only rule simply because Python hosts the call.
The fixture defines an explicit model wrapper that raises a harmless sentinel exception. It is not an actual model client. The network control points at a deliberately invalid example address but is patched before invocation. These choices let the test detect forbidden behavior without creating external requests or relying on credentials.
Start with a declared pure comparator
The approved function receives completed closes only. It outputs one when the current close exceeds the mean of the current and previous two observations; the first two outputs are zero. For ten, ten and twelve, the result is [0,0,1]. It reads no clock, random number, environment variable or outside data.
Repeated calls begin from the same function input and return the same sequence. That observation is useful but incomplete by itself: a hidden clock might happen not to change between calls, or a cached model response might mask a dependency. The test therefore combines repeatability with explicit dependency blocking and controlled variation.
Preserve the input list before and after if the real implementation can mutate it. An unchanged function signature does not prevent hidden state from accumulating in an object or module. This fixture’s list comprehension creates a new output and keeps no mutable strategy instance between calls.
Block the enumerated external dependencies
Python’s unittest.mock documentation describes replacing looked-up objects and checking their use. The script patches the exact referenced clock, module-level random draw, environment lookup, URL opener and synthetic model wrapper. Each patched dependency raises if called.
Run the approved function twice inside those patches. It must still return [0,0,1], and every mock must report no call. This establishes that the exercised path does not need the five declared dependencies. It is stronger than making external services unavailable and interpreting a lack of visible errors as success.
Patching must occur where the object is looked up. A function that previously imported a different alias may bypass a patch on the original module name. Inventory those aliases and patch the appropriate namespace. The pack is a targeted test of listed lookup points, not a universal sandbox capable of intercepting arbitrary generated code.
Prove that the detector can catch violations
Five control functions intentionally read one hidden input each. The clock control reads current time, the random control draws from the shared generator, the environment control reads a fixture-only variable, the network control invokes the patched opener and the model control calls the sentinel wrapper.
Each must raise the corresponding blocked-dependency exception. The test records all five names and reasons. If a control silently passes, either the patch location or the detector is incomplete. Keeping known failing controls prevents a reassuring no-call result from being confused with a test harness that never observed the relevant call.
The environment example does not read an .env file or user secrets. It uses only a synthetic key and a mock. The network and model exceptions occur before a real request. Report those boundaries accurately rather than describe the fixture as having tested a live provider outage or a model’s reproducibility.
Vary a hidden clock while holding prices fixed
Outside the blocking test, supply clock return zero and then one through controlled patches. The clock-dependent function returns all zeros for the first value and all ones for the second, despite identical close inputs. That is a direct counterexample to the claim that the output is determined by the declared prices.
This difference is not market noise or a backtest matching issue. It is a single-function dependency on an undeclared runtime value. Once isolated, decide whether to remove it, pass a historical timestamp explicitly or redefine the strategy as a time-dependent rule with a saved input. Do not fix it merely by rerunning until two outputs happen to agree.
Keep the dependency’s purpose in the decision. A legitimate trading-session timestamp should come from the historical data contract, while today’s wall clock usually does not belong in a historical price-only comparator. The appropriate repair changes input ownership, not just a random seed or exception handler.
Treat permitted randomness explicitly
Python’s random documentation distinguishes shared module state from local generator instances and describes seeding. The fixture creates a fresh local generator with seed seven for each three-draw demonstration. The two sequences match and contain more than one distinct draw.
That is an example of declared reproducibility, not a reason to add randomness to the approved rule. If randomness is necessary, archive algorithm, seed, initialization point and Python/dependency version. A fixed seed does not control a model server, external data revisions or clock input, so it cannot replace the full dependency inventory.
Reset a local generator per independent run when that is the intended contract. Resetting before every draw would repeat the first value rather than produce the intended sequence; sharing it across runs would make call order part of the hidden state. These are distinct implementation choices with testable consequences.
Run the isolation pack
Download the complete example, save it as R581.py and run python R581.py with Python 3.12 or later. It uses only the standard library and synthetic inputs. The source is docs/growth/research/round-5/examples/R581.py.
Assertions check repeated pure signals, no forbidden calls, all five detected controls, changed clock outputs and equal local-seed sequences. The output explicitly states that no real network or model call occurred. It verifies a narrow function-level claim rather than a complete backtest or a security boundary.
Hand off a bounded dependency result
Archive the source version, declared inputs, dependency table, patched lookup points, runtime version and actual assertion output. List untested paths and cached or aliased dependencies discovered during review. A passing test under one branch cannot justify deleting those unresolved items from the report.
After removing or explicitly injecting hidden inputs, rerun both the pure fixture and the known controls. Then perform separate tests for temporal leakage and whole-run reproducibility if needed. Completion means the exercised fixed-rule function can compute from its declared inputs while forbidden listed calls are detected, with clear limits on everything the isolation did not test.
The blocking scope begins around the function call. It does not prove that an arbitrary strategy module performed no external work during an earlier import. Review import-time execution separately and keep that stage’s evidence distinct. Likewise, an unlisted file read or a cached alias is not intercepted merely because these five controls fail as expected. A useful handoff lists those limits rather than expands a successful targeted test into a universal isolation claim. New dependencies should enter the inventory before they enter an allegedly fixed-rule implementation.
FAQ
Does calling a model to write code make the strategy nondeterministic?
Not necessarily. Runtime model calls and model-assisted authorship are different dependencies.
Does repeating the same signal prove there are no hidden inputs?
No. Combine repetition with dependency inventory, controlled variation and targeted blocking.