Prepare local stock data for a backtest loader
Local stock data are ready for a backtest only when the consumer can read the intended fields in the intended order and units. A clean-looking CSV is not the same object as a mapping of stock identifiers to indexed price tables. Build a narrow adapter between those formats, verify its output with a minimal reader and stop malformed inputs before strategy logic starts.
This guide tests a standard-library representation of that handoff. It also describes the additional pandas boundary used by the statically inspected Vibe reference code. The fixture does not import Vibe, start a backtest or verify CoTrading integration. Its result proves row conversion and rejection rules, not an operational product loader or strategy performance.
Write down the consumer contract
The reference interface inspected for this article expects a per-symbol data_map, with OHLCV tables consumed by SignalEngine.generate(data_map). The inspected loader protocol returns a dictionary of pandas DataFrames. These are static implementation observations; the presence of an interface does not establish that a particular local-file source is registered or runnable.
For our independent handoff, each symbol maps to a list of date-and-bar pairs. Each bar has lowercase open, high, low, close and volume fields. Dates must be unique within a symbol and ascending after conversion. The entire batch uses one currency. The adapter rejects a mixture rather than pretending that common numeric prices imply a shared currency.
| Boundary | Required evidence |
|---|---|
| File to normalized rows | accepted fields, types, currency and adjustment metadata |
| Rows to per-symbol object | same row count, unique dates, ascending sequence |
| Object to actual engine | expected DataFrame type, index, columns and loader registration |
| Strategy execution | separately authorized and tested behavior |
Keep these boundaries distinct. A row converter can pass while the engine-facing type is still wrong. Conversely, a DataFrame can be structurally accepted while its volume unit is economically incorrect. Both require their own checks.
Validate identity, units and dates before grouping
Retain a market-qualified identifier and declared quote currency for each stock. Use consistent raw or adjusted OHLC values, and retain that choice in provenance. Do not use an adjusted close beside raw high and low without an explicit consistent transformation. The synthetic fixture uses CNY and raw stock prices throughout.
The sample input has two AAA rows, deliberately ordered 20 October before 19 October 2026. Open is 10, high 12, low 9 and volume 1000 on both dates. Close is 11 on 20 October and 10 on 19 October. The adapter parses ISO dates, creates numeric bars and sorts the dates. The Python datetime reference documents the date parser; daily date parsing does not establish an intraday timezone convention.
Check that all five numeric fields are finite, prices are positive, volume is nonnegative and high-low values enclose open and close. Reject missing mandatory fields. Sorting is allowed because the input contract permits unordered rows; duplicate dates are rejected because no source precedence has been agreed. Never use sorting as a substitute for checking uniqueness.
Build the object and read it back
The core adapter groups only validated bars and returns a fresh, ordered structure. This keeps caller-owned input rows from being modified during preparation.
output = {}
for row in rows:
symbol, day = row["symbol"], date.fromisoformat(row["date"])
bars = output.setdefault(symbol, {})
if day in bars:
raise ValueError("duplicate_date")
bar = {key: D(str(row[key]))
for key in ("open", "high", "low", "close", "volume")}
bars[day] = bar
data_map = {symbol: [(day.isoformat(), bars[day]) for day in sorted(bars)]
for symbol, bars in output.items()}The full fixture includes all the validation checks and is saved as docs/growth/research/round-5/examples/R538.py. It actually executes the adapter, reads the resulting close sequence and asserts [10, 11]. Its output reports AAA, two rows, start 2026-10-19 and end 2026-10-20. It also blocks duplicate_date, mixed_currency and a missing close field in three separate injected inputs.
Download the complete example. Use Python 3.12+ with the standard library only; all inputs are synthetic. After downloading, run python R538.py.
Compare input count and output count before any later filtering. Here two accepted rows become two output bars. If a future adapter intentionally drops rows, publish the drop reasons and counts instead of describing the result as an unchanged handoff. Sample the first and last bar and compare their values to the original accepted records.
Cross the pandas boundary deliberately
A real consumer expecting pandas cannot consume the fixture’s list representation unchanged. Build each DataFrame from the validated bar records, construct the datetime index from the paired dates, select OHLCV columns explicitly and convert numeric fields to the supported numeric dtype. The DataFrame.from_records reference documents record conversion. The DatetimeIndex reference documents the index object.
At that boundary, assert the exact columns, supported types, a unique increasing index and unchanged row counts. Preserve currency, volume unit, adjustment basis and source revision in consumer-supported metadata. Check that downstream code actually reads those metadata rather than assuming that placing them in DataFrame attributes enforces accounting compatibility.
That pandas construction and real loader registration were not executed in this standard-library fixture. Before using the reference engine, a separate integration test must return real DataFrames through the selected loader, call a minimal read-only consumer and compare the same two dates and closes. Passing this article’s fixture is a prerequisite example, not a claim that this additional integration has been completed.
Block errors that strategy code cannot repair
The different-mechanism counterexample adds a USD row to the CNY batch. Every price field may be valid and the dates may be unique, yet a single-currency accounting consumer cannot combine them without a defined FX layer. The adapter rejects the batch before grouping. Converting the USD number to a float cannot solve that financial mismatch.
The duplicate-date case instead concerns identity and precedence, while missing close concerns schema completeness. Give each a separate error so the owner knows whether to obtain currency support, resolve a source conflict or repair a field mapping. Do not fill missing close with zero simply to satisfy a numeric column requirement.
Deliver the adapter version, source snapshot, per-symbol reconciliation table, rejection results and the remaining integration checks. Reopen the handoff tests when the loader contract, library version, timezone convention or source format changes. Once the consumer has passed those checks, strategy logic can be tested on a declared input object; this preparation alone says nothing about execution costs, market rules or profitability.
Keep a handoff acceptance record
Record one row per symbol with input count, adapted count, first date, last date, required-column result and the declared currency and volume unit. The minimal reader should request the exact fields that the eventual strategy needs. If a strategy additionally requires VWAP or fundamentals, do not fabricate those columns to make its import succeed; expand the input contract and acquire or explicitly define them.
Perform a deliberate negative read as well: remove close, repeat a date and mix a currency, then verify that the adapter stops at the intended boundary. Save the resulting error category beside the fixture revision. This distinguishes an expected data rejection from an unexpected consumer crash.
When the pandas integration is eventually tested, append its library version and actual readback to the same record. Keep the earlier standard-library evidence separate so reviewers can identify precisely which boundary has been verified and which remains pending.
The minimal consumer should read fields without generating trading signals. If it reads closes in reverse order, inspect date sorting and value binding first; if both closes become identical, inspect mapping or overwriting. A plausible equity curve can conceal those basic input defects and cannot replace direct reconciliation. For several engines, keep the frozen normalized observations shared while maintaining a separate final adapter for each consumer. One engine’s column naming or numeric-type requirement should not silently redefine the underlying data for another engine. Recheck that last conversion whenever the relevant interface changes.
FAQ
Is the standard-library object a pandas DataFrame?
No. It is an independently tested handoff representation. A pandas consumer needs the additional conversion and integration check described above.
Can mixed currencies be fixed by scaling every price to a similar range?
No. Currency conversion needs dated rates and an accounting design. This workflow rejects the mixture.