Block errors in automated stock research with structured checks
An automated research result needs an executable acceptance gate. Give extraction and calculation branches structured outputs, join them to the same source identities, and publish only rows that satisfy the join and numerical checks. Then deliberately inject plausible errors and observe whether the gate blocks them. Two agents repeating the same PE in prose do not establish that either the inputs or arithmetic were independently checked.
This tutorial builds a complete small join for three fictional A-share-style valuation rows. It goes beyond the principle of separating calculation from writing: the downloadable program actually merges branch results, produces accepted and blocked rows, injects an incorrect multiple and a wrong year, and replays the corrected data. It is a local synthetic workflow; no Vibe scheduler, model or CoTrading automation is enabled.
Prepare verified source rows and a stable identity
Begin with reviewed annual forecast EPS cells and a separate dated price input. Each source row needs company, target fiscal year, report-row identifier, currency, EPS basis and price date. Preserve the original document reference and extraction location outside this compact fixture. A report-row identifier such as row-1 is not globally unique: AAA and BBB can both have a row with that local label.
The fixture uses FY2027, diluted CNY EPS and prices dated 2026-10-26. IAS 33’s official overview explains why an explicit EPS basis matters. It does not supply our forecasts. The input rows are manually verified synthetic table cells, not evidence that the script can read an arbitrary PDF or identify an issuer’s preferred adjusted earnings measure automatically.
Use the full key (company, fiscal_year, source_row) to check the intended match. Also retain (company, source_row) as an origin anchor. This second identity makes a wrong-year mutation detectable: the program can see that the same source origin has acquired a different year, rather than treating it as an unrelated extra row and silently losing the original observation.
Make the two branches produce different evidence
The extraction branch reads the supplied cells and returns reported EPS and reported PE with the source identity. The calculation branch separately takes the verified raw EPS and the external price and computes price divided by EPS. It never uses the extracted or reported PE as its formula input. If the reported PE is wrong, the calculation remains anchored to price and verified EPS.
This independence is limited and explicit. Both branches depend on the reviewed raw EPS. If that common input has the wrong unit or year, arithmetic agreement alone will not discover every common-source mistake. Source verification and scope checks remain necessary. A second model that merely restates the first branch’s claimed multiple would add no formula independence at all, regardless of whether its prose sounds confident.
Python Decimal supports numerical comparison, with finite positive checks on price, EPS and the supported PE fields. Reject NaN and infinities rather than letting them enter a difference test. The acceptance tolerance for the reported-versus-computed multiple is 0.01 PE units. This tolerance is declared in advance and addresses rounding, not errors in financial scope.
Perform a full, anchored merge
Validate the common identity schema and each branch’s required numeric fields before matching. Index each branch by company-qualified origin. Duplicate origins inside a branch are rejected, even if their rows happen to agree, because the compact example requires one reviewed row per origin. Different companies sharing row-1 remain legitimate separate entries. Do not implement last-write-wins deduplication for conflicting financial evidence.
For every original source row, emit exactly one audit outcome. A missing branch is missing_branch; unexpected origins stop the merge. A different year on either branch is year_mismatch. Compatible years must also agree with the source’s currency, dilution basis and price date. After those checks, compare extracted EPS with verified EPS and reported PE with independently computed PE.
if key(extracted) != key(source) or key(calculated) != key(source):
reasons.append("year_mismatch")
elif abs(reported_pe - computed_pe) > D("0.01"):
reasons.append("arithmetic_mismatch")The excerpt shows two central decisions; the full file includes schema, missing-row, scope and finite-value checks. An audit row retains source identity, status and reasons. accepted_pe is null whenever any reason exists. This separation lets diagnostics show why a result failed without accidentally exposing the unverified number as a publishable valuation.
Establish the clean baseline
AAA has price 24, EPS 1.2 and reported PE 20. BBB has price 30, EPS 1.5 and reported PE 20. CCC has price 18, EPS 1 and reported PE 18. The calculation branch independently reproduces 20, 20 and 18. All three origin anchors match FY2027 and the agreed scope, so the baseline contains three accepted rows.
The program asserts the number and status of baseline rows before injecting errors. This provides a control: a gate that rejects every input would block errors but would not satisfy the workflow. Retain both the accepted baseline and the mutated run, along with the source fixture and code version, so a reviewer can inspect whether the checks are selective rather than merely restrictive.
Download the complete example. It uses Python 3.12+, the standard library and synthetic data. Run python R555.py to produce baseline, injected, repaired and missing-branch audit results. Python JSON documentation describes the structured output format; the script supplies the financial and identity checks that JSON itself does not provide.
Inject an arithmetic error and a year mismatch
First change only BBB’s extracted reported PE from 20 to 21. Its price and verified EPS remain 30 and 1.5, so the calculation branch still returns 20. The merge reports arithmetic_mismatch and leaves accepted PE null. AAA remains accepted. A batch gate should not erase unaffected valid rows, but the failed row cannot enter a downstream summary as if its reported multiple were trusted.
For a different mechanism, change only CCC’s extracted target year to 2028. The original source anchor and calculation branch remain FY2027. The merge reports year_mismatch, even though CCC’s numbers could otherwise agree perfectly. Averaging the two branches, asking a model which year seems plausible or dropping the unmatched record would conceal the actual defect. The source identity, not a numerical vote, determines the required match.
Reverse both mutated branch lists and run the merge again. The result must be identical in source-row order and status. The example asserts this property. It also tests a missing extraction row, a duplicate origin and nonfinite reported multiples. These cover alignment, cardinality and numeric failures beyond the two headline mutations.
Repair, replay and define the publication boundary
Restore the extracted data from the verified fixture and independently recompute again. The repaired output must equal the clean baseline. Keep the failed run rather than overwriting it with the passing version. A corrected final answer alone does not establish that the system would have blocked the earlier bad result; the injected-run evidence demonstrates the blocking behavior directly.
Downstream publication should consume only accepted fields, accompanied by the data contract and coverage. In this example that is a local data structure, not an external report sender. A real automation needs durable run identifiers, source storage, authorization and a reviewed publication policy. The script establishes the structured merge and failure gate for these inputs, while leaving those product capabilities unclaimed.
When a source row, forecast year, price date or EPS convention changes, rerun both relevant branches and their merge. Preserve the dependency identity so a new extraction cannot silently inherit an old calculation. The operational result is a clear answer for every expected source row: accepted valuation, or blocked valuation with a concrete reason and a defined next review step.
Keep the row-level audit beside any aggregate. An accepted count measures usable valuations, while the total audit count measures whether every expected origin received an outcome. They should not be collapsed into a single success percentage.
FAQ
Are two model responses enough for independent verification?
No. The calculation branch must actually recompute from verified inputs and avoid relying on the claimed result it checks.
Can a wrong-year row pass if PE agrees?
No. Identity and scope matching precede arithmetic acceptance.