AI document comparison: verify changes against both versions
AI document comparison should identify a candidate change, then verify it against both original versions. Keep textual differences separate from changes in business facts. A renamed heading, a moved table and a revised accounting definition are different events, even if a model describes all three as an update to the report.
Establish the two versions before comparing
Record issuer, document title, publication date, reporting period and version identifier for each file. Make sure one is actually the predecessor of the other. Comparing a quarterly presentation with an annual report may be useful, but it is not a simple revision comparison. State which comparison you are making before asking what changed.
Python’s difflib documentation explains tools for comparing sequences and generating differences. A text diff can help locate edits; it does not decide whether an edit changes an economic claim. The workflow below combines candidate detection with a separate meaning check. It does not require Python, and it is not a claim that an AI summary has performed a complete formal diff.
Use a change classification table
| Class | What to check | Typical consequence |
|---|---|---|
| Formatting or relocation | Same wording and values at another position? | Update location, not the claim |
| Terminology | Has a label changed while the definition stayed constant? | Preserve a mapping and verify comparability |
| Definition or scope | Did included entities, operations or customers change? | Reconsider the comparison basis |
| Numeric correction | Is a previously reported figure restated? | Recompute dependent calculations |
| New observation | Does the later period add genuinely new information? | Update the evidence, retaining the earlier record |
Require the model to quote the relevant short passages from each supplied version and provide page or section positions. Then open those positions yourself. “The company improved disclosure” is not a substitute for showing the actual change. When a claimed deletion is not found, search for relocation and renamed headings before concluding that information disappeared.
A hypothetical margin comparison
Assume an invented report version A shows a 20 percent segment margin including a small discontinued activity. Version B shows an assumed 22 percent margin after excluding that activity, and the note explains the changed scope. A model says profitability improved by two percentage points. The numbers differ, but the statement about improvement is not established on a like-for-like basis.
Record both values with their definitions. Check whether B restates A on the new basis. If it does, use the comparable restated pair and retain the original for history. If it does not, label the trend unresolved and explain the missing bridge. Do not manufacture a restated value from the apparent difference. A changed margin denominator can matter more than the visible movement in the percentage.
Trace the change into existing research
A verified source correction can affect a chart, a ratio, a thesis condition and a memo written from the old value. List those dependencies rather than updating only the latest summary. Keep the previous conclusion as a dated record, attach the correction, and state whether the new evidence changes the conclusion or only its presentation.
Use separate statuses for candidate, verified, rejected and unresolved. A candidate found by a model is a work item, not a confirmed source change. When the original files are scans, inspect extraction differences before interpreting a missing minus sign or a broken table row as a reporting revision. Normalizing text can reduce irrelevant whitespace, but retain the original files so normalization cannot hide meaningful punctuation.
Limits of an automated comparison
A useful diff cannot prove that no substantive changes remain. Model omissions, extraction errors and moved tables can all escape a short comparison. Narrow the scope to the sections relevant to your question and state that scope in the result. Important deleted or revised claims deserve manual checks against both versions, even if the rest of the report looks unchanged.
FAQ
Is every changed number a business change?
No. Restatements, reclassifications, currency presentation and corrections can change the reported value without representing the same underlying business movement.
Can I replace the old file with the new one?
Keep both when prior research depends on the old version. Record which one is current and why, while preserving the evidence used in earlier work.