Dataset snapshots: freeze rows as well as documents
A reproducible research snapshot freezes a row population and its transformation rules, not merely the PDFs that supplied it. Another reviewer should be able to identify the same input objects, apply the same selections and obtain the same observation keys. Otherwise an unchanged memo can rest on a silently changing dataset.
W3C DCAT version 3 provides vocabulary for datasets, distributions and version relationships. It supports clear dataset description; it does not prescribe this worksheet or prove any snapshot is complete. The operational checks below remain the researcher’s responsibility.
Define the snapshot boundary
Name the question, population, cutoff and dataset grain. A company-year table and a filing-fact table have different row identities. State which one the snapshot contains and which upstream sources it depends on. A folder full of files without a row definition is not a frozen analytical sample.
Retain raw input identities, retrieval parameters, schema and transformation version. Include filtering, identifier crosswalks, unit conversions and revision-selection rules. A dataset hash can identify a file, but it cannot explain how rows were selected or whether the source claims are accurate.
Preserve both a key-set fingerprint and basic quality checks. Row count alone can stay constant when one entity disappears and another enters. Aggregate totals can also remain unchanged after compensating errors. Several independent checks make silent substitutions easier to detect without pretending they guarantee truth.
Hypothetical equal-count replacement
Assume fictional Snapshot Pine contains 100 company-year rows. A later refresh also contains 100, so a simple count check passes. Yet one company was dropped because of a changed identifier match and a different company was added. The sample changed even though the row count did not.
Compare sorted observation keys: 99 are shared, one is removed and one added. The symmetric difference contains two keys. Investigate whether the substitution is an intended universe update or an accidental crosswalk change before reusing the original conclusion.
Suppose the keys are identical but 12 values changed after a revised-source refresh. A key-set check now passes while the value comparison fails. Keep both differences in the snapshot report. All counts and names in this scenario are hypothetical, and no software feature is being claimed.
Snapshot manifest worksheet
| Component | What to preserve |
|---|---|
| Analytical grain | Meaning of one row and its unique key |
| Population and cutoff | Membership and information-eligibility rules |
| Raw inputs | Object identities, retrieval details and permitted copies |
| Transformations | Script or documented steps, parameters and crosswalk versions |
| Structural checks | Row counts, duplicate keys, key-set differences and schema |
| Value checks | Selected totals, changed cells and unresolved exceptions |
Write a rerun instruction that starts from the frozen inputs. A command that fetches today’s latest data is a refresh, even if the script name says reproduce. If an input cannot be retained or recovered, state the reproduction limit and preserve enough metadata to identify what is missing.
Separate reproduction from refresh
Use different snapshot identifiers for the original analysis and a later update. Compare them explicitly rather than overwriting one file. An intentional refresh can change conclusions; the point is to make the change attributable to data, transformation or analytical interpretation.
Ask AI to summarize the difference report only after mechanical key and value checks. The model should not infer unchanged data from unchanged prose or identical counts. Nor should a matching hash be described as independent factual verification: it establishes identity of the checked bytes under the selected algorithm. The final packet should let a reviewer reopen inputs, reproduce row selection and see every unresolved exception.
FAQ
Is saving the original documents enough to reproduce a dataset?
Not usually. Row grain, filters, crosswalks, conversions and version-selection rules also determine the result.
Does the same row count prove a refresh left the sample unchanged?
No. Compare observation keys and values; substitutions and revisions can preserve the count.