Onchain data windows: blocks, finality, missing history
An onchain statistic is reproducible only when its network, block boundaries, finality convention, query method, and missing-data policy are recorded. “Last month” is not a complete specification. Two correct queries can disagree because they count different blocks or have access to different historical information.
Ethereum’s JSON-RPC reference documents methods and block parameters used to retrieve chain information. A research query must still verify that its chosen provider supports the required history and parameters. A method described in the reference is not proof that every endpoint offers identical retention or service behavior.
Define time as a block selection rule
Write the intended time interval in a stated timezone, then document how it maps to blocks. Decide whether the opening boundary is included and the closing boundary excluded, or use another explicit convention. That small decision prevents duplicated observations when adjacent periods are combined.
Save network identifiers, endpoint class, query parameters, retrieval time, and the selected block hashes when relevant. For a historical state question, distinguish a value read at the target block from a current value attached to an old timestamp in a spreadsheet. They are not interchangeable.
| Query record | Required detail |
|---|---|
| Research interval | Exact start, end, timezone, inclusion rule |
| Chain scope | Network and relevant asset or contracts |
| Block range | Numbers and selected identity references |
| State convention | Historical, latest, or another explicit choice |
| Coverage | Successful ranges and missing responses |
| Revision policy | Treatment of reorganizations and reruns |
The record should allow another researcher to reconstruct the question without guessing what the original script intended.
Hypothetical midnight boundary
Assume a fictional dataset counts transfers between 00:00 on Monday and 00:00 on Tuesday. One script includes both endpoints, and the next day’s script also includes Tuesday’s opening block. A transfer at the shared boundary is counted twice when daily totals are added.
A half-open convention can prevent that particular overlap: include the start and exclude the end. The important point is to write and consistently apply the convention, not to claim that every data service already uses it. Test the boundary with a small sample before processing a long history.
Now suppose an endpoint returns incomplete results for one block segment. Treating the empty response as zero activity would understate the count. Keep that segment marked missing until the response is validated or another suitable source fills the gap. Every number in this example is illustrative rather than a claim about an actual endpoint.
Audit coverage before interpreting change
Create a coverage ledger showing requested ranges, successful ranges, retries, and unresolved gaps. If the request must be split, verify that the pieces have neither gaps nor overlap. Count response records before and after deduplication and retain the deduplication key.
For recent observations, state the finality convention and whether a later rerun may revise the result. For historical questions, confirm whether the data source can return the necessary state rather than only transaction history. Some research questions cannot be answered by logs alone.
Independently reproduce a narrow segment using another suitable method. The goal is to test field interpretation and completeness, not merely to obtain a similar grand total. Two scripts sharing the same extraction error can agree. Inspect specific observations at the start, middle, and end of the chosen interval.
Limits and common mistakes
Provider limitations, chain reorganizations, undocumented contract coverage, and unavailable historical state can prevent a complete result. In those circumstances, report the observed coverage and avoid a precise whole-period claim. A partial series should not be silently interpolated into complete history.
Common mistakes include confusing UTC with local calendar dates, counting shared endpoints twice, interpreting request failures as zero, using current contract state for historical balances, and omitting the network identifier. A research note should state which observations are final under its convention and which remain revisable. This is evidence engineering for spot research, not a trading signal.
FAQ
Is a transaction timestamp enough to reproduce a dataset?
No. Also retain the network, block selection, query method, contract scope, and missing-data treatment.
Should an empty response always be recorded as no activity?
No. First establish that the query succeeded with complete coverage. An unavailable or truncated response is a missing-data condition, not a measured zero.