Backfill restores a missing range in an existing dataset; it needs a precise boundary and a reconciliation policy.
Record the last accepted event identifier or timestamp and the point where live processing resumed. Overlap the boundary deliberately when necessary, then remove duplicates using an agreed event key rather than price equality.
Compare source, symbol mapping and schema versions across the live and recovered data. If bars were built while events were missing, regenerate affected intervals and expose revisions to downstream chart or research consumers.
Bring the proposed instrument list, source documentation and consumer workflow to a technical discussion. The agreed integration specification defines coverage, fields, usage permissions and acceptance tests.
| Dataset definition | Confirm raw ticks, bid/ask events, trades or OHLC bars and the source instrument. |
|---|---|
| Time and ordering | Define UTC boundaries, timestamp precision, sequence ordering and duplicate policy. |
| Quality and gaps | Document missing intervals, corrections and any backfill or normalization applied. |
| Export and retention | Agree format, pagination, permitted storage and the consumers allowed to use the dataset. |
Historical data for your instrument universe needs an instrument definition and provenance. Tick history, aggregated bars and replay events are different outputs. Record which prices form each bar, how gaps are represented and whether corrections can change an earlier result.