Free lesson · Point-in-time data
Coverage, missingness and stale values
Open interactive lessonPractice calculationsExplore labs
Start with the idea
A complete-looking table can hide that many observations were never measured or are too old to support the decision.
Symbols, units & horizon
- c: dimensionless coverage fraction within a declared sample window
- n_observed: eligible nonmissing observations
- n_expected: expected eligible observations, positive
- s: staleness in seconds
- t: decision time
- a: last usable observation time on the same clock
When and why to use this
Detect feed outages and sample selection before model fitting or live feature use.
A complete-looking table can hide that many observations were never measured or are too old to support the decision.
Coverage needs a declared denominator: all expected entity-time observations, all active instruments, or all eligible documents. A missing value is not zero activity. Forward filling keeps an old measurement alive; it does not create new information.
Measure coverage by venue, sector, date and outcome class. If small or distressed assets are less covered, a model trained only on available rows learns a selected population. Keep missingness indicators and test economic sensitivity to the missing cases.
Coverage, missingness and stale values
- Define the expected universe and measurement schedule before counting.
- Divide nonmissing eligible observations by expected count.
- For each carried observation subtract usable time from decision time and compare to a declared maximum age.
Eight observations out of ten expected give .8 coverage. A feature last updated at second 100 used at second 160 is 60 seconds stale.
Apply it in a strategy
- Freeze inputs at the stated decision time and record their units.
- Detect feed outages and sample selection before model fitting or live feature use.
- Recompute the example, then change the material assumption and explain the difference.
Research deliverable
Coverage, missingness and stale values: produce the worked calculation, a timestamped input record and a written decision addressing this limitation: High aggregate coverage can conceal systematic gaps in the most economically important subgroup.
These are synthetic mechanics examples, not historical performance or paper replications. Module evidence and research boundaries record the 12 September 2026 review.
Python implementation
Self-contained teaching example. Python 3.10+; dependencies and input conventions are shown in the code and notation. Run in your own Python environment.
# Python 3.10+; standard library and NumPy only.
# Synthetic teaching inputs; conventions and units are defined in the notation above.
def quality_summary(observed,expected,decision,last_available):
if expected<=0 or not 0<=observed<=expected or last_available>decision: raise ValueError('Invalid counts or future observation')
return observed/expected,decision-last_available
assert quality_summary(8,10,160,100)==(.8,60)
print(quality_summary(8,10,160,100))Continue learning
Data: Availability, Revisions & Reproducible Research — all lessons- Four clocks for one observation
- An as-of join that never selects the future
- First prints and revisions can reverse a signal
- Reconcile splits and cash distributions
- Universe membership and disappearing assets
- Coverage, missingness and stale values
- Purging labels that cross a test boundary
- Reproducible snapshots and discrepancy ledgers
Quantitative finance and development glossary · Python resources and libraries · Research sources and limitations