Free lesson · Quant strategy development

09 / Decide whether the edge is real enough to continue

Open interactive lesson Practice calculations Explore labs

Start with the idea

When performance changes, separate prediction error from execution cost and risk exposure. A strategy can keep forecasting direction while losing its economic advantage because fills, spreads or borrow costs worsen.

Symbols, units & horizon

Rₜ: realised same-horizon return
R̂ₜ: forecast saved before the outcome
Cₜ: realised execution cost
Ĉₜ: cost forecast saved before trading
Error: actual minus forecast
Positive cost error: costs were underestimated
t: evaluation period
Bars and hats: average and estimate respectively

When and why to use this

Use attribution at predefined review times to decide whether a signal, cost model or operational component needs investigation. A revision should start a newly documented validation cycle.

Use a written promotion process: hypothesis → research replay → locked out-of-sample study → paper operation → a separately authorised live evaluation. Each stage answers different questions. Paper operation reveals plumbing problems and model timing; its fill assumptions still cannot establish realised profitability.

{f o r e c a s t e r r o r}_{t} = R_{t} - {\hat{R}}_{t}, {c o s t e r r o r}_{t} = C_{t} - {\hat{C}}_{t}

Algebra and arithmetic

Separate forecast and cost surprises

Return error is realised return minus forecast; cost error is realised cost minus estimated cost. Rearranging gives $R = \hat{R} + e_{R}$ and $C = \hat{C} + e_{C}$ .
Subtract the identities: $R - C = (\hat{R} - \hat{C}) + e_{R} - e_{C}$ . A positive cost error hurts net performance.

Work it by hand

Forecast 12 bp, estimated cost 4 bp, realised return 10 bp, actual cost 7 bp: expected net 8 bp, return error −2 bp, cost error +3 bp, realised net 3 bp.

Track forecast and cost errors separately. A model can retain predictive power while becoming uneconomic because spread, borrow or impact rose. It can also look profitable because a common risk factor rallied even while the original signal stopped working. Attribution helps distinguish these cases.

Prediction: bucket calibration, IC, residuals and changes in feature distributions.
Economics: net return, turnover, shortfall, realised markouts, borrow and capacity.
Risk: factor exposure, concentration, stress losses, drawdown and available cash.
Operations: reconciliation breaks, stale data, order rejects, duplicates, latency and recovery events.

Final strategy dossier: one economic hypothesis; a feature dictionary; exact decision and execution timing; chronological splits; all trials; simple baselines; net results with uncertainty; ablations; cost and capacity curves; portfolio and liquidity limits; and a reproducible incident replay. The conclusion may be “reject”, “collect more evidence”, or “continue paper evaluation”.

Research sources, review dates and limitations

Extend the research question

Write who pays the strategy and why the pressure might persist. Distinguish guaranteed state-payoff relationships from uncertain convergence and compensation for risk.

Continue with the connected research module →

Python implementation

Self-contained teaching example. Python 3.10+; dependencies and input conventions are shown in the code and notation. Run in your own Python environment.

def forecast_errors(realized_return, saved_return_forecast, actual_cost, saved_cost_forecast):
    """All four inputs use consistent return units and horizons."""
    return realized_return-saved_return_forecast, actual_cost-saved_cost_forecast

print(forecast_errors(.001,.003,.0015,.001))

Continue learning

Quant Strategy Development — all lessons

Quantitative finance and development glossary · Python resources and libraries · Research sources and limitations