Free lesson · Portfolio construction

Optimise under realistic constraints

Open interactive lesson Practice calculations Explore labs

Start with the idea

An optimiser formalises trade-offs; it does not improve the quality of its inputs automatically. Costs discourage unnecessary trades, constraints express the mandate, and shrinkage reduces sensitivity to noisy estimates.

Symbols, units & horizon

w: vector of signed portfolio weights
μ: expected excess-return vector
Σ: return covariance matrix
λ: positive risk aversion
κ: nonnegative turnover penalty
w_pre: pretrade weights after price drift
|| ||₁: sum of absolute vector entries
1: vector of ones
n₀: required net exposure
G: gross-exposure limit
lᵢ,uᵢ: lower and upper asset bounds
S: sample covariance matrix
F: shrinkage target matrix
δ: fraction assigned to target, between 0 and 1
Σ̂: blended estimate

When and why to use this

Use constrained optimisation to convert multiple forecasts into a risk-budgeted book and compare the result against simple allocations.

\max_{w} μ^{𝖳} w - \frac{λ}{2} w^{𝖳} Σ w - κ ‖ w - w_{pre} ‖_{1}

Differentiation + subgradient reasoning

Differentiate the smooth portfolio objective

Ignoring turnover temporarily, gradient of $μ^{T} w - \frac{λ w^{T} Σ w}{2}$ is $μ - λ Σ w$ for symmetric Σ. An unconstrained optimum solves $λ Σ w = μ$ .
The absolute-value turnover penalty is nonsmooth. Away from zero trade its derivative contributes $κ sign (w - w_{p r e})$ ; at zero use a subgradient between −κ and κ. This creates a no-trade region.

Work it by hand

For one asset with μ=.01, λ=2 and variance=.04, the no-cost solution=.125. Nonzero transaction costs can make retaining the existing position preferable.

𝟏^{𝖳} w = n_{0}, ‖ w ‖_{1} \leq G, l_{i} \leq w_{i} \leq u_{i}

Constraint arithmetic

Check an allocation against its constraints

Add signed weights to check $𝟏^{T} w = n_{0}$ . Add absolute weights to check $‖ w ‖_{1} \leq G$ . Compare each coordinate with its lower and upper bounds.
For two assets with fixed net n₀, substitute $w_{2} = n_{0} - w_{1}$ into the objective. This reduces a small hand problem to one variable, but the gross constraint remains piecewise.

Work it by hand

Weights .6,−.2 have net .4 and gross .8. They satisfy net=.4 and gross≤1, but fail a .5 individual long limit.

$μ$ is the forecast vector for the optimisation horizon; $Σ$ is covariance for that same horizon. Risk aversion $λ$ penalises variance, and $κ$ penalises turnover. Net exposure $n_{0}$ , gross limit G, and position bounds keep the mathematical solution within a mandate. Include cash explicitly when interpreting a fully invested constraint.

Unconstrained weights often magnify small differences in noisy expected returns or nearly collinear assets. Do not treat an optimiser’s precision as forecast accuracy. Shrink expected returns toward zero or a prior; shrink covariance toward a stable target.

\hat{Σ} = (1 - δ) S + δ F, 0 \leq δ \leq 1

Algebra and arithmetic

Blend sample covariance with a target

For every matrix entry compute $(1 - δ) S_{i j} + δ F_{i j}$ . Coefficients sum to 1, so this is a convex combination.
For any vector x, $x^{T} \hat{Σ} x = (1 - δ) x^{T} S x + δ x^{T} F x \geq 0$ when both input matrices are positive semidefinite.

Work it by hand

S diagonal .04,.09 and covariance .03; diagonal target F with the same variances; δ=.5 leaves variances unchanged and reduces covariance to .015.

S is sample covariance and F is a structured target such as a diagonal matrix. The shrinkage intensity trades variance for bias. Choose it without consulting the final holdout, then compare weights and out-of-sample behaviour with simple equal-weight and inverse-volatility baselines.

Further reading: CFA Institute: Portfolio Risk and Return ↗

Python implementation

Self-contained teaching example. Python 3.10+; dependencies and input conventions are shown in the code and notation. Run in your own Python environment.

import numpy as np  # dependency: numpy

def smooth_optimum(expected_returns, covariance, risk_aversion):
    """Unconstrained, zero-turnover-penalty solution only; invertible covariance."""
    return np.linalg.solve(np.asarray(covariance), expected_returns)/risk_aversion

def objective(w, mu, cov, risk_aversion, turnover_penalty, pretrade):
    w, mu, cov, pre = map(np.asarray, (w,mu,cov,pretrade))
    return float(mu@w-.5*risk_aversion*(w@cov@w)-turnover_penalty*np.abs(w-pre).sum())

def feasible(w, required_net, gross_limit, lower, upper, tolerance=1e-9):
    w, lower, upper = map(np.asarray, (w,lower,upper))
    return bool(abs(w.sum()-required_net)<=tolerance
                and np.abs(w).sum()<=gross_limit+tolerance
                and np.all(w>=lower-tolerance) and np.all(w<=upper+tolerance))

def shrink_covariance(sample_cov, target_cov, fraction):
    if not 0 <= fraction <= 1:
        raise ValueError("Shrinkage fraction must be in [0,1]")
    return (1-fraction)*np.asarray(sample_cov)+fraction*np.asarray(target_cov)

print(feasible([.6,.4], 1, 1, [0,0], [1,1]))

Continue learning

Portfolio Construction & Factors — all lessons

Quantitative finance and development glossary · Python resources and libraries · Research sources and limitations