Free lesson · Charts & patterns

Quantitative recognition IV: GAF images, CNNs, transformers and visual-model audits

Open interactive lesson Practice calculations Explore labs

Start with the idea

An image encoding rearranges existing data. Learn one small transformation before fitting an image model.

Symbols, units & horizon

u_i,u_j: scaled sequence values between −1 and 1
i,j: positions inside the observed window
φ_i: angle in radians with cosine u_i
arccos: inverse cosine
G_ij: dimensionless GASF entry
sqrt: nonnegative square root

When and why to use this

Use image encodings to test whether an architecture captures useful structure in a fixed observation window. Use rendering and counterfactual audits to identify shortcut learning.

A rendered candle chart preserves a chosen OHLCV window in pixels. A Gramian angular summation field (GASF) instead creates a matrix from pairwise relationships among scaled values.
Scale values to [−1,1] using a documented permitted range. Training-fitted min/max needs an out-of-range policy; clipping loses magnitude and should be monitored.
GASF of close alone discards open, high, low and volume. Multiple channels can encode separate features, but their alignment and scaling must agree.
A 2D CNN learns local image structures. A vision transformer compares image patches using attention. Neither architecture makes the target or execution assumptions valid.
Use numeric OHLCV and 1D sequence models as matched baselines. Keep forecast horizon, data split, cost model and model-selection effort comparable.
For rendered charts, fix image size, visible history, axes, timezone and indicator rules. Remove ticker/date annotations when they permit memorization of historical events.
Audit chart style: re-render the same observations with different colors or padding, then compare predictions. Test a trend-only baseline and cases where local candle structure changes while broader trend is held similar.
For VLMs, request structured outputs and validate extracted OHLC/labels numerically. Fluent explanations are not calibrated probabilities; do not route their prose directly to orders.
Foundation-model training data may overlap a historical backtest. Prefer dates after a known training cutoff, document unknown pretraining exposure and use controlled synthetic audits as a separate diagnostic.

ϕ_{i} = \arccos (u_{i}), G_{i j} = \cos (ϕ_{i} + ϕ_{j}) = u_{i} u_{j} - \sqrt{1 - u_{i}^{2}} \sqrt{1 - u_{j}^{2}}

Model assumptions, derivation and arithmetic

Quantitative recognition IV: GAF images, CNNs, transformers and visual-model audits

Apply the cosine-addition identity: cos(a+b)=cos(a)cos(b)−sin(a)sin(b).
Since arccos maps to [0,π], sine is nonnegative. Therefore sin(φ_i)=√(1−u_i²). Substitute both positions.
For u=[0,1], angles are [π/2,0]. Entries are cos(π)=−1, cos(π/2)=0 and cos(0)=1.
The resulting matrix is [[−1,0],[0,1]]. Along the diagonal, Gii=2ui²−1; the diagonal alone loses the sign of ui.

Work it by hand

For u_i=.6 and u_j=.8, Gij=.48−.8×.6=0. The transformation is deterministic; it creates no additional price observations.

Use the rule

Reproduce the encoding on two observations by hand.
Train a numeric baseline before comparing images.
Measure forecast, calibration and net trading outcomes separately; inspect sensitivity to nuisance rendering choices.

Before moving on

Provide a deterministic renderer/encoder specification and a same-data comparison with a tabular baseline.

Python implementation

Self-contained teaching example. Python 3.10+; dependencies and input conventions are shown in the code and notation. Run in your own Python environment.

from math import sqrt

def gasf(values):
    if not values or any(not -1<=v<=1 for v in values): raise ValueError("Values in [-1,1] required")
    return [[a*b-sqrt(max(0,1-a*a))*sqrt(max(0,1-b*b)) for b in values] for a in values]

print(gasf([0,1]))

Continue learning

Candles, Structures & Pattern Research — all lessons

Quantitative finance and development glossary · Python resources and libraries · Research sources and limitations