Spiral Positional-Encoding Benchmarks

The experiment

Task: predict New York City hourly temperature 24 hours ahead from the previous 96 hours. Data: 35,064 hours of Open-Meteo ERA5 reanalysis (2021–2024, UTC). Split: train on 2021–2023, test on all of 2024 — strictly chronological, no leakage. Model: 2-layer transformer (d_model 64, 4 heads), 4 epochs, best-validation checkpoint, 3 random seeds per variant.

Benchmark 1 — Forecast accuracy

Mean absolute error on unseen 2024 data, in °C (lower is better; mean of 3 seeds):

The spiral encoding cut forecast error by 22% vs. a raw timestamp and 16% vs. calendar integers, and won on every metric we tracked — including the boundary cases where integer features jump discontinuously while sin/cos stays smooth:

Benchmark 2 — The advantage compounds with history

Same task, but training only on the most recent slice of history (chronological tail of 2021–2023; 2 seeds per point). At small data budgets the three encodings are statistically indistinguishable — differences are within run-to-run noise. The spiral variant separates clearly only at the full three years:

At the full three years of history the spiral variant is 14% ahead of calendar features and 20% ahead of a raw timestamp. A plausible reading — consistent with, but not proven by, these four points — is that the richer 24-dimensional encoding needs enough repeated cycles (seasons, weeks, days) in the training data before the model can exploit it.

Honest footnotes: each point is only 2 seeds, so small-data differences (including spiral's 3.6- and 9-month points and the calendar-integer 5.43 outlier at ≈9 months) are within noise. We report every measured number, including the awkward ones.

Benchmark 3 — Selecting the band of scales

The spiral coordinate spans 12 scales, but a given dataset can only teach the scales it actually contains: hourly samples make the sub-hour rungs constant, and three years of history contains zero completed decade cycles. So we tested a focus-band variant — keep only the scales whose period fits between the sampling cadence and the training span (here: the hour-through-year rungs — 10 dims):

On the full training set (3 seeds: 2.92, 2.78, 2.73) the focus-band spiral reaches MAE 2.81 ± 0.08 °C — another 14% below the all-scales spiral, 27% below calendar integers, and 33% below a raw timestamp. Scale selection is not a hack around the encoding; it is the encoding used well — the API exposes exactly this via its trusted-scales output, so a model consumes only the rungs its data can support.

Honest footnote: focus-banding did not rescue the small-data regime — at ≈3.6 months it was slightly worse than the full spiral (4.64 vs 4.33) and at ≈9–18 months slightly better (4.08 / 3.86 vs 4.38 / 4.03), all within seed noise. Its decisive win is at realistic data volumes.

Benchmark 4 — How many dimensions does the accuracy cost?

Same task, full 36-month history, but sweeping nested bands of the spiral encoding from 2 dims (the daily rung alone) up to all 24 (3 seeds per band):

What these eight measured points show. The 4-dim hr–day band alone score 2.85 ± 0.02 — within about 0.1 °C of the best row in the table. The three rows furthest from the best are the 2-dim band (3.97), the 12-dim min–yr band (3.07 ± 0.11), and the 24-dim all-scales encoding (3.27 ± 0.14). In this sweep, more dimensions did not mean better accuracy: every band from 4 to 14 dims outscored the full 24.

Honest footnotes: the best single row here is the 14-dim s–yr band (2.76 ± 0.03), slightly below the 10-dim focus band (2.81 ± 0.08) — the seed spreads of the two rows overlap. We ship the focus band because it falls out of a principled rule (cycle ≥ 2× sampling interval, cycle ≤ training span) rather than post-hoc picking the best row. The 12-dim min–yr point (3.07 ± 0.11) sits above both of its neighbours; we report it as measured.

Reproduce it

The full pipeline is three files and one public dataset — no proprietary data, no cherry-picking:

1. data     Open-Meteo ERA5 archive — NYC hourly temperature,
            2021-01-01 → 2024-12-31 UTC (35,064 hours)
2. encode   gen_spiral.mts — runs the production encodeSpiral()
            over every timestamp → 24-dim (sin, cos) features
3. train    train.py / train_efficiency.py / train_adaptive.py /
            train_dims_sweep.py — PyTorch, identical 2-layer
            transformer per variant, seeds 0-2, best-validation
            checkpoint → results.json

Want the spiral coordinate for your own model? Read the spiral docs or call the API in one request.

More benchmarks are in progress, including GPU-side (WebGPU) throughput measurements — this page will grow.