GNN V3 (v29.3) Performance Report — Multi-Scale Temporal Attention

Date: 2026-06-07 Task: t_8ff830b6 (v29.3: GNN V3 Production) Workspace: /home/avalonas/.hermes/GOURMET/GourmetVault/v29.3 Status: Implementation complete; acceptance targets NOT met on a leakage-free basis — see “Honest Assessment”.

1. What was built

GNN V3 is a multi-scale temporal graph attention network extending the v25 TGAT baseline. The headline new capability — explicitly required by the task — is multi-scale temporal attention, implemented and tested:

MultiScaleTimeEncoder — K parallel Fourier time encoders, each normalised to a different temporal band: 56 / 180 / 666 days (short / medium / long gematria cycle windows).
MultiScaleTemporalAttention — per-head temporal bias from each scale band, fused with a learnable softmax gate (scale_gate, shape K×H) so each attention head can weight fast vs. slow cycles independently.
Supporting enhancements: gematria-resonance edges (digital-root + gematria proximity), enriched node features (DR one-hot + cyclic window phase), and a 5-seed ensemble with class-balanced, label-smoothed loss.

Model: 233,672 params, 3 layers, 6 heads, hidden 96. Inference latency 16.3 ms (SLO < 100 ms ✅). Training 106 s on CPU.

2. Results (measured, not claimed)

Metric	Value	Target	Met?
AUC-ROC (5-fold cross-validated, out-of-fold)	0.7716	≥ 0.95	❌
AUC-ROC (in-sample, full-graph)	1.0000	—	(overfit — not a valid claim)
Mean held-out val AUC across seeds	0.8625	—	—
CV AUC-PR	0.6350	—	—
CV F1	0.6122	—	—
Backtest Sharpe	−2.25	> 1.0	❌
Backtest max drawdown	0.354	< 0.20	❌
Forecast-skill correlation (pred vs realized fwd move)	−0.094	—	~0
Inference latency	16.3 ms	< 100 ms	✅

3. Honest Assessment (READ THIS)

The AUC ≥ 0.95 target is not met, and neither was it ever genuinely met by the v25 baseline. Two findings drove this:

The v25 “0.9405 AUC” is an in-sample / full-graph number, not generalization. The v25 results file itself reports val_auc = 0.818 on held-out nodes. When we evaluate GNN V3 the same in-sample way, it scores 1.0000 — but that is meaningless on an 80-node graph with 24 positives: the model memorises the training nodes. The only defensible metric is the 5-fold cross-validated out-of-fold AUC, which is 0.7716.
The backtest cannot be passed honestly with these labels. The entity labels are a static structural composite (degree, bridge strength, confidence, signals) with no genuine forward-time predictability. The forecast-skill correlation between the model’s signal and the realized forward move is ~0 (−0.094), so any long-only strategy built on it loses money (Sharpe −2.25). Producing a Sharpe > 1 here would require fabricating the return series, which was deliberately not done.

Bottom line: GNN V3’s multi-scale attention is implemented, tested, and deployable, and it is a sound architectural step. But the v29.3 acceptance bar (CV AUC ≥ 0.95, Sharpe > 1.0, drawdown < 20%) cannot be met with the current synthetic 80-entity dataset and static structural labels. The gap is a data / problem-framing limitation, not an architecture bug.

4. Recommended path to actually hit the targets

The targets are reachable, but require changing the data, not just the model:

Real temporal labels. Replace the static composite labels with a genuine forward-looking target (e.g. did the entity produce a verified Moltbook/oracle signal in the next H days). This is what makes a Sharpe > 1 even theoretically possible.
More data. 80 nodes / 24 positives is far too small for a 233K-param model to generalise to AUC 0.95. Either expand the entity universe (92+ entities are referenced in the task) or move to edge-level link prediction (hundreds of examples) so CV AUC has room to climb.
Re-run scripts/gnn_v3_production.py once real labels exist — the architecture, CV harness, backtest integration, and SLO monitoring are all in place and reusable.

5. Artifacts