TinyCast: Probabilistic Zero-Shot Forecasting with Computed Periodicity

arXiv:2608.157672026-08-15

一个只有14.6万参数的时间序列预测AI,精度不减,还能在嵌入式芯片上完整运行

TinyCast是一个零样本预测模型,意味着它无需针对新的信号单独训练就能直接预测,总共只用146,505个参数。它不用神经网络去学习周期性规律,而是用一个不含参数的频谱检测器直接计算出主导周期,再按周期相位把数据折叠对齐,之后由膨胀卷积编码器和分位数解码器完成剩余的建模。这样的模型在GIFT-Eval榜单上比所有已知参数量的零样本模型都小,在概率预测精度上刷新了体积与效果的最优边界,并且仅用整数运算就能在嵌入式开发板上完成端到端预测,无需针对每个信号单独调参。

他们做了什么

不通过神经网络学习周期性(季节性规律),而是用一个零参数的频谱检测器直接计算出主导周期,并按其相位把输入数据折叠对齐
剩余的建模工作由膨胀卷积编码器和按区块自回归运行的分位数解码器完成,整个模型完全不使用注意力机制
在GIFT-Eval基准测试中,该模型是所有可查明参数量的零样本模型里最小的一个,并在概率预测精度指标nWQL上重新定义了体积与精度的最优前沿
在声明无测试数据泄露的零样本模型中,它是唯一一个参数量低于140万、却能输出完整概率分布(而非单一点预测)的模型,所有表现更好的模型参数量都至少达到140万
由于模型仅由卷积和矩阵乘法构成,可导出为INT8整数格式,并在没有神经网络加速器的STM32H753嵌入式开发板上完成端到端预测,无需针对具体信号做额外拟合

Figure 1: TinyCast architecture. (a) A single encoder block. (b) The pipeline for one 48-step block: the normalized context and the period-conditioned positional encoding pass through the encoder and decoder to nine quantiles at all block positions in parallel; the dashed arrow is the median feedback that chains blocks. (c) The decoder, whose three readouts compose the per-horizon query qh.

Table 1: Zero-shot GIFT-Eval; lower is better, best per column in bold among the learned models. All three metrics are ratios to seasonal naive, which scores 1.000 on each. Parenthesized values are point errors for models not emitting predictive distributions.
Model	Params ↓	nGMASE↓	nWQL↓	nMSIS↓
TinyCast (ours)	146 K	0.774	0.545	0.554
Reverso-Nano (Fu et al. 2026)	200 K	0.760	(0.661)	(2.035)
Reverso-Small (Fu et al. 2026)	550 K	0.726	(0.626)	(1.945)
TTM-R3 (Ekambaram et al. 2024)	1.4 M	0.724	0.520	0.501
Reverso (Fu et al. 2026)	2.6 M	0.711	(0.610)	(1.905)
Toto-2.0-4m (Khwaja et al. 2026)	4.1 M	0.757	0.524	0.455
YingLong-6m (Wang et al. 2025a)	7.3 M	0.880	0.609	0.534
FlowState-9.1M (Graf et al. 2026)	9.1 M	0.726	0.502	0.563
Kairos-10m (Feng et al. 2025)	9.9 M	0.753	0.554	0.776
AutoARIMA (Hyndman & Khandakar 2008)	0	1.074	0.912	0.948
AutoTheta (Assimakopoulos & Nikolopoulos 2000)	0	1.090	1.244	1.199
AutoETS (Hyndman et al. 2002)	0	1.212	7.489	8.635
FLAIR (Honda 2026)	0	0.838	0.587	0.538

Figure 2: GIFT-Eval point accuracy (left) and nWQL (right) versus parameter count; lower is better. TinyCast∗ is the same checkpoint at the firmware profile (Section 4.3). Dashed staircase: the Pareto frontier over the census at the host profile, which the static-W8A8 point does not enter.

Table 2: TinyCast parameter budget: D=64, 10 encoder blocks, FFN expansion 1.0, K=4 periods, nb=16 phase bins, nine quantile levels.
Stage	Module (shape)	Params
input	Linearin (14→64)	960
encoder	depthwise conv (10×(64,1,3)+bias)	2,560
	pointwise 1×1 (10×(64,64,1)+bias)	41,600
	shared SwiGLU (up 64→128, down 64→64)	12,480
	RMSNorms (20×(64))	1,280
	encoder subtotal	57,920
phase	Wphase (256→64)	16,448
query	Wq (205→64)	13,184
decoder	SwiGLU + RMSNorm	12,544
future-conv	Wfc-in (14→64)	960
	depthwise conv (6×(64,1,3)+bias)	1,536
	pointwise 1×1 (6×(64,64,1)+bias)	24,960
	shared SwiGLU	12,480
	RMSNorms (12×(64))	768
	Wfc-out (64→64, zero-init)	4,160
	future-conv subtotal	44,864
output	Wout (64→9)	585
Total		146,505

Figure 3: Phase binning of the context with a dominant detected period of p1=24. (a) Context values against position relative to the forecast start, t−L; the stripe below assigns every position to one of nb=16 phase bins. (b) Cycle template: the mean of the context values sharing a bin. The arrow indicates the phase template selection for the marked future offset h∗.

Table 3: Training configuration of the deployed TinyCast.
Optimization
Optimizer	AdamW (Loshchilov & Hutter 2019)
Weight decay	0.01
Gradient clipping	1.0
Precision	bf16-mixed
Random seed	42
Total samples	150 M
Effective batch size	4096
Accelerators	eight RTX 3090, DDP
Wall clock	7.8 h, about 62 accelerator-hours
Final weights	average of the last eight checkpoints
Learning-rate schedule
Shape	warmup-stable-decay
Peak learning rate	3×10−3
Minimum learning rate	1×10−5
Warmup fraction	5% (linear to peak)
Stable fraction	60% (held at peak)
Decay fraction	35% (to the minimum)
Sequence layout
Context length L	2048
Horizon unit p	48
Objective and data
Loss	nine-quantile pinball + gated committing term (weight 0.3)
Scheduled sampling	four blocks; feedback probability ramps to 0.5
Corpus	GIFT-Eval-Pretrain + Chronos KernelSynth + four synthetic shards, band-balanced
Augmentations	temporal flip, sign flip, downsample, mixup; each p=0.5

Figure 4: Empirical cumulative distribution over all 97 configurations of the relative change from the matched unquantized reference to the quantized host profile, both carrying the two host-side strategies, in nGMASE (left) and nWQL (right), for all configurations (black) and by forecast term (light to dark: 55 short, 21 medium, 21 long). Positive is a degradation and the dashed line is zero change; the panels share one x range. Dots mark the median and the P90; the filled marker where a curve reaches one is the worst configuration, named with its change.

Table 4: The eight scored configurations: one checkpoint, the same 97 GIFT-Eval configurations, all scored on the host. Lower is better. “Exact” W8A8 calls a host build of the same C integer core sources the firmware executes rather than a fake-quantization emulation, and the board runs the fidelity chain of Appendix E, not the benchmark.
Profile	Arithmetic	Symmetrization	Alignment	nGMASE	nWQL	nMSIS
Host	bf16	yes	yes	0.7738	0.5454	0.5541
No alignment	bf16	yes	no	0.7858	0.5567	0.5786
No symmetrization	bf16	no	yes	0.7816	0.5515	0.5638
Single pass	bf16	no	no	0.7935	0.5629	0.5889
Quantization reference	fp32-strict	yes	yes	0.7736	0.5457	0.5535
Single-pass reference	fp32-strict	no	no	0.7918	0.5622	0.5865
Quantized host	exact W8A8	yes	yes	0.7901	0.5526	0.5632
Firmware configuration	exact W8A8	no	no	0.8328	0.5807	0.6243

Figure 5: The same construction as Figure 4, but at the firmware profile: the relative change from the single-pass reference to exact static W8A8, with neither sign symmetrization nor period alignment on either side. Lower is better and positive is a degradation.

Table 5: Per-dataset supervised models in the pinned snapshot, all 97 configurations, ratios to seasonal naive. Lower is better. These are trained on the series they forecast and are therefore outside the zero-shot census of Table 1.
Model	nGMASE↓	nWQL↓	nMSIS↓
TinyCast (ours)	0.774	0.545	0.554
PatchTST	0.849	0.587	0.574
iTransformer	0.893	0.620	0.613
TFT	0.915	0.605	0.656
N-BEATS	0.938	0.816	2.512
DLinear	1.061	0.846	2.841
TiDE	1.091	0.772	0.906
DeepAR	1.343	0.853	0.933
Crossformer	2.574	1.637	6.892

Figure 6: Zero-shot forecasts at the host profile on ten GIFT-Eval tasks, spanning all seven domains, four sampling frequencies and three forecast terms. Truth (blue), median forecast (orange, dashed), shaded 10–90% predictive interval, seasonal naive (grey, dotted); the vertical dotted line marks the forecast start and grey bands mark missing observations. Titles give configuration, domain, nGMASE and, where the horizon is only partly drawn, “first n of H”; y is in each series’ own units. The panel-selection and x-range rules are stated below.

Table 6: Paired bootstrap intervals for every difference in Table 1 with a per-configuration comparator record. Positive Δ means the comparator scores lower and therefore leads. The three StatsForecast baselines are omitted: all are resolved against us, by margins of 0.30 to 8.08 units. FLAIR is absent for the reason given above, having no per-configuration entry in the snapshot. † marks a difference whose cluster interval spans zero. The Reverso rows on nWQL and nMSIS carry the same caveat as in Table 1: those models emit no predictive distribution, so the quantity differenced is a point error.
				95% CI
	Model	Params	Δ	configs	base datasets
nGMASE	Reverso-Nano	200 K	+0.0140†	[+0.0030,+0.0245]	[−0.0022,+0.0283]
Reverso-Small	550 K	+0.0475	[+0.0366,+0.0581]	[+0.0322,+0.0613]
TTM-R3	1.4 M	+0.0498	[+0.0348,+0.0656]	[+0.0290,+0.0695]
Reverso	2.6 M	+0.0626	[+0.0479,+0.0782]	[+0.0437,+0.0843]
Toto-2.0-4m	4.1 M	+0.0172†	[−0.0081,+0.0382]	[−0.0102,+0.0448]
YingLong-6m	7.3 M	−0.1064	[−0.1446,−0.0733]	[−0.1702,−0.0530]
FlowState-9.1M	9.1 M	+0.0476	[+0.0327,+0.0627]	[+0.0300,+0.0651]
Kairos-10m	9.9 M	+0.0211	[+0.0008,+0.0383]	[+0.0021,+0.0374]
nWQL	Reverso-Nano	200 K	−0.1156	[−0.1324,−0.1002]	[−0.1408,−0.0922]
Reverso-Small	550 K	−0.0811	[−0.0954,−0.0678]	[−0.1006,−0.0634]
TTM-R3	1.4 M	+0.0259	[+0.0127,+0.0399]	[+0.0085,+0.0459]
Reverso	2.6 M	−0.0649	[−0.0827,−0.0466]	[−0.0886,−0.0382]
Toto-2.0-4m	4.1 M	+0.0213†	[+0.0001,+0.0401]	[−0.0036,+0.0457]
YingLong-6m	7.3 M	−0.0636	[−0.0894,−0.0405]	[−0.1031,−0.0260]
FlowState-9.1M	9.1 M	+0.0435	[+0.0304,+0.0582]	[+0.0268,+0.0629]
Kairos-10m	9.9 M	−0.0087†	[−0.0283,+0.0086]	[−0.0278,+0.0092]
nMSIS	Reverso-Nano	200 K	−1.4807	[−1.6688,−1.3123]	[−1.7867,−1.1911]
Reverso-Small	550 K	−1.3911	[−1.5651,−1.2281]	[−1.6846,−1.1185]
TTM-R3	1.4 M	+0.0528	[+0.0219,+0.0843]	[+0.0076,+0.0935]
Reverso	2.6 M	−1.3506	[−1.5205,−1.1952]	[−1.6226,−1.0875]
Toto-2.0-4m	4.1 M	+0.0989	[+0.0684,+0.1304]	[+0.0668,+0.1309]
YingLong-6m	7.3 M	+0.0200†	[−0.0131,+0.0522]	[−0.0300,+0.0579]
FlowState-9.1M	9.1 M	−0.0091†	[−0.0413,+0.0244]	[−0.0488,+0.0215]
Kairos-10m	9.9 M	−0.2224	[−0.2852,−0.1660]	[−0.2967,−0.1549]

Table 7: Configuration of the three ablation families and of the deployed model. Deltas are valid within a family and absolute scores are not comparable across families. Q is the number of quantile levels; the single-quantile families are scored on nMAD, and the component family runs at about a fifth of the deployed sample budget. Every family reads GIFT-Eval-Pretrain and Chronos KernelSynth; the component family adds four locally generated length-4096 KernelSynth shards, which its synthetic-family arm replaces with the four Reverso-family shards the deployed model trains on.
Family	Control	Parameters	Steps	Q	Encoder
Architecture	dilated-conv base	340,545	7,500	1	untied, non-separable, FFN 1.5
	+ phase binning	361,089
	+ recency gate	393,921
	substituted-recency arm	410,241
Component	architecture family’s best arm	394,441	30,000	9	untied, non-separable, FFN 1.5
	+ future-conv	445,513			future-conv stack separable, shared FFN
Optimization	peak LR 3×10−3, 100 M samples	–	–	1	effective batch 4096
Deployed	–	146,505	36,621	9	tied, separable, FFN 1.0

Table 8: Architecture family, dilated-conv base at 340 K, 7,500 steps. Lower is better on nGMASE and nMAD, so a negative Δ is an improvement. Arms are not parameter-matched, and each row gives its count. nMAD is the single-quantile point error defined above. † marks a delta whose 95% paired bootstrap interval over configurations spans zero.
	params	nGMASE	Δ	nMAD	Δ
detector off, recency path substituted	410 K	1.1537	+0.1312	0.9774	+0.1160
control (dilated-conv base)	340 K	1.0225		0.8614
causal padding	340 K	1.0161	−0.0064†	0.8562	−0.0052†
+ phase binning	361 K	0.9248	−0.0977	0.7791	−0.0823
+ phase binning + causal	361 K	0.9250	−0.0975	0.7886	−0.0728
+ phase binning + recency gate	393 K	0.9092	−0.1133	0.7694	−0.0920

Table 9: Component family, 394 K nine-quantile line, 30,000 steps. Lower is better on nGMASE and nWQL, so a negative Δ is an improvement. Arms carry the control’s parameter count except future-conv, which adds 51 K. † marks a delta whose 95% paired bootstrap interval over configurations spans zero.
	nGMASE	Δ	nWQL	Δ
control (394 K line)	0.8123		0.5699
+ future-conv correction	0.7972	−0.0151	0.5614	−0.0085
+ synthetic-family blend	0.8060	−0.0062†	0.5669	−0.0030†
+ synthetic-family blend (tuned dose)	0.8052	−0.0071†	0.5623	−0.0076†
+ gated committing loss	0.8100	−0.0023†	0.5690	−0.0009†
+ future-conv + synthetic families	0.7860	−0.0263	0.5529	−0.0170
8 training AR chunks (vs 4)	0.8132	+0.0010†	0.5648	−0.0051†
backtest-selected period (vs raw detection)	0.8167	+0.0044†	0.5749	+0.0050†
MASE-weighted loss	0.8146	+0.0023†	0.5719	+0.0020†

Table 10: Optimization family: learning-rate peak and sample budget. Lower is better, so a negative Δ is an improvement. nMAD is a point error, as in Table 8. † marks a delta whose interval spans zero.
	nGMASE	Δ	nMAD	Δ
control (peak LR 3×10−3, 100 M samples)	0.8562		0.7252
peak LR 1e-3	0.8957	+0.0395	0.7460	+0.0208
peak LR 2e-3	0.8665	+0.0103	0.7300	+0.0048†
peak LR 4e-3	0.8745	+0.0183	0.7304	+0.0052†
budget 50 M samples	0.8933	+0.0371	0.7479	+0.0227
budget 150 M samples	0.8533	−0.0029†	0.7217	−0.0035†

Table 11: Single-variable overrides on the architecture family’s best arm. All 97 GIFT-Eval configurations; a negative Δ is an improvement.
	nGMASE	Δ
control (phase binning + recency gate)	0.9092
phase bins, 16 to 32	0.9047	−0.0045
cross-horizon convolution, width 5	0.9085	−0.0007
encoder kernel width, 3 to 5	0.9100	+0.0008
gated encoder convolution	0.9178	+0.0086
harmonics per period, 1 to 2	0.9178	+0.0086
period cap, 4 to 8	0.9233	+0.0141
recency-weighted phase fold	0.9242	+0.0150

Table 12: Interventions aimed at point accuracy, tested and rejected. nGMASE throughout, lower is better. Groups 1 and 2 are scored against an archived control and group 1’s per-configuration results are released. Group 3’s arms have no archived control, and groups 4 to 6 are reported by direction.
Group	Intervention	Scored on	Result
1 Computed values	seasonal-naive draft as input value	6-config probe	0.8825
	+ all significant periods	6-config probe	0.8930
	+ linear trend term	6-config probe	0.9178
	control (dilated-conv base)	6-config probe	0.8760
	trend-seasonal decomposition as channels	97 configs	0.9177
2 Learned rule	learned per-period weight, variant 1	97 configs	0.9114
	learned per-period weight, variant 2	97 configs	0.9190
	hard reliability rule	97 configs	0.9002
	no rule	97 configs	0.9092
3 Per-instance adaptation	4 methods, best is an oracle gate	97 configs	0.818
	amortized model	97 configs	0.774
4 Tail-robust objective	worst-α per-sample losses	probe	worse at every α
5 Horizon-weighted loss	linear long-horizon up-weighting	probe	no net change
6 Median up-weighting	q0.5 term up-weighted	probe	no gain at convergence

Table 13: Interval score on GIFT-Eval at the host profile, all 97 configurations, normalized to seasonal naive. Lower is better. Positive Δ means the comparator scores lower and therefore leads. Intervals are paired bootstraps over configurations, and † marks a difference whose cluster interval spans zero. Parenthesized values are point errors for models not emitting predictive distributions, as in Table 1, and their differences carry the same caveat.
Model	Params ↓	nMSIS↓	Δ	95% CI
TinyCast (ours)	146 K	0.554
Reverso-Nano	200 K	(2.035)	−1.4807	[−1.6688,−1.3123]
Reverso-Small	550 K	(1.945)	−1.3911	[−1.5651,−1.2281]
TTM-R3	1.4 M	0.501	+0.0528	[+0.0219,+0.0843]
Reverso	2.6 M	(1.905)	−1.3506	[−1.5205,−1.1952]
Toto-2.0-4m	4.1 M	0.455	+0.0989	[+0.0684,+0.1304]
YingLong-6m	7.3 M	0.534	+0.0200†	[−0.0131,+0.0522]
FlowState-9.1M	9.1 M	0.563	−0.0091†	[−0.0413,+0.0244]
Kairos-10m	9.9 M	0.776	−0.2224	[−0.2852,−0.1660]
FLAIR	0	0.538	–	no per-configuration record

Table 14: Chronos-ZS at the host profile, all 27 tasks, ratios to the published seasonal-naive reference. Lower is better. The statistical methods need no training data and have no position on a parameter axis.
Model	Params	rel. MASE	rel. WQL
TinyCast	146 K	0.880	0.722
AutoARIMA	–	0.869	0.741
AutoTheta	–	0.859	0.793
AutoETS	–	0.943	0.814
TTM-R2	805 K	1.120	1.134
Chronos-Tiny	8.4 M	0.874	0.703
Chronos-Bolt-Tiny	8.7 M	0.849	0.668

Table 15: fev-bench at the host profile, all 100 tasks. Ratios are to the vendored seasonal-naive reference and lower is better. Win rate is TinyCast’s share of tasks won against that model, computed by the benchmark’s own pairwise function, so it rests on no comparator pool. AutoARIMA returns 96 of the 100. ∗CITRAS-FM releases no checkpoint, so its parameter count is the one stated in its paper rather than one we instantiated.
		MASE	WQL
Model	Params ↓	ratio	win rate	ratio	win rate
TinyCast	146 K	0.819	–	0.658	–
AutoARIMA	0	0.879	0.688	0.746	0.719
AutoTheta	0	0.890	0.650	0.922	0.810
Toto-2.0-4m	4.1 M	0.720	0.190	0.553	0.140
CITRAS-FM	7.2 M∗	0.707	0.140	0.540	0.080
FlowState	9.1 M	0.702	0.140	0.525	0.110

Table 16: Cold start: accuracy against the number of observed samples, all 97 configurations rescored on a truncated context at the no-alignment profile, since an alignment factor computed from a truncated context would change the effective context length as history shrinks. Relative MAE, lower is better; 1.0 is parity with seasonal naive.
Observed samples	64	128	256	512	1024	2048
Relative MAE	1.013	0.975	0.911	0.845	0.786	0.759

Table 17: Deployment record of the TinyCast static-W8A8 core, with FP32 islands, on an STM32H753 Cortex-M7 at 480 MHz. The RAM rows are not additive: the first 8 KiB of the heap row is already reserved in the linker-section row, and the accounted total removes the overlap.
Latency
Core call (full L=2048 re-encode)	4.08 s
Per-position encoder step (×L per core call)	1860 μs
Decoder (in isolation)	0.20 s
Flash
INT8 matrix and convolution coefficients	138.1 KiB
Complete benchmark image (incl. 8 KiB context)	365.5 KiB
RAM
Encoder causal rings	128.5 KiB
Phase-fold context tensor	128 KiB
Linker RAM sections and reservation	508.6 KiB
Persistent model heap payload	217.0 KiB
Accounted unique RAM (lower bound)	717.6 KiB
Peak occupancy (statics, heap, stack high-water)	730.7 KiB

为什么重要

在每个部署现场都单独收集数据、训练并维护一个专属模型,人力成本高昂且不会随算力变便宜而降低,而这项工作证明小到可以塞进嵌入式芯片的模型依然能对陌生信号直接做出可靠预测。它还填补了一个空白:此前体积如此小的模型通常只能给出单一预测值,而它能同时给出带不确定性的概率预测。

本文术语

零样本预测 · 无需针对新数据额外训练,直接进行预测
概率分布/分位数预测 · 给出一系列可能取值及其概率,而不是单一的预测数字
膨胀卷积 · 跳跃采样的卷积操作,能用较小计算量覆盖较长范围,捕捉长距离规律
INT8量化 · 把模型中的数值用8位整数表示,以减少计算量和存储占用
相位折叠 · 把处于同一周期相同位置的数据点归为一组并取平均

论文原文摘要（英文）

We introduce TinyCast, an attention-free zero-shot forecaster that emits a predictive distribution from 146,505 parameters, on the premise that at this size the periodic structure of a context is worth computing rather than learning. A zero-parameter spectral detector supplies the dominant periods, the context is folded on their phase, and a dilated convolutional encoder and a block-autoregressive quantile decoder model the rest. It is smaller than every zero-shot entry on the GIFT-Eval board whose parameter count can be established. On probabilistic accuracy it defines the size-accuracy frontier. Among zero-shot entries declaring no test-data leakage it is the only one below 1.4M parameters that emits a predictive distribution, and every entry scoring better carries at least that budget. On Chronos-ZS and fev-bench every neural model ahead of it carries at least 28 times its parameters. Because the mixing path is convolutions and matrix multiplications only, it exports to static INT8 and forecasts end to end on an embedded device without per-signal fitting.

作者 · Armin Steinhauser

在 arXiv 阅读

METAL LAB 最新报道

图片来源： Armin Steinhauser et al., arXiv:2608.15767, CC BY 4.0

TinyCast: Probabilistic Zero-Shot Forecasting with Computed Periodicity

他们做了什么

为什么重要

本文术语

最新论文

METAL LAB 最新报道