In-depth analysis · About 6 min read
Granite combines four forecasters, but its uncertainty still needs testing
IBM’s ensemble recipe reduces model-selection work; small point-error gains and mixed probabilistic results require separate checks.
- Published here
- Source published
- Source checked
IBM Research’s October 22 article introduces a Granite time-series ensemble built from four pretrained models. It offers an equal-influence probabilistic default and an adaptive IQR alternative without retraining or fitting aggregation weights. The reported results are IBM’s benchmark evidence, not an AISubHub test.
Listen to the news
Read this article aloud or keep listening to the news.
Preparing the reader…
Device voice
Speech is handled by your browser or device. AISubHub does not send the text to an audio API. Voice availability and playback with a locked screen vary by device.
Resuming may repeat a short phrase.
At a glance
- The recipe combines four checkpoints at inference time; it does not release one merged-weight model.
- Across 97 configurations, Prob scores 0.683 MASE versus 0.685 for PatchTST-FM-r2, while that single member retains the lower aggregate CRPS/MWQL.
Four architectures remain four running models
Choosing a pretrained forecaster remains a task-specific decision: a smooth sensor signal and irregular demand need not favor the same architecture. IBM combines Granite PatchTST-FM-r1, a patched Transformer; PatchTST-FM-r2, a patched Conformer; FlowState-r1.1, a state-space model; and TTM-r3, a Tiny Time Mixers MLP mixer. Their shared quantile interface permits aggregation without first choosing one member. Architectural variety is a reason to test complementary errors, not proof that their mistakes are independent.
The downloadable recipe stores configuration rather than merged weights and fetches four member checkpoints at runtime. The first load therefore includes extra downloads. The example requires Granite TSFM 0.3.11 or newer and uses 1024 context observations with a 24-step forecast; those are example settings, not universal limits. Outputs include a point forecast (the median) and quantiles normally spanning 0.1–0.9. The interval between 0.1 and 0.9 has nominal 80% coverage; its actual coverage must be measured.
Probability pooling and spread weighting answer different questions
The recommended Probabilistic Ensemble gives each member equal influence, pools forecasts in probability space and extracts the requested quantiles from the resulting distribution. This should not be simplified into averaging four values at each quantile level. Fixed influence removes a weight-fitting stage and weighting hyperparameters for each new series. It does not remove the need to check whether the combined distribution represents the application’s uncertainty.
The IQR alternative approximates each member’s interquartile range using the available quantiles nearest the 25th and 75th percentiles. Narrower spreads receive more influence, separately for each sample, target and horizon step. Neither method fits weights from historical forecast errors; IQR uses isotonic regression to prevent crossing quantiles. A higher temperature moves its weights toward uniform, and an optional cap limits one member’s influence. Narrowness can also reflect overconfidence, so it does not establish accuracy or calibration.
The benchmark does not choose one winner for every metric
IBM reports zero-fine-tuning evaluation on 97 GIFT-Eval dataset configurations, with no test targets used to choose or fit aggregation. Scores and ranks are from October 20, 2026. MASE measures point errors, while CRPS/MWQL is the benchmark’s mean_weighted_sum_quantile_loss. Scores are normalized to Seasonal Naive and geometrically averaged. Prob, IQR and PatchTST-FM-r2 respectively score 0.683, 0.684 and 0.685 MASE; their average MASE ranks are 47.175, 48.773 and 51.371. Lower is better. The 0.002 gap to the best individual member is small and does not establish statistical significance.
Prob and IQR both score 0.471 aggregate CRPS/MWQL, compared with the better 0.467 for PatchTST-FM-r2. Their average CRPS ranks are respectively 50.948, 47.619 and 48.918. Thus IQR has the best average CRPS rank of these three while the single member has the best aggregate probabilistic error. An average rank describes relative placement across configurations; it is not an integer overall position. Equal rounded scores also do not imply identical per-configuration behavior. The practical choice depends on which error and uncertainty properties matter.
A filtered comparison is not the unrestricted leaderboard
The displayed reference comparison selects eligible systems with replication code and no declared test-data leakage, excluding fine-tuned entries and known non-permissive licenses or member dependencies. It retains ranks from the full leaderboard snapshot rather than recalculating them after filtering. Better-scoring excluded entries exist. The figure therefore supports a bounded comparison, not a claim that Granite leads the whole benchmark or holds an unqualified global podium position. Adding leaderboard entries can also change average ranks.
AISubHub analysis: evaluate Prob, IQR and a strong individual member on identical historical forecast windows that use only information available at each forecast origin. Match the needed horizon and target series, then measure both point error and quantile loss. Compare the empirical coverage of the 0.1–0.9 interval with its nominal 80%. Only after that comparison should a team decide whether a modest aggregate gain justifies operating four models rather than one.
Deployment and license boundaries remain part of the decision
The current recipe forecasts channels independently and does not model information across channels. Loading four checkpoints increases inference time and memory relative to a single member. Context limits, supported horizons, preprocessing and missing-value handling vary by member. Neither aggregation rule guarantees calibration or robustness after a distribution shift. Monitoring should therefore continue after deployment; an apparently confident interval is not a substitute for checking realized errors and coverage.
IBM describes the recipe and its default Granite members as permissively licensed. Expanded recipes can add checkpoints with non-commercial or research-use restrictions, so that statement cannot become a blanket license for every combination. Compatibility, licensing, latency and memory need checking together. We read this article and its necessary architecture and rank figures; we did not execute code or independently inspect the linked research, model cards or notebooks. Our interpretation concerns a testable starting point for forecasting, not a verified production service, price or consumer entitlement.
Why it matters
AISubHub analysis: a common forecast interface can reduce the initial choice among architectures. Whether using four models is worthwhile depends on local accuracy, interval coverage, runtime and memory together.
Limits and availability
The evidence is an IBM article and its diagrams, with an October 20 leaderboard snapshot. We did not run the recipe, reproduce results or independently read the linked papers, notebooks and model cards. No universal calibration, deployment cost or license for expanded recipes is established.
Sources and evidence
Source published: · Source checked:
AI-assisted editorial articles checked against linked primary sources. Vendor claims are attributed; inclusion is not endorsement.