TabPFN-Wide

TabPFN-Wide is a tabular foundation model adapted for the high-dimensional, low-sample-size regime that characterizes much of biomedical data. Omics experiments routinely produce tables with only tens or hundreds of samples but many thousands of measured features — gene expression, methylation, or proteomic readouts — a shape that defeats most classical machine-learning pipelines and requires aggressive feature selection. TabPFN-Wide targets this setting directly, letting practitioners fit predictive models on wide omics matrices without discarding most of the measured signal.

The model builds on TabPFN, a prior-data fitted network that performs supervised learning by in-context inference: rather than training a new model for each dataset, a single pretrained transformer conditions on the labeled examples provided in its context and predicts labels for new rows in a single forward pass. The base TabPFNv2 is limited in the number of features it can ingest. TabPFN-Wide removes this ceiling through continued pre-training on synthetic datasets drawn from a customized prior tailored to extreme feature counts. It was developed by researchers at the University of Tübingen and released as a preprint in October 2025.

By preserving the in-context, zero-refit workflow of TabPFN while extending it to wide feature spaces, TabPFN-Wide makes foundation-model-style tabular prediction practical for the noisy, feature-rich datasets common in genomics and single-cell biology.

#Key Features

• Extreme feature scaling: Handles tables that scale beyond 50,000 features while matching or exceeding the base model's accuracy, removing the feature-count ceiling of TabPFNv2.
• Continued pre-training on a customized prior: Rather than architectural changes, the extension comes from further pretraining on synthetic data sampled from a prior designed for high-dimensional inputs.
• In-context, zero-refit inference: Predictions are produced in a single forward pass by conditioning on labeled examples, with no per-dataset gradient training required.
• Robustness to noise: The model exhibits improved robustness to noisy and uninformative features relative to its base, an important property for noisy omics measurements.
• Retained interpretability: Feature-importance signals remain accessible, supporting the interpretability expectations of biomedical analysis.

#Technical Details

TabPFN-Wide is a transformer-based prior-data fitted network derived from TabPFNv2. Its core mechanism is amortized in-context learning: the network is meta-trained across large collections of synthetic supervised tasks so that, at inference time, it approximates Bayesian prediction by attending over the in-context training rows. The contribution of this work is a continued pre-training stage on synthetic datasets generated from a customized prior emphasizing very high feature dimensionality, which lets the model ingest wide inputs directly. In evaluations on biomedical and synthetic tabular data, the model matches or exceeds the base TabPFN across feature regimes and scales past 50,000 features regardless of noise level, while maintaining accessible feature-importance information. The work is a preprint awaiting peer review.

#Applications

TabPFN-Wide is aimed at researchers building predictive models from wide, small-sample biomedical tables — for example classifying disease status or subtype from bulk or single-cell transcriptomes, methylation arrays, or proteomic panels without first collapsing thousands of features down to a handful. Its zero-refit inference makes it convenient as a strong baseline and rapid prototyping tool, and its retained interpretability suits settings where practitioners need to know which features drove a prediction.

#Impact

TabPFN-Wide extends the tabular-foundation-model paradigm — one pretrained network reused across datasets via in-context learning — into the high-dimensional omics regime where such models have historically underperformed dedicated pipelines. By showing that continued pre-training on a tailored synthetic prior can lift the feature ceiling without sacrificing accuracy, noise robustness, or interpretability, it broadens the practical reach of in-context tabular learning for biomedical discovery. As a recent preprint, its adoption and independent benchmarking are still emerging.