OmicsFM

Between a transcript and the protein that does the work sit translation, folding, post-translational modification, trafficking, assembly and degradation. Single-cell foundation models such as scGPT, Geneformer and scPRINT learn from mRNA counts and inherit that gap. Proteomics measures the executing molecule directly, but public mass-spectrometry data is deposited as raw spectra with inconsistent metadata — not as the model-ready abundance matrices ARCHS4 and the CELLxGENE Census supply for RNA. The corpus did not exist, so neither did the model.

OmicsFM, from the CompOmics group at the VIB-UGent Center for Medical Biotechnology, builds the corpus first. The group reprocessed 78,601 spectral files from 1,397 public PRIDE projects through matched DDA and DIA pipelines, reconstructed run-level metadata with an agentic extraction pipeline over publications and repository records, and filtered to 48,837 quality-passing abundance profiles from 1,143 projects. A pre-norm transformer is then pretrained on it by masked abundance reconstruction: rank the detected features by abundance, bin them, hide a subset, predict the hidden bins from the rest.

The design is deliberately matched rather than tuned: identical architecture, parameter count and training configuration, fitted independently to proteomics, to 680,216 bulk transcriptomes from ARCHS4 and to 4,550,106 cells from the CELLxGENE Census, so modality differences cannot be attributed to modelling differences. On that footing the proteomics model rivals both transcriptomic counterparts, on 14- to 93-fold fewer profiles by the authors' count — a corpus-size comparison they note is confounded with modality.

#Key Features

• Attention as an association network: the transformer is the only place information moves between proteins, so ranking each feature's partners by attention weight yields a zero-shot association network that recovers curated relationships across CORUM, BioPlex, HuRI, STRING, Reactome, KEGG, GO and OmniPath.
• Tissue-conditioned relationships: attention is not a fixed relational map. Across 30 human tissues, tissue-specialized pathways show significantly elevated within-pathway attention precisely where they are active — synaptic in brain, secretory in pancreas, cytoskeletal in muscle.
• Dropout-robust sample embeddings: a prepended Sample Summary Token compresses a profile into one vector. Excess self-correlation stays above 0.9 when 80% of detected proteins are removed, where PCA and log-abundance profiles decay toward zero.
• Matched cross-modality checkpoints: seven released checkpoints span proteomics, bulk transcriptomics and single-cell transcriptomics, each with learned or ESMC-derived feature identities, sharing a 20,272-accession UniProt vocabulary so a feature index means the same protein across modalities.

#Technical Details

Each feature is one token summing a feature-identity embedding and a rank-based abundance-bin embedding, following the binning and dual reconstruction objective of scGPT with the sequence-grounded identity embeddings of UCE. A token-type mask lets every token attend only to the Summary Token and the observed set. The transformer backbone is 4.94M parameters and identical across all checkpoints; the feature-identity table is counted separately because it scales with vocabulary rather than capacity — 5.19M trained for the learned variants, 23.35M frozen for the ESM-C ones. Corpora are split 90/5/5 by project, so no held-out study is seen in pretraining.

On project-held-out test sets, OmicsFM attention achieved the highest enrichment across all nine reference databases in all three modalities, above co-expression and network-inference methods fitted directly on the benchmark data (FAVA, Pearson, GENIE3, DeepSEM) and above pretrained single-cell models, scGPT second. Proteomic networks reached roughly 40-fold mean CORUM complex enrichment against about 13-fold for transcriptomic ones, while OmniPath transcription-factor-target enrichment stayed higher in transcriptomics. Frozen feature-identity embeddings predicted DepMap 24Q4 Chronos essentiality across 17,692 genes at r = 0.706 (proteomics) and 0.713 (bulk) against 0.674 for ESM-C alone, and r = 0.784 with all three modalities combined. Fine-tuned models beat an identically configured from-scratch model, GEARS and a linear baseline on perturbation-specific response across the Replogle K562, Adamson and Norman CRISPR screens.

#Applications

The attention networks work as a hypothesis engine: the strongest partners of a poorly characterized protein propose candidate functions by association, and the 30-tissue networks are deposited for direct reuse. Sample embeddings support neighbor search, out-of-distribution screening and comparison of profiles with very different protein-detection depth — the usual obstacle to pooling heterogeneous MS studies.

#Impact

OmicsFM establishes that self-supervised abundance reconstruction transfers beyond transcriptomics, and the reprocessing behind it converts a decade of PRIDE deposits into a reusable resource; the matched transcriptomic checkpoints give the field a controlled reference point alongside models like BulkFormer. On cell-type classification, L1-regularized logistic regression on raw protein abundances matched the fine-tuned model, so pretraining helps against an untrained encoder but not against conventional classifiers on marker-driven tasks. Every model tested underestimated perturbation amplitude. Sample embeddings retain study-specific structure and should not be read as batch-integrated. The corpus excludes isoforms, post-translational modifications and protein groups, and its metadata were reconstructed rather than curated. The work is a preprint awaiting peer review, and the reprocessed proteomics corpus is the one artifact not yet public; the authors state it will be released on publication.