Phenformer

Genome-wide association studies relate individual single nucleotide polymorphisms to disease, and polygenic risk scores aggregate a few hundred of them into a per-person estimate. Both share two weaknesses: they see variants stripped of their sequence context, and they inherit the ancestry composition of the cohort they were derived from, which is overwhelmingly European. A typical genome differs from the reference at roughly 20 million positions, so a score built on hundreds of independent SNPs uses a small slice of the available signal.

Phenformer attacks this by predicting disease risk end-to-end from raw sequence. It follows the direction of biological information flow — sequence, then cell context, then expression, then phenotype — so that the intermediate representations are interpretable as mechanistic hypotheses rather than opaque features. Given an individual's genome, it produces both a per-disease risk prediction and a ranking of the cell types, tissues, and gene sequence windows that drive it. The work is led by GSK, with collaborators at the Max Planck Institute for Intelligent Systems and ELLIS Institute Tübingen, Harvard Medical School, ETH Zurich, and the University of Oxford, and was posted as a preprint in January 2025.

The sequence context it handles is the headline number. Where Nucleotide Transformer works at 6 kb, Evo at 131 kb, and HyenaDNA up to 1 Mb, Phenformer integrates almost 88 Mb — around 3% of a personal genome, an order of magnitude beyond the largest prior genetic language model. It achieves this by not modelling the genome uniformly: gene-centred windows run in parallel through a frozen pretrained backbone, and the model reasons over the resulting embeddings.

#Key Features

• Multi-scale sequence-to-phenotype architecture: One model spans DNA sequence, cell-type-specific expression, and organism-level disease risk, so attributions can be traced back through every intermediate stage.
• Mechanistic hypotheses from sequence alone: The model ranks disease-associated cell and tissue types without requiring single-cell RNA sequencing or any other experimental assay alongside the genome.
• Ancestry transportability: Because predictions rest on sequence context rather than linkage-disequilibrium structure learned in one population, gains over conventional polygenic scores are largest in non-European ancestry groups.
• Molecular subtyping: Latent embeddings of individuals cluster into subgroups with significantly different comorbidity rates, separating patients by the molecular processes underlying their diagnosis rather than by the diagnosis alone.

#Technical Details

Phenformer takes 512 windows of 196 kb of DNA, each centred on a transcription start site, covering almost 88 Mb in total. Each window passes through a frozen Enformer backbone, pretrained to predict gene expression and chromatin accessibility across many cell types, producing a 3,072-dimensional embedding per window. Shared projection layers and Fourier position encodings feed these into transformer encoder layers, which aggregate across windows using Pooling by Multihead Attention before a prediction head emits per-disease risk. A separate model is trained for each disease.

Training used whole-genome sequencing from 150,076 UK Biobank participants and about 12,500 GPU-hours, covering six conditions: psoriasis, type 1 diabetes, type 2 diabetes, diabetic retinopathy, chronic obstructive pulmonary disease, and hypothyroidism. On identifying disease-associated cell types, Phenformer beats five published methods that additionally require single-cell data, by 0.08 to 0.47 average F1. Ensembling its predictions with Lassosum, LDpred2, PRS-CSx, p-value thresholding, and clumping-and-thresholding improves AUROC significantly over those methods alone in 86.7% of disease–method combinations in mixed-ancestry test individuals and 96.7% in non-European ancestry individuals, with average gains of up to 4.2% and 11.19% AUROC respectively. Restricted to the same 3% of the genome Phenformer sees, it outperforms those baselines directly by up to 5.49% and 14.59% average AUROC. The authors note that this 3% subset is biased toward regions near transcription start sites, a consequence of the sequence-to-expression backbone, so causal variants outside those windows are invisible to the model.

#Applications

The immediate uses are genetic risk stratification and target discovery. Risk predictions that hold up across ancestries address a well-documented failure mode of polygenic scores in clinical deployment, and the cell-type attributions give a starting point for wet-lab follow-up — the model independently surfaced liver involvement in psoriasis and small-intestine signal in type 1 diabetes, both clinically observed but mechanistically unexplained. The subtyping capability is relevant to trial enrichment, where splitting a heterogeneous diagnosis into molecular subgroups can sharpen endpoints.

#Impact

Phenformer demonstrates that end-to-end sequence-to-phenotype modelling at multi-megabase scale is computationally feasible and that it pays off in both interpretability and predictive accuracy. Its attributions are explicitly framed as hypotheses rather than causal claims, and the 196 kb windows frequently overlap several genes, so a highlighted window does not cleanly implicate a single one. The practical constraint on reuse is availability: no code, weights, or hosted inference have been released, and the training data is itself access controlled. Phenformer therefore stands as an architecture and a result rather than a tool other groups can run, and the work remains a preprint awaiting peer review.