GPN-Star

GPN-Star (Genomic Pretrained Network with Species Tree and Alignment Representation) is a genomic language model developed by Yun S. Song's group at UC Berkeley, with collaborators at The Jackson Laboratory and DKFZ, and posted to bioRxiv in September 2025. It predicts the functional constraint on every position and allele in a genome by reading whole-genome alignments together with the species tree relating them.

The model is the direct successor to GPN-MSA from the same lab and fixes three limitations of that design. GPN-MSA masked only the human sequence; GPN-Star masks nucleotides across many species in the alignment, increasing both the volume and the diversity of training signal. GPN-MSA's generic sequence transformer treated aligned species as unordered rows, which forced the authors to drop most primates because closely related genomes degraded performance; GPN-Star encodes phylogenetic distances explicitly, so arbitrarily close species can be included. And where GPN-MSA was tuned to one alignment, GPN-Star accepts alignments of any composition with minimal tuning — the repository now lists GPN-MSA as deprecated in its favor.

Three human models were trained, one per evolutionary timescale: GPN-Star (V) on a 100-vertebrate MULTIZ alignment, GPN-Star (M) on a 447-mammal Cactus alignment, and GPN-Star (P) on the 243-primate subset. Five further models cover mouse, chicken, Drosophila melanogaster, Caenorhabditis elegans, and Arabidopsis thaliana.

#Key Features

• Phylogeny-informed cross-attention: Each encoder block pairs sequence-wise self-attention over the target genome with a cross-attention module that weights contributions from other species using evolutionary distances read off the species tree, rather than treating alignment rows as interchangeable.
• Clade-based alignment pooling: Species are partitioned into clades by single-linkage clustering on pairwise phylogenetic distance, and each clade is condensed into one representative embedding per alignment column, keeping cost largely independent of how many closely related genomes are supplied.
• Timescale-matched models: Separate vertebrate, mammal, and primate models expose a consistent pattern — coding and rare large-effect variants are best predicted at deeper timescales, non-coding and common small-effect variants at shallower ones.
• Mutation-rate calibration: A post-hoc procedure removes the correlation between raw model scores and context-dependent mutation rate estimates, isolating selection from mutational bias and improving most downstream benchmarks.
• Open weights and genome-wide scores: All checkpoints are released on HuggingFace under the MIT license, alongside precomputed calibrated log-likelihood-ratio and entropy scores and a multi-assembly UCSC Genome Browser track hub.

#Technical Details

GPN-Star is an encoder-only transformer trained with a masked language modeling objective on windows of whole-genome alignment. Inputs split into target sequences, whose masked nucleotides supply the training loss, and source sequences from all aligned species, which supply evolutionary context together with their tree distances. The largest human checkpoints have 200M parameters (16 layers, 1024 hidden dimensions, 16 heads) and were trained for several days on 8 NVIDIA A100 GPUs; 85M and 25M variants are also released. Context is 128 bp for the vertebrate model and 256 bp for the mammal and primate models; training uses the top 5% most conserved windows plus a random 0.1% of the rest, with chromosome 21 for validation and chromosome 22 held out.

On human benchmarks, GPN-Star (V) leads all genome-wide predictors on ClinVar missense pathogenicity and on COSMIC somatic variants, matching ESM-1b on the former, and outperforms Nucleotide Transformer, Evo 2, and GPN-MSA throughout. GPN-Star (M) leads on non-coding pathogenic variants from OMIM and HGMD and on UK Biobank fine-mapped coding and non-coding variants, ahead of the sequence-to-function models Enformer, Borzoi, and AlphaGenome. In stratified LD score regression meta-analyzed over 106 complex traits, GPN-Star (P) exceeds the heritability enrichment of primate PhastCons, the previous best-performing conservation annotation. Added to DeepRVAT's annotation set, GPN-Star scores raised rare-variant association discoveries on UK Biobank exomes from 383 to 402 genes on average.

#Applications

The precomputed scores support variant prioritization in rare disease diagnostics, functionally informed GWAS fine-mapping, heritability partitioning, and rare-variant burden testing, and are browsable as UCSC tracks by groups without GPU access. Because the framework needs only a multispecies alignment, it also gives model-organism communities a genome-wide constraint score where curated pathogenicity resources are scarce — validated here against population allele frequencies in all five species and against experimentally validated lethal mutations in fly and worm.

#Impact

GPN-Star argues that explicit evolutionary structure is a better use of model capacity than scale: 200M parameters trained on 8 GPUs outperform billion-parameter alignment-free models, and beat PhyloP and PhastCons at each of the three timescales tested. The heritability results also strengthen the case that primate-specific constraint carries distinctive information about human complex traits. The work is a preprint and has not yet been peer reviewed. Limitations are real: inference requires a whole-genome alignment, so unaligned or lineage-specific sequence is out of reach; context windows stay short because alignment blocks fragment at longer ranges; and the models capture between-species constraint rather than selection within human populations.