Phyla

Protein language models are routinely described as having learned evolution, on the reasoning that masked-token training over hundreds of millions of homologous sequences must internalise the process that produced them. Phyla comes out of a paper that tests this directly and finds it false in the way practitioners actually use these models. When frozen or fine-tuned embeddings are compared by distance — the standard recipe — models including ESM-2 and ProGen2 fail to recover phylogenetic structure, even though the same embeddings do well on masked-token and contact prediction.

Phyla is the constructive half of that argument: a model built to do the thing the others cannot. It is a hybrid state-space and transformer network from Marinka Zitnik's lab at Harvard Medical School, trained not with masked language modelling but with a tree-based objective over curated phylogenies, and designed from the start to process a set of related sequences jointly rather than one sequence at a time. Given a FASTA file it emits a tree, with no multiple sequence alignment and no guide tree anywhere in the pipeline.

The work was posted to bioRxiv in January 2025 and accepted as a NeurIPS 2025 poster. Its title changed across preprint versions as the framing shifted from "towards a foundation model for phylogenetic inference" to the negative result about standard usage; the model's name stayed Phyla throughout.

#Key Features

• Joint multi-sequence processing: The model ingests a whole set of sequences together, so representations are shaped by the relationships between sequences rather than assembled post hoc from independent embeddings.
• Quartet-topology objective: Training rewards embeddings whose induced distances preserve the correct four-taxon topology. Masked language modelling was tested and dropped after it degraded performance, leaving the tree loss as the sole pretraining signal.
• Alignment- and guide-tree-free: The two most expensive stages of a classical phylogenetics pipeline are absent, which is where the runtime advantage comes from.
• Interleaved recurrence and attention: Bidirectional Mamba layers mix information across sequences with linear-time recurrence, while a masked attention layer at the end of each block confines each token to its own sequence, preserving per-sequence context.
• Small enough to be cheap: At 24 million parameters it runs on one GPU and stays competitive with models orders of magnitude larger.

#Technical Details

Phyla stacks three inter-sequence blocks, each containing 16 BiMamba layers followed by a single sparsified attention layer, at a hidden dimension of 256 — 24 million parameters in total. Training used 3,321 high-quality multiple sequence alignments curated from OpenProteinSet, with pairwise distances derived by counting matching positions over alignment length and used to supervise the quartet loss. An earlier alpha release trained on 13,696 uncleaned trees; the released beta checkpoint uses the smaller, filtered set and a revised loss.

On tree reconstruction Phyla reaches a normalised Robinson–Foulds distance of 0.73 on TreeBASE and 0.58 on TreeFam, cutting normRF by 13.4% against the strongest protein language model baseline and beating models with 12 to 266 times more parameters. On taxonomic clustering over GTDB it improves species-level homogeneity by 23.2% over the best baseline, and it reaches a Spearman correlation of 0.64 on ProteinGym. Classical pipelines still win on raw accuracy — MAFFT with FastTree records 0.65 on TreeBASE and 0.32 on TreeFam — and Phyla closes roughly half of that gap while changing the cost profile entirely: the classical workflow needed about two hours on TreeBASE and 66 hours on TreeFam across multiple CPU nodes, where Phyla finished both in under an hour on a single H100. Reconstructing a 3,084-sequence tree of life took 16 hours against 3,840 for the traditional route, though at that scale the model was run on CPUs with roughly 1 TB of memory.

#Applications

The natural users are groups doing phylogenetics at a scale where alignment is the bottleneck: large-scale comparative genomics, microbial ecology, protein family curation, and outbreak genomics. Applied to real data, Phyla places sequences into their correct domains across the tree of life and resolves genome-scale relationships among Mycobacterium tuberculosis clinical isolates. The repository ships a pip-installable package that turns a FASTA file into a Newick tree, plus the three-task Tree Reasoning Benchmark and its datasets on Harvard Dataverse.

#Impact

Phyla's most durable contribution may be the negative result it is packaged with: a clear demonstration that evolutionary reasoning does not emerge from sequence-scale pretraining under the embedding-and-distance protocol most downstream work assumes, which is a caution for anyone reading phylogenetic meaning into protein language model representations. The positive claim — that explicit phylogenetic supervision buys that capability at a fraction of the parameters — is supported but early: the released checkpoint is described as a beta under active development, the repository carries no licence file, and classical alignment pipelines remain more accurate where their runtime is affordable.