Masked protein language model with an INT8 GPU inference engine, matching 600M-class encoders on zero-shot fitness while embedding about 2x faster.
No providers recorded yet. Browse all providers
Protein language models now score variant libraries and embed metagenomes at the scale of billions of sequences, so training and serving costs matter as much as accuracy. Recent 600M-class encoders were trained on four to six trillion tokens, and their reference implementations leave much of a GPU idle, so one embedding pass over a large sequence database can occupy a multi-GPU server for weeks. HyPLM-Flash aims to reach the accuracy of those encoders with far less training data and to serve it with an inference stack written around the hardware.
HyPLM-Flash was released in November 2026 by the Hyra team at Tencent Hunyuan, a research group within Tencent, and described in the HyPLM Technical Report. The report states that the model was trained by Hyra 2.0, the team's autonomous research agent; Hyra is the system that produced the model, not part of the model itself. HyPLM-Flash is a 652-million-parameter bidirectional Transformer encoder trained with masked language modeling. It keeps the 33-token alphabet and RoBERTa-style output head of ESM-2, so code written against ESM-2 650M can use it with minimal changes, but it modernizes the Transformer block and the optimizer.
The authors evaluate it against 28 public checkpoints from 13 model families on 32 tasks under one protocol, as a peer of ESM-2 650M, ESM C 600M and E1 600M. It is the first member of a planned HyPLM family, with a larger HyPLM-Pro in training.
The encoder has 32 layers of width 1,280, 20 attention heads and a SwiGLU feed-forward width of 3,584, with RMSNorm pre-normalization, rotary position embeddings and per-head query–key normalization, which keeps attention logits bounded at the high learning rates used with the Muon optimizer. The training corpus combines UniRef90, the UniRef50 representatives absent from it, MGnify90, active UniParc entries and JGI IMG isolate proteomes: 2.6 billion deduplicated sequences (659 billion residues), with exact-match removal of benchmark sequences including all ProteinGym wild types and variants. Pretraining covered 713 billion tokens on crops of up to 1,022 residues; a 13-billion-token second stage extended crops to 8,190 residues and added structure objectives on 1.6 million experimental and predicted chains, filtered by MMseqs2 against the test proteins. Total compute is about 2.8 × 10²¹ FLOPs by the 6ND estimate.
On the 217 substitution assays of ProteinGym v1.3, masked-marginal scoring gives a Spearman correlation of 0.409, statistically tied with ESM-2 650M, ESM C 600M, DPLM-650M and E1-600M, and 0.406 on organismal fitness, the highest among self-supervised single-sequence models on the leaderboard. Linear probes on frozen representations give a long-range contact precision at L/5 of 0.650 on ProteinNet, the best of all 29 models evaluated including ESM-2 15B and ESM C 6B, and 0.882 three-state secondary-structure accuracy on CB513. Masked-residue negative log-likelihood on PDB chains released after the training cutoff is 1.148, below ESM-2 650M, ESM C 600M and E1-600M. INT8 inference stays within 0.01 of these BF16 results; the authors recommend BF16 mode for regression on near-identical variants.
HyPLM-Flash suits workloads where an ESM-2-class encoder is already used and throughput is the constraint: zero-shot ranking of mutagenesis libraries before wet-lab testing, embedding proteomes or metagenomic catalogs for clustering and annotation, and supplying frozen features to predictors of stability, localization, enzyme class or Gene Ontology terms. The inference engine requires a CUDA GPU, with Hopper-generation cards recommended, and the model reads sequence alone, without structure or retrieved homologs.
HyPLM-Flash shows that the accuracy of current 600M-class protein encoders does not require multi-trillion-token training budgets, and that inference engineering can roughly halve serving cost at that scale. Its matched evaluation of 29 checkpoints with paired confidence intervals is useful in its own right, since it shows many headline differences between similar encoders are not statistically significant. Weights and inference code are released under the MIT License. The work is a technical report that has not been peer reviewed, its benchmarks are computational, and it is notable as a protein model trained by an AI research agent.
Much of this page is generated or calculated automatically. Flag anything that looks off and we will re-run it.