Plant protein language model family: ESM-2 checkpoints with continued masked-language pretraining on 19.9M Viridiplantae sequences from TrEMBL.
No providers recorded yet. Browse all providers
PlantPLM is a family of protein language models built by taking Meta's general-purpose ESM-2 checkpoints and continuing their masked-language-model pretraining on plant proteins only. It was released on Hugging Face in April 2026 by Dipayan Sarkar, a PhD research scholar in the Computational Systems Biology Lab in the Department of Bioinformatics at the University of North Bengal, India. There is no accompanying paper; the Hugging Face model cards and the public training repository are the primary documentation.
The problem it targets is one of representation. ESM-2 learned protein grammar from UniRef50, where green-plant (Viridiplantae) proteins make up about 4.64% of member sequences, far behind bacteria at roughly 63%. Plant proteins also have fewer close homologs in that corpus, so features that matter for plant biology, such as chloroplast transit peptides, cell-wall biosynthesis enzymes and plant secondary-metabolism families, are diluted in the embedding space. Rather than training a plant model from scratch, PlantPLM applies domain-adaptive continued pretraining: every weight of an existing ESM-2 checkpoint is updated on plant sequences at a deliberately low learning rate, shifting the model toward plant statistics while retaining what it learned from the full protein universe.
The name differs by one letter from PlantPTM, an unrelated plant post-translational-modification predictor from a different group. PlantPLM is a general-purpose embedding model, not a task-specific classifier.
EsmForMaskedLM class in Hugging Face Transformers.Three checkpoints are published: PlantPLM-8M (6 layers, hidden size 320, 7.5M parameters), PlantPLM-35M (12 layers, hidden size 480, 33.5M parameters) and PlantPLM-150M (30 layers, hidden size 640, 148M parameters), initialized from the matching ESM-2 checkpoints. The model cards describe a 650M variant as part of the family, and the training repository contains configurations for it, but no 650M checkpoint has been released. All variants use full-parameter masked-language-model training with 15% masking and the AdamW optimizer. The 8M and 35M models train on the raw 19.9M-sequence corpus (about 6.76 billion amino-acid tokens) for token budgets of 800M and 546M tokens, respectively, which is well under one pass. The 150M model instead completes one epoch over the deduplicated corpus, about 1.11 billion tokens in 90,000 optimizer steps. Validation perplexity on a 5% holdout is 9.92 for the 8M model, 7.96 for the 35M model and 8.98 for the 150M model; the last is measured on a different, deduplicated distribution.
Downstream evaluation uses frozen [CLS] embeddings and a logistic-regression linear probe on reviewed Arabidopsis thaliana proteins from UniProt SwissProt. For the 35M model, 9-class subcellular localization accuracy rises from 91.87% for vanilla ESM-2 to 94.28%, and macro-AUROC over the 50 most frequent GO terms rises from 94.26% to 94.82%, on test sets of 332 and 396 proteins. The 8M model shows similar gains: 91.6% to 93.7% on localization and 94.7% to 95.0% on GO terms. The 150M checkpoint has no downstream evaluation.
PlantPLM is aimed at plant biologists and crop scientists who need sequence representations for proteins from species that general models cover poorly. Typical uses are embedding plant proteomes for clustering, retrieval and similarity search; linear probes or fine-tuning for GO-term annotation, subcellular localization and signal-peptide detection; and serving as a starting point for small labeled plant datasets where a better-matched prior matters most. The model cards scope it to Viridiplantae: non-plant proteins should use the original ESM-2 weights, and structure prediction belongs to tools such as ESMFold.
PlantPLM is a small, openly released test of a question that recurs across biological language models: whether a general model retrained on a neglected clade outperforms the general model on that clade. The early answer is a consistent but modest improvement, a few points of localization accuracy and under one point of GO-term AUROC, on single-species benchmarks of a few hundred proteins, with no peer-reviewed study behind it yet. Its practical value is accessibility: MIT-licensed weights that load as ESM-2 checkpoints, the full adaptation code, and the filtered 19.9M-sequence plant corpus as an HDF5 file are all public, so others can extend the evaluation to more species and tasks or reuse the recipe for other underrepresented lineages. The deduplicated 4.37M-sequence set the 150M variant actually trained on is not released, and the MMseqs2 clustering step that produced it is described on the model card rather than scripted in the repository, so reproducing that variant means re-running the clustering.
Much of this page is generated or calculated automatically. Flag anything that looks off and we will re-run it.