Masked protein language model pretrained on 1.56 billion UniRef and metagenomic sequences, released under MIT as an open ESM Cambrian-style backbone.
No providers recorded yet. Browse all providers
The protein language models that set the pace in 2024 and 2025 were not equally usable. ESM Cambrian showed that a modernized transformer trained on a metagenome-heavy corpus could match much larger predecessors on structural signal, but its larger checkpoints came with licensing terms that ruled out many commercial uses. AtlasLM-3B is an independent reimplementation of that recipe, trained from scratch and released with its weights under the MIT License, so that the same class of representation can sit inside tools that are redistributed or used commercially.
AtlasLM-3B is the language-model member of the Atlas family, developed by Seonghwan Seo and Woo Youn Kim's group at KAIST with HITS Inc. as part of Korea's K-Fold initiative, and described in a September 2026 preprint. It reads a protein sequence and returns residue-level hidden states and pairwise attention maps; nothing in it is trained on structures. That frozen encoder is what the two folding models in the same release build on: AtlasFold for monomers and AtlasFold-Multimer for complexes. Both project its attention maps into their pair representations rather than running an alignment search.
AtlasLM was developed before the full ESMC release, so its data preparation and training schedule follow ESM3 instead. Where ESMC drew on a two-billion-cluster JGI metagenome collection that proved impractical to reconstruct, AtlasLM substitutes the smaller public MetaClust set.
pip install atlasfold and loaded through an
embed_sequences call that returns embeddings, per-layer hidden states and, on request,
attention maps.AtlasLM-3B has 3.06 billion parameters across 48 layers with a hidden width of 2,304 and 36 attention heads. Training sequences are selected as representatives of 90%-identity clusters and sampled through 70%-identity clusters: 188 million UniRef sequences (83 million clusters), 727 million MGnify (375 million) and 644 million MetaClust (400 million). The first stage runs 1.5 million AdamW updates at context length 512 and global batch size 8,192, sampling the three sources at 36/32/32%; the second runs 0.5 million updates at context 2,048 and batch 4,096, shifting the mix to 62.5/25/12.5% and decaying the learning rate from 4×10⁻⁴ to 4×10⁻⁵. Contact precision was measured with the attention-probe protocol of Rao et al. on partition 4 of the ESM structural superfamily split (2,370 scored proteins, ten independent probes). Long-range P@L rose from 0.555 after 0.42 trillion processed tokens to 0.715 after 10.5 trillion. AtlasLM-3B exceeds ESM-2 at 3B and at 15B parameters and the 300M and 600M ESMC models on P@L and P@L/5, while ESMC-6B remains higher on both.
AtlasLM-3B is a general sequence encoder: its embeddings can feed property predictors, variant-effect models, retrieval over metagenomic protein sets, or new structure models, and its attention maps give contact estimates without building a multiple sequence alignment. Its most direct use is as a swap-in backbone for groups that want a folding or design model whose language component carries no downstream licensing restriction. Its pseudo-perplexity is also a useful difficulty signal: in the Atlas evaluation, several of the largest gaps between AtlasFold and alignment-based AlphaFold 2 fell on high pseudo-perplexity sequences, most clearly on CASP14.
AtlasLM-3B's significance is its release terms more than its ranking. The final weights, the model code and an intermediate checkpoint from the end of the first training stage are published under MIT, which gives the field an openly licensed protein language model at the 3B scale with a documented training trajectory. The pretraining pipeline and the processed pretraining corpus are not part of the release, so the model can be used and fine-tuned but not retrained end to end from the published materials. The authors are candid that it does not reach ESMC-6B's contact precision, a gap they attribute to dataset scale and training settings. A 600M-parameter sibling was also released and has since been deprecated by its authors in favor of ESMC-600M. The work is a bioRxiv preprint and has not been peer reviewed.
Much of this page is generated or calculated automatically. Flag anything that looks off and we will re-run it.