Protein function annotation that reshapes language model embeddings with a neural-collapse loss so rare EC, Pfam, and GO classes stay separable.
No providers recorded yet. Browse all providers
Assigning function to a protein sequence — an Enzyme Commission number, a CATH structural domain, a Pfam family, a Gene Ontology term — is a classification problem with a pathological label distribution. In the Swiss-Prot EC training data, fewer than 10% of the more than 5,000 unique EC numbers account for over 84% of annotated proteins, and the rest have fewer than 100 sequences each. Models trained on that distribution optimize almost entirely for the head of the curve. The functions researchers most want predicted, the understudied ones, sit in the tail where accuracy collapses.
MSRep, from Jiaqi Luo and Yunan Luo at the Georgia Institute of Technology, attacks the imbalance in the geometry of the embedding space rather than in the classifier. It builds on neural collapse, the empirical observation that well-trained networks on balanced data converge to a state where same-class embeddings collapse onto their class mean and those means arrange into a simplex equiangular tight frame — vectors of equal length, maximally and equally separated. On imbalanced data that structure does not emerge on its own; minority-class centers crowd into similar directions and the space is dominated by overrepresented classes.
MSRep induces the structure explicitly. It takes frozen ESM-1b embeddings and learns a small projection network that maps them into a space where a neural-collapse-inspired loss forces class centers apart at equal angles regardless of how many proteins carry each label. Every function class, common or rare, gets an equal share of the representation space.
ESM-1b produces a 1,280-dimensional sequence embedding, which the projection network maps into a space of dimension at least K−1 for K classes — a requirement for a simplex ETF to exist. Training data is Swiss-Prot filtered to sequences within ESM-1b's 1,022-residue limit, split temporally at 25 May 2022, with sequence-identity-controlled test sets (EC-S50, Gene3D-S50, Pfam-S50) plus the Price-145 enzyme benchmark. Task scale ranges from roughly 5,000 EC and Gene3D classes to 14,723 Pfam families and 3,000–17,000 terms per GO sub-ontology.
On EC-S50, MSRep exceeds CLEAN, Protein-Vec, ProteInfer, and DeepEC on precision, recall, and F1, and the margin widens on classes with fewer than 30 training proteins. Binned by class frequency it improves F1 in every bin, holding F1 above 0.4 even for classes with fewer than 10 associated proteins. On Gene3D it beats Protein-Vec and BLAST, with the largest gains below 100 occurrences and parity at the head. On Pfam it leads ProtCNN, ProtENN, and the Vec baselines across all frequency bins, and its advantage grows as sequence identity to the training data is tightened below 30%. GO evaluation follows CAFA conventions with Fmax and AUPR. The repository ships five checkpoints per task under different random seeds, combined by majority vote.
MSRep is aimed at newly sequenced or poorly characterized proteins whose closest functional relatives are themselves sparsely annotated: enzymes from metagenomic assemblies, orphan proteins in non-model organisms, and biocatalyst candidates where the target chemistry has few known examples. Because inference is nearest-neighbour retrieval against a user-supplied lookup set, groups can point it at their own annotated collections without retraining, and the GO pipeline includes CAFA-style hierarchy backpropagation.
MSRep imports a result from the deep learning theory literature — that class-balanced training converges to a specific, describable geometry — and turns it into a practical remedy for the long tail that pervades biological annotation databases. Its framing generalises beyond protein function to any imbalanced biological classification task built on a frozen foundation-model backbone. The work is a preprint that has not been peer reviewed; the code is MIT-licensed but checkpoints, training data, and precomputed embeddings are distributed through Dropbox links rather than a durable archive, and the required embedding dimension scales with the class count, raising the cost of very large label spaces.
Much of this page is generated or calculated automatically. Flag anything that looks off and we will re-run it.