CLSS

Maps of the protein universe have historically been hierarchical: expert-curated databases such as ECOD, CATH, and SCOP sort structural domains into nested tiers of class, architecture, and homologous group. Those tiers encode decades of human insight, but they force every domain into one branch and cannot express the partial, sub-domain-sized similarities that link folds across lineages. CLSS (Contrastive Learning Sequence-Structure) instead learns a single continuous space in which a domain's sequence and its structure land in the same place, so that distance between any two points is a joint statement about sequence and structural similarity.

CLSS was developed by Guy Yanai and Rachel Kolodny at the University of Haifa with Gabriel Axel and Nir Ben-Tal at Tel Aviv University and Liam Longo at the Earth-Life Science Institute of the Institute of Science Tokyo, and posted to bioRxiv in September 2025. The design is borrowed from CLIP: two encoder towers, one per modality, trained with a contrastive objective that pulls matched pairs together and pushes mismatched pairs apart. Where CLIP paired images with captions, CLSS pairs a full domain structure with a randomly chosen segment of its own sequence.

That segment-level pairing is the substantive departure from existing protein language models. Because training contrasts short sub-sequences against whole structures, CLSS embeddings carry information about which structural context a fragment fits — not just what an entire chain looks like.

#Key Features

• Overlapping sequence and structure maps: Embeddings of a domain's structure, full sequence, and a random 20-60 residue segment fall in nearly the same location, giving t-SNE maps that are almost identical across all three modalities.
• Sub-domain resolution: Training on segments of 10 residues or longer means short fragments embed near the domains that host them, a property ablations show is lost when training uses full sequences only.
• No supervision from classifications: ECOD and CATH labels were never used in training, so agreement between embedding distance and hierarchy level is emergent rather than fitted.
• Compact and cheap: Only ~36M trainable parameters and 32-dimensional output vectors, small enough to retrain on modest hardware and to store and search at scale.
• Interactive protein-space viewer: A public web viewer plots roughly 1.81 million ECOD-AF2 domains, searchable by domain identifier, evolutionary group, or name, with selections exportable as CSV.

#Technical Details

The structure tower is the frozen ESM3 1.4B encoder (1536-dimensional per-residue output); the sequence tower is initialized from the 35M-parameter ESM-2 model (480-dimensional) and updated during training. Each tower mean-pools over residues, applies a single trainable linear adapter to 32 dimensions, and L2-normalizes, leaving ~36M trainable parameters. Training used 1 million domains sampled from the ECOD classification of AlphaFold 2 predicted structures, split 950,000/50,000 train and validation, with batches of 1,440 domains sharded across 8 NVIDIA A100 40GB GPUs, a fixed softmax temperature of 0.5, and 80 epochs over roughly 4.5 days.

Evaluation used two reference sets: 31,696 ECOD domains from the 109 best-characterized X-groups spanning 16 architectures, and 9,899 CATH domains from Røgen's set of sequence-similar but topologically distinct pairs. Embedding distances separate cleanly by ECOD level — H-group, X-group, architecture, class — for structure, sequence, and sub-sequence inputs alike. On the same maps, ESM3, ProstT5, and ProTrek place each modality in distinct regions rather than overlapping, and their distance distributions discriminate hierarchy levels far less sharply, with sub-sequence embeddings largely uninformative.

#Applications

The joint space supports database search where a query can be a structure, a sequence, or a fragment, returning neighbors regardless of which modality is available. Evolutionary analysis is the paper's own demonstration: the maps place α+β four-layer sandwiches inside the Rossmannoid region and split β-helix H-groups between β and α/β territory, cases where structural inspection supports the embedding over the hierarchy. Colored by ligand contacts, the map also shows cofactor-binding domains occupying one broad region, concentrated among α/β and α+β folds. Characterizing which sequence motifs fit a given structural context is directly relevant to library-based protein design.

#Impact

CLSS argues that hierarchical classifications are a harder and more revealing benchmark for protein language models than supervised readouts such as EC number or GO term prediction, since they probe the global organization of the embedding space rather than family-level features. It also makes a case for smaller models: a 36M-parameter trainable head over a frozen structure encoder beats much larger models at reproducing known evolutionary relationships. The work remains a preprint awaiting peer review, and its evaluation is comparative rather than a standardized benchmark suite. Because training used ECOD domains, the natural input unit is a domain or segment rather than a full-length chain, and fragments extracted from α/β architectures sometimes embed far from their parent domain. Code, training scripts, and both checkpoints (CLSS-sub and CLSS-full) are released under Apache 2.0, with the model installable as the clss-model package.