Scorpio

Scorpio is a contrastive learning framework for nucleotide sequences from Gail Rosen's group at Drexel University, first posted to bioRxiv in July 2024 and published in Communications Biology in 2025. It addresses what makes DNA harder to model than protein: evolutionarily related nucleotide sequences diverge far more than their translations, k-mer and codon usage shift between genomes, and the selective constraints on coding DNA are poorly characterized. Embeddings drawn straight from a genomic language model therefore tend to cluster by nuisance factors rather than by the gene or taxon a biologist cares about.

The framework reshapes those embeddings with a triplet network trained under contrastive loss on a hierarchy of labels. Rather than fitting one classifier per taxonomic rank and another for gene identity, Scorpio learns a single 256-dimensional space in which gene function and taxonomy are both organized, then answers queries by nearest-neighbour retrieval against a FAISS index. Because classification is retrieval rather than a fixed softmax over training classes, extending the reference set is an indexing operation, not a retraining run.

That design is what lets Scorpio reach sequences it has never seen. The authors built explicit held-out splits — a Genes-Out set of novel genes absent from related genera, and a Taxa-Out set of familiar genes drawn from excluded phyla — and measured transfer to both, a stronger test than a random held-out fold.

#Key Features

• One embedding space for gene and taxonomy: A single training run optimizes a loss spanning all hierarchical gene-taxonomy levels, so one model answers both gene-family and phylum-to-family questions instead of needing separate classifiers.
• Retrieval-based inference with FAISS: Precomputed embeddings are stored in a FAISS index; a query is embedded once, its nearest neighbours are retrieved, and their metadata supplies the hierarchical prediction.
• Calibrated confidence scores: A small perceptron trained on query-to-neighbour distances and class probabilities converts raw distances into a confidence value between 0 and 1 at each hierarchical level, so low-support calls can be flagged rather than silently reported.
• Three interchangeable encoders: Scorpio-6Freq encodes 6-mer frequency vectors, Scorpio-BigEmbed uses a frozen BigBird backbone, and Scorpio-BigDynamic fine-tunes BigBird's embedding layer — a spread that trades in-distribution accuracy against out-of-domain generalization.
• Transfer without fine-tuning: The gene-taxa model was applied unchanged to antimicrobial resistance prediction on MEGARes and CARD, showing that the learned space carries functional information beyond its training labels.

#Technical Details

The gene-taxa training corpus comprises 800,318 full-length coding sequences curated with the Woltka pipeline from 1,929 bacterial and archaeal genomes — one species representative per genus, drawn from 7.2 million CDS and filtered to the 497 named genes with more than 1,000 instances. Sequences are encoded either as 4,096-dimensional 6-mer frequency vectors or as 768-dimensional BigBird embeddings from MetaBERTa, a BigBird variant the same group pretrained on gene sequences. Anchor, positive, and negative examples pass through a shared stack of linear layers projecting 768 to 512 to 256 dimensions, and the final 256-dimensional branch output is the released embedding.

On the memorization test set, Scorpio-BigDynamic reaches 98.8% gene-level accuracy and 89.0% at phylum. The generalization results matter more: on the Taxa-Out set, where genes are familiar but phyla are excluded, Scorpio-BigDynamic reaches 95.5% accuracy and 94.7% F1-macro on gene labels; on the Genes-Out set, Scorpio-6Freq holds 73.8% phylum accuracy where MMseqs2 falls to 4.3% and Kraken2 to 1.1%. Applied to antimicrobial resistance without retraining, Scorpio-BigEmbed scores 94.2% on gene family, 98.8% on resistance mechanism, and 96.9% on drug class, against 87.97% average for MMseqs2 and 34.22% for Abricate. Fine-tuned for Escherichia coli sigma70 promoter detection, it improves accuracy and MCC by more than 18% over the raw BigBird backbone.

#Applications

Scorpio suits metagenomic and pangenomic workflows built on assembled coding sequences, where the analyst needs both "what gene is this" and "where does it come from" and where a meaningful share of contigs have no close reference. Microbiome researchers can annotate genes and assign taxonomy in one pass with per-level confidence, surveillance groups can screen assemblies for resistance determinants without curating a resistance database, and the embeddings support clustering, visualization, and fine-tuning for tasks such as promoter detection. Cross-attention maps over resistance genes give a route to inspecting which sequence regions drive a call.

#Impact

Scorpio's contribution is less a new backbone than a demonstration that contrastive reshaping of an existing genomic language model, combined with retrieval, buys generalization that neither alignment tools nor the raw language model provides alone. Alignment methods remain the stronger choice when a close reference exists — MMseqs2 leads on the memorization test — so the framework is best read as complementary rather than a replacement. The released artifacts are split across hosts: the Scorpio-BigDynamic and Scorpio-BigEmbed checkpoints are archived on Zenodo, while the MetaBERTa backbone and the gene-taxa training data sit on Hugging Face.