LA4SR

A freshly assembled microalgal genome arrives as a pile of translated open reading frames, and roughly two-thirds of them — 65.3% in the datasets used here — return no hit at all against NCBI's non-redundant protein database. Homology search therefore cannot say whether those "dark proteome" sequences are real algal genes or bacterial contamination carried through the culture. Algal genomes are the hardest case: endosymbiotic lineage merging and pervasive horizontal gene transfer leave genuinely algal genes looking bacterial, so alignment-based tools discard real biology as contamination.

LA4SR (pronounced "laser", for language modeling with artificial intelligence for algal amino acid sequence representation) reframes the question as next-token prediction. A causal language model reads an amino acid string and emits a tag naming its origin, so the decision rests on the sequence's internal statistics rather than on finding a database match — which is why it still works on sequences no database contains. It comes from the Laboratory of Algal, Synthetic, and Systems Biology at New York University Abu Dhabi, with collaborators at New York University and the Biotechnology Research Center of the Technology Innovation Institute.

Rather than a single checkpoint, LA4SR is a family, re-engineering a dozen open architectures — nanoGPT, GPT-NeoX, Pythia, BLOOM, DistilRoBERTa, ELECTRA, ByT5, Mistral 7B, and the S6 state-space model Mamba — from 70M to 12B parameters. That makes it a different animal from a masked protein language model such as ESM-2 or a generator like ProGen2: the backbones are general text models, and the biology enters through the corpus.

#Key Features

• Terminal-information-free training: A second corpus scrambles the start and stop regions of every sequence, forcing classification on internal patterns alone. Accuracy holds, which makes the approach usable on the fragmentary gene models that dominate draft assemblies.
• Classification without a database: Inference is generation, not alignment, so sequences with no homolog anywhere are scored like well-characterized ones — the dark proteome stops being a separate, unresolvable bucket.
• Two routes to a checkpoint: Small models such as algaGPT, the terminal-inclusive nanoGPT variant, are pre-trained from scratch on the algal corpus; models above 300M parameters are post-trained with QLoRA from published open weights, which is why they converge on a fraction of the data.
• Interpretability shipped alongside: HELIX runs layer-wise PCA on hidden states, DeepLift LA4SR computes windowed residue attributions, and DeepMotifMinerPro ranks influential motifs. In roughly 15% of algal sequences called bacterial-like, these surfaced ATP/GTP-binding motifs consistent with horizontal gene transfer.

#Technical Details

The training corpus is built from translated ORFeomes of 166 microalgal species spanning ten phyla, paired 1:1 with bacterial, archaeal, and fungal sequences drawn from NCBI nr, for roughly 77 million distinct sequences. It ships in two forms: 58,650,525 sequences (6.4 billion amino acid characters) terminal-information-free, and 17,880,279 (4.2 billion characters) terminal-information-inclusive. Candidate genomes were themselves screened with BLAST-based contaminant detection and kept only below a 10% threshold.

The best terminal-free model uses the Pythia 70m architecture and reaches 0.9031 precision, 0.8983 recall, and an F1 of 0.9006 on algal sequences, with near-identical numbers on bacterial ones — a compact model capturing most of the separating signal. A Mistral 7B model post-trained with QLoRA reached F1 near 88 after 2,000 steps, against roughly 20,000 for pre-training from scratch; Mamba adaptors from 130m to 2.8B reached F1 up to 88. Against ultra-sensitive Diamond BLASTP across 166 genomes, LA4SR averaged 82.9x faster with essentially complete recall versus 34.7%; the published benchmark reports queries processed approximately 10,701x faster than NCBI BLASTP+, comparing an NVIDIA A100 GPU against an AMD EPYC 7742 CPU. Validation used data outside the training set, including a new complete Hi-C/PacBio Chlamydomonas reinhardtii assembly and genomes from unseen genera published after training.

#Applications

The immediate use is quality control and annotation for non-model algal genomics: deciding which translated ORFs in a new assembly are the organism's own and which are contaminants, at a throughput that makes whole-proteome screening routine. The terminal-free regime extends this to metagenomic and draft-assembly ORFs whose ends are missing. The attribution tooling serves a second audience: researchers treating the motifs a model relies on as hypotheses about horizontal gene transfer and lineage-specific amino acid usage.

#Impact

LA4SR is an argument that general-purpose language model backbones, given a well-constructed biological corpus, transfer to sequence classification cheaply — the largest models needed less than 2% of the available data to pass F1 86, and a 70m model matched them. Its deployed task is narrow: a binary algal-versus-contaminant call, not the downstream repertoire of a general protein language model. The released code is a research snapshot carrying no license that grants reuse, its inference script requires a GPU and emits raw decoded text rather than calibrated probabilities, and the Hugging Face checkpoints are numerous and unevenly documented. Corpus, checkpoints, and interpretability tools are nonetheless public, making the terminal-free idea straightforward to test on other clades.