ChatMultiOmics

General-purpose language models are fluent about biology and nearly useless at biological sequences. Ask GPT-4o whether a given promoter and enhancer interact, hand it the two sequences, and it performs at chance. The specialist alternative — a BERT-style encoder with a task-specific prediction head — works, but each new task needs its own head and its own fine-tuning run, and the model cannot explain itself or hold a conversation. ChatMultiOmics, developed by Shanghai AI Laboratory with collaborators at the University of Science and Technology of China, the University of Sydney, the University of Toronto, the Chinese University of Hong Kong, Shanghai Jiao Tong University, Fudan University, and the Shanghai Innovation Institute, tries to get both: one conversational model that reads DNA, RNA, protein, and multi-molecule sequences and answers questions about them in natural language.

The model is trained on Biology-Instructions, the instruction-tuning corpus introduced in the same paper. That corpus spans 21 tasks across four omics types — 6 DNA, 6 RNA, 5 protein, and 4 multi-molecule — drawn from GEO, Rfam, the PDB, UniProt, and published benchmarks, with over 3 million training samples. Seventeen tasks take a single molecule; four involve interactions between two sequences, such as enhancer-promoter interaction, RNA-protein interaction, and antibody-antigen neutralization.

The paper's most useful negative result is that instruction tuning alone does not work. Applying LoRA fine-tuning on Biology-Instructions directly to a chat model leaves classification and regression performance at roughly random. That finding motivates the three-stage pipeline that gives ChatMultiOmics its capability, and it distinguishes this line of work from sequence-encoder-plus-LLM hybrids such as ChatNT and from protein-only chat models like ProLLaMA.

#Key Features

• Three-stage training: Continued pretraining on unlabeled sequences, then large-scale instruction tuning, then reasoning instruction tuning — each stage measurably necessary in the ablation.
• Sequence-grounded, not exam-grounded: Every task takes an actual nucleotide or amino acid sequence as input, rather than testing recall of textbook biology.
• Cross-omics joint training: Training all four omics types together improves performance within individual omics domains rather than diluting it.
• Reasoning outputs: A final stage trained on chain-of-thought data teaches the model to analyze the sequence and explain the evidence before committing to an answer.
• Multi-molecule interaction tasks: Four tasks require reasoning over two sequences at once, a setting single-omics specialist models are not built for.

#Technical Details

The base model is Llama-3.1-8B-Instruct. Stage 1 continues pretraining on unlabeled human DNA from the Genome Reference Consortium assembly, human non-coding RNA from RNACentral, and protein sequences from UniRef50, using LoRA+ on all linear layers with a learning-rate multiplier of 4 and training the normalization layers alongside the adapters. Stage 2 runs instruction tuning on Biology-Instructions minus the reasoning subset, prepending a task label to 30% of examples to help the model distinguish tasks under a skewed task distribution. Stage 3 fine-tunes on 8,000 reasoning examples plus 3,000 non-reasoning validation samples to preserve classification and regression accuracy.

The ablation isolates each stage's contribution. On mouse transcription-binding-site detection, Matthews correlation moves from -1.42 for the untuned base model and 0.13 for instruction tuning alone to 27.94 after stages 1 and 2, and 32.21 after all three. Mean ribosome loading R² rises from about 0 to 29.12 and then 47.44; RNA-protein interaction MCC rises from 3.82 to 70.80 and then 74.26. Comparison baselines include Qwen2-7B, GLM4-9B-Chat, Galactica-1.3B, GPT-4o and GPT-4o-mini, and the biology-specialized InstructProtein-1.3B and BioMedGPT-LM-7B.

#Applications

The intended user is a biologist who wants a predictive answer and an explanation in the same response, without assembling a per-task pipeline. A single deployed checkpoint covers promoter and enhancer characterization, epigenetic mark prediction, APA isoform usage, siRNA efficiency, protein solubility and thermostability, and antibody-antigen neutralization, which suits exploratory work where the question changes faster than a fine-tuning run can keep up.

#Impact

The paper is framed around the dataset, with ChatMultiOmics presented as the baseline demonstrating what the dataset enables, and it is fair to read it that way — the strongest contribution is the corpus and the evidence that continued pretraining on raw sequences is the prerequisite for instruction tuning to take hold. It was published at EMNLP 2025 Findings. The practical limitation is availability: training and inference code are open in the MyTransformers repository, and the stage-3 data ships with the Biology-Instructions repository, but the trained checkpoint has not been released, and neither repository carries a license file. Stage-2 training data is distributed through a Google Drive link rather than a versioned host.