RNA-GPT

Hand a general-purpose language model a 70-character string of A, C, G and U and ask what it is, and it will answer — confidently — that the sequence is DNA. That exact failure appears in RNA-GPT's own case studies, and it names the gap the model was built to close: the answer is latent in the sequence, but a text-only model has no channel through which to read it.

RNA-GPT, from UCLA and Georgia Tech and presented at the NeurIPS 2024 Machine Learning for Structural Biology workshop, supplies that channel. A user-submitted RNA sequence is embedded by the frozen RNA-FM encoder, the embedding is pushed through a trained linear projection into the token space of a Llama-3 8B Instruct backbone, and the language model generates its answer conditioned on that projected representation alongside the question.

The design decision worth noticing is what stays fixed. RNA language models such as RNA-FM and RiNALMo already produce strong sequence representations, but expose them only through task-specific heads — promoter prediction, secondary structure, function classification. ChatNT wraps a conversational surface around genomic tasks while keeping the task taxonomy fixed. RNA-GPT leaves both the encoder and the LLM alone and trains only the bridge between them, targeting free-form dialogue about what is known of an RNA: its class, its host species, the pathways it participates in, the diseases it has been associated with. The recipe is the one protein-text models like ProtST established, carried over to RNA.

#Key Features

• Frozen encoder, trained bridge: the RNA-FM encoder block is frozen throughout; only the linear projection that maps RNA embeddings into the LLM's representation space is optimized, keeping the trainable surface small.
• Alignment before instruction: stage one trains the projection with no text prompt at all, using only the projected RNA representation against the full RNACentral annotation, so the bridge learns sequence-to-text grounding before any conversational format is imposed.
• Answers sized to the question: earlier protein chat models were instruction-tuned on whole annotations and returned verbose, loosely relevant text. RNA-GPT decomposes each annotation into targeted question-answer pairs, so the tuned model replies with a specific fact rather than a paraphrased abstract.
• RNA-QA, a 407,616-RNA instruction corpus: an automated pipeline pairs each RNA with an annotation synthesized from its published literature, plus 5 to 14 question-answer pairs.
• Divide-and-conquer literature summarization: rather than feeding an entire literature set to one summarizer and losing detail to the context limit, LDA topic modeling groups papers per RNA, each group is summarized separately, and the summaries are then merged.

#Technical Details

The sequence encoder is RNA-FM: 12 transformer layers pretrained self-supervised on 23 million RNAs from RNACentral, with a 1,024-nucleotide input limit that also bounds what RNA-GPT can accept. The backbone is Meta's Llama-3 8B Instruct. Stage one optimizes the projection layer for 10 epochs with AdamW, batch size 1, weight decay 0.05, 2,048 warm-up steps and cosine annealing from a 1e-4 learning rate; stage two instruction-tunes for a further 10 epochs with warm-up reduced to 200 steps. Both stages together ran in one week on two 40 GB A100s.

RNA-QA starts from the roughly 420,000 RNACentral entries indexed under "Lit Scan," filters to sequences of 1,024 nucleotides or fewer to leave 407,616, then scrapes and summarizes each entry's associated abstracts with GPT-4o-mini. The reported model was trained on a 5,000-RNA, 121,000-QA subset of it. On held-out RNAs it reaches BERTScore F1 0.8494 against 0.7424 for the raw sequence passed as text and 0.7403 for the modality-aligned model without instruction tuning, and ROUGE-L 0.4747 against 0.2037 and 0.2091 respectively; on the divide-and-conquer split, F1 0.8609 and ROUGE-L 0.4405.

#Applications

The target user is a researcher holding an uncharacterized or partially characterized transcript who wants an orientation before committing to experiments: what class of RNA is this, which organism does it come from, what regulatory role has been reported, which diseases has it been implicated in. Because the model is queried with a sequence rather than an identifier, it also applies to transcripts that a literature search cannot reach by name, and its conversational form lets a wet-lab biologist follow up in plain language rather than composing database queries.

#Impact

RNA-GPT is among the first attempts to give RNA the sequence-grounded chat interface that proteins already have, and its clearest contribution may be RNA-QA's construction pipeline — topic-grouped, divide-and-conquer literature summarization is reusable for any biological entity with a scattered publication record. The claims should be read at workshop-paper scale. Evaluation is text-similarity against literature-derived reference answers, not a factuality audit or experimental validation, and the reported metrics come from a model trained on roughly one percent of RNA-QA, with the full-corpus run still in progress at publication. No code, weights, or release of the RNA-QA dataset accompanies the paper, so the results cannot be independently reproduced and the model cannot currently be run.