GeneLLM (cfRNA)

Cell-free RNA circulating in plasma reflects gene expression across tissues, which makes it an attractive substrate for non-invasive cancer screening. The conventional pipeline aligns sequencing reads to a reference genome, counts reads per annotated gene, runs differential expression, and trains a classifier on the resulting matrix. Every step is gated by annotation: reads from repetitive regions, transcribed ultra-conserved regions, and other unannotated stretches of the genome's "dark matter" are discarded before the classifier sees them, even though such sequences have shown biomarker potential.

GeneLLM removes the annotation dependency by treating the raw reads themselves as the language. Developed by a consortium led by Peking University First Hospital with the Institute of Artificial Intelligence at Beihang University and OxTium Technology, it is a transformer pretrained on unlabelled cfRNA reads tokenised into 7-mers, then applied with fixed weights to mine every read a patient contributes. The model learns a vocabulary of recurring sequence patterns it calls pseudo-biomarkers, and a patient is described by how their reads distribute across that vocabulary rather than across a gene list.

First posted to bioRxiv in July 2024 and revised in March 2025, the work was published in Nature Communications in July 2026 under a CC BY-NC-ND 4.0 license. No source code, weights, or inference code have been released; the preprint stated these would follow publication, and none have appeared since. The raw sequencing data, including the shallow-depth libraries, is deposited in the NCBI Sequence Read Archive.

#Key Features

• Annotation-free classification from raw reads: Trimmed reads go directly into the model, so no reference alignment, gene counting, or differential expression step stands between the assay and the prediction.
• Pseudo-biomarker vocabulary: After pretraining, read embeddings are aligned to a learned set of Gaussian-parameterised prototype vectors; the fraction of a patient's reads matching each prototype becomes the classification feature.
• Discovery in unannotated regions: Because prototypes are learned from sequence rather than from a gene model, many of the most informative pseudo-biomarkers fall outside annotated genes, and the top-scoring ones are cancer-type specific.
• Robust to shallow sequencing: Retraining and testing the whole pipeline on 1 GB libraries — a fraction of the standard 6 GB depth — preserved predictive accuracy, which matters for the cost of a population screening assay.
• Multi-centre validation: Samples were collected at three independent hospitals, with non-cancer controls drawn from all three to calibrate batch effects.

#Technical Details

The pretrained core is a six-block Transformer decoder trained autoregressively: each read is a sentence, each 7-mer a token, with [SOS]/[EOS] markers, padding to a common batch length, and triangular masking so each token attends only to its predecessors. Pretraining ran for roughly 15 days on 64 NVIDIA A100 GPUs at a batch size of 1 per GPU. Weights are then frozen and each of a patient's approximately 40 million reads is encoded to its final output vector; those vectors are aligned to trainable pseudo-biomarker prototypes, and the resulting proportion vector feeds a stacked skip-connection feed-forward network that outputs the disease call.

The clinical cohort comprised 496 plasma samples: 102 colorectal and 102 stomach cancers from Peking University First Hospital, 81 liver cancers from Sir Run Run Shaw Hospital, 47 lung cancers from Beijing Hospital, and 164 non-cancer controls, split 5:1:4 into training, validation, and test sets. On the held-out test set, AUC exceeded 0.9 for every individual cancer type and for pan-cancer detection, with 83.0% average accuracy in assigning the correct cancer type. Retrained on two public cfRNA datasets, the model reached pan-cancer AUC 0.96 against the 0.91 reported by the original study, and pancreatic ductal adenocarcinoma AUC 0.99 against 0.95.

#Applications

The target application is early multi-cancer detection from a single blood draw, at a per-sample cost low enough for screening rather than confirmation. Removing the bioinformatics pipeline also removes its computational overhead and its annotation assumptions, which is attractive for cohorts where reference annotation is incomplete. Beyond diagnosis, the pseudo-biomarkers themselves are outputs: sequences that discriminate cancers yet map to unannotated regions are candidates for follow-up as transcripts, therapeutic targets, or probes into what the extracellular transcriptome carries between cells. The authors describe the same framework extending to intracellular bulk RNA-seq and single-cell RNA-seq.

#Impact

GeneLLM argues that annotation is a lossy bottleneck in liquid biopsy, and that a sequence model reading raw reads recovers signal gene-level quantification throws away — a claim supported by improved AUCs on two public datasets reanalysed with the same pipeline. Its evidence base is bounded: 496 samples, four cancer types, and no independent reproduction. Peer review in Nature Communications strengthens the result but does not substitute for reanalysis, and without released code or weights the results cannot yet be verified or built upon; the CC BY-NC-ND terms on both the preprint and the version of record also restrict redistribution of derivative material. Should the checkpoint appear, the annotation-free formulation is the part most likely to generalise, since nothing in it is specific to cfRNA.