RefineNovo

De novo peptide sequencing reads amino acid sequences straight out of tandem mass spectra, without matching against a reference database. It is how proteomics reaches the parts of a sample no database covers: antibodies from an immunized animal, peptides from a non-model organism, neoantigens, or metaproteomes. Non-autoregressive transformers have become the strongest architecture for the task, predicting every position of the peptide at once with bidirectional attention instead of decoding left to right, which makes them far faster than autoregressive decoders.

That speed comes with a training problem. Non-autoregressive peptide models are trained with connectionist temporal classification loss, which marginalizes over all alignments between spectrum peaks and output tokens. CTC is notoriously brittle: on unfavourable data distributions it produces loss explosions, severe overfitting, and runs that simply fail to converge. RefineNovo, from Fudan University with collaborators at the University of British Columbia, Shanghai AI Laboratory, and Zhejiang University, addresses this with a curriculum: the model estimates its own current capability on each peptide and is fed easy sequences first, ramping to hard ones as it improves. Reported training failures drop by more than 90%.

A second contribution operates at inference. A self-refining module recycles the model's own token embeddings through the decoder several times, letting each pass correct the previous one. Together the two changes put RefineNovo ahead of all published methods on both versions of the nine-species benchmark, extending the same line of work that produced catalog entries such as OmniNovo and PLMNovo.

#Key Features

• CTC curriculum learning: Training difficulty is scheduled per peptide from a sampled estimate of the model's generative capability, moving from simple to complex sequences and cutting training failures by over 90% across data distributions.
• Iterative self-refinement: At inference the whole predicted sequence is re-encoded through learned non-autoregressive token embeddings and re-decoded, recycling full sequences rather than partial tokens so the downstream mass-control step sees a consistent distribution.
• Non-autoregressive decoding: All residue positions are predicted in a single forward pass with unmasked self-attention, avoiding the sequential bottleneck of autoregressive peptide decoders.
• Precise mass control: A dynamic-programming decoding unit constrains the output peptide to match the precursor mass, and can be disabled when measured masses are unreliable.
• Released checkpoint: The RefineNovo-30M model trained on MassIVE-KB is distributed through the group's shared de novo sequencing repository under an MIT license.

#Technical Details

Peaks, precursors, peptides, and amino acids are all embedded into a shared 400-dimensional space. The backbone is a 9-layer transformer with 8 attention heads per layer and a feed-forward dimension of 1,024, trained for 30 epochs with AdamW at a peak learning rate of 4e-4 under cosine decay and a spectrum batch size of 1,600. Training data is MassIVE-KB, a human proteomics collection whose high-quality subset holds more than 30 million peptide-spectrum matches. On the original nine-species benchmark, RefineNovo averages 0.800 amino acid precision and 0.653 peptide recall across the nine held-out species, against 0.788 and 0.638 for the previous non-autoregressive state of the art, PrimeNovo, and 0.739 and 0.539 for the autoregressive Casanovo V2. On the stricter nine-species-V2 revision it reaches 0.907 average amino acid precision — the best on all nine species — and 0.790 peptide recall, leading on eight of nine. Ablations attribute 0.645 peptide recall to curriculum learning alone, with refinement and difficulty annealing supplying the remainder.

#Applications

The model suits any proteomics workflow where a search database is incomplete or absent: antibody and nanobody sequencing, immunopeptidomics and neoantigen discovery, environmental and clinical metaproteomics, and characterization of proteins from organisms with no annotated genome. Non-autoregressive decoding makes it fast enough for large spectral libraries, and the released checkpoint runs inference on unlabeled MGF files directly.

#Impact

The paper was accepted at ICML 2025, and the training-stability result is arguably as useful as the accuracy gain: CTC-based non-autoregressive models had been difficult to train reproducibly, and a curriculum that removes most failures lowers the barrier to building on them. The framing is deliberately general — the authors present peptide sequencing as a case study in curriculum learning for biological sequence prediction. Practical constraints are real: the implementation requires NVIDIA GPUs because the mass-control decoder is written against CUDA cores directly, so it will not run on Apple silicon or non-NVIDIA hardware, and the released checkpoint is trained on human proteomic data, which shapes what it generalizes to.