ETAP-CLF

A protein language model returns one vector per residue, so a 200-residue protein and a 2,000-residue protein reach a classifier as matrices of different height. The standard remedy is mean or max pooling, which fixes the length but weights a catalytic site and a disordered linker identically. ETAP-CLF replaces that fixed operation with a learned one: a lightweight transformer re-reads the frozen residue embeddings in context, then a single trained query vector scores every valid position and takes an attention-weighted sum.

The framework was built by a group centred on the women's and children's hospital of Chongqing Medical University, with collaborators at the University of Toronto, the Lunenfeld-Tanenbaum Research Institute, Columbia University, the MD Anderson Cancer Center and the Georgia Institute of Technology. ESM-3 supplies the representation and is never updated; only the 2.5-million-parameter stack above it is trained. That stack — architecture and hyperparameters alike — was held fixed across ferroptosis-, senescence- and pyroptosis-associated protein prediction, with separate weights per task: the paper offers a reusable downstream design, not one multi-task checkpoint.

That framing puts ETAP-CLF alongside the same group's SenSeqNet, which pairs ESM-2 embeddings with a BiLSTM-CNN head for senescence alone — and which supplies the benchmark partition ETAP-CLF is measured on.

#Key Features

• Learned attention pooling: A single query vector assigns each residue a softmax weight, so the protein-level representation is an attention-weighted sum rather than an average. UMAP projections of the ferroptosis test sequences separate the classes more cleanly than mean- or max-pooling the same embeddings.
• Residue-level attention scores: Pooling weights are normalized to sum to one within each protein, giving a per-residue prioritization signal. The authors treat these as ranking scores, not causal explanations of residue function.
• One configuration, three tasks: The three classifiers share an identical four-layer encoder and hyperparameter set, differing only in learned weights. Pyroptosis-associated protein prediction had no prior dedicated sequence-based method.
• Gene-disjoint external evaluation: Alongside the random sequence split, ferroptosis performance is measured on 28,079 sequences from 29 genes excluded entirely from development, with hard negatives from related cell-death processes rather than housekeeping proteins.
• Installable command-line tool: The classifier ships as the MIT-declared etap-clf PyPI package — MIT appears in the packaging metadata, but the repository carries no license file — which embeds FASTA input with ESM-3, trains a head and scores new sequences, optionally plotting attention by amino acid, motif, gene and position. Note that it truncates to 1,536 residues by default, where the paper's dataset filter dropped proteins over 2,048 residues, so the shipped tool is not an exact reproduction of the paper's pipeline.

#Technical Details

Sequences are embedded by esm3_sm_open_v1, the open 1.4-billion-parameter ESM-3 checkpoint, giving a 1,536-dimensional vector per residue; proteins over 2,048 residues or carrying unsupported tokens are dropped. A linear projection with layer normalization maps each residue to 256 dimensions, sinusoidal positional encodings are added, and four pre-layer-normalized transformer encoder layers (eight heads, 512-dimensional feed-forward, dropout 0.1) contextualize the sequence under padding masks. Attention pooling, layer normalization, dropout and a linear layer produce two class logits — 2,503,682 trainable parameters in all. Training uses AdamW at 3 × 10⁻⁴, weight decay 1 × 10⁻⁴, batch size 64, at most 30 epochs and early stopping on validation AUROC. MMseqs2 clustering at 30% identity left 116,845 sequences for the ferroptosis task, 76,054 for senescence and 31,831 for pyroptosis. On the ferroptosis internal test set (n = 23,369) the model reaches AUROC 0.980, accuracy 0.931, sensitivity 0.946 and specificity 0.893, ahead of retrained PLM-FRP (0.956), FRP-XGBoost (0.912) and FeroConCap (0.794); on the gene-disjoint cohort every method degrades and ETAP-CLF leads at 0.801. Senescence reaches AUROC 0.949 against SenSeqNet's reported 0.940 on the same partition, and pyroptosis reaches 0.909.

#Applications

The practical use is candidate prioritization. A researcher holding a list of uncharacterized proteins — or a whole proteome — can score each sequence for association with a cell-death or cell-state programme and rank the output for follow-up, with no wet-lab assay for the process itself. Because the design is task-agnostic, the package can be pointed at any binary problem with positive and negative FASTA sets. Genes recovered consistently carried the expected biology: ferroptosis hits overlapped the WikiPathways ferroptosis set at 48.53-fold enrichment, and pyroptosis hits recovered an inflammasome–interleukin-1β axis.

#Impact

The contribution is methodological rather than a new checkpoint: learned aggregation over frozen ESM-3 embeddings matches or beats architectures built for one task, and the pyroptosis result supplies a first sequence-based baseline where none existed. The caveats are substantial, and the authors state most of them. Labels are propagated from genes to all their protein sequences, so isoforms and fragments inherit one annotation; 30% clustering reduces redundancy without giving family- or structure-disjoint evaluation; the residue attention scores were never experimentally validated; and results come from single training runs. No trained ETAP-CLF weights have been released — the PyPI package trains a head from user-supplied FASTA — and the processed datasets and full analysis code are promised on journal acceptance, so the reported numbers cannot yet be reproduced end to end. Part of the data is already public: the repository carries the ferroptosis representative sequences as FASTA (85,342 positives across 171 genes and 31,214 negatives) alongside a sequence metadata table, so the ferroptosis inputs — but not the senescence or pyroptosis ones, nor the training and evaluation code behind the reported numbers — are in hand today.