PhosF3C

Phosphorylation is the most studied post-translational modification in eukaryotic biology, and mapping which serine, threonine and tyrosine residues actually get phosphorylated is a standing problem in proteomics. Experimental phosphoproteomics is expensive and incomplete, so sequence-based predictors fill the gap — but most of them either work from hand-engineered composition descriptors or extract frozen embeddings from a protein language model and hand them to a conventional classifier, which leaves the language model's representation unaligned with the task.

PhosF3C is an applied adaptation of ESM-2 that closes that gap in two places. First, rather than freezing the encoder, it applies Low-Rank Adaptation to the 650-million-parameter ESM-2 checkpoint, training a small adapter so the representation shifts toward phosphosite discrimination without the memory cost of a full fine-tune. Second, it replaces the usual single-branch classifier head with a conformer — a convolutional branch for local sequence motifs alongside a transformer branch for long-range context — and couples the two with a Feature Coupling Unit that exchanges information between them at every block rather than only concatenating at the end.

The work comes from Yuhuan Liu, Tianchi Lu and colleagues at City University of Hong Kong, Lanzhou University, the Institute of Automation, Chinese Academy of Sciences and Xi'an Jiaotong University. It was posted to bioRxiv in December 2024 and published in Briefings in Bioinformatics in 2025.

#Key Features

• LoRA-adapted backbone: Low-rank adapters are trained on the ESM-2 encoder and merged back into the checkpoint, which is then used as a feature extractor — task alignment without retraining 650 million weights.
• Conformer head: A CNN branch and a transformer branch predict in parallel, so local motif structure and global sequence context are modeled separately rather than being forced through one inductive bias.
• Feature Coupling Unit: FCU aligns the dimensions of the two branches and passes features down and up between them at each stage, progressively closing the semantic gap instead of fusing once at the output.
• Linear Regression Tomography: A top-down probe that fits linear directions in the representation space to physicochemical properties — hydrophobicity, surface charge, isoelectric point — measuring what the encoder actually encodes.
• Cross-dataset evaluation: Training uses only the general phosphosite set; the PhosAF and DeepIPS benchmarks are held out entirely and scored without refitting.

#Technical Details

The backbone is esm2_t33_650M_UR50D. The general training set fuses phosphosite annotations assembled by MusiteDeep and DeepPSP, drawing on UniProt/SwissProt, PhosphoSitePlus and Phospho.ELM; 1,500 full-length sequences, about 10% of the corpus, are isolated as an independent test set, and all models are compared at a fixed window size of 31 residues around the candidate site. On the general test set PhosF3C reports AUCs of 79.52%, 76.3% and 71.36% at S, T and Y sites, with Matthews correlation coefficients of 0.3449, 0.2591 and 0.2263. Ablations separate the two contributions: the base conformer alone reaches AUCs of 77.26%, 73.81% and 69.01%; adding LoRA lifts these to 78.76%, 76.06% and 70.11%; adding the FCU on top produces the final figures. Linear Regression Tomography was run across 65 window sizes, with larger windows yielding property directions that agree more closely with values computed by Biopython.

#Applications

The model is a screening tool for phosphoproteomics: given a protein sequence, it ranks candidate S, T and Y residues so that mass-spectrometry follow-up or mutagenesis can be targeted at the most likely sites. Because it is trained on general rather than kinase-specific data and evaluated on two independent benchmarks without refitting, it is usable on proteins with no prior phosphosite annotation. The interpretability probe is useful separately from prediction — it gives a way to ask which chemical properties a protein language model's embeddings carry, and at what sequence-window scale, which generalizes to other residue-level tasks built on the same backbones.

#Impact

PhosF3C is a careful piece of engineering on top of an existing foundation model rather than a new pretrained backbone, and the entry is best read that way: the contribution is the LoRA-plus-conformer recipe and the tomography probe, not new pretraining. The code is on GitHub under an MIT license with prediction and training entry points, though the LoRA and conformer weights are distributed through a Google Drive folder rather than a versioned release, and no hosted web server accompanies the release. Gains over the conformer baseline are consistent but modest in absolute terms, and tyrosine sites — the scarcest class in the training data — remain the weakest, with an MCC of 0.2263.