LoRA-DR-suite

A large share of any proteome consists of segments that never adopt a stable fold yet remain fully functional. These intrinsically disordered regions (IDRs) drive signalling, regulation, and molecular recognition, and their misbehaviour is implicated in neurodegeneration and cancer. Predicting them from sequence is a mature field with its own biannual benchmark, the Critical Assessment of Intrinsic protein Disorder (CAID).

LoRA-DR-suite, from Sorbonne University's LCQB with Universidad Complutense de Madrid, makes two moves. The first is architectural: rather than build a bespoke network, it attaches low-rank adaptation layers to the attention modules of a frozen protein language model and puts a single linear classification head on the per-residue output. Training touches a small fraction of parameters, avoids catastrophic forgetting, and can be reverted by disabling the adapters.

The second move is conceptual. Alongside intrinsic disorder the suite predicts soft disorder — residues that are missing or have high B-factors in at least one of several crystal structures of the same sequence, but not in all of them. Soft disorder correlates with protein interaction sites and captures the disorder-to-order transitions that a binary IDR label discards. To support it the authors built SoftDis, a database covering the bulk of the crystallographic PDB, and released it alongside the models.

#Key Features

• Adapter-based training: LoRA layers are inserted into the query, key, and value projections of every multi-head attention block; the pretrained weights stay frozen and only the adapters and the linear head are trained.
• Four interchangeable backbones: ESM-2 35M, ESM-2 650M, ProtT5-XL-U50, and Ankh-large are each adapted for both tasks, letting users pick a point on the speed–accuracy curve.
• Soft disorder prediction: A second task head trained on SoftDis predicts flexibility across crystal ensembles, a signal that correlates with binding interfaces and is not captured by circular-dichroism-derived IDR labels.
• The SoftDis database: 229,376 PDB structures, 484,044 chains, clustered into 64,285 groups at 90% identity and coverage, with per-residue labels for soft disorder, missing residues, disorder-to-order transitions, and interfaces.
• Binding region transfer: On the CAID3 binding subset the adapted models beat dedicated binding predictors without ever being trained on binding annotations.

#Technical Details

Intrinsic disorder models are trained on DisProt 7.0 as preprocessed by flDPnn (445 training, 100 validation sequences), with the CAID3-era checkpoints additionally trained on CAID1 and CAID2 data. Sequences are truncated at 1,024 residues; training uses cross-entropy with balanced class weights, AdamW, and Optuna random search over 40 trials on a single NVIDIA A100.

ESM-2 650M-LoRA is the strongest configuration throughout. On CAID1-DisProt it reaches ROC AUC 0.833 and PR AUC 0.510, ahead of flDPnn (0.814), rawMSA (0.780), and SPOT-Disorder2 (0.760). On CAID2_NOX it is competitive with flDPnn2 and Dispredict3 at ROC AUC 0.837. On CAID3_NOX with full residue coverage it leads on every metric — ROC AUC 0.880, F1 0.649, MCC 0.524, PR AUC 0.721 — and on the reduced 53.8%-coverage subset reaches ROC AUC 0.896. For soft disorder on the SoftDis test set it records ROC AUC 0.839, F1 0.655, MCC 0.476, and PR AUC 0.729. Logistic probes confirm the adapters do the work: LoRA-adapted embeddings score ROC AUC 0.881 for intrinsic disorder against 0.832 for the unmodified ESM-2 650M representation.

#Applications

Disorder profiles guide experimentalists deciding which constructs to crystallise, which loops to truncate, and which regions are plausible interaction sites. Soft disorder predictions are the more distinctive product: because they track flexibility across an ensemble rather than a single structure, they highlight surface regions that rearrange on binding, which is directly useful for interface prediction and for interpreting the low-pLDDT regions of AlphaFold models. The authors show strong agreement between soft disorder scores and negative pLDDT across 2,009 chains, including case studies on MEFV, synuclein, and serum amyloid P-component.

#Impact

The work is a clean demonstration that parameter-efficient adaptation of a frozen protein language model matches or beats specialised disorder architectures at a fraction of the training cost, and that the adapted embeddings themselves become better disorder detectors than the base model. SoftDis is the more durable contribution — it makes a definition of disorder that had existed only as an analysis in two earlier papers available as a labelled, machine-learning-ready resource. The paper appeared in the ISMB/ECCB 2025 proceedings issue of Bioinformatics under CC BY. Models and dataset are distributed through HuggingFace with deployment scripts; the training and evaluation code lives on a Sorbonne GitLab instance rather than a public mirror, and neither the model repositories nor the dataset repository states explicit reuse terms.