Segment Transformer

Temperature stability decides whether an enzyme is usable in an industrial process, and measuring it is slow and expensive. The gap between what is sequenced and what is characterized is severe: UniProt holds more than 230 million enzyme sequences, while the BRENDA enzyme database contains fewer than 30,000 experimentally determined temperature stability records. Existing predictors narrow the problem to stay tractable: some are tuned to a single enzyme family, others require the source organism's optimal growth temperature as an input, which is unavailable for most sequences.

Segment Transformer, released in July 2025 by researchers at Jiangnan University with collaborators at Shanghai Jiao Tong University, Macquarie University, Nantong University, and two industrial biotechnology partners in Shandong, attacks the problem from the representation side. Its premise is that different regions of an enzyme contribute unequally to thermal behavior, so residue-level tokens are the wrong granularity. The model takes frozen embeddings from ESM-2 and converts them into segment-level features — tokens that each summarize a short contiguous stretch of sequence — before applying attention.

Alongside the model the authors release a curated benchmark: 27,216 raw BRENDA records reduced to 3,454 deduplicated sequence-temperature pairs, partitioned so that training and validation sets share little sequence identity. The study then takes its own output into the lab, using the model's segment importance scores to design a thermostability-improved cutinase variant.

#Key Features

• Segment-level tokens: Amino-acid embeddings are segmented into fixed-length blocks and convolved into single tokens, so attention operates over sequence regions rather than individual residues.
• Dual grouped segment attention: Each DGSA block reorganizes segments into groups and runs two attention passes — one within short-range groups, one across long-range groups — then sums them, capturing local motifs and global dependencies in one layer.
• Multi-scale prediction with a fluctuation range: Two resolution scales each produce a prediction; their mean is the point estimate and their spread reports a range, giving users an explicit confidence signal rather than a bare number.
• Segment importance scores: Attention-pooling weights are averaged across scales into an interpretable per-region profile, which is what drives mutation-site selection.
• Imbalance-aware training: A weighted RMSE loss upweights the sparse extremes of the temperature distribution, where only 175 samples fall at or below 19 °C and 76 at or above 100 °C.

#Technical Details

ESM-2 (the 150M-parameter, 30-layer checkpoint, kept frozen during training) encodes a sequence into an L × d feature map. Strided 1D convolutions produce a second, downsampled scale; each scale is reshaped into fixed-length segments and passed through 2D convolutions to yield segment tokens. Two DGSA blocks refine these, followed by attention pooling and per-scale regression heads. An architecture sweep over one to three scales and one to three DGSA blocks selected the two-scale, two-block configuration.

Training used 2,798 sequences, with 311 for validation and a fixed 345-sequence held-out test set; train/validation splits were built by MMseqs2 clustering within temperature bins. On the test set the model reaches RMSE 24.03, MAE 18.09, and Pearson and Spearman correlations of 0.33, compared against BiLSTM, CNN, Transformer, Light Attention, RNN, DeepET, Seq2Topt, and TemStaPro trained under the same loss. BiLSTM is marginally better on RMSE (23.96); Segment Transformer leads on MAE and correlation and, more importantly, is the most balanced across thermal regimes, with grouped MAE of 22.68 / 13.54 / 19.16 below 45 °C, 45–70 °C, and at or above 70 °C, where baselines that do well in one regime degrade sharply in another.

#Applications

The target user is an enzyme engineer choosing where to mutate. In the case study, a cutinase from Humicola insolens was profiled and 17 candidate mutations selected from the highest-scoring positions in its six most important segments. The A78E variant showed a 1.64-fold increase in relative activity after heat treatment at 60 °C and extended its half-life from 6.8 to 29.5 minutes, with no loss of catalytic activity; N79G reached 1.75-fold. Retrospective classification of published mutations in three other cutinases was correct for 12 of 19, 4 of 4, and 8 of 10 cases. Beneficiaries are industrial biotechnology groups working on detergents, food processing, textile finishing, and polyester degradation.

#Impact

The work is a preprint, not yet peer reviewed. Its most durable contribution may be the curated, cluster-split BRENDA benchmark, which gives the small enzyme-thermostability modeling community a shared evaluation set. The performance ceiling is real and the authors say so: correlations of 0.33 on a 345-sequence test set are modest, mutation-effect prediction is effectively zero-shot because no large annotated mutation-stability dataset exists, and several of the 17 designed cutinase mutations reduced activity because the model scores thermostability alone and never sees catalytic function. Segment-level features also blunt sensitivity to single-residue changes — the same abstraction that improves whole-sequence prediction. Code is MIT-licensed, with weights and source data on Zenodo.