IgBert-HIC

Herceptin is a fixed molecule, but injected as a control onto the same hydrophobic interaction chromatography (HIC) column across many runs, its retention time came back anywhere from 11.87 to 18.45 minutes — a spread of more than six minutes, standard deviation 1.27, on an antibody that never changed. A model trained on raw HIC retention times therefore fits the reagent lot, the instrument and the operator alongside the sequence. IgBert-HIC's central move is to take that out: every training label is a ratio, the candidate's retention time divided by the Herceptin retention time from the same batch.

The effect is decisive. Fine-tuned on raw retention times, the model does worse than predicting the mean, with an R² of -0.18. Fine-tuned on Herceptin-adjusted ratios under identical hyperparameters, it reaches a cross-validated R² of 0.63. The model is a fine-tune of IgBert, Exscientia's paired antibody language model, pretrained on more than two billion unpaired antibody sequences and roughly two million paired heavy–light chains from the Observed Antibody Space — the same backbone Igloo builds on. Yangtze River Pharmaceutical Group appended a fully connected regression head to the pooled representation, giving a single scalar output, and fine-tuned the 420M-parameter stack on 359 in-house antibodies with measured HIC retention times. Input is paired VH/VL sequence; no structure, homology model or descriptor set is involved.

What comes out is a Herceptin-referenced ratio under this lab's specific protocol — a 1.5 M ammonium sulfate gradient on a MAbPac HIC-10 column — not an absolute retention time in minutes. The authors state that transferring such a model between labs would require stable internal controls, captured assay metadata, and shared normalization standards.

#Key Features

• Internal-standard labels: Training targets are batch-matched RT ratios rather than absolute minutes, stripping out the instrument- and operator-dependent drift that otherwise dominates real-world chromatography.
• Sequence-only prediction: Paired heavy and light chain sequence is the entire input, sidestepping the structural descriptors competing HIC predictors depend on and that are rarely available early in discovery.
• A fixed released checkpoint: The fine-tuned weights are published on HuggingFace and loaded by an inference script that predicts without any retraining step.
• Calibrated as a screening band: In practice the scalar is read qualitatively — ratios roughly between 0.9 and 1.3 mark acceptable candidates, and the model is used to exclude the poor ones rather than to rank finely within the band.

#Technical Details

The backbone is 30 transformer layers at a hidden dimension of 1024 with 16 attention heads; the released checkpoint carries 419,932,161 float32 parameters as a BertForSequenceClassification with a single regression label. Fine-tuning ran up to 40 epochs at batch size 32, initial learning rate 1e-5 decayed by 0.2 every 10 epochs, early stopping at patience 3 and minimum improvement 0.01; convergence came after 8 epochs. Each epoch resplit the data 90/10, amounting to a repeated 10-fold cross-validation, which returned the R² of 0.63 — above the 0.49 reported for the ensemble-plus-structure method AbLEF. Correlation between predicted and measured ratios was 0.916 on the 359-antibody training cohort and 0.714 on an independently collected batch of 75 antibodies. The sequence-only baseline Protein-Sol correlated at 0.662 on the 135 clinical-stage antibodies it was developed against and 0.407 on 348 B cell-derived mAbs, but only 0.174 and 0.198 on this lab's internal retention times and ratios respectively; the internal cohort spans a far wider retention range, 10.63 to 25.01 minutes, than the public sets. Twenty clinical-stage antibodies were separately expressed as IgG1 in CHO cells and measured; predicted ratios ran from 0.976 to 1.246.

#Applications

The intended use is triage: scoring candidate libraries on hydrophobicity before committing material and column time, dropping molecules prone to non-specific binding and accelerated clearance. It slots into multi-parameter developability panels alongside thermostability, viscosity and immunogenicity readouts, and is relevant to antibody-drug conjugate work, where added hydrophobicity from the payload drives conjugate stability and pharmacokinetics. It serves teams running their own HIC assay with a consistent internal control, the setting the released checkpoint is calibrated to.

#Impact

The contribution is methodological rather than architectural: the same backbone and hyperparameters swing from an R² of -0.18 to 0.63 purely on how the labels were normalized — an argument that assay design deserves as much attention as model choice when fine-tuning on small biophysical datasets. The scope is correspondingly narrow — one scalar output, one chromatography protocol, and 359 labeled antibodies, with no evaluation against a HIC assay other than the authors' own. The training data is available only from the corresponding author on request, and the work is a preprint awaiting peer review. Licensing differs across artifacts: the bioRxiv posting permits no reuse, while the code and the released weights are both declared MIT, and MIT is what governs the model itself.