BelugaMultiplexer

Asking a chromatin model which bases in a regulatory element matter means mutating every one of them and rerunning the network. For Beluga and a 2,000 bp window that is 6,001 forward passes — one reference sequence plus three alternative bases at each of 2,000 positions — through a 150-million-parameter convolutional network, for a single locus. The scan, not the model, becomes the cost.

BelugaMultiplexer removes the loop instead of optimizing it. Given the unmutated reference sequence alone, it emits a tensor of shape 2,002 × 4 × 2,000: the predicted chromatin effect of every base at every position, for every profile Beluga predicts. It gets there by distillation. Beluga is run offline over a sampled sequence and all 6,000 of its single-nucleotide variants, the log-odds difference between each alternative and the reference becomes a training target, and a separate network is fit to reproduce that whole table from the reference sequence alone. The teacher's weights are never touched; what is learned is the mapping from a sequence to its own mutational response surface.

Dennis Tang, Chenlai Shi and Jian Zhou built it in the Lyda Hill Department of Bioinformatics at UT Southwestern Medical Center and presented it at the 2023 ICML Workshop on Computational Biology. Zhou first-authored the DeepSEA line Beluga belongs to, so student and teacher come from the same group. "Multiplexer" is the general recipe — any base sequence model can be distilled this way, and the repository ships a notebook for doing it — while the released, pretrained artifact is the Beluga student.

#Key Features

• One forward pass for a whole mutagenesis scan: One reference sequence in, the full saturation mutagenesis table out, so cost no longer scales with the number of variants.
• Base-pair-resolution output through a U-Net: Skip connections sum the outputs of blocks three and two into blocks six and seven, reintroducing detail that dilated convolutions spanning the whole window would otherwise blur away.
• Explicit positional encoding: A fifth input channel carrying min(0.001p, 2 − 0.001p) gives early convolutions the absolute position they otherwise lack — which matters because the base model labels only the central 200 bp of its window.
• Distilled from model outputs, not experimental labels: Targets are generated on demand by the frozen base model, so no new chromatin assay data is needed and every target is exact.
• Command line tool and hosted server: CLI.py predict takes a chromosome and a position and loads the pretrained checkpoint by default, plot renders the 4 × 2,000 heatmap, and a web server runs the model with no local install.

#Technical Details

Seven convolutional blocks, each a pair of Conv1d–BatchNorm1d–ReLU stages at kernel width 8, widen 5 input channels to 640 and then 1,280, with dilations of 1, 4, 16, 64, 16 and 4 across blocks one to six; a final 1 × 1 block projects 1,280 channels to 8,008 and reshapes to 2,002 × 4 × 2,000. That is 202,273,376 parameters. Training sequences were sampled genome-wide from hg19 with chromosomes 8 and 9 held out, minimizing mean squared error against the log-odds targets with Adam at a learning rate of 1e-4 and batches of 16 regenerated each epoch, stopping on a fixed 288-sequence validation set after 50,000 epochs. It computes ISM 94× faster than the base model and 13× faster than Yuzu's compressed-sensing ISM; a position-specific variant predicting only the average effect per position reaches 142× and 20×. On allele-imbalanced variants from ATAC-seq, DNase-seq and H3K27ac/H3K4me1 QTL studies — disjoint from anything either model saw — the student matches the teacher at calling which allele opens chromatin.

#Applications

The output is a per-base map of predicted regulatory consequence, which makes two expensive jobs cheap. The first is interpretation: a heatmap over a 2 kb window recovers the motif a variant disrupts, as with the GATA element carrying a δ-thalassaemia mutation at chr11:5255790 and the CTCF site created by a familial skewed X-inactivation mutation at chrX:73072592. The second is throughput — prioritizing GWAS or eQTL variants, or scanning candidate enhancers, at a scale where 6,001 passes per locus is prohibitive.

#Impact

The result worth carrying beyond this model is that the mutational response surface of a sequence network is itself learnable: a 202M-parameter student reproduces its teacher's saturation scan closely enough that external accuracy is unaffected, even though the correlation between the two is imperfect — a gap the authors read as distillation's usual regularization. Practical limits are inherited wholesale from the base model: hg19 coordinates, a fixed 2,000 bp input, a fixed 2,002-profile head, and a --diff mode that needs Beluga's own weights locally. The repository has been quiet since August 2023 and carries no license file, though the Zenodo checkpoint bundle is CC BY 4.0; the paper remains a non-archival workshop submission.