Shorkie

Shorkie is a semi-supervised sequence-to-expression model for budding yeast, developed at Calico Life Sciences with Johns Hopkins University and posted as a preprint in September 2025. It targets a gap in regulatory genomics: the deep learning models that map DNA sequence to expression have been built almost entirely for human and mouse, where hundreds of thousands of functional genomics tracks support supervised training. Saccharomyces cerevisiae is the workhorse of eukaryotic gene-regulation research but has a far smaller labeled corpus, and models trained on it from random initialization leave accuracy on the table.

The answer is a two-stage recipe. A masked DNA language model, Shorkie LM, is first pretrained on 165 Saccharomycetales genomes closely related to budding yeast, learning conserved regulatory grammar from evolution rather than from labels. That trunk is then fine-tuned on thousands of yeast transcriptomic and epigenomic coverage profiles — including 3,053 high-resolution transcriptional-regulator induction RNA-seq timepoints generated for this study on miniaturized chemostats — to yield Shorkie, which predicts RNA-seq coverage and scores sequence variants.

Architecturally, Shorkie is a compact member of the family running from Basenji through Enformer to Borzoi, all from the same group: a convolutional tower with subsampling, self-attention over the pooled representation, and upsampling back toward base resolution. Its self-supervised objective echoes the multi-species masked modeling of the Species-Aware DNA Language Model.

#Key Features

• Evolutionary-scale pretraining: The language model is trained on 165 fungal genomes with 15% masking, reverse-complement augmentation, and exonic and repetitive regions down-weighted 0.1× in the loss so that capacity goes to regulatory sequence.
• Purpose-built induction atlas: 3,053 new RNA-seq induction timepoints, extending the Induction Dynamics gene Expression Atlas, give the fine-tuning stage dense temporal coverage of what happens downstream when a regulator is switched on.
• Ablation-controlled pretraining gain: A from-scratch twin, Shorkie_Random_Init, shares the architecture exactly, isolating the contribution of self-supervision rather than of model capacity.
• Variant effect scoring: An eight-fold ensemble produces logSED scores for single nucleotide variants, benchmarked against three independent yeast cis-eQTL resources and a massively parallel reporter assay.
• Released weights and reproducible pipeline: All three checkpoints — LM, fine-tuned ensemble, and random-init ablation — are downloadable with published checksums, alongside Apache-2.0 code and per-figure reproduction notebooks.

#Technical Details

Shorkie LM has 13.7M parameters and reads 16,384 bp windows: a convolutional tower with subsampling, eight self-attention blocks operating at 128 bp resolution, and a U-Net decoder that upsamples seven times back to single-nucleotide predictions over the four bases. Fine-tuning drops the final four upsampling layers and attaches heads for 5,215 tracks — 3,053 induction time-course RNA-seq, 1,014 strain-panel RNA-seq, 1,128 ChIP-exo, and 20 ChIP-MNase histone profiles — giving coverage predictions in 896 bins of 16 bp. Pretraining corpora at four phylogenetic breadths (the R64 reference alone, 80 S. cerevisiae strains, 165 Saccharomycetales, and 1,341 fungi) were compared, and the 165-genome corpus performed best.

Against its random-initialized twin, Shorkie raises median bin-level Pearson correlation from 0.67 to 0.78 and gene-level correlation from 0.74 to 0.88, improving on 87.8% of genes. On the Random Promoter DREAM Challenge MPRA it reaches Pearson r of 0.70 on native yeast promoters and 0.74 on random sequences, r = 0.82 on motif perturbations, and AUROC and AUPRC above 0.95 separating high- from low-expression sequences; DREAM-RNN, trained directly on that assay, remains stronger on the MPRA itself. On endogenous regulation the ranking reverses: Shorkie beats the DREAM-CNN, DREAM-RNN, and DREAM-Attention models on cis-eQTL classification across TSS-distance bins, evaluated on 1,901 local eQTLs from roughly 1,000 sequenced isolates, 683 high-resolution mapped variants, and 142 MPRA-validated core-promoter variants.

#Applications

The model is a working tool for yeast functional genomics: prioritizing noncoding variants from population sequencing, interpreting cis-eQTLs whose causal SNP is ambiguous, and screening designed promoters in silico before synthesis. Because the fine-tuned heads cover induction time courses, attribution analyses recover canonical transcription factor binding motifs and track how their usage shifts after a regulator is switched on — a readout on regulatory dynamics that static expression models cannot give. A minimal command-line example scores a single SNP end-to-end on CPU, and released notebooks show how to fine-tune the language model on a user's own tracks.

#Impact

Shorkie's broader argument is methodological: for organisms where labeled functional genomics is scarce, pretraining on a tight clade of related genomes recovers much of what large supervised corpora buy in human genomics. The controlled ablation makes that claim unusually clean, since the only difference between the two models is whether training started from the language model's weights. The work remains a preprint awaiting peer review, and its scope is deliberately narrow — the model is specific to yeast regulatory sequence. The released code is Apache-2.0 while the weights on the public bucket carry their own terms, and the stack is pinned to TensorFlow 2.15 and Python 3.9.