DeepFM-Crispr

A Cas9 guide mostly succeeds or fails on sequence: the protospacer either matches the genomic locus or it does not. A Cas13d guide has a harder problem, because its target is RNA, and RNA folds. A guide can be perfectly complementary to its site and still knock nothing down if that site is buried in a stable hairpin. Guide-design tools for RNA-targeting CRISPR therefore have to reason about a structure nobody has measured for most transcripts.

DeepFM-Crispr answers that with borrowed structural knowledge rather than measured structure. Developed by Condy Bao at St. Mark's School and Fuxiao Liu at the University of Maryland, College Park, and posted as a preprint in September 2024 before appearing at the NLP4Science workshop that November, it takes frozen embeddings from RNA-FM — a 12-layer bidirectional transformer encoder pretrained on non-coding RNA — and uses them twice: once as input to a ResNet that labels every nucleotide of the guide paired or unpaired, and once as the sequence representation itself. Both streams are fused and reduced to a single continuous efficacy score.

The design is deliberately a task head over a released RNA language model rather than a model pretrained from scratch on CRISPR data, which is what lets it work at the scale Cas13d screens allow: a screen yields thousands of labelled guides, not the millions a sequence model needs, so the evolutionary signal has to come from elsewhere.

#Key Features

• Frozen RNA language model backbone: RNA-FM supplies per-nucleotide embeddings that already encode base-pairing propensity and evolutionary context, so the CRISPR head never has to learn RNA grammar from screen data alone.
• Predicted structure as a first-class input: A ResNet reads the RNA-FM representation and emits a per-position paired/unpaired probability matrix, giving the predictor an explicit structural view of each guide instead of leaving folding implicit.
• Two-stream DenseNet fusion: Seq-DenseNet and SS-DenseNet process the sequence and secondary-structure channels separately, with dense connectivity keeping features flowing between the two representations.
• Positional transformer refinement: A positional-encoding transformer encoder over the fused features captures dependencies between distant positions, which matters because pairing partners are rarely adjacent.
• Single trained head: The predictor is trained once and applied to new guide sequences with no per-screen refit, producing a continuous score rather than a categorical call.

#Technical Details

Guides enter as one-hot nucleotide vectors, pass through RNA-FM, and exit an MLP as a scalar efficacy score. Training used Adam at a learning rate of 1e-4 decayed 10% every two epochs, batch size 128, L2 regularization, dropout of 0.1 per transformer layer, and attention masks over padding so variable-length guides are handled cleanly, for up to 50 epochs with early stopping. Three sgRNA counts appear in the paper and describe different things: the model is validated against 22,599 Cas13d sgRNAs attributed to Cheng et al. (2023); the screening library comprises 10,830 sgRNAs across 426 genes (192 protein-coding, 234 lncRNA, including 94 essential and 14 non-essential genes at roughly 30 guides per gene); and the head-to-head evaluation runs over three public Cas13d tiling screens totalling 5,726 sgRNAs. The underlying proliferation screen is a two-vector CRISPR/Cas13d experiment in A375 melanoma cells following the protocol and library design of Cheng et al. (2023), analysed with MAGeCK.

Under five-fold cross-validation, treating guides with log-fold change below −0.5 as efficient, DeepFM-Crispr reaches an average AUC of 0.88, matching DeepCas13 and above the 0.78–0.85 range of Random Forest, XGBoost, SVM and AdaBoost baselines built on 185 curated features. Its clearer margin is in precision-recall: an average AUPR of 0.69 against 0.45–0.58, on a set where efficient guides are the minority class.

#Applications

The practical use is guide selection: before committing a Cas13d knockdown experiment or a tiling library to synthesis, a lab can score candidate guides against a target transcript and keep the ones predicted to give strong knockdown, which matters most for long non-coding RNAs, where RNA interference is unreliable. Because the score is continuous and fit against log-fold changes from proliferation screens, it supports ranking rather than simple filtering, useful when a screen budgets a fixed number of guides per gene.

#Impact

DeepFM-Crispr shows that a released general-purpose RNA language model can carry the representational load for a narrow CRISPR design task, letting a two-author team match a purpose-built baseline on classification and beat it on precision-recall. Its reach is bounded: the reported evaluation covers on-target efficacy only, retrospectively and in silico against published screens rather than by prospective wet-lab validation; the work is a workshop paper rather than a full journal study; and no code, weights or checkpoint have been released, so the scorer cannot be run without reimplementing it. The name is also shared with an unrelated recommender-systems architecture, and the paper calls the model DeepFM-Cas13d once in its introduction.