PENNE

An RNA-seq time course never actually watches a cell change. Sequencing lyses whatever it measures, so every time point comes from a different well, and any difference between two points confounds real dynamics with batch effects and population drift. Live-cell microscopy has the opposite property — the same field can be imaged every thirty minutes for days without perturbing it — but a phase-contrast image reports morphology, not molecules.

PENNE (Phase-to-Expression Neural Network Estimator) bridges the two by borrowing its training signal from a third assay. Spatial transcriptomics already produces large volumes of H&E image patches paired with measured expression, so PENNE learns the morphology-to-expression mapping there and then transports it onto unstained phase-contrast microscopy (PCM) through adversarial domain adaptation. A translator network splits each image embedding into a domain-invariant "biological" vector and a domain-specific one, and a gradient-reversal discriminator is trained until it can no longer tell whether a biological vector came from H&E or from PCM. Only that domain-invariant half reaches the expression head, so a single regression serves both modalities. Earlier PCM-to-expression methods were fit on small in-house paired datasets and re-fit for each new experiment; PENNE ships one fixed checkpoint.

It comes from Gregory Schwartz's group at the Princess Margaret Cancer Centre, University Health Network, with collaborators at the University of Toronto and the Vector Institute.

#Key Features

• Transcriptome-wide readout from an unstained image: Each PCM patch yields expression across 18,085 genes, not the handful of transcripts a fluorescent reporter can track.
• Non-destructive time courses: Imaging leaves the culture intact, so the same cells can be profiled repeatedly, turning snapshots of different populations into a real trajectory.
• Domain disentanglement with adversarial alignment: A Hilbert-Schmidt Independence Criterion term and a cosine-orthogonality penalty keep biological and domain features independent, while a correlation-alignment loss matches the two domains' distributions.
• Marker-gene supervision on the unlabeled domain: PCM images have no ground-truth expression, so a margin-ranking loss over cell-line marker genes and a cell-type classifier regularize the biological features directly in the target domain.
• Runs as a fixed pretrained model: A pip-installable package and a command-line tool fetch the released checkpoint automatically, and a Dockerfile pins the inference environment.

#Technical Details

The frozen front end converts a PCM patch into an H&E-like image with the same lab's SPAGHETTI style-transfer model, then embeds it with Phikon-v2, a histopathology vision transformer, producing 1,024-dimensional features for both real H&E and converted PCM. Five trainable modules follow: a translator factoring those features into a 960-dimensional biological vector and a 64-dimensional domain vector, a domain classifier, an adversarial domain discriminator behind a gradient-reversal layer in the DANN style, a cell-type classifier, and a gene-expression predictor of gated multi-layer perceptron blocks whose spatial gating units act as attention over gene dimensions. Expression is fit with a combined Huber and cosine loss.

Training ran in three stages — H&E features to expression, then domain adaptation, then marker-constrained fine-tuning — over 46,161 paired 224x224 px H&E patches and expression for 18,085 genes from nine public 10x Genomics Visium samples, plus 2,743 PCM crops from LIVECell, on an 80/20 split with AdamW. Evaluation spanned four independent PCM datasets. Predicted marker expression separated six LIVECell cell lines and held for U373 glioblastoma cells absent from training entirely; permuting the input images destroyed the signal. In 1:1 MCF10A/HCT116 co-cultures, inferred expression predicted the MCF10A fraction with a coefficient of determination of 0.510, and apical-junction expression tracked pseudo-time across confluency at 0.930, falling to 0.342 on permuted images. Against bulk RNA-seq across 18 samples, 92 high-confidence genes emerged, enriched for cell division and cell-cycle processes.

#Applications

PENNE suits experiments where the interesting question is what happens between time points: how a culture's expression program shifts with confluency, how a normal epithelial line and a p53-null tumour line diverge after a radiation dose, or how cell-cycle genes move across a geminin-marked time lapse without adding further reporters. Because inference needs only phase-contrast frames, it slots into existing live-cell imaging platforms and scales to plate-level screens, and its cell-type inference annotates mixed cultures without manual scoring or fluorescent labeling.

#Impact

The wider claim here is that label-free imaging carries enough molecular information to be read quantitatively, and that the training pairs can come from an entirely different assay. That reframes spatial transcriptomics as a supervision source for microscopy rather than an endpoint in itself, and the authors point the same recipe at bright-field and other unstained modalities. The caveats are real: training covered a limited range of human cell types and morphologies, predictions are made at patch rather than single-cell resolution, and the work is a preprint awaiting peer review. Code is GPL-3.0 and the pretrained checkpoint is published in the repository, so the no-retraining claim is one other groups can test.