US-JEPA

Most of a B-mode ultrasound image is interference. Speckle — the granular texture filling every scan — is a stochastic artifact of coherent wave backscatter rather than anatomy, and its exact pattern changes when the sonographer shifts the probe a centimeter or the machine's gain is re-set. A masked-image-modeling objective that asks a network to repaint missing pixels therefore spends much of its capacity memorizing acquisition noise tied to one transducer — the signal least likely to transfer to the next clinic. US-JEPA, from the Biomedical Artificial Intelligence Research Lab at UCLA, moves the prediction target out of pixel space: following the Image-based Joint-Embedding Predictive Architecture (I-JEPA), it predicts the latent representations of masked target blocks from a single visible context block within the same frame.

Its departure from I-JEPA is where the teacher comes from. A standard JEPA derives its targets from an online teacher maintained as an exponential moving average of the student, a coupling that is expensive and notoriously sensitive to hyperparameters. US-JEPA instead adopts the Static-teacher Asymmetric Latent Training (SALT) objective, freezing an existing ultrasound encoder — URFM, itself distilled from BiomedCLIP — as the target network. Gradients flow only through the student and predictor, training dynamics stabilize, and the student is pushed to extend rather than merely reproduce the teacher's semantic priors.

The paper pairs the model with two curation efforts. OPUS (OPen UltraSound) aggregates public ultrasound data into a single pretraining corpus, and the authors extend the public UltraBench suite so that every publicly available ultrasound foundation model can be compared under one frozen-backbone linear-probing protocol — the first such head-to-head evaluation of the family that includes USFM, UltraSAM and USF-MAE. A companion variant, USrc-JEPA, adds region conditioning; it shares the encoder recipe and differs only in which patches it is allowed to sample.

#Key Features

• Static teacher in place of an EMA teacher: the frozen URFM encoder supplies fixed latent targets, decoupling student and teacher optimization and removing the moving-average update entirely.
• Latent prediction over pixel reconstruction: Smooth L1 loss on predicted versus teacher embeddings directs capacity toward organ morphology and tissue texture rather than speckle.
• Ultrasound region conditioning: the USrc-JEPA variant derives a binary mask of the scan cone through a deterministic thresholding, morphological-closing and convex-hull pipeline, then restricts both context and target sampling to valid anatomy.
• A fully public pretraining corpus: OPUS is assembled entirely from open repositories, and its eight UltraBench evaluation datasets are held disjoint from pretraining.
• Label efficiency under linear probing: on FATTY LIVER with fewer than half the labels, the models average 19 macro-F1 points above the URFM and USFM baselines.

#Technical Details

Both the student context encoder and the frozen teacher use a ViT-B/16 backbone at 224×224 resolution with embedding dimension 768; the predictor is a narrower 12-layer transformer of width 384 with a linear projection back to 768 before the Smooth L1 loss. Each image contributes one context block at scale [0.85, 1.0] and four target blocks at scale [0.075, 0.125], with non-overlapping masks enforced. Training runs 100 epochs at batch size 128 with AdamW and cosine decay. OPUS comprises 5,123,697 frames curated from 50 public datasets spanning 22 anatomies, with 79.7% drawn from video sequences or volumetric scans; a per-dataset sampling cap of 50,000 frames per epoch keeps EchoNet-scale cardiac collections from swamping smaller organ sets. Across UltraBench's eight classification tasks, US-JEPA takes the best mean macro F1 on AUL (69.6), MMOTU (52.2, against URFM's 41.9) and POCUS (93.1), while USrc-JEPA leads on BUSBRA (76.0), FATTY LIVER (89.2) and GBCU (70.2). URFM retains thyroid nodule classification on TN5000 at 78.1 versus 73.1.

#Applications

The intended use is as a frozen feature extractor for image-level ultrasound classification: liver mass characterization, breast, gallbladder, ovarian and thyroid malignancy prediction, lung pathology triage from point-of-care scans, and anatomical view recognition. Because a linear head on frozen features recovers most of the performance, groups without the labeled volume or compute to fine-tune a full encoder can build task-specific classifiers from a few hundred annotated images — the regime most clinical ultrasound research operates in.

#Impact

US-JEPA's contribution is as much methodological as architectural. Its controlled comparisons show that MAE and EMA-teacher I-JEPA trained on the same OPUS corpus both underperform the SALT variant, isolating the objective from the data, and its standardized linear probe gives the ultrasound foundation model literature a common yardstick it previously lacked. The limits are stated plainly: robustness is measured against synthetic blur, contrast and speckle corruptions rather than real scanner heterogeneity, and OPUS's thin thyroid representation (5.2% of frames) leaves URFM ahead on that organ. The work remains a preprint under review, and no code, weights or assembled corpus have been released, so the reported results cannot yet be reproduced independently.