HAVEN

Roughly 60% of human infectious diseases originate in animals, and there may be as many as 1.6 million animal-infecting viruses whose host range is unknown. Determining which species a newly sequenced virus can infect is slow and expensive in the lab, and existing computational predictors are narrow: most cover a single viral family or protein group, many pose the question as a binary "does it infect humans," and few can say anything about a reservoir host that never appeared in their training data.

HAVEN — Hierarchical Attention for Viral protEin-based host iNference — was developed at Virginia Tech as part of the NSF COMPASS Center, with clinical virology input from the University of Michigan. It is a BERT-style viral protein language model pretrained by masked language modeling on the full universe of viral protein sequences in UniRef90, then fine-tuned for multi-class host prediction. Its distinguishing architectural choice is a two-level hierarchy: rather than truncating or discarding proteins longer than a fixed context window, HAVEN splits each sequence into overlapping segments, encodes each one, and then runs a second self-attention layer across the segment embeddings so that every part of a protein can inform every other.

Paired with a prototypical-network few-shot classifier, the same pretrained encoder extends to hosts represented by a handful of labeled sequences, to hosts absent from training entirely, and to viruses the model has never seen.

#Key Features

• Hierarchical self-attention over segments: A six-layer BERT segment encoder (segment length 256, stride 64) produces one CLS embedding per segment, and a sequence-level attention layer mixes them, handling proteins from 11 to 13,556 amino acids without a fixed context ceiling.
• Virus-specific pretraining corpus: Pretraining on 1,207,317 Viridae sequences from UniRef90, clustered at 90% identity, targets a taxon that is badly underrepresented in the general protein databases used to train larger language models.
• Few-shot generalization to rare hosts: A prototypical-network classifier built on HAVEN embeddings predicts hosts at prevalences as low as 0.09%, computing class prototypes from as few as five labeled support sequences.
• Parameter efficiency: At 18M parameters, HAVEN matches or beats foundation models 65 to 5,000 times larger, including ProtT5 (1.2B), ProstT5 (1.2B), and ESM3 (98B).

#Technical Details

Host labels come from the European Nucleotide Archive, restricted to vertebrate hosts per the WHO definition of zoonoses: 267,860 protein sequences from 3,779 viruses infecting 1,314 hosts, dominated by human hosts and by immunodeficiency viruses. The authors split this into a non-immunodeficiency-virus set (47,792 sequences, 1,304 hosts) and an immunodeficiency-virus set (220,068 sequences, 40 hosts), the latter reserved to test generalization to unseen viruses. Fine-tuning optimizes a focal loss to handle severe class imbalance, and evaluation uses area under the precision-recall curve rather than accuracy or AUROC, which flatter majority-class predictions.

On the five common hosts of the non-IV set, HAVEN reaches a median macro-AUPRC of 0.67, on par with ProtT5 (0.67) and above ESM3 (0.65). A 3-way, 5-shot prototypical classifier on rare classes achieves a median AUPRC of 0.68 despite prevalences below 1%. An ablation isolating pretraining, segmentation, and hierarchical self-attention confirms all three contribute, with the full model at mean macro-AUPRC 0.68. In a focused case study, HAVEN fine-tuned on 681 Coronaviridae spike sequences across eight hosts reaches mean macro-AUPRC 0.88 versus 0.87 for ProtT5, 0.85 for ProstT5, and 0.73 for ESM3; asked to identify the host of 16 SARS-CoV-2 variants of concern, it correctly returns Homo sapiens for fourteen, while the three baseline language models assign highest probability to the Chinese rufous horseshoe bat for fourteen.

#Applications

The immediate use is triage for the flood of viral sequences now arriving from clinical and environmental metagenomics: given a single protein from an uncharacterized virus, HAVEN ranks candidate vertebrate hosts, including reservoir species with almost no labeled data. That supports outbreak surveillance, spillover-risk assessment, and prioritization of which viruses merit experimental host-range characterization. Because the pretrained encoder is released as a general-purpose viral protein representation, it can also be fine-tuned for other viral sequence properties.

#Impact

HAVEN is an argument that a domain-matched corpus can substitute for scale: an 18M-parameter model trained only on viral proteins holds its own against billion- and hundred-billion-parameter general protein language models, and beats them where viral specificity matters most. Code is on GitHub under GPL-3.0 with pretrained and fine-tuned checkpoints and the full pretraining and fine-tuning datasets deposited on Zenodo; the preprint itself carries the more restrictive CC-BY-ND license. Limitations are stated plainly by the authors: predictions rest on a single protein sequence with no host-virus network or ecological context, performance on the dominant human class is far stronger than on any other, and the two SARS-CoV-2 variants HAVEN misses both carry mutations near the spike furin cleavage site, a region already known to modulate host range.