BioCLIP

BioCLIP is a CLIP-style vision foundation model for the tree of life: given a photograph of a plant, animal, or fungus, it produces image and text embeddings that support zero-shot and few-shot classification of the organism's species. It is the original model in the BioCLIP line, released in November 2023 by the NSF-funded Imageomics Institute, led by The Ohio State University, and is the direct predecessor to BioCLIP 2 (2025).

Identifying organisms from images is a foundational problem across ecology, conservation, and systematics, but biological classification is fine-grained and long-tailed: many species look nearly identical to close relatives, most are rare, and conventional image models trained on flat class indices generalize poorly to the hundreds of thousands of taxa that make up the tree of life. BioCLIP addresses this by curating TreeOfLife-10M, the largest and most diverse machine-learning-ready dataset of biology images at the time, and training a CLIP model with a knowledge-guided objective that aligns each image with its full Linnaean taxonomic hierarchy rendered as text rather than an arbitrary label.

The work, by Samuel Stevens, Jiaman Wu, Matthew J. Thompson, Elizabeth G. Campolongo, and colleagues, was presented as an oral at CVPR 2024 and received the conference's Best Student Paper award. It established that contrastive vision-language training over taxonomically structured text yields a strong fine-grained biological classifier, a result the Imageomics Institute later scaled in BioCLIP 2.

#Key Features

• Tree-of-life-scale training: BioCLIP is trained on TreeOfLife-10M, over 10 million organism images spanning 454,000 taxa drawn from the Encyclopedia of Life, iNaturalist 2021, and BIOSCAN-1M.
• Knowledge-guided taxonomic text: Rather than treating species as independent labels, the model contrasts images against their flattened seven-rank Linnaean taxonomy (kingdom through species) plus common names, injecting the nested structure of biological classification into the embedding space.
• Zero-shot and few-shot classification: The learned image-text space classifies species by supplying candidate taxon names, including taxa unseen during training, with no task-specific retraining.
• Strong fine-grained generalization: Across ten biologically relevant benchmarks, BioCLIP substantially outperforms general-domain CLIP baselines in both zero-shot and few-shot regimes.
• Open release: Model weights and code are released under the MIT license, with a HuggingFace model card, the TreeOfLife-10M dataset, and an interactive demo.

#Technical Details

BioCLIP uses a ViT-B/16 image encoder paired with an autoregressive Transformer text encoder, initialized from OpenAI's CLIP ViT-B/16 checkpoint and trained with the standard CLIP contrastive objective using the open_clip library. Its central design choice is textual: taxonomic labels are serialized into strings spanning the full Linnaean hierarchy, optionally combined with scientific and common names, so the text tower encodes hierarchical relationships among taxa. TreeOfLife-10M comprises roughly 10.4 million images, of which about 8.5 million carry complete taxonomic labels, covering 454,000 taxa. Evaluated on ten benchmarks spanning birds, insects, plants, and other organisms, BioCLIP outperforms general-domain baselines by 16 to 17 percent absolute on average, raising mean accuracy to 39.4 percent versus 21.9 percent for OpenAI CLIP and reaching 72.1 percent on Birds 525 compared with 49.9 percent for CLIP.

#Applications

BioCLIP is aimed at ecologists, conservation biologists, systematists, and biodiversity-informatics teams who need to identify organisms at scale. Its zero-shot and few-shot capabilities suit camera-trap surveys, museum specimen digitization, citizen-science platforms such as iNaturalist, and rapid biodiversity assessment where labeled training data for the target taxa is scarce. Because classification is driven by taxon names rather than a fixed label set, practitioners can apply the model to new species and clades without collecting task-specific training data.

#Impact

BioCLIP launched a line of biodiversity foundation models and demonstrated that aligning images with taxonomically structured text produces a fine-grained classifier that generalizes across the tree of life. Its Best Student Paper award at CVPR 2024 and its openly released MIT-licensed weights and code lowered the barrier for the ecology and computer-vision communities to build on organism-scale representations, while TreeOfLife-10M became a widely used community dataset. The Imageomics Institute subsequently released BioCLIP 2, which scales the same recipe to 214 million images and 952,000 taxa, positioning the original BioCLIP as the foundational entry in an increasingly adopted family of models for AI applied to the natural world.