Pestle-27B-Ternary

The records that make a medical language model useful are the ones least able to leave the building. A 27B-class model held in 16-bit weights occupies 55.56 GB before any runtime cache, which in practice means a datacenter accelerator and, for most institutions, an external inference service between a clinician and their own data.

Pestle-27B-Ternary attacks that constraint at the level of the weight representation rather than the architecture. Built by Doses AI and released in August 2026, it converts the general-purpose Qwen3.6-27B checkpoint to ternary weights under a calibration weighted toward retaining medical capability. The topology is untouched — the same 64-layer hybrid-attention core, head, chat template and serving interface. What changes is the footprint: deployed text weights fall from 55.56 GB to 6.75 GB, packaged as one 8.48 GB GGUF that a laptop with 18 GB of unified memory can hold.

Pestle is therefore a compressed derivative rather than a model pretrained from scratch, and the framing is capability per deployed gigabyte rather than capability outright: every matched medical score but one sits below its 16-bit parent's — MMLU Virology alone reaches parity, at 56.02 for both — with Doses AI reporting a 96.48% median across the 17 measures it paired. Its reported medical figures land near those published for MedGemma 27B, a purpose-built medical model roughly six times the download, though those are external references under their own protocols. Doses AI keeps the compression algorithm as company IP and treats the release as a reference implementation for compressing hospital- and pharma-owned specialist models the same way.

#Key Features

• Ternary weights with one parent-precision block: The decoder is compressed to a 1.79-bit nominal representation, except the final block, which stays at BF16 because its measured quality gain outweighed its footprint cost.
• One runnable file: The published GGUF is the complete model — no overlay assembly, no weight conversion, no separate base checkpoint to fetch.
• Local-first runtime: Mortar, an MIT-licensed llama.cpp-compatible engine shipped with the model, picks Metal, CUDA or CPU automatically and applies the evaluated generation defaults; context extends to 262K tokens.
• Optional medical vision: A separately downloadable 0.93 GB projector feeds the parent's vision encoder into the compressed language core, adding radiology visual question answering to the same checkpoint.
• One fixed evaluation protocol: Every reported figure comes from greedy decoding with thinking disabled at temperature 0 and seed 20260805, with row-level results retained.

#Technical Details

Pestle carries 26,895,998,464 text parameters at 1.79 nominal and 2.52 effective deployed bits per weight. Holding the final decoder block at BF16 lowered full WikiText-2 perplexity from 12.677 to 12.302 at a cost of 0.745 GB; the production conversion ran on a single NVIDIA H100 for about 40 hours. Doses AI reports 89.79% on the full 1,273-question MedQA test split, 86.89% row-weighted across MMLU medical subjects, 76.70% accuracy on PubMedQA, 75.28 token-F1 on BioASQ, 90.15 MRR on PharmaRAG, 68.85% on MedMCQA, 32.49% on MedXpertQA and 61.72% on ChemBench, against 83.53 on MMLU-Redux and 87.20 pass@1 on HumanEval+ for general capability. Image-conditioned results are 51.03 macro-F1 on CheXpert expert validation, 53.49 token-F1 on VQA-RAD and 54.81 on SLAKE. Throughput is about 10 tokens per second on an 18 GB M3 MacBook Pro and about 39 single-stream on A100-class hardware. Every one of these numbers is the developer's own; the training corpus is not described, the compression recipe is unpublished, and no third party has reproduced the suite.

#Applications

Doses AI aims the model at settings where the data governs the deployment: hospitals running an internal model API beside governed clinical records, pharma teams querying proprietary safety and regulatory material, and community pharmacies and small clinics on workstation-class hardware. The advertised uses are assistive — formulary and guideline retrieval, question answering over approved local documents, patient-letter drafting and coding support — with a qualified professional in the review path. Doses AI states that the release is a research preview, not a medical device, and not released for clinical use.

#Impact

Pestle's contribution is that a 27B-class medical assistant can be served from an 8.48 GB file on hardware a single practice already owns — it changes who can run one at all rather than how well it performs. The caveats are substantial and the developer names most of them: the evaluation record is self-reported and unverified, no training data description exists, and the conversion pipeline is withheld, so the result is reproducible as an artifact but not as a method. A release-stage repair for pathological repetition cleared the loop in 99 of 152 held-out cases, leaving 53 that still need an exact-loop guard and an output ceiling in production; image-conditioned performance is uneven. What is durable is the operating point itself: evidence that ternary representations can hold most of a medical model's measured capability while removing the accelerator room from the deployment.