Multi-User Speech BCI Decoder

Intracortical brain-computer interfaces (BCIs) can restore communication to people with vocal tract paralysis by decoding cortical activity recorded during attempted speech into text. The strongest systems pair a neural-to-phoneme decoder with a phoneme-to-word language model and have reached word error rates (WERs) as low as 1% — but only after thousands of sentences of training data have been collected from that one person. That per-person calibration burden is the practical bottleneck between surgery and usable communication.

The multi-user (MU) speech decoder, introduced in the preprint "A generalizable speech neuroprosthesis" from the University of California, Davis Neuroprosthetics Lab and collaborating clinical sites, breaks that one-model-per-person pattern. It is a transformer decoder trained jointly on neural data from six intracortical speech BCI participants. Each participant's recordings pass through their own learned input projection, mapping that person's array placement and signal statistics into a shared latent space on which a single shared decoder operates. The paper gives the model no product name, calling it simply the multi-user model.

It is the pretrain-broadly, adapt-cheaply recipe of foundation models arriving in a domain where datasets are measured in individual people rather than millions of samples. Non-invasive decoders such as Brain2Qwerty and the EEG foundation model for BCI have probed cross-subject transfer with scalp recordings; the MU decoder shows the same pooling logic holds for human intracortical data, despite variability in electrode placement, disease etiology, and speaking strategy across recipients.

#Key Features

• Joint pretraining across six users: One decoder trained on pooled data from six intracortical speech BCI participants, turning a set of small per-person datasets into a single larger corpus.
• Per-user projection into a shared latent space: Subject-specific input layers absorb differences in channel layout and signal statistics, so the shared decoder sees a consistent representation regardless of whose cortex produced it.
• Rapid calibration on a held-out user: Finetuning on fewer than 200 sentences from a participant excluded from pretraining yields a WER below 7%, against thousands of sentences for a from-scratch per-user decoder.
• Frozen-decoder few-shot mode: A linear alignment fit on a small amount of new data adapts the model with no decoder retraining, making the pretrained checkpoint reusable as a fixed backbone.
• Consistent across participant differences: The multi-user model beat single-user models for every participant, irrespective of sex, disease etiology, or attempted speaking strategy.

#Technical Details

The decoder is a transformer that emits phoneme probabilities from intracortical activity recorded by microelectrode arrays in speech motor cortex, feeding the standard two-stage brain-to-text pipeline in which a phoneme-to-word language model converts those probabilities into sentences. Averaged over the six participants, the multi-user model reduced relative WER by more than 50% against models trained on each individual's data alone. On a held-out user it reached a WER below 7% after finetuning on fewer than 200 sentences, and the frozen-decoder linear-alignment variant adapts with no decoder update at all.

The recordings come from participants in a clinical trial of an intracortical BCI, so the training data is regulated human research data rather than a public corpus. Neither code nor weights for the multi-user decoder have been released; separately available are the lab's Apache-2.0 phoneme-to-word decoding stack — CTC beam search over an n-gram language model with optional LLM rescoring — and the reproduction repository for its earlier single-user system. The preprint carries an all-rights-reserved bioRxiv license and has not been peer reviewed.

#Applications

The immediate application is clinical: shortening the interval between implant and accurate communication for people with ALS, brainstem stroke, or other causes of severe speech loss. A shared pretrained decoder lets a new recipient be calibrated in a fraction of the sessions previously required, which matters most for people whose condition is progressing. More broadly, it is a template for multi-site BCI research: neural data from separate trials and separate implant recipients can be pooled into one model instead of spent on parallel, non-transferable decoders.

#Impact

This work reframes a core assumption of intracortical BCI research — that decoders must be refit from scratch for each person — as an artifact of small data rather than a biological necessity. It extends a lineage of UC Davis speech-neuroprosthesis results, including the single-participant system reported in the New England Journal of Medicine in 2024, by making the trained model a reusable asset rather than a one-off fit. The evidence base is six participants — large for an implanted BCI study, small by machine-learning standards — and the results await peer review. Because no weights or code for the decoder are public and the training data is clinical-trial material, independent replication will depend on the field building comparable multi-site data-sharing arrangements.