Researchers Have Trained an AI to Decode Human Thoughts 

Researchers at the University of Texas at Austin have developed a new artificial intelligence system that can translate a person’s brain activity into a continuous stream of text. 

Called a semantic decoder, the system is a noninvasive method that first involves measuring brain activity using an fMRI scanner, an imaging machine that tracks blood flow across different parts of the brain. The semantic decoder is trained through this imaging while the patient listens to hours of podcasts in the scanner. 

For the study, three people listened to podcasts from headphones for up to 16 hours each in the fMRI scanner. Much of the listening material consisted of stories from "The Moth Radio Hour," the popular public radio show with a weekly podcast. 

After the decoder is trained in this method, if the patient is open to having their thoughts decoded, they can listen to a new story or imagine telling a story and the machine will generate corresponding text from brain activity alone, according to a report from UT News. 

For its predictive text generation, the decoder uses a transformer language model comparable to the large language model powering ChatGPT. Instead of a verbatim transcript of the patient’s thoughts, the system produces text that only partially matches the intended meanings of the original words. The UT News report included the following example: 

These example segments were manually selected and annotated to demonstrate typical decoder behaviors, says UT News. (Credit: University of Texas at Austin)

This study was led by Jerry Tang, a doctoral student in computer science, and Alex Huth, an assistant professor of neuroscience and computer science at UT Austin. The study results were published in a paper in the journal Nature Neuroscience. The paper addresses concerns about patient privacy and the possibility for misuse of this technology. Decoding was only possible with patients who had willingly participated in training the decoder. The study notes that results were incoherent for individuals who did not train with the decoder or who purposefully thought about other things. 

The researchers say they are taking privacy and safety concerns seriously and want to make sure people only use this technology voluntarily and to help others, according to UT News. 

Though the system is not practical due to its need for a bulky fMRI machine, the researchers believe the technology could shift to a more portable brain imaging format like functional near-infrared spectroscopy (fNIRS). The technology could be a solution for patients who are left unable to speak due to health issues like strokes or neurological diseases. Read the original article and listen to a podcast that dives into the research methods at this link.