Mind – Reading Made Possible: Scientists Use GPT AI in Breakthrough Study

According to a recent study, scientists created a language parser that uses an AI tool similar to ChatGPT to convert speech into text. The finding is notable because it is the first time that continuous language has been non-invasively recovered from a person’s brain activity using a functional magnetic resonance imaging (fMRI) device. Using fMRI brain patterns, the tool was able to interpret the main points of stories that human subjects read, watched, or listened to—or even just pictured—in a way that safely lets it read people’s minds. Scientists expect that this tech, which is still in its infancy, may one day make it easier for people with neurological diseases that impair speech to chat easily with others.

Mind – reading tools could be used for evil

The team behind the decoder warns that mind – reading tools might potentially be used for evil acts. One of these could be illegal monitoring by the government. According to a report published on Monday in Nature Neuroscience, the authors stated that “brain-computer interfaces should respect mental privacy,” even if they noted that their tool needs a team effort of human subjects to function.

Jerry Tang, a graduate student in computer science at the Univ. of Texas at Austin who led the study, in a press briefing held last Thursday said

“Currently, language – decoding is done using implanted devices that require neurosurgery, and our study is the first to decode continuous language, meaning more than full words or sentences, from non-invasive brain recordings, which we collect using functional MRI … The goal of language – decoding is to take recordings of a user’s brain activity and predict the words that the user was hearing or saying or imagining,” he noted. “Eventually, we hope that this technology can help people who have lost the ability to speak due to injuries like strokes, or diseases like ALS.”

How the new AI tool work

Three people who each spent 16-hours listening to stories in an fMRI tool helped Tang and others create their tool. To connect the semantic aspects of the recorded stories with the brain activity noted in the fMRI data, the team trained an AI model known as GPT-1. It might then learn which words and phrases are linked to specific brain patterns.

The participants then listen to brand – new stories that were not part of the training dataset. While doing this, they had their brains imaged using an fMRI. Although these interpretations often employed use different semantic structures from the original recordings, the tool was able to translate the audio storylines into text as people heard them. For instance, using fMRI readers, the listener’s thoughts were translated from a tape of a speaker saying … “I don’t have my driver’s license yet” to “She has not even started to learn to drive yet.”

These shaky readings result from a crucial distinction between the new tool and established methods. The old standard methods invasively implant electrodes in the brain. While Tang’s team focused on the blood flow through the brain, which is what is captured in fMRI machines. The electrode – based tools typically predict text from motor activities. This include movements of a person’s mouth as they try to speak.

Tool does not give exact words

Alexander Huth, an assistant prof. of neuroscience and computer science at UT Austin and senior author of the new study, says in a press briefing

“Our system works at a very different level … Instead of looking at this low – level motor thing, our system really works at the level of ideas, of semantics, and of meaning. That’s what it’s getting at … This is the reason why I think what we get out is not the exact words that somebody heard or spoke, it’s the gist … It’s the same idea but expressed in different words.”

The team’s new tool allows them to push past the limits of mind – reading tech. This it did by testing whether the tool could translate the thoughts of subjects as they looked at silent films. It also did the process again as the subjects just created stories in their heads. In both cases, the decoder did get some pretty good results. Whether they were creating stories in their minds or they were looking at a film, the decoder did well.

With respect to the imagined speech, the decoder gave better results all through the testing with audio recordings. But it was still able to infer a few basic facts about unspoken thoughts from the brain activity. As a case study, a subject imagined the statement … “went on a dirt road through a field of wheat and over a stream and by some log buildings,”. The decoder produced a text that reads … “he had to walk across a bridge to the other side and a very large building in the distance.”

Huge privacy issues are bound to arise

All of these tasks were given to study members while they were inside an fMRI machine. This is a bulky and fixed piece of lab gear. Due to this, Tang and his team believe that the decoder is not yet ready to be used. This means that for people with speech issues, this tool is not feasible at the moment. However, they do believe that future versions of the device may be altered to work with simpler tools. Tools such as fNIRS sensors that can be worn on a patient’s head.

The team which did the study warns that the decoders raise moral questions about mental privacy. Though, they alluded to the potential of this tech as a new form of dialogue

Tang’s team said in the study

“Our privacy analysis suggests that subject cooperation is currently required both to train and to apply the decoder … However, future developments might enable decoders to bypass these requirements. Moreover, even if decoder predictions are inaccurate without subject cooperation, they could be intentionally misinterpreted for malicious purposes … “For these and other unforeseen reasons, it is critical to raise awareness of the risks of brain decoding tech. There is a need to also enact policies that protect each person’s mental privacy,”