Using AI to Decode Speech from Brain Activity

Apologies in advance for sharing PR hype from Meta (formerly known as Facebook),but I want to comment on the essence of the idea, which is using AI to decode speech from brain activity. It seems to imply that one would apply supervised machine learning to train a system to map speech to brain activity as illustrated by the image below.

Podcast: Audio rendition of this page content
To decode speech from noninvasive brain signals, we train a model with contrastive learning to align speech and its corresponding brain activity

The dataset would require the captured patterns of a large enough sample size. In this case, it appears to have been some 417 volunteers.

Activations of wav2vec 2.0 (left) map onto the brain (right) in response to the same speech sounds. The representations of the first layers of this algorithm (cool colours) map onto the early auditory cortex, whereas the deepest layers map onto high-level brain regions (e.g. prefrontal and parietal cortex).

This feels like it could have many commercial, consumer, and industrial uses including removing other human-computer interface devices, notably keyboards, but perhaps even mouses. Yes, I said mouses. Sue me.

Given hypotheses related to language and cognition, I am wondering what can be gleaned by mapping different multiple native language speakers to cognitive processes in order to remap them to speech output if it would be able to arrive at some common grammar that could then output a given thought stream into any known (and mapped) language, allowing for instantaneous “translation”.

Of course, a longer-term goal would be to skip the external devices and interface brain to brain. This sounds rogue science fiction scary, as one might imagine an external device trained on a brain to read its contents. One of the last things this world needs is to have to worry about neuro-rights and about being monitored for thought crimes. Come to think of it, isn’t there already a book on this? Nevermind. Probably not.

Technology is generally not inherently harmful or helpful, as that is determined by use. Humans do seem to tend toward the nefarious. Where do you think this will go? Leave a comment.

Advertisement

2 thoughts on “Using AI to Decode Speech from Brain Activity

  1. I seriously want my life to be very short. Stuff like that just makes me realize how I don’t belong here 😝. Ironically. I am pretty positive I will be doomed to live till I’m at least 100 years old.

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s