10 July 2023
From thoughts to speech: Exploring inner speech recognition
You cannot read minds – or can you? For those who cannot communicate at all, advanced speech prosthetics controlled by 'mind-reading' could be the way out. A ground-breaking study conducted at Luleå University of Technology has added an important piece to the puzzle that will propel the technology forward.
Foteini Liwicki, Associate Professor of Machine Learning, together with her research group has created a prodigious study about inner speech recognition. Utilising both temporal and spatial means to gather enriched information from the research subjects. The study attracted much attention and was published by Nature, in their journal ‘Scientific Data’.
The aim of the study was to create a publicly available bimodal dataset on inner speech to contribute to the existing body of literature which could potentially turn into advanced speech prosthetics.
– This is great news, and we are very excited. The future for decoding inner speech just became brighter and hopefully, with time, this will help a lot of people that suffers from locked-in syndrome.
The research data was collected from four healthy, right-handed participants during an inner-speech task involving words in a numerical or social category. Each 8-word stimulus was evaluated in 40 trials, resulting in a total of 320 trials in each modality for each participant.
What is ground breaking is the method that has been used, Foteini Liwicki explains.
Two methods are better than one
The study used two methods, electroencephalography (EEG) and functional magnetic resonance imaging (fMRI). EEG is an excellent method to measure electrical activity in the brain with high temporal resolution – it can capture rapid changes in the brain activity with millisecond precision. However, it does not yield as good spatial results as fMRI due to the electrical signals which has to travel through the scalp, skull and brain tissue.
This combination of methods, along with a great study and supplementary papers, has been rewarded by being published by Nature, in their journal ‘Scientific Data’.
– By combining EEG and fMRI we see a much higher accuracy, the study reports a mean accuracy of 71,72%. Compared to 62,81% and 56.17% when using EEG, respectively fMRI alone. This is a very interesting find as it perfectly demonstrates that a bimodal approach is a promising direction for inner speech decoding, Foteini Liwicki says.
Contact
Foteini Liwicki
- Associate Professor
- 0920-491004
- foteini.liwicki@ltu.se
- Foteini Liwicki
Published:
Updated: