AI language recognition during awake brain surgery

Thanks to stophersentumoren, the linguistic team can start a pilot study on AI and spontaneous speech.

Awake surgery and language testing

Currently, patients who undergo awake brain surgery are assessed with language tests before, after and during surgery to monitor their language level. In addition, spontaneous speech (e.g. talking about hobbies) is also transcribed and classified, for example number of incomplete sentences or the number of unique words.

“That takes a lot of time, but it is very important,” says Elke. Spontaneous speech is more sensitive and shows mild language problems that do not always emerge from standardized language tests. Djaina: ‘If we do not analyze spontaneous speech properly, we may miss language errors, while patients do complain about language problems at home and at work. These patients are often sent home while they actually still need follow-up with more specific tests and speech therapy.’

Aim

With the pilot, the researchers are working on AI models to automatically transcribe spontaneous speech of brain tumor patients. The problem with existing AI automatic speech recognition tools such as Whisper etc. is that they are developed for typical speech. They automatically correct for pauses, slip of the tongues and errors. They do not work well with atypical speech such as aphasia (language disorder caused by damage to the brain). 'But as researchers and clinicans, we are interested in these subtle language errors which are typical for brain tumor patients with mild aphasia. “If the model can also score in addition to transcribing, perhaps even during an awake brain operation, we will not only be faster, but also more objective,” says Elke. Herefore, we work together with the University of Delft. With a larger consortium within EMC, we are applying for larger grants to automatically transcribe and classify (score) speech of various neurological and psychiatric patient populations.