Anna-Riikka Smolander – Teflon Project

31/08/202217/12/2022

Greetings from Aalto University, Otaniemi!

Mikko Kurimo is the coordinator of Teflon and leads the research group at Aalto University. Prof. Kurimo has been the head of the automatic speech recognition (ASR) group at Aalto since 2000. He has led the group in many national and international machine learning and ASR projects. Kurimo’s work is internationally best known for unsupervised subword language modeling for morphologically complex languages such as Finnish, Estonian, Turkish and Arabic. His research interests include deep learning methods for automatic speech recognition and spoken language modeling.

Aalto’s team is in charge of the ASR technology that is required in Teflon and will also provide the game platform for Teflon. The platform is based on the previous projects of Mikko Kurimo and Sari Ylinen where English was the target language. Now the goal in Teflon is to apply it to Finnish, Swedish and Norwegian.

The Aalto team consists of:

Tamas Grósz is a post-doctoral researcher whose specification area is ASR and computational paralinguistics. His background lies in developing machine learning based ASR systems.
Ragheb Al-Ghezi is a doctoral researcher whose main research focus lies in applying ASR and machine learning in language assessment. He has a versatile background in natural language processing and second language (L2) learning.
Ekaterina Voskoboinik is a doctoral researcher whose main research focus lies in applying statistical language modeling and machine learning in ASR and spoken L2 assessment. Her background is on learning and analysing the representations of words and subwords for morphologically rich languages.
Yaroslav Getman is a doctoral researcher whose main research focus lies in applying self-supervised learning and pre-trained models for ASR in low-resource tasks like L2 learners’ ASR for spoken language assessment.
Aku Rouhe is a doctoral researcher whose main research focus is in ASR. He has a versatile background speech and spoken language modeling including speaker adaptation, voice activity detection, subword language modeling using statistical morphemes, speech translation and decoding algorithms and miscue tolerant ASR in L2 reading tutoring.
Nhan Phan is a master’s student whose research topic is developing a mobile app to give feedback for beginner level L2 learners’ in read aloud tasks. His skills also include game programming using Unity and his task is to modify the game platform for Teflon tasks.

We are excited and looking forward to the experiment with the Nordic languages.

Aalto University is the coordinating partner of the Teflon project. Aalto University describes itself as a meeting place for the fields of science, art, technology, and business. Today the university is one of the most prestigious ones in Finland. The university started operating in 2010 and the goal was to merge together the old Helsinki School of Economics, Helsinki University of Technology and the University of Art and Design Helsinki and create a new multidisciplinary university. The Aalto university campus in Otaniemi is situated in Greater Helsinki in the city of Espoo, just a metro ride away from central Helsinki.

The campus was originally built almost in the middle of the forest in Otaniemi. The rebuilding after the Second World War in Finland led to a need to educate more engineers – and hence to a need to build a new campus area and laboratory spaces for the Helsinki University of Technology and VTT Technical Research Centre of Finland Ltd. The Otaniemi campus is a park campus design from the 1950s and the city plan of the area is the work of the Finnish architect Alvar Aalto who has along with other esteemed Finnish architects such as Reima and Raili Pietilä and Heikki and Kaija Sirén also designed multiple buildings in the campus. The name of the university is a tribute to Alvar Aalto who himself graduated from the former Helsinki University of Technology.

List of recent publications:

Anssi Moisio, Dejan Porjazovski, Aku Rouhe, Yaroslav Getman, Anja Virkkunen, Ragheb AlGhezi, Mietta Lennes, Tamás Grósz, Krister Lindén and Mikko Kurimo. Lahjoita puhetta: a large-scale corpus of spoken Finnish with some benchmarks. Language Resources & Evaluation (2022).
Kathania, Hemant; Kadiri, Sudarsana; Kadyan, Virender; Kurimo, Mikko. Data augmentation using spectral warping for low resource children ASR. Journal of Signal Processing Systems. Accepted for publication.
Kathania, H , Kadiri, S , Alku, P & Kurimo, M. A Formant Modiﬁcation Method for Improved ASR of Children’s Speech. Speech Communication (2022), vol. 136 , pp. 98 – 106.
Kathania, Hemant; Kadiri, Sudarsana; Alku, Paavo; Kurimo, Mikko. Using data augmentation and time-scale modification to improve ASR of children’s speech in noisy environments. Applied Sciences (Springer International Publishing AG), 2021.
Ylinen Sari, Smolander Anna-Riikka, Karhila Reima, Kakouros Sofoklis, Lipsanen Jari, Huotilainen Minna, Kurimo Mikko. The Effects of a Digital Articulatory Game on the Ability to Perceive Speech-Sound Contrasts in Another Language. Frontiers in Education, 2021 ; Vol 6.

Ragheb Al-Ghezi, Yaroslav Getman, Ekaterina Voskoboinik, Mittul Singh, Mikko Kurimo. Automatic Rating of Spontaneous Speech for Low-Resource Languages. IEEE Spoken Language Technology Workshop (SLT 2022), 2022.
Grósz, T., Porjazovski, D., Getman, Y., Kadiri, S., Kurimo, M. Wav2vec2-based Paralinguistic Systems to Recognise Vocalised Emotions and Stuttering. ACM Multimedia 2022 Conference : Grand Challenges.
Yaroslav Getman, Ragheb Al-Ghezi, Katja Voskoboinik, Tamás Grósz, Mikko Kurimo, Giampiero Salvi, Torbjørn Svendsen, and Sofia Strömbergsson. Wav2vec2-based Speech Rating System for Children with Speech Sound Disorder. Proc. Interspeech 2022.
Ragheb Al-Ghezi, Yaroslav Getman, Aku Rouhe, Raili Hildén and Mikko Kurimo. Self-supervised End-to-End ASR for Low Resource L2 Swedish. Proc. Interspeech 2021.

https://www.aalto.fi/fi/aalto-yliopisto

https://research.aalto.fi/fi/persons/mikko-kurimo

11/08/202217/12/2022

Researchers of Teflon project: Sofia Strömbergsson

Author: trainee Elina Aittokallio, Tampere University

Greetings from Tampere!In the following blogpost we will briefly introduce one of the researchers working on the Teflon project, Sofia Strömbergsson from Karolinska Institutet (KI), the largest center of medical academic research university in Sweden.

Kuvan esikatselu

Strömbergsson is an associate professor of Karolinska Institutet. She is a teacher and conductor of research concerning children’s speech and language disorders. Strömbergsson earned her doctoral degree in April 2014 at the Royal Institute of Technology (KTH), Stockholm. Strömbergsson also has a medical science degree in SLP (Lund 2007) and in computational linguistics (Uppsala, 2000).

We’ve had the honor to hold an interview on Sofia’s job as a researcher:

What sparked your interest into the field of logopedics? How did your career path form?

I’ve always been interested in language, and particularly in its spoken form – different ways of speaking, and how that’s perceived by listeners. I first started out in the field of speech technology, and I worked a few years in the industry developing text-to-speech for Swedish. I really enjoyed the task of trying to “teach” the computer how to read. After some time, though, I wanted to make use of my interest in speech and language in ways that would benefit others more directly. That’s when I decided to embark on a study program in logopedics. When I finished, an opportunity opened up for pursuing PhD studies at KTH, where I had the chance to combine my two interests into a thesis project on how children perceive automatically “corrected” versions of their own recorded misarticulation. In 2014, after I finished, I joined the Division of Speech-Language Pathology at KI, where I still am today.

How do you find Karolinska Institutet as a place to work and do research at?

When I joined KI and the Division of Speech and Language Pathology in 2014, it felt a little bit like coming home. Although I had really enjoyed my time at KTH, I had been a bit of an odd bird, being the only SLP among engineers (and, admittedly, also some phoneticians). At KI, I experienced that I could contribute with my background in computational linguistics as a new perspective, but to a research core that lied closer to my own interests. I thrive here, where I can focus on research centered around clinical relevance, and to increase our understanding of if, how and why certain intervention methods work, and others don’t.

Regarding the ongoing Teflon project:

What is your role in the project?

I lead the part of TEFLON that focuses on trying out the speech training app in a clinical population.

What type of joys and challenges have you faced during the project?

Up until now, I’ve been working alone at my site on the project. Although we’ve had regular meetings in the bigger group, it hasn’t always been easy to manage the KI part of the project single-handedly. Therefore, I’m thrilled to have a project assistant, Magdalena Pettersson, joining me now in October!

Has the project raised any new questions for future research?

As we are still in the beginning of the project, I can’t say it has. (But I might have more to say about it in 2024!)

Aside from Teflon, are you involved in other research projects at the moment?

Yes, I run a project called SPETS (http://ki.se/spets). In SPETS, we explore and compare different types of intervention for preschool-aged children with Developmental Language Disorder (DLD). It’s a longitudinal study, following children with DLD for 2 years, to explore what intervention they receive, and how their language skills, communicative participation, and quality of life change over time.

Lastly, as you have been very active in the field of research and must have a lot of other work besides it, we’d like to know what is your way to cool down from work.

Do you have any non-work-related passion projects, hobbies, or goals to achieve?

Although I try to guard my limits, research has its way of intruding on one’s free time. And indeed, I also let it, and often enjoy it! But that said, cooling down and doing other things is important. And there are a lot of things I enjoy doing to relax! Spending time with my family and friends is of course one and seeking wild-life experiences is another. I love to go hiking and cross-country skiing in the Norwegian mountains, and I run a lot, and most of all enjoy running in new places.

We wish Strömbergsson all the success for the project and future research. In addition, we have listed below some research Strömbergsson has been part of and also the PhD from 2014.

Recent research:

Simulating Speech Error Patterns Across Languages and Different Datasets

Strömbergsson S, Götze J, Edlund J, Nilsson Björkenstam K

Language and speech 2022;65(1):105-142

A survey of Swedish speech-language pathologists’ practices regarding assessment of speech sound disorders

Wikse Barrow C, Körner K, Strömbergsson S

Logopedics, phoniatrics, vocology 2021;():1-12

Canonical babbling ratio-Concurrent and predictive evaluation of the 0.15 criterion

Nyman A, Strombergsson S, Lohmander A

Journal of communication disorders 2021;:106164-

Sofia Strömbergsson’s PhD from 2014:

Children’s perception of their synthetically corrected speech production

Strombergsson S, Wengelin A, House D

Clinical linguistics & phonetics 2014;28(6):373-95

Sources:

https://staff.ki.se/people/sofia-strombergsson

https://nyheter.ki.se/ny-docent-i-logopedi-vid-clintec-0

28/02/202228/02/2022

TEFLON in Tampere

Author: trainee Kamilla Hyytiäinen, Tampere University

Greetings from Tampere!

One of the Finnish research partners of the Nordic Teflon project is based at Tampere University. In this blogpost you will get a brief introduction to the city of Tampere, the university and its various fields of science as well as the local research participants of the Teflon project and their academic achievements.

Nordic cooperation between Sweden and Finland dates back to the Swedish era in the 18th century, when the city of Tampere, among other cities in Finland, was established by King of Sweden Gustav III in 1779, on the bank of The Tammerkoski rapids. Today, Tampere is the third largest city in Finland and the largest inland centre in the Nordic countries with roughly 238 000 inhabitants. Tampere has been an industrial pioneer in Finland since the very beginning and is still the centre of Finnish industry today. Versatile research, education and cooperation between companies and universities have maintained and developed the competitiveness of the region’s industry.

At Tampere University, multidisciplinary research is conducted across the boundaries of fields of science with over 2,800 researchers. The focal areas of research lie in the fields of health, technology and society. In addition to basic research, new fields of research emerge at the university including e.g. gamification, augmented reality and sustainable cities. Multidisciplinarity is well represented in the Teflon project group: we consist of specialists in the field of logopedics, neuroscience, speech technology, language pedagogy and linguistics.

Sari Ylinen has been working as an Associate Professor at Tampere University since August 2021 and she is currently leading the Teflon project group in Tampere. Ylinen has comprehensive experience in project management: she has previously coordinated various research projects funded by Academy of Finland, Business Finland and University of Helsinki. In addition to managing research projects, Ylinen has extensive work experience as a neuroscientist for over a decade and she has specialised in brain plasticity in language learning, among other things. In recent years, she has studied articulation with the help of speech technology and gamification, which she aims to apply to children’s language learning and correcting their pronunciation errors.

Other Teflon project members in Tampere include a language technologist and research assistants. Anna Smolander has worked in Ylinen’s projects for several years as a language technologist developing digital platforms for various learning difficulties. The Teflon Tampere research team has two research assistants, Kamilla Hyytiäinen and Saara Telinkangas, who are eager to familiarise themselves with how international research projects operate. Telinkangas studies Scandinavian languages and politics and is finishing her Bachelor’s degree while Hyytiäinen is an English and Swedish teacher student in the Master’s programme. The main tasks of the assistants include analysing Swedish speech samples produced by Finnish children in order to develop the speech recognition for language learning applications.

We are all looking forward to diving deep into the world of digital language learning with our fellow Nordic partners and aim at scientific breakthroughs together!

Sources:

https://www.tampere.fi/en/city-of-tampere/information-on-tampere.html

https://www.tuni.fi/en/research/research-tampere-university

27/02/202217/12/2022

After the Easter it was time to get started!

The 6th of April was a great day. All the Teflon team members from Finland, Norway and Sweden were meeting for the first time. The spritits were high! Thank you Nordforsk for financing this project!

Planning is a crucial part of such a big project and the first three days we were going through the schedules, plans and cooperation possibilities.

Here’s a quick introduction of the multidisciplinary Teflon team partners:

Mikko Kurimo Aalto University, Finland
Sari Ylinen, University of Helsinki, Finland
Minna Lehtonen, University of Oslo, Norway (Turku, Finland)
Pernille Hansen, Inland Norway University of Applied Sciences, Norway
Giampiero Salvi, Norwegian University of Science and Technology, Norway
Torbjørn Svendsen, Norwegian University of Science and Technology, Norway
Sofia Strömbergsson, Karolinska Institutet, Sweden

Research for a reason

Nordic countries face the challenge of social integration of immigrants to the society. Language learning and especially oral language skills are critical for their integration. Immigrants with poor language skills are at risk of social exclusion, facing difficulties in contacting local people and finding employment. Also immigrant children should learn the language of their environment quickly to adapt to their new environment and to use the language in school and with their peers. The demand for learning Nordic languages as a second language (L2) is high: In 2018, about 132 000 immigrants moved to Sweden (Sweden and migration, 2019), 52 000 to Norway (Statista, 2019) and 31 000 to Finland (Statistics Finland, 2019).

Acquiring proficiency in a new language is challenging for many learners. To speed up learning but to avoid high costs of individual tutoring for the Nordic societies, new techniques of teaching languages suitable for scalable digital applications are needed (Meltzoff et al 2009).

New solutions

Digital applications enable language learning already before immigration. Moreover, digital applications are urgently needed in all learning, as shown by the current coronavirus epidemic outbreak, resulting in closing of schools in many countries. With digital applications, teaching can be provided by means of distance education (i.e., remote learning without physical contact). Such applications are likely to be in increasing demand in the future and many of them may require speech technology that is robust enough for non-native speakers which is currently not available for Nordic languages. While the importance of well-performing speech technology is growing rapidly in all languages, the smart speakers, robots, games and toys are emphasizing the importance of a new user group: the children.

Covid-19 is causing it’s own barriers for this project as well but we truly hope that the recruitments and pilots will be succesful and achievable within the targeted schedule.

The recruitment is active at the moment. Later there will be an introduction post of research groups from each university. We are very excited to see who are the new talent joining our experts of Teflon project.