Brief introduction of the challenge
Learning the pronunciation of foreign or second language (L2) requires a lot of practise and accurate feedback. Mobile apps that have automatic pronunciation assessment (APA) technology let learners practise pronunciation at their own time, place and pace.
Challenges: The developing and implementing APA has several challenges. First and worst of all is the lack of speech data for L2 learners that would be annotated for pronunciation accuracy. This is particularly the case of children and the learners of low-resource target languages. Second, if such data were available, it is usually heavily unbalanced for different skill levels and the provided reference scores suffer from noise and inter-annotator disagreement. Finally, to make a useful app, the scoring has to happen in seconds to provide real-time feedback with minimal delay to encourage a lot of repetitions.
Data: Recordings of children of 5 – 12 years repeating the words in Norwegian that were played to them. The children include L1 speakers, beginner learners of L2 Norwegian and children who had no previous exposure to Norwegian. For each word in the data we provide the correct orthographic transcription and the speech accuracy assessment score of 1 – 5 given by human experts. For more details, see [1].
Task: Develop an automatic speech assessment system and use it to predict the score of each utterance in the given test data.
Please contact mikko.kurimo@aalto.fi for more information.
Data sets to be used and, if applicable, planned baseline systems
Dataset: TBA
Baselines: TBA
Rules for participation
TBA
Relevant dates
- Team registration starts: March 19 (planned).
- Training and test data released: March 19 (planned).
- Evaluation system opens: April 2.
- Baseline paper (and code) released: April 27.
- Paper submission deadline: May 20.
- Evaluation system closes, and the leaderboard is public: May 20.
- Post-Challenge evaluation phase: from May 20 until July 15.
- Notification of paper acceptance: June 24.
- Camera-ready upload July 15.
References
[1] Anne Marte Haug Olstad, Anna Smolander, Sofia Strömbergsson, Sari Ylinen, Minna Lehtonen, Mikko Kurimo, Yaroslav Getman, Tamás Grósz, Xinwei Cao, Torbjørn Svendsen, and Giampiero Salvi. 2024. Collecting Linguistic Resources for Assessing Children’s Pronunciation of Nordic Languages. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 3529–3537, Torino, Italia. ELRA and ICCL.