2012 An Intonation Clinic for Second Language Learners

In this talk I present a project proposal on a novel Computer-Aided Pronunciation Training (CAPT) system that provides corrective feedback on intonation to second language learners. The system can be used to learn the correct placement of lexical stress, but also the correct intonation for a polite request in a foreign language.

Two novelties are introduced with respect to current (experimental) systems. First, the software makes decisions on the adequacy of the learner's input (correct/wrong) based on statistical models built on a corpus of utterances spoken by native speakers. Second, corrective feedback is provided in the form of re-synthesized speech: Either the learner's wrong intonational pattern (F0 contour and segment durations) is corrected and played back in the learner's own voice, or a reference (teacher) voice is manipulated to imitate the learner's intonation mistake. In either case, the advantage is that the learner does not need to isolate his/her intonation mistake by comparing his/her own utterance with the teacher's realization, which involves several distracting dimensions of variation like phonetics, voice quality and rhythm.

Both decision making and speech manipulation are based on a modern statistical methodology called Functional Data Analysis, which allows one to model F0 contours and segment durations automatically and without introducing unwarranted simplifications (e.g. no F0 contour stylization).