EVENT

Event News

ASPIRE Human-Centered AI - UMCS Monthly Talk

Starting this month, in connection with ASPIRE Human-Centered AI, we will hold the following Monthly Talk series.

Title:

Advancing Turn-Taking Models for Dialogue AI: From Interpretable End-to-End Processing to Sign Language

Presenter:

Koji Inoue (Kyoto University, Japan)(Co-PI & ECR of UMCS)

Abstract:

0Turn-taking serves as a fundamental element not only for analyzing human-human dialogue but also for realizing natural dialogue control in spoken dialogue AI and robots. Conventionally, predicting the next speaker at the end of an utterance has been formulated as a binary classification problem: "hold" (speaker continuation) versus "shift" (speaker change). Consequently, various prediction models have been proposed utilizing prosodic and linguistic features derived from findings in conversation analysis. However, even in the current era dominated by Large Language Models (LLMs), practical systems still widely rely on simple rule-based methods, such as waiting for a fixed duration of silence (e.g., two seconds). This vividly illustrates the inherent difficulty of naturally initiating and maintaining a dialogue in real-world environments. This talk will review our previous research trajectory aimed at real-world applications, such as dialogue robots, and will elaborate on our current focus: a Transformer-based end-to-end turn-taking prediction model. While end-to-end models often raise concerns about their "black-box" nature, we have demonstrated that introducing appropriate data and annotation strategies enhances the resolution of interpretability regarding the model's behavior. Furthermore, we show that this model can be extended to serve as a foundational architecture for generating other non-linguistic behaviors, such as backchannels and head nodding. Finally, by applying our existing turn-taking research framework, we will discuss the potential for constructing turn-taking models in sign language dialogues, along with future prospects in our ASPIRE collaboration project.

Bio:

Koji Inoue received his M.S. and Ph.D. degrees from the Graduate School of Informatics, Kyoto University, Japan, in 2015 and 2018, respectively. He is currently an Assistant Professor at Kyoto University and a Specially Appointed Assistant Professor at Osaka University, both in Japan. In 2023, he was a Visiting Researcher at KTH Royal Institute of Technology in Sweden. His research interests include spoken dialogue systems, conversational robots, and multimodal interaction. His research team has developed a spoken dialogue system for the android ERICA. A series of his works was presented as a keynote talk at SIGDIAL 2024. He was the recipient of the NETEXPLO Innovation Award in 2022.