EVENT

Event News

ASPIRE Human-Centered AI - UMCS Monthly Talk - Dr. Zhidong Xiao

How can AI understand not only what we say, but also how we feel during conversation?
In this month's UMCS Monthly Talk, we will explore recent advances in real-time emotion recognition, facial expression transfer, and emotion-driven motion generation. Dr. Zhidong Xiao will discuss how multimodal AI can capture and generate expressive human behavior, and how these technologies may contribute to more natural human-computer interaction, sign language technologies, and virtual communication.

Title:

Real-time emotion recognition, facial expression transfer and motion generation in conversation

Presenter:

Dr. Zhidong Xiao (Principal Academic, National Centre for Computer Animation, Bournemouth University, UK)

Abstract:

Recent advances in artificial intelligence have enabled increasingly natural and interactive human-computer communication through the integration of facial expression analysis, speech emotion recognition, and motion generation. This talk brings together three interconnected research projects that explore the progression from real-time emotion recognition and classification to facial expression transfer and emotion-driven motion generation in conversational settings. The presented framework combines marker-less facial expression tracking, multimodal emotion and stress recognition, and emotion-aware motion synthesis using only consumer-grade hardware. Facial expressions are captured through a single webcam, while speech signals are acquired through a microphone. The system employs a robust real-time facial tracking model alongside deep learning-based emotion recognition networks, including CNN-based architectures for facial emotion analysis and BiLSTM-attention models for speech emotion classification. Emotion cues extracted from facial and vocal modalities are further utilized to assess stress levels and support personalized user interactions. Building upon these recognition capabilities, a four-stage audio-to-motion generation pipeline automatically detects emotions from conversational audio streams, eliminating the need for manual emotion annotation, and generates expressive motion sequences accordingly. Together, these projects demonstrate the feasibility of creating real-time, emotionally aware digital humans and interactive avatars for applications in sign language representation, human-computer interaction, virtual communication, entertainment, and game development.

Bio:

Dr. Zhidong Xiao is Co-Programme Leader for MSc Artificial Intelligence for Media, a Fellow of Higher Education Academy (FHEA), a Fellow of the British Computer Society (FBCS), Member of EPSRC, ESRC and EDI Hub+ Peer Review College. He has over 10 years leadership experience at HEI, and had worked as Head of Education, Deputy Head of Department at National Centre for Computer Animation, Faculty of Media, Science and Technology, Bournemouth university. His research interests are in the areas of Computer Graphics, Computer Animation, Motion Capture, Artificial Intelligence, Machine Learning, Motion Synthesis, Physics-based Simulation, Sign language Recognition, Virtual Reality and Robotics. He has supervised seven PhD students to successful completion. Working as PI and Co-I, he had successfully secured and completed a few research council funded projects and commercial projects for industry partners.