||We examine the possibility of predicting the occurrence of latent misunderstandings
in psychotherapeutic conversations from gestural patterns.
Although most misunderstandings occurring in conversations can be easily
detected from the content of superficial expressions in utterances, there are
some misunderstandings that are not immediately evident from the verbal
interaction. Such latent misunderstandings can only be noticed when we
observe the conversation carefully; thus, they may be overlooked while
the therapy is being carried out. However, if we observe the conversation
multimodally, there will be some cases where we can find latent misunderstandings
immediately even though the content of the utterance does not
contain any clues of misunderstanding. Motivated by this assumption, we
investigated the applicability of machine learning as a means of detecting
misunderstanding from gestural patterns observed in the conversations.
We constructed classifiers using different features taken from the gesture
data. In the process, we identified the gesture features that are useful
in terms of classification accuracy. Then, by using the extracted gesture
features, we predicted misunderstandings that were not immediately observable
from the verbal queues. In an experiment using gesture features,
we found a few misunderstandings that could only be discovered by careful
observation of the utterance context.