|
Designing a spoken dialogue system involves many non-trivial decisions. This is in part due to a variety of dialogue phenomena that occur during a dialogue. Being able to detect and handle such phenomena has a big impact on the success of a dialogue system. For example, detecting and handling speech recognition problems is crucial for a dialogue system.
Instead of looking at dialogue phenomena in isolation, my research attempts to understand the inherent interactions that exist between these phenomena. For example, a string of speech recognition problems is likely to result in a frustrated user. Several phenomena are being investigated: speech recognition problems, user affect (e.g. certainty, frustration), user state (e.g. correctness), discourse transitions (e.g. crossing a discourse segment boundary). An empirical approach is being used: statistical dependencies between dialogue phenomena are mined from a corpus of dialogues. Analyses of these dependencies offer additional insights about the dialogue phenomenon and suggests new handling strategies.
The image below summarizes the interactions we explored. Clicking on an arrow will reveal/hide the appropriate reference(s) at the bottom of the image.
| · |
Katherine Forbes-Riley, Mihai Rotaru, Diane J. Litman, Joel Tetreault (2007) “Exploring Affect-Context Dependencies for Adaptive System Development”. In Proceedings of HLT/NAACL 2007 (late-breaking news award).
[abstract]
|
|
| |
We use the Chi Square test to investigate the context dependency of student affect in our computer tutoring dialogues, targeting uncertainty in student answers in 3 automatically monitorable contexts. Our results show significant dependencies between uncertain answers and specific contexts. Identification and analysis of these dependencies is our first step in developing an adaptive version of our dialogue system.
|
|
| · |
Mihai Rotaru (2007) “Applications of Discourse Structure for Spoken Dialogue Systems”. Ph.D. Thesis Proposal, Computer Science Department, University of Pittsburgh.
[abstract]
|
|
| |
Due to the relatively simple structure of dialogues in previous spoken dialogue systems, discourse structure has seen limited applications in these systems. We investigate the utility of discourse structure for spoken dialogue systems in complex domains (e.g. tutoring). Two types of applications are being pursued: on the system side and on the user side. On the system side, we investigate if the discourse structure information is useful for various spoken dialogue system tasks: performance analysis, characterization of user affect and characterization of speech recognition problems. On the user side, we investigate whether the discourse structure information is useful for users of a spoken dialogue system through a graphical representation of the discourse structure.
|
|
|
| · |
Mihai Rotaru and Diane J. Litman (2006) “Dependencies between Student State and Speech Recognition Problems in Spoken Tutoring Dialogues”. In Proceedings of the Joint Conference of the International Committee on Computational Linguistics and the Association for Computational Linguistics (Coling/ACL), Sydney, Australia.
[abstract]
|
|
| |
Speech recognition problems are a reality in current spoken dialogue systems. In order to better understand these phenomena, we study dependencies between speech recognition problems and several higher level dialogue factors that define our notion of student state: frustration/anger, certainty and correctness. We apply Chi Square analysis to a corpus of speech-based computer tutoring dialogues to discover these dependencies both within and across turns. Significant dependencies are combined to produce interesting insights regarding speech recognition problems and to propose new strategies for handling these problems. We also find that tutoring, as a new domain for speech applications, exhibits interesting tradeoffs and new factors to consider for spoken dialogue design.
|
|
|
| · |
Mihai Rotaru, Diane J. Litman, and Katherine Forbes-Riley (2005) “Interactions between Speech Recognition Problems and User Emotions”. In Proceedings of the 9th European Conference on Speech Communication and Technology (Interspeech-2005/Eurospeech), Lisbon, Portugal.
[abstract]
|
|
| |
Understanding how speech recognition problems affect the interaction with the user is a topic of great interest for the spoken dialogue community. In this paper, we examine the dependencies between speech recognition problems in adjacent turns. We also examine the dependencies between speech recognition problems and student emotions within a turn and in adjacent turns. We apply Chi Square analysis to a corpus of speech-based computer tutoring dialogues to discover these dependencies. We find that rejections are followed by more rejections than expected if there was no dependency between rejections, and that misrecognitions are followed by more misrecognitions than expected. We also find a strong dependency between recognition problems in the previous turn and user emotion in the current turn: after a system rejection there are more emotional user turns than expected. Surprisingly, in our data, we find no relationship between user emotions and recognition problems within a turn nor between previous turn user emotions and current turn recognition problems.
|
|
|
| · |
Mihai Rotaru and Diane J. Litman (2006) “Discourse Structure and Speech Recognition Problems”. In Proceedings of Interspeech 2006, Pittsburgh, USA.
[abstract]
|
|
| |
We study dependencies between discourse structure and speech recognition problems (SRP) in a corpus of speech-based computer tutoring dialogues. This analysis can inform us whether there are places in the discourse structure prone to more SRP. We automatically extract the discourse structure by taking advantage of how the tutoring information is encoded in our system. To quantify the discourse structure, we extract two features for each system turn: depth of the turn in the discourse structure and the type of transition from the previous turn to the current turn. The Chi Square test is used to find significant dependencies. We find several interesting interactions which suggest that the discourse structure can play an important role in several dialogue related tasks: automatic detection of SRP and analyzing spoken dialogues systems with a large state space from limited amounts of available data.
|
|
|
|
