AboutPublicationsNewsPeopleProjectsSponsors |
About:Spoken dialogue is a natural and highly desirable form of student-computer interaction, which provides both opportunities and challenges to both dialogue-based tutoring systems, and to spoken language systems. The goal of this research is to wed spoken language technology with instructional technology, in order to promote learning gains by enhancing communication richness. For further details, see the project descriptions and papers below, as well as information on the tutoring back-end (Why2, a text-based tutoring system in the domain of qualitative physics).
Publications:News:
People:
Research AssociatesPhD StudentsUndergraduate StudentsProgrammerAlumniPast Visitors
Currently Funded Projects:Tutoring Scientific Explanations via Natural Language Dialogue (project homepage) (Jan. 2004 - Dec. 2007)It is widely acknowledged, both in academic studies and the marketplace, that the most effective form of education is the professional human tutor. A major difference between human tutors and computer tutors is that only human tutors understand unconstrained natural language input. Recently, a few tutoring systems have been developed that carry on a natural language (NL) dialogue with students. Our research problem is to find ways to make NL-based tutoring systems more effective. Our basic approach is to derive new dialogue strategies from studies of human tutorial dialogues, incorporate them in an NL-based tutoring system, and determine if they make the tutoring system more effective. For instance, some studies are determining if learning increases when human tutors are constrained to follow certain strategies. In order to incorporate the new dialogue strategies into our existing text and spoken NL-based tutoring systems, two completely new modules are being developed. One new module will interpret student utterances using a large directed graph of propositions called an explanation network, which is halfway between the shallow and deep representations of knowledge that are currently used. The second new module uses machine learning to improve the selection of dialogue management strategies. The research is thus a multidisciplinary effort whose intellectual merit lies in new results in the cognitive psychology of human tutoring, in the technology of NL processing, and in the design of effective tutoring systems. Improved NL-based tutoring systems could have a broad impact on education and society. This grant is in collaboration with Kurt VanLehn, Micheline Chi, and Pamela Jordan at the Learning Research and Development Center, University of Pittsburgh, and with Carolyn Rose (now at CMU). Adapting to Student Uncertainty over and above Correctness in A Spoken Tutoring Dialogue System (Sep. 2006 - Aug. 2009)This research investigates whether responding to student uncertainty over and above correctness improves learning during computer tutoring. The investigation is performed in the context of a spoken dialogue tutoring system, where student speech provides many linguistic cues (e.g. intonation, pausing, word usage) that computational linguistics research suggests can be used to detect uncertainty. Intelligent tutoring systems research suggests that uncertainty is part of the learning process, and has hypothesized that to increase system effectiveness, it is critical to respond to more than correctness. However, most existing tutoring systems respond only to student correctness, and few controlled experiments have yet investigated whether also responding to uncertainty can improve learning. This research designs and implements two different enhancements to the spoken dialogue tutoring system, to test two hypotheses in the tutoring literature concerning how tutors can effectively respond to uncertainty over and above correctness. The first hypothesis is that student uncertainty and incorrectness both represent learning impasses, i.e., opportunities to improve understanding. This hypothesis is addressed with an enhanced system version that treats uncertainty in the same way that incorrectness is currently treated (i.e., with additional subdialogue to increase understanding). The second hypothesis is that more optimal responses can be developed by modeling how human tutor responses to correctness change when the student is uncertain. This hypothesis is addressed by analyzing human tutor dialogue act responses (i.e. content and presentation) to student uncertainty over and above correctness in an existing tutoring corpus, then implementing these responses in a second enhanced system version. Two controlled experiments are then performed. The first tests the relative impact of the two adaptations on learning using a Wizard of Oz version of the system, with a human (Wizard) detecting uncertainty and performing speech recognition and language understanding. The second experiment tests the impact of the best-performing adaptation from the first experiment in the context of the real system, with the system processing the speech and language and detecting uncertainty in a fully automated manner. The major intellectual contribution of the research is to demonstrate whether significant improvements in learning are achieved by adapting to student uncertainty over and above correctness during tutoring, to advance the state of the art by fully automating and evaluating user uncertainty detection and adaptation in a working spoken dialogue system, and to investigate any different effects of this adaptation under ideal versus actual system conditions. Cohesion in Tutorial Dialogue and its Impact on Learning (Oct. 2006 - Sep. 2009)Research on the factors that make one-on-one tutoring a very effective mode of instruction has converged on an important finding: that the critical term in "tutorial interaction" is "interaction." That is, what the tutor says or does during tutoring, and what the student says or does are less important than the dynamic, coordinated interplay between their dialogue turns. It is now important to identify the discourse mechanisms that drive highly interactive human tutoring, so that these mechanisms can be simulated by natural-language dialogue engines in intelligent tutoring systems (ITSs). In the first stage of this project, we will analyze a corpus of naturalistic tutorial dialogues to accomplish this goal. Specifically, we will identify the mechanisms that achieve cohesion in tutorial dialogues, since highly interactive tutorial dialogue is intrinsically highly cohesive. In the second stage, we will run a series of controlled studies to test the hypothesis that more highly cohesive tutorial dialogue is more effective for promoting learning than less cohesive dialogue, and to assess the effectiveness of a few selected mechanisms of cohesion. Finally, in the third stage of the project, we will explore the extent to which database tools developed by the computational linguistics community (e.g., WordNet and FrameNet) can automatically tag cohesion in tutorial dialogue. We will also extend these tools and develop algorithms that will allow them to be used to automatically generate cohesive tutor turns for a small sample of student turns, as a first step towards developing a natural-language dialogue engine that can use these tools to generate highly cohesive tutorial dialogue. This grant is in collaboration with Sandra Katz, LRDC. Completed Projects:Adding Spoken Language to a Text-Based Dialogue Tutor (Nov. 2003 - Sep. 2006)The goal of this research is to generate an empirically-based understanding of the ramifications of adding spoken language capabilities to text-based dialogue tutors, and to understand how these implications might differ in human-human and human-computer spoken interactions. This research will explore the relative effectiveness of speech versus text-based tutoring in the context of ITSPOKE, a speech-based dialogue system that uses a text-based system for tutoring conceptual physics (VanLehn et al, 2002) as its ``back-end.'' The results of this work will demonstrate whether spoken dialogues yield increased performance compared to text with respect to a variety of evaluation measures, whether the same or different student and tutor behaviors correlate with learning gains in speech and text, and how such findings generalize both across and within human and computer tutoring conditions. These results will impact the development of future dialogue tutoring systems incorporating speech, by highlighting the performance gains that can be expected, and the requirements for achieving such gains. TuTalk: Infrastructure for authoring and experimenting with natural language dialogue in tutoring systems and learning research (project homepage) (Dec. 2004- Nov. 2006)The focus of our proposed work is to provide an infrastructure that will allow learning researchers to study dialogue in new ways and for educational technology researchers to quickly build dialogue based help systems for their tutoring systems. Most tutorial dialogue systems that to date have undergone successful evaluations (CIRCSIM, AutoTutor, WHY-Atlas, the Geometry Explanation Tutor) represent development efforts of many man-years. These systems were instrumental in pushing the technology forward and in proving that tutorial dialogue systems are feasible and useful in realistic educational contexts, although not always provably better on a pedagogical level than the more challenging alternatives to which they have been compared. We are now entering a new phase in which we as a research community must not only continue to improve the effectiveness of basic tutorial dialogue technology but also provide tools that support investigating the effective use of dialogue as a learning intervention as well as application of tutorial dialogue systems by those who are not dialogue system researchers. We propose to develop a community resource to address all three of these problems on a grand scale, building upon our prior work developing both basic dialogue technology and tools for rapid development of running dialogue systems. This grant is led by Pamela Jordan at the University of Pittsburgh and Carolyn Rose at Carnegie Mellon University.Does Treating Student Uncertainty as a Learning Impass Improve Learning in Spoken Dialogue Tutoring (Oct. 2006 - May 2007)Most existing tutoring systems respond based only on the correctness of student answers. Although the tutoring community has shown that incorrectness and uncertainty both represent learning impasses (and thus opportunities to learn), and has also shown correlations between uncertainty and learning, to date there have been very few controlled experiments investigating whether system responses to student uncertainty improve learning. We thus propose a small controlled study to test whether this hypothesis holds true, under "ideal" system conditions. The study uses a Wizard of Oz (WOZ) version of a qualitative physics spoken dialogue tutoring system, where the human Wizard performs speech recognition, natural language understanding, and recognition of uncertainty, for each student answer. In the experimental condition, the Wizard then tells the system that correct but uncertain answers are incorrect, causing the system to respond to both uncertain and incorrect student answers in the same way, namely with further dialogue, thereby reinforcing the student's understanding of the principle(s) under discussion. In the first control condition, the system responds only to incorrect student answers in this way. In the second control condition, the system responds to a percentage of correct answers in this way, to control for the additional tutoring in the experimental condition. Monitoring Student State in Tutorial Spoken Dialogue (Sep. 2003 - Aug. 2007)This research investigates the feasibility and utility of monitoring student emotions in spoken dialogue tutorial systems. While human tutors respond to both the content of student utterances and underlying perceived emotions, most tutorial dialogue systems cannot detect student emotions, and furthermore are text-based, which may limit their success at emotion prediction. While there has been increasing interest in identifying problematic emotions (e.g. frustration, anger) in spoken dialogue applications such as call centers, little work has addressed the tutorial domain. The PIs are investigating the use of lexical, syntactic, dialogue, prosodic and acoustic cues to enable a computer tutor to automatically predict and respond to student emotions. The research is being performed in the context of ITSPOKE, a speech-based tutoring dialogue system for conceptual physics. The PIs are recording students interacting with ITSPOKE, manually annotating student emotions in these as well as in human-human dialogues, identifying linguistic and paralinguistic cues to the annotations, and using machine learning to predict emotions from potential cues. The PIs are then deriving strategies for adapting the system's tutoring based upon emotion identification. The major scientific contribution will be an understanding of whether cues available to spoken dialogue systems can be used to predict emotion, and ultimately to improve tutoring performance. The results will be of value to other applications that can benefit from monitoring emotional speech. Progress towards closing the performance gap between human tutors and current machine tutors will also expand the usefulness of current computer tutors. This grant is in collaboration with Julia Hirschberg and her group at Columbia University.
Current and Past Sponsors:
|
