DOC PREVIEW
Combining Bayesian Networks and Formal Reasoning for Semantic Classification of Student Utterances

This preview shows page 1-2-3 out of 8 pages.

Save
View full document
View full document
Premium Document
Do you want full access? Go Premium and unlock all 8 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 8 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 8 pages.
Access to all documents
Download any document
Ad free experience
Premium Document
Do you want full access? Go Premium and unlock all 8 pages.
Access to all documents
Download any document
Ad free experience

Unformatted text preview:

Combining Bayesian Networks andFormal Reasoning for SemanticClassification of Student UtterancesMaxim Makatcheva,1and Kurt VanLehnbaThe Robotics Institute, Carnegie Mellon UniversitybLearning Research and Development Center, University of PittsburghAbstract. We describe a combination of a statistical and symbolic approaches forautomated scoring of student utterances according to their semantic content. Theproposed semantic classifier overcomes the limitations of bag-of-words methods bymapping natural language sentences into predicate representations and matchingthem against the automatically generated deductive closure of the domain givens,buggy assumptions and domain rules. With the goal to account for uncertainties inboth symbolic representations of natural language sentences and logical relationsbetween domain statements, this work extends the deterministic symbolic approachby augmenting the deductive closure graph structure with conditional probabilities,thus creating a Bayesian network. By deriving the structure of the network for-mally, instead of estimating it from data, we alleviate the problem of sparseness oftraining data. We compare the performance of the Bayesian network classifier withthe deterministic graph matching-based classifiers and baselines.Keywords. Dialogue-based intelligent tutoring systems, Bayesian networks,formal methods, semantic classification1. IntroductionModern intelligent tutoring systems attempt to explore relatively unconstrained interac-tions with students, for example via a natural language (NL) dialogue. The rationale be-hind this is that allowing students to provide unrestricted input to a system would triggermeta-cognitive processes that support learning (i.e. self-explaining) [1] and help exposemisconceptions. WHY2-ATLAS tutoring system is designed to elicit NL explanations inthe domain of qualitative physics [7]. The system presents a student a qualitative physicsproblem and asks the student to type an essay with an answer and an explanation. A typ-ical problem and the corresponding essay are shown in Figure 1. After the student sub-mits the first draft of an essay, the system analyzes it for errors and missing statementsand starts a dialogue that attempts to remediate misconceptions and elicit missing facts.Although there are a limited number of classes of possible student beliefs that are ofinterest to the system (e. g., for the Pumpkin problem, about 20 correct and 4 incorrect1Correspondence to: Maxim Makatchev, The Robotics Institute, Carnegie Mellon University, 5000Forbes Ave., Pittsburgh, PA, 15213, USA. Tel.: +1 412 268 3474; Fax: +1 412 624 7904; E-mail:[email protected]: Suppose you are running in a straight line at constant speed. You throw a pumpkinstraight up. Where will it land? Explain.Explanation: Once the pumpkin leaves my hand, the horizontal force that I am exerting on it nolonger exists, only a vertical force (caused by my throwing it). As it reaches it’s maximum height,gravity (exerted vertically downward) will cause the pumpkin to fall. Since no horizontal forceacted on the pumpkin from the time it left my hand, it will fall at the same place where it left myhands.Figure 1. The statement of the problem and a verbatim explanation from a student who received no follow-updiscussions on any problems.beliefs), for each class there are multiple examples of NL utterances that are semanticallyclose enough to be classified as representatives of one of these classes by an expert. Typ-ically the expert will classify a statement belonging to a certain class of student beliefsif either (1) the statement is a re-phrasal of the canonical textual description of the beliefclass, or (2) the statement is a consequence (or, more rarely, a condition) of an inferencerule involving the belief. An example of the first case is the sentence “pumpkin has nohorizontal acceleration” as a representative of the belief class “the horizontal accelera-tion of the pumpkin is zero.” An example of the second case is the sentence “The hor-izontal component of the pumpkin’s velocity will remain identical to that of the man’sthroughout” as a representative of the belief class “The horizontal average velocities ofthe pumpkin and man are equal”: the letter can be derived in one step from the formervia a domain rule. The second case occurs due to the coarseness of the classes: the expertwould like to credit the student’s answer despite the fact that it doesn’t match any of thepre-specified semantic classes, based on its semantic proximity to the nearby semanticclasses. To summarize, utterances assigned to a particular semantic class by an expertcan have different syntactic and semantic features.While bag-of-words methods have seen some successful applications to the problemsof semantic text classification [4], their straightforward implementations are known toperform weakly when the training data is sparse, the number of classes is large, or classesdo not have clear syntactic boundaries1[6]. This suggests using syntactic and semanticparsers and other NLP methods to convert the NL sentences into symbolic representa-tions in a first-order predicate language [7]. However, uncertainty inherent in the variousNLP methods that generate those representations [5] means that the same utterances canproduce representations of different quality and structure. Figure 2, for example, showstwo representations for the same utterance, produced by different NLP methods.The uncertainty in parsing and in other NLP components adds to the syntactic andsemantic variability among representatives of a semantic class. Encoding these sourcesof variabilities explicitly appears to be infeasible, especially since the properties of theNLP components may be difficult to describe, and the reasoning behind an expert as-signing a semantic class label to an input may be hard to elicit. A method to combinedifferent sources of uncertainty in an NLP pipeline using a Bayesian network has beenproposed in [3]. Our objective is to adapt this idea to the scenario when semantic classesthemselves, as well as relationships between them are uncertain. In particular, we would1Syntactic features alone become insufficient for classification when classes depend on the semantic struc-ture of the domain (as described in the previous paragraph), including the sensitivity to conditionals and nega-tion.Representation I:(position pumpkin ...)(rel-position ...pumpkin


Combining Bayesian Networks and Formal Reasoning for Semantic Classification of Student Utterances

Download Combining Bayesian Networks and Formal Reasoning for Semantic Classification of Student Utterances
Our administrator received your request to download this document. We will send you the file to your email shortly.
Loading Unlocking...
Login

Join to view Combining Bayesian Networks and Formal Reasoning for Semantic Classification of Student Utterances and access 3M+ class-specific study document.

or
We will never post anything without your permission.
Don't have an account?
Sign Up

Join to view Combining Bayesian Networks and Formal Reasoning for Semantic Classification of Student Utterances 2 2 and access 3M+ class-specific study document.

or

By creating an account you agree to our Privacy Policy and Terms Of Use

Already a member?