CS 182Sections 103 - 104slide credit toEva Mok and Joe MakinUpdated by Leon BarrettApril 25, 2007Announcements•a9 due Tuesday, May 1st, in class•final exam Tuesday May 8th in class• final paper due Friday May 11th, 11:59pm•final review sometime next weekSchedule•Last Week– Constructions, ECG•This Week– Models of language learning– Embodied Construction Grammar learning•Next Week– Open lecture– Wrap-UpQuestions1. Why is learning language difficult?2. What are the “paucity of the stimulus” and the “opulence of the substrate”?3. What is Gold's Theorem?4. How does the analyzer use the constructions to parse a sentence?5. How can we learn new ECG constructions?6. What are ways to re-organize and consolidate the current grammar?7. What metric is used to determine when to form a new construction?Difficulty of learning language•What makes learning language difficult?– How many sentences do children hear?– How often are those sentences even correct?– Even when they're correct, how often are they complete?– How often are they corrected when saying something wrong?– How many possible languages are there?Larger context•War!– Is language innate?– Covered in bookQuestions1. Why is learning language difficult?2. What are the “paucity of the stimulus” and the “opulence of the substrate”?3. What is Gold's Theorem?4. How does the analyzer use the constructions to parse a sentence?5. How can we learn new ECG constructions?6. What are ways to re-organize and consolidate the current grammar?7. What metric is used to determine when to form a new construction?Paucity and Opulence•Poverty of the stimulus– Coined to suggest how little information children have to learn from• Opulence of the substrate– Opulence = “richness”– Coined in response to suggest how much background information children haveQuestions1. Why is learning language difficult?2. What are the “paucity of the stimulus” and the “opulence of the substrate”?3. What is Gold's Theorem?4. How does the analyzer use the constructions to parse a sentence?5. How can we learn new ECG constructions?6. What are ways to re-organize and consolidate the current grammar?7. What metric is used to determine when to form a new construction?Gold's Theorem•Suppose that you have an infinite number of languages– language = “set of legal sentences”•Suppose that for every language Ln, there is a bigger language Ln+1– makes every sentence, and then some• There's some language Linfinity– contains all the sentences in all other grammars• You can arrange data so that no one ever learns Linfinity– http://www.lps.uci.edu/~johnsonk/Publications/Johnson.GoldsTheorem.pdfQuestions1. Why is learning language difficult?2. What are the “paucity of the stimulus” and the “opulence of the substrate”?3. What is Gold's Theorem?4. How does the analyzer use the constructions to parse a sentence?5. How can we learn new ECG constructions?6. What are ways to re-organize and consolidate the current grammar?7. What metric is used to determine when to form a new construction?“you”youschema Addresseesubcase of HumanFORM (sound)MEANING (stuff)Analyzing “You Throw The Ball”“throw”throwschema Throwroles:throwerthrowee“ball”ballschema Ballsubcase of Object“block”blockschema Blocksubcase of Objectt1 before t2t2 before t3Thrower-Throw-Objectt2.thrower ↔ t1t2.throwee ↔ t3“the”AddresseeThrowthrowerthroweeBallAnother way to think of the SemSpecAnalyzing in ECGcreate a recognizer for each construction in the grammarfor each level j (in ascending order)repeatfor each recognizer r in jfor each position p of utteranceinitiate r starting at puntil we don't find anything newRecognizer for the Transitive-Cn• an example of a level-1 construction is Red-Ball-Cn• each recognizer looks for its constituents in order (the ordering constraints on the constituents can be a partial ordering)agt v objI get cuplevel 0level 2level 1Constructions(Utterance, Situation)1. Learner passes input (Utterance + Situation) and current grammar to Analyzer.AnalyzeSemantic Specification,Constructional Analysis2. Analyzer produces SemSpec and Constructional Analysis.1. Learner updates grammar:Hypothesizea. Hypothesize new map.Reorganizeb. Reorganize grammar (merge or compose).c. Reinforce(based on usage).Learning-Analysis Cycle (Chang, 2004)Questions1. Why is learning language difficult?2. What are the “paucity of the stimulus” and the “opulence of the substrate”?3. What is Gold's Theorem?4. How does the analyzer use the constructions to parse a sentence?5. How can we learn new ECG constructions?6. What are ways to re-organize and consolidate the current grammar?7. What metric is used to determine when to form a new construction?AcquisitionReorganizeHypothesizeProductionUtterance(Comm. Intent, Situation)GenerateConstructions(Utterance, Situation)AnalysisComprehensionAnalyzePartialUsage-based Language LearningBasic Learning Idea• The learner’s current grammar produces a certain analysis for an input sentence• The context contains richer information (e.g. bindings) that are unaccounted for in the analysis• Find a way to account for these meaning relations (by looking for corresponding form relations)SemSpecr1r2r1r2r3r1r2Contextr1r2r1r2r3r1r2“you”“throw”“ball”youthrowball“block”blockschema Addresseesubcase of HumanFORM (sound)MEANING (stuff)lexical constructionsInitial Single-Word Stageschema Throwroles:throwerthroweeschema Ballsubcase of Objectschema Blocksubcase of Object“you”youschema Addresseesubcase of HumanFORM MEANINGNew Data: “You Throw The Ball”“throw”throwschema Throwroles:throwerthrowee“ball”ballschema Ballsubcase of Object“block”blockschema Blocksubcase of Object“the”AddresseeThrowthrowerthroweeBallSelfSITUATIONAddresseeThrowthrowerthroweeBallbeforerole-fillerthrow-ballRelational Mapping ScenariosAfBfAmBmABform-relationrole-fillerthrow ball throw.throwee ↔ ballAfBfAmBmABform-relationrole-fillerXrole-fillerput ball down put.mover ↔ balldown.tr ball↔AfBfAmBmABform-relationrole-fillerYrole-fillerNomi ball possession.possessor ↔ Nomipossession.possessed ball↔Questions1. Why is learning language difficult?2. What are the “paucity of the stimulus” and the “opulence of the substrate”?3. What is Gold's Theorem?4. How does the analyzer use the constructions to parse a sentence?5. How can we learn new ECG