6.863J Natural Language ProcessingLecture 21: Language Acquisition Part 1Robert C. [email protected]/9.611J Lecture 21 Sp03The Menu Bar• Administrivia:• Project check• The Twain test & the Gold Standard• The Logical problem of language acquisition: the Gold theorem results• How can (human) languages be learned?• The logical problem of language acquisition• What is the problem• A framework for analyzing it6.863J/9.611J Lecture 21 Sp03The Twain test• Parents spend….6.863J/9.611J Lecture 21 Sp03The Logical problem of language acquisitionInitialConstraints& learningmethod“Correct”target languagenoiseAll PossibleHuman languagesInput DataTargetlanguageInput DataInput DataInput Data6.863J/9.611J Lecture 21 Sp03The problem• From finite data, induce infinite set• How is this possible, given limited time & computation?• Children are not told grammar rules• Ans: put constraints on class of possible grammars (or languages)6.863J/9.611J Lecture 21 Sp03The logical problem of language acquisition• Statistical MT: how many parameters? How much data?• “There’s no data like more data”• Number of parameters to estimate in Stat MT system -6.863J/9.611J Lecture 21 Sp03The logical problem of language acquisition• Input does not uniquely specify the grammar (however you want to represent it) = Poverty of the Stimulus (POS)• Paradox 1: children grow up to speak language of their caretakers• Proposed solution: target choice of candidate grammars is restricted set• This is the theory of Universal Grammar (UG)• (Paradox 2: why does language evolve?)6.863J/9.611J Lecture 21 Sp03The illogical problem of language changeLangagis, whos reulis ben not writen as benEnglisch, Frensch and many otheres, ben channgidwithynne yeeris and countrees thatoon man of theoon cuntre, and of the oon tyme, myghte not, orschulde not kunne undirstonde a man of the otherekuntre, and of the othere tyme; and al for this, thatthe seid langagis ben not stabili and fondamentaliwritenPecock (1454) Book of Feith6.863J/9.611J Lecture 21 Sp03Information needed• Roughly: sum of given info + new info (data) has to pick out right language (grammar)• If we all spoke just 1 language – nothing to decide – no data needed• If just spoke 2 languages (eg, Japanese, English), differing in just 1 bit, 1 piece of data needed• What about the general case?6.863J/9.611J Lecture 21 Sp03Can memorization suffice?• Can a big enough table work?• Which is largest, a <noun>-1, a <noun>-2, a <noun>-3, a <noun>-4, a <noun>-5, a <noun>-6, or a <noun>-7?• Assume 100 animals• # queries = 100 x 99 x …94 = 8 x 1013• 2 queries/line, 275 lines, 1000 pages inch =• How big?6.863J/9.611J Lecture 21 Sp03153m6.863J/9.611J Lecture 21 Sp03The inductive puzzle• Unsupervised learning• (Very) small sample complexity• 1—5 examples; no Wall Street Journal subscriptions• The Burst effect• Order of presentation of examples doesn’t matter• External statistics don’t match maturational time course6.863J/9.611J Lecture 21 Sp03The burst effecttwo-3 words,ages 1;1–1;11“full” language(some residual)? time span: 2 weeks–2months1;10 ride papa's neck 2;1.2 you watch me 1;10.3 this my rock-baby open sandbox1;11.2 papa forget this 2;1.3 papa, you like this song?2;4.0 I won't cry if mama wash my hair2;4.3 put this right here soI see it betterWhat’s the difference?1. I see red one2. P. want drink3. P. open door4. P. tickle S.5. I go beach6. P. forget this7. P. said no8. P. want out9. You play chickenMultiple choice:(a) Pidgin speakers; (b) apes;(c) feral child Genie; (d) ordinary children6.863J/9.611J Lecture 21 Sp03Answers1,5,9 = pidgin2,4,8 = apes7= Genie3,6= children6.863J/9.611J Lecture 21 Sp03Challenge: tension headachesWarlpiriChineseGerman…(4500 others)• Essentially error-free• Minimal triggering (+ examples)• Robust under noise• Still variable enough??6.863J/9.611J Lecture 21 Sp03The input…Bob just went away .Bob went away . no he went back to school . he went to work . are you playing with the plate ?where is the plate ?you 're going to put the plate on the wall ?let's put the plate on the table .the car is on your leg ?you 're putting the car on your leg ?on your other leg .that's a car.woom ? oh you mean voom . the car goes voom .cars are on the road ?thank you . the cow goes moo ?what color is the cow ?what color is the cow ?what color is the cow ?what color 6.863J/9.611J Lecture 21 Sp03A developmental puzzle• If pure inductive learning, then based on pattern distribution in the input• What’s the pattern distribution in the input?• English subjects: most English sentences overt • French: only 7-8% of French sentences have inflected verb followed by negation/adverb (“Jeanembrasse souvent/pas Marie”)• Dutch: no Verb first S’s; Obj Verb Subject trigger appears in only 2% of the cases, yet…6.863J/9.611J Lecture 21 Sp03Predictions from pure induction• English obligatory subject should be acquired early• French verb placement should be acquired late• Dutch verb first shouldn’t be produced at all – because it’s not very evident in the input6.863J/9.611J Lecture 21 Sp03The empirical evidence runs completely contrary to this…• English: Subjects acquired late (Brown, Bellugi, Bloom, Hyams…), but Subjects appear virtually 100% uniformly• French: Verb placement acquired as early as it is possible to detect (Pierce, others), but triggers don’t occur very frequently• Dutch: 40-50% Verb first sentences produced by kids, but 0 % in input (Klahsen)• So: what are we doing wrong?6.863J/9.611J Lecture 21 Sp03Can’t just be statistical regularities…acquisition time course doesn’t matchEnglish Subject use(“there” sentences)1% in Childesdiffuseand sparseregularities{French verbraising 7-8%(VfinAdv/pas)Dutch verb secondOVS 2%; 0 V1 pats20m. 2.5y% corrrect (adult) usestable inferencesand rapid time course6.863J/9.611J Lecture 21 Sp03The language identification game• black sheep• baa baa black sheep• baa black sheep• baa baa baa baa black sheep…6.863J/9.611J Lecture 21 Sp03The factsChild: Nobody don’t like me.Mother: No, say “Nobody likes me.”Child: Nobody don’t like me.Mother: No, say “Nobody likes me.”Child: Nobody don’t like me.Mother: No, say “Nobody likes me.”Child: Nobody don’t like me.[dialogue repeated five more times]Mother: Now listen
View Full Document
Unlocking...