DOC PREVIEW
MIT 6 863J - Language Acquisition

This preview shows page 1-2-3-4-5-6-7-49-50-51-52-53-54-55-98-99-100-101-102-103-104 out of 104 pages.

Save
View full document
View full document
Premium Document
Do you want full access? Go Premium and unlock all 104 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 104 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 104 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 104 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 104 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 104 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 104 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 104 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 104 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 104 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 104 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 104 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 104 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 104 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 104 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 104 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 104 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 104 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 104 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 104 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 104 pages.
Access to all documents
Download any document
Ad free experience
Premium Document
Do you want full access? Go Premium and unlock all 104 pages.
Access to all documents
Download any document
Ad free experience

Unformatted text preview:

6.863J Natural Language Processing Lecture 21: Language Acquisition Part 1The Menu BarThe Twain testThe Logical problem of language acquisitionThe problemThe logical problem of language acquisitionSlide 7The illogical problem of language changeInformation neededCan memorization suffice?PowerPoint PresentationThe inductive puzzleThe burst effectWhat’s the difference?AnswersChallenge: tension headachesThe input…A developmental puzzlePredictions from pure inductionThe empirical evidence runs completely contrary to this…Can’t just be statistical regularities…acquisition time course doesn’t matchThe language identification gameThe factsBrown & Hanlon, 1970The problem…But kids do recover (well, almost)OvergeneralizationPositive vs. negative evidencePositive & negative examplesContrast…Formalize this game…Gold’s result…Framework for learningSome detailsLearning algorithms & textsCriterion of success; LearnabilityGenerality of this settingWhat can we do with this?In short:The inductive inference approach (Gold’s theorem)ID in the limit - dfnse-learnability & “locking sequence/data set”Relation between this & learnability in limitGold’s thm follows…Gold’s thmPictureBut what about…But what about… (more old whine in new bottles)Stochastic extensions/Gold complaints & positive resultsResults for stochastic caseFinite language caseSimple finite caseHow many examples needed?Example inferred fsa (NP specifiers)OK smarty…English is function-argument formOther languages are the mirror-inverse: arg-functionOrdersParameter space of languagesParameter space - 14JapaneseSimple switchesParametric space – triggering learning algorithm (TLA)TLASlide 71How to get there from hereLearning driven by language triggering set differencesThe RingstrasseSlide 75TLA, 2TLA, 3Slide 78Slide 79Slide 80# of examples to hit the targetLearning with 24 parametersBut…what’s a symbol?Slide 84Slide 85Power of Gold resultStatistical learning theory approachStatistical learning theory goes further – but same resultsLearnability in statistical framework“Standard” PAC formulationMain result: VC dimension finite = learnableVC dimensionLearnability and VC dimensionSlide 94Slide 95Slide 96Slide 97Slide 98Slide 99But not 4 pointsSlide 101Slide 102Slide 103Slide 104Slide 105Slide 106Slide 107Slide 108Slide 109Slide 1106.863J Natural Language ProcessingLecture 21: Language Acquisition Part 1 Robert C. [email protected]/9.611J Lecture 21 Sp03The Menu Bar•Administrivia:•Project check•The Twain test & the Gold Standard•The Logical problem of language acquisition: the Gold theorem results•How can (human) languages be learned?•The logical problem of language acquisition•What is the problem•A framework for analyzing it6.863J/9.611J Lecture 21 Sp03The Twain test•Parents spend….6.863J/9.611J Lecture 21 Sp03The Logical problem of language acquisitionInitialConstraints& learningmethod“Correct” target languagenoiseAll PossibleHuman languagesInput DataTargetlanguageInput DataInput DataInput Data6.863J/9.611J Lecture 21 Sp03The problem•From finite data, induce infinite set•How is this possible, given limited time & computation?•Children are not told grammar rules•Ans: put constraints on class of possible grammars (or languages)6.863J/9.611J Lecture 21 Sp03The logical problem of language acquisition•Statistical MT: how many parameters? How much data?•“There’s no data like more data”•Number of parameters to estimate in Stat MT system -6.863J/9.611J Lecture 21 Sp03The logical problem of language acquisition•Input does not uniquely specify the grammar (however you want to represent it) = Poverty of the Stimulus (POS)•Paradox 1: children grow up to speak language of their caretakers•Proposed solution: target choice of candidate grammars is restricted set•This is the theory of Universal Grammar (UG)•(Paradox 2: why does language evolve?)6.863J/9.611J Lecture 21 Sp03The illogical problem of language changeLangagis, whos reulis ben not writen as ben Englisch, Frensch and many otheres, ben channgid withynne yeeris and countrees that oon man of the oon cuntre, and of the oon tyme, myghte not, or schulde not kunne undirstonde a man of the othere kuntre, and of the othere tyme; and al for this, that the seid langagis ben not stabili and fondamentali writenPecock (1454) Book of Feith6.863J/9.611J Lecture 21 Sp03Information needed•Roughly: sum of given info + new info (data) has to pick out right language (grammar)•If we all spoke just 1 language – nothing to decide – no data needed•If just spoke 2 languages (eg, Japanese, English), differing in just 1 bit, 1 piece of data needed•What about the general case?6.863J/9.611J Lecture 21 Sp03Can memorization suffice?•Can a big enough table work?•Which is largest, a <noun>-1, a <noun>-2, a <noun>-3, a <noun>-4, a <noun>-5, a <noun>-6, or a <noun>-7?•Assume 100 animals•# queries = 100 x 99 x …94 = 8 x 1013• 2 queries/line, 275 lines, 1000 pages inch =•How big?6.863J/9.611J Lecture 21 Sp03153m6.863J/9.611J Lecture 21 Sp03The inductive puzzle•Unsupervised learning•(Very) small sample complexity•1—5 examples; no Wall Street Journal subscriptions•The Burst effect•Order of presentation of examples doesn’t matter•External statistics don’t match maturational time course6.863J/9.611J Lecture 21 Sp03The burst effecttwo-3 words,ages 1;1–1;11“full” language(some residual)? time span: 2 weeks–2months1;10 ride papa's neck 2;1.2 you watch me 1;10.3 this my rock-baby open sandbox1;11.2 papa forget this 2;1.3 papa, you like this song? 2;4.0 I won't cry if mama wash my hair 2;4.3 put this right here so I see it betterWhat’s the difference?1. I see red one2. P. want drink3. P. open door4. P. tickle S.5. I go beach6. P. forget this7. P. said no8. P. want out9. You play chickenMultiple choice:(a) Pidgin speakers; (b) apes;(c) feral child Genie; (d) ordinary children6.863J/9.611J Lecture 21 Sp03Answers1,5,9 = pidgin2,4,8 = apes7= Genie3,6= children6.863J/9.611J Lecture 21 Sp03Challenge: tension headachesWarlpiriChineseGerman…(4500 others)• Essentially error-free• Minimal triggering (+ examples)• Robust under noise• Still variable enough??6.863J/9.611J Lecture 21 Sp03The input…Bob just went away . Bob went away . no he went back to school . he went to work


View Full Document

MIT 6 863J - Language Acquisition

Documents in this Course
N-grams

N-grams

42 pages

Semantics

Semantics

75 pages

Semantics

Semantics

82 pages

Semantics

Semantics

64 pages

Load more
Download Language Acquisition
Our administrator received your request to download this document. We will send you the file to your email shortly.
Loading Unlocking...
Login

Join to view Language Acquisition and access 3M+ class-specific study document.

or
We will never post anything without your permission.
Don't have an account?
Sign Up

Join to view Language Acquisition 2 2 and access 3M+ class-specific study document.

or

By creating an account you agree to our Privacy Policy and Terms Of Use

Already a member?