Lecture 19: Language Acquisition II Professor Robert C. Berwick [email protected] SP11 The Menu Bar • Administrivia: lab 5-6 due this Weds! • Language acquisition – the “Gold standard” & basic results or “the (Evil) Babysitter is Here” (apologies to Dar Williams) • Informal version • Formal version • Can we meet the Gold standard? What about probabilistic accounts? • Stochastic CFGs & Bayesian learning6.863J SP11 Conservative Strategy • Baby’s hypothesis should always be smallest language consistent with the data • Works for finite languages? Let’s try it … • Language 1: {aa,ab,ac} • Language 2: {aa,ab,ac,ad,ae} • Language 3: {aa,ac} • Language 4: {ab} Babysitter Baby aa L3 ab L1 ac L1 ab L1 aa L1 … aa ab ac ad ae6.863J SP11 Evil Babysitter • To find out whether Baby is perfect, we have to see whether it gets 100% correct even in the most adversarial conditions • Assume Babysitter is trying to fool Baby • although she must speak only sentences from LT • and she must eventually speak each such sentence • Does C-Baby’s strategy work on every possible ‘fair sequence’ for every possible language? In finite # of languages case, Yes – why?6.863J SP11 A learnable (“identifiable”) family of Languages • Family of languages: • Let Ln = set of all strings of length < n, over some fixed alphabet = {a, b} • What is L0? • What is L1? • What is Ln? • Let the family L= {L0, L1, …, Ln} • No matter what the Li can Babysitter really follow rules? • Must eventually speak every sentence of L. Is this possible? • Yes: ∅; a, b; aa, ab, ba, bb; aaa, aab, aba, abb, baa,6.863J SP11 An Unlearnable Family of Languages: so-called “Superfinite” family • Let Ln = set of all strings of length < n • What is L0? • What is L1? • What is L∞? • Our (infinite) family is L = {L0, L1, …,Ln, …, L∞} • A perfect C-baby must be able to distinguish among all of these depending on a finite amount of input • But there is no perfect C-baby …6.863J SP11 An Unlearnable Family • Our class is L = {L0, L1, …, L∞} • C-Baby adopts conservative strategy, always picking smallest possible language in L • So if Babysitter’s longest sentence so far has 75 words, baby’s hypothesis is L75. • This won’t always work for all languages in L What language can’t a conservative Baby learn? So, C-baby cannot always pick smallest possible language and win…6.863J SP11 An Unlearnable Family • Could a non-conservative baby be ‘almost’ a perfect C-Baby, and eventually converge to any of the languages in the family of languages? • Claim: Any perfect C-Baby must be “quasi-conservative”: • If the true language is L75, and baby posits something else, baby must still eventually come back and guess L75 (since it’s perfect). • So if longest sentence so far is 75 words, and Babysitter keeps talking from L75, then eventually baby must actually return to the conservative guess L75. • Agreed?6.863J SP11 The “Evil Babysitter” If longest sentence so far is 75 words, and Babysitter keeps talking from L76, then eventually a perfect C-baby must actually return to the conservative guess L75. • But suppose the true target language is L∞. • Evil Babysitter can then prevent our supposedly perfect C-Baby from converging to it • If Baby ever guesses L∞, say when the longest sentence is 75 words: • Then Evil Babysitter keeps talking from L75 until Baby capitulates and revises her guess to L75 – as any perfect C-Baby must. • So Baby has not stayed at L∞ as required. • Then Babysitter can go ahead with longer sentences. If Baby ever guesses L∞ again, she plays the same trick again (and again)6.863J SP11 The “Evil Babysitter” If longest sentence so far is 75 words, and Babysitter keeps talking from L76, then eventually a perfect C-baby must actually return to the conservative guess L76. • Suppose true language is L∞. • Evil Babysitter can prevent our supposedly perfect C-Baby from converging to it in the limit • If Baby ever guesses L∞, say when the longest sentence is 75 words: • Then Evil Babysitter keeps talking from L76 until Baby capitulates and revises her guess to L76 – as any perfect C-Baby must. • So Baby has not stayed at L∞ as required. • Conclusion: There’s no perfect Baby that is guaranteed to converge to L0, L1, … or L∞ as appropriate. If C-Baby always succeeds on finite languages, Evil Babysitter can trick it on infinite language; if C-Baby succeeds on the infinite L∞ then Evil Babysitter can force it to never learn finite L’sWhat does this result imply? • Any family of languages that includes all the finite languages and at least this one super-finite language is not identifiable in the limit from positive-only evidence • This includes the family of all finite-state languages; the family of all context-free languages; etc. etc. 6.863J SP116.863J SP11 Is this too ‘adversarial’? • Should we assume Babysitter is evil? • Maybe more like Google…. • Perhaps Babysitter isn’t trying to fool the baby - not an adversarial situation6.863J SP11 Formally: Notation & definitions6.863J SP11 Notation & definitions6.863J SP11 Notation and definitions6.863J SP11 Notation and definitions6.863J SP11 The “locking sequence” (evil babysitter) theorem g After ‘lock sequence’ seen, then “happily ever after” inside the sphere of radius epsilon…6.863J SP11 Proof6.863J SP11 Construct “Evil babysitter” text6.863J SP11 To get classical result for exact identification (0–1 metric) 1/26.863J SP11 Classic Gold Result (“Superfinite theorem”) Proof: By contradiction.Suppose A can identify the family L.Therefore, A can identify the infinite language L∞.Therefore, ∃ finite locking sequence for L∞, c al l i t σinf.But L = range(σinf)isafinite language, and so L ∈ LThen tk= σinf, k=length(σinf) is a te x t for L.Since A lear ns L on all fair tex t s for L, it must converge to L on tk.Therefore, A does not ide ntify L∞, a contradiction.6.863J SP11 Extensions reveal the breadth of Gold’s result6.863J SP11 What happens if we ‘go probabilistic’? • Everyone always complains about the Gold results… • Gold is
View Full Document
Unlocking...