6.863J/9.611J SP03 Lecture 16.863J Natural Language ProcessingLecture 1: Introduction Instructor: Robert C. [email protected]/9.611J SP03 Lecture 1The Menu Bar! Administrivia! All on web page: www.ai.mit.edu/courses/6.863! Stellar web site:http://stellar.mit.edu/S/course/6/sp03/6.863j/! What this course is about! Why NLP is hard, and interesting! The ingredients of language! Why language and computation?! What you have to do in the course! Till next time…6.863J/9.611J SP03 Lecture 1What is this course all about?! Computational methods for working with natural (human) languages! Applications of computer science & AI! Linguistic theory! Natural (psycholinguistics) or artificial computation (natural language processing, NLP)6.863J/9.611J SP03 Lecture 1Simple model: relation of sound-meaning sound‘meaning’Aristotle, e.g., only 2500years old…6.863J/9.611J SP03 Lecture 1•Only interfaces ‘remain’sound(outside)meaning (inside world)Language = Pairing sound & meaning6.863J/9.611J SP03 Lecture 1Natural language at the heart of human intelligence! The first Turing test:“Rabbah Zoreh made a Gollum and brought it to Rabbah; he bid it to talk.Rabbah replied: ‘It cannot speak; return it unto the flames’(Manhet Sahedrin, Babylonian Talmud, approx. 400 BCE)6.863J/9.611J SP03 Lecture 1Human language: special character! Pop-quiz (multiple choice): who produced the following ‘sentences’ (Names changed to protect the innocent):! (1) I see red one ! (2) P. want drink ! (3) P. open door ! (4) P. tickle S. ! (5) I go beach! (6) P. forget this! (7) P said no! (8) P. want out! (9) You play chicken! Multiple choice: (a) Pidgin speakers; (b) apes (signing); (c) Feral child Genie; (d) ordinary children6.863J/9.611J SP03 Lecture 1Applications! Lightweight / AI-complete! Line breakers, hyphenators, spell checkers, grammar & style checkers! Information Retrieval (IR) / Question-answering systems! Sentence/dialog understanding! Document summarization! Machine translation6.863J/9.611J SP03 Lecture 1But what’s inside the black box? Lightweight / AI-completefoxesfox + sLightweight:6.863J/9.611J SP03 Lecture 1AI-completeEnglishJapanese6.863J/9.611J SP03 Lecture 1But the most important reason of all…6.863J/9.611J SP03 Lecture 1It’s the year 2003Dave Bowman: Open the pod bay doors, HALHAL: I’m sorry Dave, I’m afraid I can’t do that.6.863J/9.611J SP03 Lecture 1Why study NLP?How the mind works: the Twain test! How do people look up words?! How do people parse sentences?! How do people learn language?! How does language evolve?6.863J/9.611J SP03 Lecture 1The human sentence processor! Properties! Garden-path (Blind alleys)! Sue told the person that she hired a story! Sue told the person that she hired an assistant! Non-uniform processing cost! The reporter who the senator attacked admitted the error! The reporter who attacked the senator admitted the error6.863J/9.611J SP03 Lecture 1Sentence analysis – can be subtle –what is the knowledge?! Representation of events! John is too stubborn [to talk to ]! Event(e) & agent(x, e)! John is too stubborn [to talk to Mary]! Who is the agent now?6.863J/9.611J SP03 Lecture 1What makes NLP interesting (and difficult)! Complex phenomenon arising out of the interaction of many distinct kindsof knowledge!Whatis this knowledge? (data structures - linguistics)!Howis it put to use? (algorithms)! Example: “the dogs ate ice-cream”6.863J/9.611J SP03 Lecture 1Knowledge of language: What do we knowabout this sequence?!Do.. begins a valid word of English, but no English word begins with ptk; the s on dogsmarks it as plural! Words must appear in a certain order: *Dogs ice-cream ate! Parts and divisions: the dogs is the Subject; ate ice-cream,the Predicate. Distinct parts or constituents(phrases)! Who did what to whom: the dogsis the Agent of the action ate, while ice-cream is the Object6.863J/9.611J SP03 Lecture 1But wait, there’s more… (you also get…)! The two sentences John claimed the dogs ate ice-creamand John denied the dogs ate ice-creamare logically incompatible! Sentence & the world: know whether the sentence is trueor not - perhaps whether in some particular situation (possible world) the dogs did indeed eat ice-cream! Know that it would sound fine if it were to follow I had espresso this morning, but…! However, odder if it were to follow John is intelligent6.863J/9.611J SP03 Lecture 1The linguistic pipeline! We need data representation (linguistic) primitives to represent sounds, sound pieces, words, word pieces, sentences, sentence pieces (compare to vision), so….! Primitives only contain partialinformation, unique (proprietary) to their own “level”, so they must combine in non-arbitrary ways! Levels must be connnected! What is the knowledge in each level?6.863J/9.611J SP03 Lecture 1The “spiral notebook” modelthe dogs ate ice-creamθε dawgz…Sentence‘surface’formNoun phrase Verb phraseVerb Noun Phraseate ice-creamthe dogzλx, xε{dogs}, ate(x, i-c)‘sound’form‘phrase’form‘logical’form6.863J/9.611J SP03 Lecture 1Sentence knowledge is subtle! A book was given Mary! Mary was given a book! A book was given to Mary! Mary was given a book to6.863J/9.611J SP03 Lecture 1Word knowledge is subtle! He arrived at the station! He chuckled at the station! He arrived drunk! He chuckled drunk! He chuckled his way through the meeting! He arrived his way through the meeting6.863J/9.611J SP03 Lecture 1Invisible knowledge! I want to solve the problem! I wanna solve the problem! Displacement: I understand these students! These students I understand! I want these students to solve the problem! These students I want [x] to solve the problem [x]= these studentsNotice that contraction of want+to is now blocked!6.863J/9.611J SP03 Lecture 1What is the character of this knowledge?! Some of it must be memorized (obviously so):! Singing-> Sing+ing; Bringing-> bring+ingDuckling ->?? Duckl +ingSo, must know duckl is not a wordBut it can’t all be memorized…Because there is too much to know6.863J/9.611J SP03 Lecture 1Besides memory, what else do we need? ! English plural:! Toy+s -> toyz ; add z! Book+s -> books ; add s! Church+s -> churchiz ; addiz! Box+s-> boxiz ; addiz! What if a novelword?! Bach’s many cantatas! Which pronounciation is it? Sor IZ ?Bachs many cantatas NOT BachIZ despiteAnalogy/similarity to ‘box’ - why?6.863J/9.611J SP03 Lecture 1Conclusion: must be a rule system to
View Full Document
Unlocking...