6.863J/9.611J Natural language & computers Lecture 1: Walking the walk, talking the talk Professor Robert C. Berwick [email protected] Course web page: http://www.mit.edu/~6.863/spring2011/ 6.863J/9.611J Spring 2011 The Menu Bar • The Rules of the Game: goals of the field and the course (yes, Anne Hunter, it is an AUS this term) • Mark Twain’s problem • The “WYSIWYG” language problem • We haven’t had a sale in 40 years • For next time… (predict what word comes next!) 6.863J/9.611J Spring 2011 Goals of the field: the problem of natural language understanding Computers would be a lot more useful if they could handle our email, do our library research, talk to us … How can we tell computers about language? (Or help them learn it as kids do?) 6.863J/9.611J Spring 2011 Natural language processing (NLP) • We know what we want from computer software • “killer applications” – those that can make sense of language data • retrieve language data: (IR) • summarize knowledge contained in language data • answer questions (QA), make logical inferences • translate from one language into another • recognize speech: transcribe medical records,… • In short: we want computers to be smart about language and pass the Turing test… • Well, perhaps not that smart…6.863J/9.611J Spring 2011 Some examples • Answering questions: – Bin Laden is too angry to talk to – Q: Can we talk to Bin Laden easily? – Bin Laden is too angry to talk to Cheney • Extracting information – from biological papers, can a computer figure out what proteins interact? (‘data mining’) • And, the usual Holy Grail… 6.863J/9.611J Spring 2011 Killer Application: computer translation Has Google solved this? How does this work? Does it work? 6.863J/9.611J Spring 2011 Language understanding? 6.863J/9.611J Spring 2011 Goals of the course • Introduce you to natural language processing problems • Gain working knowledge of the two key approaches have been to solving NLP problems: linguistics & statistics By the end of the course you should: • Agree that language is subtle & interesting • Feel some ownership over the formal & statistical models • Understand research papers in the field6.863J/9.611J Spring 2011 Course organization, I • Me: Bob Berwick, [email protected] • TA: Igor Malioutov [email protected] • Course web page for all readings/assignments/code: http://web.mit.edu/6.863/spring2011/ • 2 Lectures/week; office hours & lab hours if needed; ‘official’ lab hours if needed • Lab oriented, w/ a few Winston-style “reading and responses” (R&R) • No final exam • 6-7 Labs, 1 final project lab; final project lab is joint • Labs typically 2-3 weeks long • R&R, typically out on Wednesday, due Sunday 5pm/Monday next class for mandatory in-class discussion • All work can be done jointly, but you must write-up your own reports, identifying who you worked with • Email pdf/web URLs for Lab write-ups to: [email protected] 6.863J/9.611J Spring 2011 Course organization, II • Grade determined by: – 60% Lab assignments & Competitive Grammar (CGW) – 20% Reading & Response & class participation – 20% final project • Late days to help with time management (7 days) – see web page for details (1 day = 24 consecutive hours) • So you won’t have to ask me for ‘extensions’ • Use your late days wisely! 6.863J/9.611J Spring 2011 (Required!) Textbook (Jurafsky &Martin) New and improved for 2008 (2nd edition) Nearly 1000 pages (full year’s worth…) 25 chapters Divided into 5 parts I. Words II. Speech III. Syntax IV. Semantics and Pragmatics V. Applications For Monday: Read ch 1.; pp.83-94; 114-116. (online; also on Barker reserve P98.J87 2009); 6.863J/9.611J Spring 2011 So, not so fast, re language being ‘solved’…? More Problems… • Mark Twain’s problem: Parents spend… • The non-WYSIWYG problem: language is not ‘WYSIWIG’ (required information is sometimes just not there – not what appears ‘on the surface’…and so…how can we learn from what is statistically invisible ‘on the surface’?) • Language-as-communication problem: Not even obviously well-designed for ‘ease of communication’ (ask any diplomat, teacher, student, …); it is ambiguous, leading to nondeterminism6.863J/9.611J Spring 2011 Human language is not ‘wysiwig’ and not designed for ‘ease of communication’ • Invisible elements – shared knowledge – a window into the human mind • Bin Laden is too angry to talk to • Bin Laden is too angry to talk to Cheney • What are the ‘invisible elements’ here? • Human language is ambiguous – sometimes on purpose We haven’t had a sale in forty years 6.863J/9.611J Spring 2011 Language ambiguity makes NLP hard: Especially in news headlines Iraqi Head Seeks Arms Juvenile Court to Try Shooting Defendant Teacher Strikes Idle Kids Stolen Painting Found by Tree Kids Make Nutritious Snacks Local HS Dropouts Cut in Half Obesity Study Looks for Larger Test Group 6.863J/9.611J Spring 2011 Levels of Description in language • Phonetics/phonology/morphology: what words (or subwords) are we dealing with? • Syntax: What phrases are we dealing with? Which words modify one another? • Semantics: What’s the literal meaning? • Pragmatics: What should you conclude from the fact that I said something? How should you react? 6.863J/9.611J Spring 2011 And even subtler ambiguity • Q: Why does my high school give me a suspension for skipping class? • A: Administrative error. They’re supposed to give you a suspension for auto shop, and a jump rope for skipping class.6.863J/9.611J Spring 2011 What makes NLP hard? The ‘Road Touring Test’ for NLP… John stopped at the donut store on his way home from work. He thought a coffee was good every few hours. But it turned out to be too expensive there. To get a donut (spare tire) for his car? 6.863J/9.611J Spring 2011 Why is this story hard? John stopped at the donut store on his way home from work. He thought a coffee was good every few hours. But it turned out to be too expensive there. store where donuts shop? or is run by donuts? or looks like a big donut? or made of donut? or has an emptiness at its core?
View Full Document