About Me CMSC 723 Computational Linguistics I Session 1 Introduction to NLP Jimmy Lin The iSchool University of Maryland NLP IR Wednesday September 2 2009 CLIP Teaching Assistant Melissa Egan About You pre requisites Must be interested in NLP Must have strong computational background Must be a competent programmer Do not need to have a background in linguistics Administrivia Text z Course webpage Class z z z Course Grade Speech and Language Processing An Introduction to Natural Language Processing Speech Recognition and Computational Linguistics second edition Daniel Jurafsky and James H Martin 2008 http www umiacs umd edu jimmylin CMSC723 2009 Fall Wednesdays 4 to 6 30pm CSI 2107 Two blocks 5 10 min break in between Out of Class Support Exams 50 Office hours by appointment Class Assignments 45 Course mailing list umd cmsc723 fall 2009 googlegroups com z z Class participation 5 z Assignment 1 warm up 5 Assignments 2 5 10 each Showing up for class demonstrating preparedness and contributing to class discussions Policy for late and incomplete work etc 1 What is Computational Linguistics Study of computer processing of natural languages Interdisciplinary field z z Let s Let s get started z Goes by various names z z z z Where does NLP fit in CS What is the goal of this endeavor z z Programming Languages Systems Networks Human Computer Interaction Artificial Intelligence Databases Machine Learning NLP Robotics Computational linguistics Natural language processing Speech language text processing Human language technology technologies Science vs Engineering Computer Science Algorithms Theory Roots in linguistics and computer science specifically AI Influenced by electrical engineering cognitive science psychology and other fields Dominated today by machine learning and statistics Goals usually in tension z Understanding the phenomenon of human language Building a better applications Analogy flight Rationalism vs Empiricism Success Stories Where does the source of knowledge reside If it works it s not AI Chomsky s poverty of stimulus argument Speech recognition and synthesis It s an endless pendulum Information extraction Automatic essay grading G Grammar checking h ki Machine translation 2 NLP Layers Speech Recognition Morphological Analysis Speech Recognition Parsing Semantic Analysis Conversion from raw waveforms into text Involves lots of signal processing It s hard to wreck a nice beach Reasoning R i Planning Speech Synthesis Morphological Realization Syntactic Realization Utterance Planning Phonology Morphology Syntax Semantics Reasoning Source Adapted from NLTK book chapter 1 Optical Character Recognition Conversion from raw pixels into text Involves a lot of image processing What if the image is distorted or the original text is in poor condition What s a word Break up by spaces right Ebay Sells Most of Skype to Private Investors Swine flu isn t something to be feared What about these Morphological Analysis Morpheme smallest linguistic unit that has meaning Inflectional z z duck s N duck plural s duck s V duck 3rd person singular s Derivational z z organize organization happy happiness Complex Morphology Turkish is an example of agglutinative language From the root uyu sleep the following can be derived uyuyorum uyuyorsun uyuyor uyuyoruz uyuyorsunuz uyuyorlar uyuduk uyuduk a uyumal y z uyumadan uyuman uyurken uyuyunca uyutmak uyutturmak uyutturtturmak I am sleeping you are sleeping he she it is sleeping we are sleeping you are sleeping they are sleeping we slept as long as somebody sleeps we must sleep without sleeping your sleeping while somebody is sleeping when somebody sleeps to cause somebody to sleep to cause somebody to cause another to sleep to cause somebody to cause some other to cause yet another to sleep From Hakkani T r Oflazer T r 2002 3 What s a phrase Coherent group of words that serve some function z z Syntactic Analysis Parsing the process of assigning syntactic structure Organized around a central head The head specifies the type of phrase Examples z z z S Noun phrase NP the happy camper Verb phrase VP shot the bird Prepositional phrase PP on the deck NP VP N N V I saw NP N det N the man I det saw the man S NP I VP saw NP the man Semantics Different structures same meaning z z z z I saw the man The man was seen by me The man was who I saw z Scoping issues z z Everyone on the island speaks two languages Two languages are spoken by everyone on the island Ultimately what is meaning z Simply pushing the problem onto different sets of SYMBOLS First order predicate logic x MAN x SEE x I TENSE past Semantic frames and roles PREDICATE see EXPERIENCER I PATIENT man Lexical Semantics Any verb can add able to form an adjective z z z Semantic representations p attempt p to abstract meaning g z Semantics More Complexities I taught the class The class is teachable I loved that bear The bear is loveable I rejected the idea The idea is rejectable Pragmatics and World Knowledge Interpretation of sentences requires context world knowledge speaker intention goals etc Example 1 z Association of words with specific semantic forms z z z John noun noun masculine masculine proper the boys noun masculine plural human load smear verbs specific restrictions on subjects and objects z Could you turn in your assignments now command Could you finish the assignment question command E Example l 2 2 z z I couldn t decide how to catch the crook Then I decided to spy on the crook with binoculars To my surprise I found out he had them too Then I knew to just follow the crook with binoculars the crook with binoculars vs the crook with binoculars 4 Discourse Analysis Discourse how multiple sentences fit together Pronoun reference z Multiple reference to same entity Inference and other relations between sentences z So easy The professor told the student to finish the exam He was pretty aggravated at how long it was taking him to complete it z Why is NLP hard George Bush Clinton The bomb exploded in front of the hotel The fountain was destroyed but the lobby was largely intact At the word level Part of speech z z At the syntactic level PP Attachment ambiguity z z z z I saw the man on the hill with the telescope z I cooked her duck Visiting relatives can be annoying Time flies like an arrow arrow I went to the bank to deposit my check I went to the bank to look out at the river river I went to the bank of windows and chose the one for complaints Difficult cases Requires world knowledge z Structural