CS 124 LINGUIST 180 From Languages to Informa on Dan Jurafsky Stanford University Introduc on and Course Overview Dan Jurafsky What this course is about Automa6cally extrac6ng meaning and structure from Natural language text Speech Web pages Social networks and other networks Genome sequences Dan Jurafsky Commercial World Lots of exci6ng stu going on Dan Jurafsky Ques on Answering IBM s Watson Dan Jurafsky Informa on Extrac on and Sen ment Analysis hIp www bing com search q canon powershot go form QBLH qs n Sen6ment analysis AIribute detec6on Rela6on extrac6on Dan Jurafsky Sen ment Emo6onal Spell Check New York Times 10 big ideas of 2010 hIp video ny6mes com video 2010 12 15 magazine 1248069422438 emo6onal spell check html scp 1 sq emo6onal 20spell 20check st cse Dan Jurafsky Blog Analy cs Data mining of blogs discussion forums message boards user groups and other forms of user generated media Product marke6ng informa6on Poli6cal opinion tracking Social network analysis Buzz analysis what s hot what topics are people talking about right now Dan Jurafsky Livejournal com I me my on or aRer Sep 11 2001 Cohn Mehl Pennebaker 2004 Linguistic markers of psychological change surrounding September 11 2001 Psychological Science 15 10 687 693 7 2 7 0 6 8 6 6 6 4 6 2 6 0 5 8 B s14 s12 s18 s16 Graph from Pennebaker slides s22 s20 o2 o8 s24 o30 n5 o16 o22 Dan Jurafsky September 11 LiveJournal com study We us our Cohn Mehl Pennebaker 2004 Linguistic markers of psychological change surrounding September 11 2001 Psychological Science 15 10 687 693 1 1 1 0 9 8 7 6 5 B s14 s12 s18 s16 Graph from Pennebaker slides s22 s20 o2 o8 s24 o30 n5 o16 o22 Dan Jurafsky Machine Transla on Fully automa6c Helping human translators Enter Source Text Transla6on from Stanford s Phrasal This is only a maIer of 6me Dan Jurafsky Google Translate Fried ripe plantains hIp laylita com recetas 2008 02 28 platanos maduros fritos Dan Jurafsky Informa on Extrac on Event Curriculum mtg Date Jan 16 2012 Subject curriculum mee ng Start 10 00am Date January 15 2012 End 11 30am Where Gates 159 To Dan Jurafsky Hi Dan we ve now scheduled the curriculum mee6ng It will be in Gates 159 tomorrow from 10 00 11 30 Chris Create new Calendar entry Dan Jurafsky Computa onal Biology Finding Genes 5 Intron 1 Intron 2 Exon 3 Exon 1 Exon 2 Start codon ATG Splice sites 3 Stop codon TAG TGA TAA Pictures from Serafim Batzoglou Dan Jurafsky Computa onal Biology Comparing Sequences AGGCTATCACCTGACCTCCAGGCCGATGCCC TAGCTATCACGACCGCGGTCGATTTGCCCGAC AGGCTATCACCTGACCTCCAGGCCGA TGCCC x TAG CTATCAC GACCGC GGTCGATTTGCCCGAC Sequence comparison is key to Finding genes Determining function Uncovering the evolutionary processes Slide stuff from Serafim Batzoglou Dan Jurafsky Ambiguity Resolving ambiguity is a crucial goal throughout string and language processing Dan Jurafsky Ambiguity Find at least 5 meanings of this sentence I made her duck Dan Jurafsky Ambiguity Find at least 5 meanings of this sentence I made her duck I cooked waterfowl for her bene t to eat I cooked waterfowl belonging to her I created the plaster waterfowl she owns I caused her to quickly lower her head or body I waved my magic wand and turned her into undi eren6ated waterfowl Dan Jurafsky Ambiguity is Pervasive I caused her to quickly lower her head or body Syntac c category duck can be a Noun or Verb I cooked waterfowl belonging to her Syntac c category her can be a possessive of her or da6ve for her pronoun I made the plaster duck statue she owns Word Meaning make can mean create or cook Dan Jurafsky Ambiguity is Pervasive Grammar make can be Transi ve verb has a noun direct object I cooked waterfowl belonging to her Ditransi ve verb has 2 noun objects I made her into undi eren6ated waterfowl Ac on transi ve verb has a direct object verb I caused her to move her body Dan Jurafsky Ambiguity is Pervasive Phone cs I mate or duck I m eight or duck Eye maid her duck Aye mate her duck I maid her duck I m aid her duck I mate her duck I m ate her duck I m ate or duck I mate or duck Dan Jurafsky Why else is natural language understanding difficult non standard English segmenta on issues Great job jus6nbieber Were SOO PROUD of what youve accomplished U taught us 2 neversaynever you yourself should never give up either the New York New Haven Railroad the New York New Haven Railroad neologisms unfriend Retweet bromance world knowledge Mary and Sue are sisters Mary and Sue are mothers But that s what makes it fun idioms dark horse get cold feet lose face throw in the towel tricky en ty names Where is A Bug s Life playing Let It Be was recorded a muta6on on the for gene Dan Jurafsky Making progress on this problem The task is di cult What tools do we need Knowledge about language Knowledge about the world A way to combine knowledge sources How we generally do this probabilis6c models built from language data P maison house high P L avocat g n ral the general avocado low Luckily rough text features can oyen do half the job Dan Jurafsky Models Finite state machines Markov models Alignment models Genome alignment Alignment of sentence in L1 to sentence in L2 Alignment of text to speech Vector space model of IR Network models Dan Jurafsky Dynamic Programming Don t do the same work over and over Avoid this by building and making use of solu6ons to sub problems that must be invariant across all parts of the space Minimum Edit Distance The Viterbi Algorithm Baum Welch Forward Backward In parsing CKY Earley charts etc Dan Jurafsky Machine Learning Machine learning based classi ers that are trained to make decisions based on features extracted from the context Simple Classi ers Na ve Bayes Decision Trees Sequence Models Hidden Markov Models Maximum Entropy Markov Models Condi6onal Random Fields Dan Jurafsky Course logistics in brief Instructor Dan Jurafsky TAs Leon Lin Robin Melnick Evan Rosen Alden Timme Adam Vogel Time TuTh 9 30 10 45 Braunlec Requirements Online Video Lectures with embedded quizzes Homeworks In Java or Python Online Review Exercises Final Exam Class sessions Tuesdays Discussions Guest Lectures Thursdays Open group working hours Dan Jurafsky Overview of the course hIp cs124 stanford edu