PowerPoint PresentationDealing with Implicit NegativesSlide 3Slide 4Slide 5Slide 6Learning SystemSlide 8Slide 9Slide 10Slide 11Slide 12Slide 13Slide 14Slide 15Slide 16Topological RelationsIssue #2: Shift InvarianceSlide 19Slide 20Slide 21Slide 22LimitationsDemo of the Regier SystemLanguage and Thought5 levels of Neural Theory of LanguageLanguage, Learning and Neural Modeling www.icsi.berkeley.edu/AISlide 28Simulation-based language understandingSimulation SemanticsPsycholinguistic evidenceNeural evidence: Mirror neuronsArea F5cF5 Mirror NeuronsSlide 35Slide 36Slide 37Slide 38Slide 39The Mirror System in HumansThe ICSI/Berkeley Neural Theory of Language ProjectSlide 42Computing other relationsTriangle nodes and McCullough-Pitts Neurons?Representing concepts using triangle nodes“They all rose”Basic Ideas behind the modelCan we formalize/model these intuitionsSpreading activation and feature structuresSlide 50Dealing with Implicit NegativesExplicit positive for aboveImplicit negatives for below, left, right, etcin Regier: E = ½ ∑i,p (( ti,p – oi,p) * βi,p )2, where i is the node, p is the pattern,βi,p = 1 if explicit positive, βi,p < 1 if implicit negativeLearning Systemdynamic relations(e.g. into)structured connectionistnetwork (based on visual system)Topological RelationsSeparationContactCoincidence:-Overlap-Inclusion-Encircle/surroundIssue #2: Shift InvarianceBackprop cannot handle shift invariance (it cannot generalize from 0011, 0110 to 1100)But the cup is on the table whether you see it right in the center or from the corner of your eyes (i.e. in different areas of the retina map)What structure can we utilize to make the input shift-invariant?LimitationsScaleUniqueness/PlausibilityGrammarAbstract ConceptsInferenceRepresentationDemo of the Regier Systemon the English aboveLanguage and ThoughtWe know thought (our cognitive processes) constrains the way we learn and use languageDoes language also influence thought? Benjamin Whorf argues yesPsycholinguistics experiments have shown that linguistics categories influence thinking even in non-linguistics taskLanguageThoughtcognitive processes5 levels of Neural Theory of LanguageCognition and LanguageComputationStructured ConnectionismComputational NeurobiologyBiologyMidtermQuizFinalsNeural DevelopmentTriangle NodesNeural NetSpatial RelationMotor ControlMetaphorSHRUTIGrammarabstractionLanguage, Learning and Neural Modelingwww.icsi.berkeley.edu/AIScientific Goal Understand how people learn and use languagePractical Goal Deploy systems that analyze and produce language Approach Build models that perform cognitive tasks, respecting all experimental and experiential constraints Embodied linguistic theories with advanced biologically-based computational methodsphysics lowest energy statechemistry molecularminimabiology fitness, MEU Neuroeconomicsvision threats, friendslanguage errors, NTLConstrained Best Fit in Natureinanimate animateSimulation-based language understanding“Harry walked to the cafe.”Schema Trajector Goalwalk Harry cafeAnalysis ProcessSimulation SpecificationUtteranceSimulationCafeConstructionsGeneral KnowledgeBelief StateSimulation SemanticsBASIC ASSUMPTION: SAME REPRESENTATION FOR PLANNING AND SIMULATIVE INFERENCE Evidence for common mechanisms for recognition and action (mirror neurons) in the F5 area (Rizzolatti et al (1996), Gallese 96, Boccino 2002) and from motor imagery (Jeannerod 1996)IMPLEMENTATION: x-schemas affect each other by enabling, disabling or modifying execution trajectories. Whenever the CONTROLLER schema makes a transition it may set, get, or modify state leading to triggering or modification of other x-schemas. State is completely distributed (a graph marking) over the network.RESULT: INTERPRETATION IS IMAGINATIVE SIMULATION!Psycholinguistic evidenceEmbodied language impairs action/perceptionSentences with visual components to their meaning can interfere with performance of visual tasks (Richardson et al. 2003)Sentences describing motion can interfere with performance of incompatible motor actions (Glenberg and Kashak 2002)Sentences describing incompatible visual imagery impedes decision task (Zwaan et al. 2002)Simulation effects from fictive motion sentencesFictive motion sentences describing paths that require longer time, span a greater distance, or involve more obstacles impede decision task (Matlock 2000, Matlock et al. 2003)Neural evidence: Mirror neuronsGallese et al. (1996) found “mirror” neurons in the monkey motor cortex, activated whenan action was carried outthe same action (or a similar one) was seen.Mirror neuron circuits found in humans (Porro et al. 1996)Mirror neurons activated when someone:imagines an action being carried out (Wheeler et al. 2000)watches an action being carried out (with or without object) (Buccino et al. 2000)Area F5cArea F5cConvexity region of F5:Mirror neuronsF5 Mirror NeuronsF5 Mirror NeuronsGallese and Goldman, TICS 1998Category Loosening in Mirror Neurons (~60%)(Gallese et al. Brain 1996)Observed: A is Precision GripB is Whole Hand Prehension Action: C: precision gripD: Whole Hand PrehensionPF Mirror NeuronsPF Mirror Neurons(Gallese et al. 2002)1. Neuron responds to execution (grasping) but to grasping and releasing in observation. 2. Mirror neurons in parietal cortex.3. Difference in left hand and right hand.Umiltà et al. Neuron 2001A (Full vision)A (Full vision)B (Hidden)B (Hidden)C (Mimicking)C (Mimicking)D (HiddenMimicking)D (HiddenMimicking)F5 Audio-Visual Mirror NeuronsF5 Audio-Visual Mirror NeuronsKohler et al. Science (2002)Somatotopy of Action ObservationSomatotopy of Action ObservationFoot ActionFoot ActionHand ActionHand ActionMouth ActionMouth ActionBuccino et al. Eur J Neurosci 2001The Mirror System in HumansThe Mirror System in HumansBA6The ICSI/Berkeley Neural Theory of Language ProjectLearning early constructions (Chang, Mok)ECGComputing other relationsThe 2/3 node is a useful function that activates its outputs (3) if any (2) of its 3 inputs are activeSuch a node is also called a triangle node and will be useful for lots of representations.Triangle nodes and McCullough-Pitts Neurons?B CAA B CRepresenting concepts using triangle nodestriangle nodes:when two of the neurons fire, the third also fires“They all rose”triangle nodes:when two of the neurons
View Full Document