Cognitive revolution: Information processingGeorge Miller, Ulrich Neisser, Donald BroadbentKey metaphor: brain as computer• Hardware distinct fromsoftware (Marr)• Abstract knowledge• Symbol processing• Cognition as an algorithm• Modularity (Fodor)Language as rules:E → /ɛ/ (BED)IE → /ɑɪ/ (LIED)EA → /i/ (TEAM)Reading a word is just a matter ofprocessing the input…… applying the right rule… to compute the pronunciation.But of course there are exceptions DEAD, HEAD, THREAD… STEAK???There are things that rules don’t capture wellEA* → /i/ (TEAM)EAD → /ɛ/ (HEAD, THREAD)But what aboutMEAD (/i/)? THREAT (/ɛ/)?English past-tense consists of regular formsWalked Ÿ ModeledCarried Ÿ RacedAll use the rule [stem] + [-ed]+ ExceptionsŸ Ate Ÿ Sang Ÿ StuckŸ Went Ÿ Stank Ÿ BroughtAlternative view of: past-tense is quasi-regular(McCellend& Seidenberg, 1989)There are clearly regularities in language.But they’re not fully consistent.Language users must be flexible.Is there a way to capture these probabilisticrelationships? Without the rigidity of rules?Is there a way to capture language with a set of basic processes that are closer to how the brain operates?Connectionism(AKA “parallel distributed processing”, “neural networks”)Inspired by a new metaphor: neural processing.Neural processing is not:Discrete - Symbolic - SerialModular – AlgorithmicInstead it isGraded or probabilisticParallelRepresentations are distributed.Driven by local interactionsTuned by learning principles.Emergent complexity.Fundamental Concept: A connectionist networkInspired by neural principles, but usuallyvery oversimplified.Most use the term neural network to denotemore neurologically plausible systems.What is it?Set of simple mathematical principlesdescribing how neurons interact duringtraining and learning.Description of the inputs and outputs (thedata), and the rules for processing.Usually run on a computer.Model vs. TheoryConnectionist networks are models of aphenomenon, built to implement a theory.But simple math when it is implemented onlots of neurons and connections can lead tounexpected results.Often a model is used to test implicationsof a theory.Why build models at all?Simple mechanisms give rise to complex behavior.But many mechanisms arepossible.Can be easier to understandmechanism by buildingoutward, rather than observinginward.All units do one thing (thesame way): They areactive or not to various degrees.How do they get active?What makes them different? How isinformation processed?Units are active as afunction oftheirconnections toother cells.how active those other cells are.These differ between unitsallowing for specialization.How does this do anything useful?-Units stand for something-Now activation becomes a way to say “I think it’s a X…” ⎯ | / Connections become a way to pass information.SummaryIndividual units pass activation…… over weighted connections …… to other units …… that simply sum up their inputs…… to determine their own activation…CSD 3117 1st EditionLecture 7Outline of Last Lecture I. Fundamental properties of memoryII. What is memory?a. Multiple Memory Systemsb. Links to languageIII. HippocampusOutline of Current Lecture IV. Cognitive Revolutiona. Rules of languageV. ConnectionismVI. Fundamental Concepta. What is itb. Model vs theoryVII. SummaryCurrent Lecture: Connectionism Cognitive revolution: Information processing- George Miller, Ulrich Neisser, Donald Broadbent- Key metaphor: brain as computer• Hardware distinct fromsoftware (Marr)These notes represent a detailed interpretation of the professor’s lecture. GradeBuddy is best used as a supplement to your own notes, not as a substitute.• Abstract knowledge• Symbol processing• Cognition as an algorithm• Modularity (Fodor)Language as rules:o E → /ɛ/ (BED) o IE → /ɑɪ/ (LIED) o EA → /i/ (TEAM)o Reading a word is just a matter ofprocessing the input…… applying the right rule… to compute the pronunciation.But of course there are exceptions DEAD, HEAD, THREAD… STEAK???There are things that rules don’t capture well- EA* → /i/ (TEAM)- EAD → /ɛ/ (HEAD, THREAD)- But what aboutMEAD (/i/)? THREAT (/ɛ/)?English past-tense consists of regular formso Walked $ Modeledo Carried $ RacedAll use the rule [stem] + [-ed]o + Exceptionso $ Ate $ Sang $ Stucko $ Went $ Stank $ BroughtAlternative view of: past-tense is quasi-regular(McCellend& Seidenberg, 1989)- There are clearly regularities in language.- But they’re not fully consistent.o Language users must be flexible.- Is there a way to capture these probabilisticrelationships? Without the rigidity of rules?o Is there a way to capture language with a set of basic processes that are closer to how the brain operates?Connectionism- (AKA “parallel distributed processing”, “neural networks”)- Inspired by a new metaphor: neural processing.- Neural processing is not:o Discrete - Symbolic - Serial Modular – Algorithmic- Instead it iso Graded or probabilistico Parallelo Representations are distributed.o Driven by local interactionso Tuned by learning principles.o Emergent complexity.Fundamental Concept: A connectionist network- Inspired by neural principles, but usuallyvery oversimplified.- Most use the term neural network to denotemore neurologically plausible systems.What is it?o Set of simple mathematical principlesdescribing how neurons interact duringtraining and learning.o Description of the inputs and outputs (thedata), and the rules for processing.o Usually run on a computer.Model vs. Theoryo Connectionist networks are models of aphenomenon, built to implement a theory.o But simple math when it is implemented onlots of neurons and connections can lead tounexpected results.o Often a model is used to test implicationsof a theory.Why build models at all?o Simple mechanisms give rise to complex behavior.o But many mechanisms arepossible.o Can be easier to understandmechanism by buildingoutward, rather than observinginward.All units do one thing (thesame way): They areactive or not to various degrees.- How do they get active?- What makes them different? How isinformation processed?Units are active as afunction oftheir o connections toother cells.• how active those other cells are.These differ between unitsallowing for specialization.How does this do anything useful? -Units stand for something -Now activation becomes a way to say “I think it’s a X…” ⎯ | \ / Connections