NeuronsNeurons with Graded Response Have Collective Computational Properties like Those of Two-StateJ. J. Hopfield doi:10.1073/pnas.81.10.3088 1984;81;3088-3092 PNAS This information is current as of January 2007. www.pnas.org#otherarticlesThis article has been cited by other articles: E-mail Alerts. click hereright corner of the article orReceive free email alerts when new articles cite this article - sign up in the box at the top Rights & Permissions www.pnas.org/misc/rightperm.shtmlTo reproduce this article in part (figures, tables) or in entirety, see: Reprints www.pnas.org/misc/reprints.shtmlTo order reprints, see: Notes:Proc.Natl.Acad.Sci.USAVol.81,pp.3088-3092,May1984BiophysicsNeuronswithgradedresponsehavecollectivecomputationalpropertieslikethoseoftwo-stateneurons(associativememory/neuralnetwork/stability/actionpotentials)J.J.HOPFIELDDivisionsofChemistryandBiology,CaliforniaInstituteofTechnology,Pasadena,CA91125;andBellLaboratories,MurrayHill,NJ07974ContributedbyJ.J.Hopfield,February13,1984ABSTRACTAmodelforalargenetworkof"neurons"withagradedresponse(orsigmoidinput-outputrelation)isstudied.ThisdeterministicsystemhascollectivepropertiesinveryclosecorrespondencewiththeearlierstochasticmodelbasedonMcCulloch-Pittsneurons.Thecontent-addressablememoryandotheremergentcollectivepropertiesoftheorigi-nalmodelalsoarepresentinthegradedresponsemodel.Theideathatsuchcollectivepropertiesareusedinbiologicalsys-temsisgivenaddedcredencebythecontinuedpresenceofsuchpropertiesformorenearlybiological"neurons."Collectiveanalogelectricalcircuitsofthekinddescribedwillcertainlyfunction.Thecollectivestatesofthetwomodelshaveasimplecorrespondence.Theoriginalmodelwillcontinuetobeusefulforsimulations,becauseitsconnectiontogradedresponsesys-temsisestablished.Equationsthatincludetheeffectofactionpotentialsinthegradedresponsesystemarealsodeveloped.Recentpapers(1-3)haveexploredtheabilityofasystemofhighlyinterconnected"neurons"tohaveusefulcollectivecomputationalproperties.Thesepropertiesemergesponta-neouslyinasystemhavingalargenumberofelementary"neurons."Content-addressablememory(CAM)isoneofthesimplestcollectivepropertiesofsuchasystem.Themathematicalmodelinghasbeenbasedon"neurons"thataredifferentbothfromrealbiologicalneuronsandfromtherealisticfunctioningofsimpleelectroniccircuits.Someofthesedifferencesaremajorenoughthatneurobiologistsandcircuitengineersalikehavequestionedwhetherrealneuralorelectricalcircuitswouldactuallyexhibitthekindofbe-haviorsfoundinthemodelsystemevenifthe"neurons"wereconnectedinthefashionenvisioned.Twomajordivergencesbetweenthemodelandbiologicalorphysicalsystemsstandout.Realneurons(andrealphysi-caldevicessuchasoperationalamplifiersthatmightmimicthem)havecontinuousinput-outputrelations.(Actionpo-tentialsareomitteduntilDiscussion.)Theoriginalmodelingusedtwo-stateMcCulloch-Pitts(4)thresholddeviceshavingoutputsof0or1only.Realneuronsandrealphysicalcircuitshaveintegrativetimedelaysduetocapacitance,andthetimeevolutionofthestateofsuchsystemsshouldberepresentedbyadifferentialequation(perhapswithaddednoise).Theoriginalmodelingusedastochasticalgorithminvolvingsud-den0-1or1-0changesofstatesofneuronsatrandomtimes.Thispapershowsthattheimportantpropertiesoftheorigi-nalmodelremainintactwhenthesetwosimplificationsofthemodelingareeliminated.Althoughitisuncertainwheth-erthepropertiesofthesenewcontinuous"neurons"areyetcloseenoughtotheessentialpropertiesofrealneurons(and/ortheirdendriticarborization)tobedirectlyapplicabletoneurobiology,amajorconceptualobstaclehasbeenelimi-nated.ItiscertainthataCAMconstructedonthebasicideasoftheoriginalmodel(1)butbuiltofoperationalamplifiersandresistorswillfunction.FormoftheOriginalModelTheoriginalmodelusedtwo-statethreshold"neurons"thatfollowedastochasticalgorithm.Eachmodelneuronihadtwostates,characterizedbytheoutputVioftheneuronhav-ingthevaluesV?orVI(whichmayoftenbetakenas0and1,respectively).Theinputofeachneuroncamefromtwosources,externalinputsIiandinputsfromotherneurons.ThetotalinputtoneuroniisthenInputtoi=Hi=ETijVj+ii.jsi[1]TheelementTijcanbebiologicallyviewedasadescriptionofthesynapticinterconnectionstrengthfromneuronjtoneuroni.CAMandotherusefulcomputationsinthissysteminvolvethechangeofstateofthesystemwithtime.ThemotionofthestateofasystemofNneuronsinstatespacedescribesthecomputationthatthesetofneuronsisperforming.Amodelthereforemustdescribehowthestateevolvesintime,andtheoriginalmodeldescribesthis
View Full Document