DOC PREVIEW
Berkeley COMPSCI C267 - The Cat is Out of the Bag

This preview shows page 1-2-3-4 out of 12 pages.

Save
View full document
View full document
Premium Document
Do you want full access? Go Premium and unlock all 12 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 12 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 12 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 12 pages.
Access to all documents
Download any document
Ad free experience
Premium Document
Do you want full access? Go Premium and unlock all 12 pages.
Access to all documents
Download any document
Ad free experience

Unformatted text preview:

TheCat isOut of the Bag:Cortical Simulations with 109Neurons, 1013SynapsesRajagopal Ananthanarayanan1, Steven K. Esser1Horst D. Simon2, and Dharmendra S. Modha11IBM Almaden Research Center, 650 Harry Road, San Jose, CA 951202Lawrence Berkeley National Laboratory, One Cyclotron Road, Berkeley, CA 94720{ananthr,sesser}@us.ibm.com, [email protected], [email protected] the quest for cognitive computing, we have built a mas-sively parallel cortical simulator, C2, that incorporates anumber of innovations in computation, memory, and com-munication. Using C2 on LLNL’s Dawn Blue Gene/P su-percomputer with 147, 456 CPUs and 144 TB of main mem-ory, we report two cortical simulations – at unprecedentedscale – that effectively saturate the entire memory capac-ity and refresh it at least every simulated second. The firstsimulation consists of 1.6 billion neurons and 8.87 trillionsynapses with experimentally-measured gray matter thala-mocortical connectivity. The second simulation has 900 mil-lion neurons and 9 trillion synapses with probabilistic con-nectivity. We demonstrate nearly perfect weak scaling andattractive strong scaling. The simulations, which incorpo-rate phenomenological spiking neurons, individual learningsynapses, axonal delays, and dynamic synaptic channels, ex-ceed the scale of the cat cortex, marking the dawn of a newera in the scale of cortical simulations.1. INTRODUCTIONLarge-scale cortical simulation is an emerging interdisciplinaryfield drawing upon computational neuroscience, simulationmethodology, and supercomputing. Towards brain-like cog-nitive computers, a cortical simulator is a critical enablingtechnology to test hypotheses of brain structure, dynamicsand function, and to interact as an embodied being with vir-tual or real environments. Simulations are also an integralcomponent of cutting-edge research, such as DARPA’s Sys-tems of Neuromorphic Adaptive Plastic Scalable Electronics(SyNAPSE) program that has the ambitious goal of engen-dering a revolutionary system of compact, low-power neu-romorphic and synaptronic chips using novel synapse-likenanodevices. We compare the SyNAPSE objectives withthe number of neurons and synapses in cortices of mammalsclassically used as models in neuroscience1[8, 22, 29, 32].Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copiesare not made or distributed for profit or commercial advantage, and thatcopies bear this notice and the full citation on the first page. Copyrightsfor components of this work owned by others than ACM must be honored.Abstracting with credit is permitted. To copy otherwise, to republish, topost on servers or to redistribute to lists, requires prior specific permissionand/or a fee.SC09 November 14-20, 2009, Portland, Oregon, USA (c) 2009 ACM 978-1-60558-744-8/09/11. . .$10.00MouseRat SyNAPSECat HumanNeurons ×108.160 .550 1 7.63 200Synapses ×1012.128 .442 1 6.10 200Simulations at mammalian scale pose a formidable challengeeven to modern-day supercomputers, consuming a vast num-ber of parallel processor cycles, stressing the communicationcapacity, and filling all available memory and refreshing itat least every second of simulation time, thus requiring ex-tremely innovative simulation software design. Previously,using a Blue Gene/L (BG/L) [14] supercomputer, at IBMT. J. Watson Research Center, with 32, 768 CPUs and 8 TBmain memory, we reported the design and implementationof a cortical simulator C2 and demonstrated near real-timesimulations at scales of mouse [13, 3] and rat cortices [2].In this paper, we have significantly enriched our simulationswith neurobiological data from physiology and anatomy (Sec-tion 2), and have simultaneously enhanced C2 with algorith-mic optimizations and usability features (Section 3). As a re-sult of these innovations, as our main contribution, by usingLawrence Livermore National Labs’ state-of-the-art DawnBlue Gene/P (BG/P) [17] supercomputer with 147, 456 CPUsand 144 TB of total memory, we achieve cortical simula-tions at an unprecedented and historic scale exceeding thatof cat cerebral cortex (Sections 4 and 5). Our simulationsuse single-compartment phenomenological spiking neurons[19], learning synapses with spike-timing dependent plastic-ity [36], and axonal delays. Our specific results are summa-rized below:• We simulated a biologically-inspired model with 1.617 ×109neurons and 0.887 × 1013synapses, roughly 643 timesslower than real-time per Hertz of average neuronal firingrate. The model used biologically-measured gray mat-ter thalamocortical connectivity from cat visual cortex [7](Figure 1), dynamic synaptic channels, and a simulationtime step of 0.1 ms (Section 4).• We simulated a model with 0.9×109neurons and 0.9×1013synapses, using probabilistic connectivity and a simula-tion time step of 1 ms, only 83 times slower than real-timeper Hertz of average neuronal firing rate (Section 5).• We demonstrated that the simulator has nearly perfectweak scaling (Section 5) implying that doubling of mem-ory resource translates into a corresponding doubling ofthe model size that can be simulated. From a strong scal-ing perspective (Section 5), at constant model size, wedemonstrated that using more CPUs reduces the simula-tion time, closing the gap to real-time simulations.Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. SC09 November 14-20, 2009, Portland, Oregon, USA. Copyright 2009 ACM 978-1-60558-744-8/09/11 ...$10.00. 2.NEUROSCIENCE 101Here, we describe essential dynamical features from neuro-physiology and structural features from neuroanatomy; for acomprehensive overview of neuroscience, please see [21]. Thekey features incorporated in our simulations are highlightedbelow in bold.2.1 Neurophysiology: DynamicsThe computational building block of the brain is the neu-ron, a cell specialized to continuously integrate inputs andto generate signals based on the outcome of this integrationprocess. The term neuron was coined by Heinrich WilhelmGottfried von Waldeyer-Hartz in 1891 to capture the dis-crete


View Full Document

Berkeley COMPSCI C267 - The Cat is Out of the Bag

Documents in this Course
Lecture 4

Lecture 4

52 pages

Split-C

Split-C

5 pages

Lecture 5

Lecture 5

40 pages

Load more
Download The Cat is Out of the Bag
Our administrator received your request to download this document. We will send you the file to your email shortly.
Loading Unlocking...
Login

Join to view The Cat is Out of the Bag and access 3M+ class-specific study document.

or
We will never post anything without your permission.
Don't have an account?
Sign Up

Join to view The Cat is Out of the Bag 2 2 and access 3M+ class-specific study document.

or

By creating an account you agree to our Privacy Policy and Terms Of Use

Already a member?