DOC PREVIEW
Chapter 15 Prelude to ART

This preview shows page 1-2-3-22-23-24-45-46-47 out of 47 pages.

Save
View full document
View full document
Premium Document
Do you want full access? Go Premium and unlock all 47 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 47 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 47 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 47 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 47 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 47 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 47 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 47 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 47 pages.
Access to all documents
Download any document
Ad free experience
Premium Document
Do you want full access? Go Premium and unlock all 47 pages.
Access to all documents
Download any document
Ad free experience

Unformatted text preview:

Biological Signal Processing Richard B. Wells Chapter 15 Prelude to ART § 1. Adaptive Resonance Theory By most authors' accounts, the birth of adaptive resonance theory (ART) is recognized as being in 1976 with the appearance of [GROS6]. In an important sense this is true, but it diminishes the fact that ART developed over a period of years dating back into the late 1960s. Indeed, many of the key ideas used in [GROS6] will not make sense to the novice unless he has already become familiar with them from Grossberg's earlier publications. By 1976 they had become part of what is often called "the standard argument" used in the first sections of a technical journal paper. The purpose of this chapter is to present the key ideas and findings that are essential for the actual discussion of adaptive resonance theory in chapter 16. ART networks' undeserved reputation for being very complicated is due to an unfortunate historical accident. The foundational ideas that would lead to ART were discovered and published in the "dark age" of neural network research dating from 1968 until well into the 1980s. In some ways the history of ART can be compared to the Carolingian Renaissance that began with Charlemagne and flourished briefly in the ninth century before vanishing in the tumult of the tenth. Fortunately for ART, Grossberg – unlike Charlemagne – was still alive and active when the darkness lifted. (Had Charlemagne's successors been competent men, the dark ages might have ended 300 years sooner). ART's foundations never did disappear but, like the post-Carolingians, there are many younger theorists who came into the neural network field in the 1980s and 1990s, and who are simply too young to know about the propaedeutic work of the late 1960s and early 1970s [GROS2], [GROS3], [GROS11-18], [ELIA], [GROS4-6]. ART can looked at as the fusion of two major themes: recurrent on-center/off-surround networks and Outstars. There is, of course, more to an ART network than just these two elements, but they are the central elements and everything else exists to support their function. The on-center/off-surround structure is found in abundance in the central nervous system. Its basic form consists of a population of neurons that is tightly coupled and self-excitatory (the on-center) surrounded by other populations with which it has lateral inhibitory connections (the off-surround). Figure 15.1 illustrates the basic on-center/off-surround schema. The designation of a population as on-center or as off-surround is relative. Every population is an on-center to itself and its neighbors are its off-surround. One population is designated as on-center in figure 15.1 for 453Chapter 15: Prelude to ART Figure 15.1: Basic on-center/off-surround network anatomy Figure 15.2: Minimal ART anatomy. Feedback projections are made via Outstars. Feedforward projections project into each V2 node using an Instar anatomy. purposes of discussion. Lateral inhibitory paths are only shown for the population designated as the on-center population. It is to be understood that the off-center populations all have this same connectivity when they are regarded as an on-center. When the input stimulus to the on-center population is greater than that to the off-surround populations, the on-center node tends to suppress the activities of the off-surround nodes. If the off-surround stimuli are greater, the on-center activity tends to be suppressed. The simplest example of this is seen in the behavior of the MAXNET, which is an on-center/off-surround anatomy. The basic minimal ART anatomy is shown in figure 15.2. It consists of two on-center/off-surround layers, V1 and V2. Each node in V1 projects to each node in V2, and the fan-in to each V2 454Chapter 15: Prelude to ART Figure 15.3: Detailed diagram of each v2 node's input/output anatomy. node uses an Instar anatomy. Each node in V2 projects back to each node in V1, and the feedback from each V2 node is made by means of an Outstar. Figure 15.3 illustrates the input/output anatomy of a V2-layer node. In general a V1-layer node, v1i, is the same except that V1 nodes do not have an Outstar output. Input weights W2j and output weights Z2j are adaptable. Each node in figure 15.2 represents a population of B neurons and has a level of excitation x, with 0 ≤ x ≤ B, representing how many neurons in the population are active. The quantity B – x therefore represents how many neurons in the population are inactive. In the most general case, each node vi can represent a different population size Bi, although the most commonly encountered ART networks typically use the same population size B for all nodes. This network anatomy is termed a lumped network [GROS14]. The meaning of this term is as follows. Each population in each node is regarded as being made up of both excitatory and inhibitory neural subnetworks. If the excitatory and the inhibitory subpopulations have the same parameters and receive the same inputs, then the two subpopulations are indistinguishable with respect to every input source and with respect to their temporal dynamics. In more advanced ART networks it is possible to divide each node into an excitatory population and an inhibitory population, and to give each population differing sets of parameters. Such a network is said to be unlumped [ELIA], [GROS15]. All parameters and variables in an ART system are non-negative. Although many of the details of the network are very similar to what we have seen in the earlier chapters of this text, ART systems employ two quite different features, and these make all the difference. The first is that ART does not use the classical Instar map model. Rather, it uses a special kind of Instar map developed by Grossberg. We call it the shunting node Instar or SNI. Second, the activation functions f(u) employed in an ART system are different from all the more classical activation functions we have seen so far. In one of his early works, Grossberg studied how the properties of the activation function affect the behavior of a network constructed from SNIs [GROS14]. He 455Chapter 15: Prelude to ART found that the details of the activation function are crucial to how the network performs – much more crucial than is typically the case with the simpler network systems we have already studied. We will explore both these unique aspects of ART in turn. § 2. Shunting Node Instars When used as a general term, the name 'shunting


Chapter 15 Prelude to ART

Download Chapter 15 Prelude to ART
Our administrator received your request to download this document. We will send you the file to your email shortly.
Loading Unlocking...
Login

Join to view Chapter 15 Prelude to ART and access 3M+ class-specific study document.

or
We will never post anything without your permission.
Don't have an account?
Sign Up

Join to view Chapter 15 Prelude to ART 2 2 and access 3M+ class-specific study document.

or

By creating an account you agree to our Privacy Policy and Terms Of Use

Already a member?