The Interactive Activation and Competition Network: How Neural Networks Process Information

Copyright © Simon Dennis, 1998.

1. Introduction
   
2. The Interactive Activation and Competition Mechanism
   
3. The Dynamics of IAC Networks *
   
   • Decay
     
   • Boundaries
     
   • Equilibrium
     
   • Competition
     
   • Resonance
     
   • Blocking and Hysteresis
     
   • Oscillation
     
4. Back to the Jets and Sharks Network
   
5. How Neural Networks Process Information
   
   • Content Addressability
     
   • Robustness to Noise
     
   • Generalization
     
   • Default Assignment
     
6. References
   
7. Endnotes
   
8. Slides

* These sections contain some mathematics which can be omitted on a first reading if desired.

Introduction

The Interactive Activation and Competition network (IAC, McClelland 1981; McClelland & Rumelhart 1981; Rumelhart & McClelland 1982) embodies many of the properties that make neural networks useful information processing models. In this chapter, we will use the IAC network to demonstrate several of these properties including content addressability, robustness in the face of noise, generalisation across exemplars and the ability to provide plausible default values for unknown variables. The chapter begins with an example of an IAC network to allow you to see a full network in action. Then we delve into the IAC mechanism in detail creating a number of small networks to demonstrate the network dynamics. Finally, we return to the original example and show how it embodies the information processing capabilities outlined above.

An IAC network consists of a number of competitive pools of units (see figure 1). Each unit represents some micro-hypothesis or feature. The units within each competitive pool are mutually exclusive features and are interconnected with negative weights. Between the pools positive weights indicate features or micro-hypotheses which are consistent. When the network is cycled, units connected by positive weights to active units become more active, while units connected by negative weights to active units are inhibited. The connections are, in general, bidirectional making the network interactive (i.e. the activation of one unit both influences and is influenced by the units to which it is connected).

Figure 1: The Jets and Sharks IAC network. The squares represent units and the arrows represent weights. Red arrows indicate positive weights and blue arrows represent negative weights.

The network in figure 1 represents information about two rival gangs - the Jets and Sharks (McClelland 1981). The central pool of units represent members of the two gangs. The pools around the edges represent features of these members including their name, occupation, marital status, gang membership, age and educational level. Within each pool the units are connected with negative weights indicating that they are mutually exclusive. If you are in your 20s you can't also be in your 30s, for example. Between the pools positive weights hold the specific information about each gang member. The instance unit corresponding to Art is connected to the units representing the name Art, Pusher, Single, Jet, 40's and Junior High (J.H.) - the characteristics of Art.

Table 1 shows the facts known about each of the gang members (Note: This is a subset of the full database presented in McClelland 1981).

Name	Gang	Age	Education	Marital Status	Occupation
Art	Jets	40's	J.H.	Single	Pusher
Rick	Sharks	30's	H.S.	Divorced	Burglar
Sam	Jets	20's	College	Single	Bookie
Ralph	Jets	30's	J.H.	Single	Pusher
Lance	Jets	20's	J.H.	Married	Burglar

Exercise 1: Click on the Art instance unit (in the central pool) to activate that unit. Now click on the cycle button. List the units that become active.

Exercise 2: Now click on the ZeroUnits button to return all of the units to zero activation. Activate the instance unit for Art and the pusher and bookie units. Now cycle. Describe what happens to the Pusher and Bookie units.

In this introductory section we have provided an overview of the IAC network. In the next section, we will examine the IAC in more detail to see exactly how it computes activation values. Then we take a step back and look at some the important properties of the way in which neural network process information using the Jets and Sharks network as an example.

---

The BrainWave homepage.