[Previous] [Next]

Why are stored patterns stable?

To understand why the Hebbian rule produces stable patterns we will start by considering a network which has stored just one pattern. For this pattern to be stable:
sgn( w_ija_i) = a_j for all j

because then the transfer function produces no changes. Now with only one pattern stored (that being the pattern currently on the units):

w_ij = a_i a_j So if n is the number of units:

sgn( wijai)	= sgn( aj ai ai)
	= sgn(n aj)
	= aj for all j

Furthermore, provided at least half of the units in the starting pattern have the correct value sgn(net_i) will give the correct value and the network will converge to the stored pattern - so the pattern is an attractor of the network.

In this simple case there are actually two attractors. The second one is the inverse of the stored pattern - the reverse state. If more than half of the units in the starting pattern are different from the stored pattern then the network will end up in the reverse state.