Hebbian Learning

• Simplest form of weight selection mechanism is known as Hebbian learning.
• Mathematical abstraction of the principle of synaptic modulation first articulated by Hebb (1949).
• Hebb's law says that if one neuron stimulates another neuron when the receiving neuron is firing, the strength of the connection between the two cells is strengthened.
• Mathematically this is expressed as:
  w_ij = a_i a_j

  That is, the change in a weight is equal to the product of the activations of the units which it connects. So if both units are on (a_i = 1) or both units are off (a_i = -1) the strength of the weight is increased, otherwise it is decreased.

  Mathematically equivalent to matrix multiplication.