The Hebbian learning rule produces weight patterns that are symmetric. wij = ai aj = aj ai = wji
So energy can be written as:
H = C - (ij) wij ai aj
where (ij) means all of the distinct pairs ij - so the pair 12 is the same as pair 21. The constant term C accumulates the (ii) pairs and is a constant which only depends of the number of units.
Now consider the affect of an activation update on the energy. Suppose ai' is the new value of ai. If ai' = ai then the energy does not change. If ai' = - ai we can pick out the terms invovling ai' and express the change in energy as:
H' - H
= C - ai' ¬ j = i wij aj - (C - ai¬ j = i wij aj)
= 2 ai¬ j = i wij aj
= 2 aij wij aj - 2 wii
Now by the activation rule:
ai = sgn(j wijai)
So:
- ai = sgn(j wijai)
And 2 aij wij aj must be negative. Also wii is equal to the number of patterns, so - 2 wii is also negative so the energy must decrease.