Hyperspace Graph Paper and Hopfield Networks

HSGP can graph the energy function defined over states of a Hopfield network. To display a Hopfield network, choose New from the File menu, and select the Hopfield Network tab. Choose the desired size of the Hopfield network (it's best if you choose an even size, start with something small, in the range 8-12) and hit OK. You should see a new tab appear, the contents of which are all black. This blackness represents the energy function over an untrained (all weights are 0) Hopfield network.

Right-click on the black area, and select the "Show guide lines" option. You should see grid lines imposed over the black area. These lines assist in navigating around the space. As you move the mouse around the black area, note the binary pattern at the bottom of the window, which tells you which vector the mouse is currently over (for Hopfield network purposes, consider that 0 means -1; HSGP does not attempt to model 0 states).

To add a vector to the memory of the Hopfield network, shift+left-click on the desired area. You should notice that areas of the graph become lighter. Areas of lighter shading represent areas of lower energy. Thus, the area that you clicked on should be the lightest shaded of all areas (it should, in fact, be white). Additional vectors can be added by shift+left-clicking on other areas. Should you want to start again by resetting the weights of the network, select the Reset network option from the graph's context menu.

Interesting Features

To keep track of the stored memories of the Hopfield network, select Highlight -> Local Optima from the context menu. This option highlights (by colour) the optima (minima) of the energy function.

To highlight those points that are in the basin of attraction of a particular local optimum, choose Highlight -> Basin of Attraction from the context menu, then select the desired local optimum.

Handy Hint

It is possible to keep track of a particular set of points (such as the local optima) while highlighting other things. Choose the set of points you want to highlight (e.g., select Local Optima), then choose Marked groups -> Save as -> Group 1. The selected points should be marked with a small triangle.

To see what memories/local optima are retrieved from probing with a given pattern, choose Highlight -> Inverse Basin from the context menu, then select the desired probe. The set of points of increasing energy are highlighted. If you have previously saved the local optima as a marked group, you can see which can be reached. (Yes, it would be nice if the Reachable Local Optima option worked, but I haven't got around to that yet.)

It is possible to detect limit-cycles in the energy function. You may have noticed that sometimes points are highlighted in green, sometimes in red and other times in yellow. This colour-code works relative to the most recently selected (left-clicked) point: reds are points of higher energy, yellows are points of the same energy, and greens are points of lower energy. A limit-cycle then, can be seen when a local optimum is selected and the basin of attraction is highlighted. If the local optimum has neighbours (in the HSGP topology) that are coloured yellow, then these points form the cycle.