[Previous] [Next]

The Activation Function

• once the net input has been calculated for all units, activations are updated according to the following equation:
  if (net_i < 0)
     a_i = (max - a _i) net_i - decay (a_i - rest)
  Otherwise,
     a_i = (a _i - min) net_i - decay (a_i - rest)

  where a_i is the amount by which we change a_i.

• net_i determines whether the activation of unit i will increase or decrease
• if the activation of a unit is equal to max then the net believes the hypothesis completely
• if the activation is equal to min then the net disbelieves the hypothesis completely
• if the activation is equal to rest then the net is in an "I don't know state"
• the (max - a _i) and (a _i - min) terms ensure that the activation remains between min and max and doesn't continue to either grow or shrink without bound
• the -decay (a_i - rest) part of the equation forces the activation to return to the rest value in the absence of external input
• typically, we choose max > 0 >= rest >= min
• by default max is set to 1, min is set to -0.2, rest is set to -0.1 and decay is 0.1
• to change any of these parameters you first create a global value for that parameter by selecting the appropriate item from the Actions menu