Exercises 5.1

This tutorial introduces two properties of memory: matching (knowing whether an item has been seen before) and retrieval (retrieving an item based on an associated or partial cue). Memory is a fast process - for many tasks, it only takes one episode to remember items (called one-shot learning).

---

Goals:

• introduction to distributed representations as vectors, and associations as matrices, including exposure to the term 'vector', and familiarity with different ways of representing vectors: as pattern sets, as weights in networks, as cartesian co-ordinates, or algebraically;
• experience with using the BrainWave simulator for memory tasks, including exposure to the concepts of pattern sets: INPUT SET,OUTPUT SET;
• ability to use vectors and matrices to model matching and retrieval in memory tasks, including exposure to the terms: matching, retrieval, recognition, recall;
• reflection on differences between neural network styles for modeling learning and memory.
• ability to plot a two-element vector on cartesian co-ordinates and explain the relationship to the network;
  
• ability to write the algebraic expression for a vector memory and explain the relationship to the network;
  
• ability to train and test a vector memory for one or more items;
  
• ability to calculate the vector memory values when two patterns are superimposed, in terms of:
  • network weights,
    
  • cartesian co-ordinates,
    
  • vector addition.

Exercise 5.1.1

Task: Build a vector memory network to learn to discriminate between items seen before and new items.

Load the simulator, BrainWave.

Set up the Hebbian options:

Set up a vector memory network: Create the data sets: The items in this exercise are Add the items to the data sets as follows: Train the network for one epoch and record the weights in the TRAIN FROG column of the attached table.

Test each of the items, FROG, TOAD and KOALA, and record the match values (the activation of the output unit) in the second table.

Train the network for one more epoch and test again recording the values in the appropriate table.

Question 1.1 What happens to the values of the weights after each training trial?

Question 1.2 Explain the match values after one trial.

Question 1.3 What happens to the match values after a second training trial? Why?




Add KOALA to the input set and an output value of 1 to the output set.
Zero the weights (using the ACTIONS menu) and retrain the network on the updated input set. Test the network as before, recording the values in the table in the TRAIN FROG & KOALA columns.

Delete KOALA from the input set and add TOAD. Zero the weights, retrain and test as above, recording the values in the TRAIN FROG & TOAD columns.

You should have six weight values and nine match values for each training trial.

Question 1.4 Create a graph of the match values after the first training trial: plot three lines, one for each test item, against the three training conditions:

Questions 1.5 Explain the shape of each line on the graph.

Question 1.6 Fill in the required information on each of the diagrams given at the end of this worksheet:

Question 1.7 There are three training conditions (FROG alone, FROG & KOALA, FROG & TOAD), and three ways of representing the information for each training trial. What information is most easily found from each type of representation:

---