COMP 200: Elements of Computer Science
Spring 2013

[Rice University]

Assignment 8:  Supplied Code

Code for training of ANNs

Stripped-down code for use in restoring trained ANNs

Both sets of code are set up to run the 7-segment display ANNs but the code to train a 2-bit negating ANN is included and commented out.  If you are having trouble with your 7-segment display ANN, try running the 2-bit negator ANN which is much simpler and easier to understand.

 

Features of the Supplied Code

The following is an overview of some of the major features in the supplied code.   Many of the features described here are either new or different than those shown in class, so please read this entire document.   For more information, please see the Python docs in the code itself.

 

Predefined constants

The following variables are provided as a convenience to help simplify one's code:

NEGATOR_TRAINING_DATA = training data for use with a 2-bit negation ANN

NEGATOR_TESTING_DATA = testing data  for use with a 2-bit negation ANN (same as NEGATOR_TRAINING_DATA )

DIGITS_TRAINING_DATA = training data for use with a 7 segment display interpretation ANN  (not available on the stripped-down code version above)

DIGITS_TESTING_DATA = testing data for use with a 7 segment display interpretation ANN (contains only the digits 0-9 from DIGITS_TRAINING_DATA )

LongRunner class

Repeatedly runs a Task or a Task subclass such as ANNTask.    When running, the display shows the number of iterations completed, the total number of iterations to  be done and the elapsed running time in hours:minutes:seconds.

LongRunner is configured with (i.e. the constructor of LongRunner takes) an object who is a sub-class of Task, e.g. ANNTask.   To start the process, the start method of LongRunner is called.

Task class

The general task that a LongRunner runs.   All specific tasks should be sub-classes of Task.    The Task class provides default behaviors for actions and behaviors to perform when the long running process is paused or stopped or ends.    The behavior to perform at each iteration when repeatedly called by LongRunner is expected to be provided by the specific sub-class of Task that will be used, e.g. ANNTask.

ANNTask class

A specific task designed for iterating through the training of an ANN.   An ANNTask is configured with (i.e. its constructor called with) a given ANN, the training data, the number of iterations to perform per invocation by the LongRunner ("block size"), and the number of trials of the training data to run through the ANN.    The testing data is typically one of the pre-defined values described above.  The total number of iterations is the number of elements in the training data times the number of trials.   The LongRunner's running display should increment the iteration count by block size.

The run method of the task provides the code to loop through a block of iterations.    The ANNTask object keeps track of what iteration count it is in and the return value of run tells the LongRunner what the current iteration count is, the total iterations it will perform and whether or not the iterations have completed.

test_ANN(anANN, testing_data, input_formatter, output_formatter) function

 This is a function to automatically test the given ANN using the given testing data.  The testing data is usually the pre-defined values described above.  The input and output formatters are functions that are used to format the input and output values for easier reading on the screen by the user.

test_ANN will print out each test it runs from the testing data, showing the input to the ANN's forward propagation method, the output of the forward propagation, and measure of the error of the output.

The error is calculated by taking the square root of the average of the squares of the differences between the actual and expected outputs of the ANN for the given inputs.    This is analogous to the standard deviation used in statistics and thus gives you an idea of how close the outputs are, on average, to the desired values.   Ideally, the error should be zero.  

test_ANN will also print out an "overall error", which is the similar to the errrors printed for each test input, but the average is performed over the entire testing data.   Note that the overall error is not the average of the individual errors.

Typical calls:

For 2-bit negator ANN:   test_ANN(ann, NEGATOR_TESTING_DATA, str, str)

For the 7-segment display ANN:  test_ANN(ann, DIGITS_TESTING_DATA, str_7seg, bits_to_num)

draw_ANN(anANN) function

This function will create a graphical display of the given ANN.

ANNStepper class

Enables the user to manually step through each forward and backward propagation step in a training run in a graphical display of the ANN.

An ANNStepper is configured with an ANN and calling its train method with the training or testing data (e.g. the pre-defined variables described above) and number of trials will start the stepping process.  

ann.save() and ANN.load() methods

ann.save() will print out the Python code necessary to recreate the ANN object exactly as it is, e.g. after it has been trained.   The dictionary of the ANN's data that is printed is also returned. 

Note that the data to recreate an ANN is so large that it including it in the full ANN training code will exceed CodeSkulptor's 64KB file size limit.   An ANN can only be restored using the stripped-down ANN code that has been supplied.

restored_ann = ANN.load(saved_ann_dict, fns) will recreate the ANN from the dictionary of saved ANN data plus the functions used to make the original ANN.

Running in regular Python

If one has a regular Python installation available, ANN training can run up to 10 times faster than in Codeskulptor.    Students are NOT required to run their code in regular Python.   This capability is provided for those students who would like to perform more extensive explorations of the ANN system.   The supplied code has comments describing what lines of code to comment out and what lines of code to uncomment for use in regular Python.   

There are 2 lines to be modified at the top of the code, one line at the ANN.load() method, and several lines at the bottom of the code, in the actual training code.

You will also need to add the line

ann.save() # change 'ann' to the variable name you are using for your ANN

after you do your training if you want the output that enables you to restore the trained ANN later.