AI-NeuralNet-Simple

 view release on metacpan or  search on metacpan

Changes  view on Meta::CPAN

Revision history for Perl extension AI::NeuralNet::Simple.

0.11  November 18, 2006
      Converted from Inline::C to XS
      No longer require 5.008.  5.005 and above should be fine.

0.10  December 29, 2005
      The following changes are all courtesy of Raphael Manfredi
      <Raphael_Manfredi [at] pobox.com>.
      Added tanh (bipolar) activation function.
      train_set() can now accept an error target to avoid over-training.
      Multiple network support.
      Persistence via storable.

0.02  September 21 2005
      Added pod and pod coverage tests
      Added Sub::Uplevel dependency to stop that annoying error failure :(

0.01  Sat Jan 31 12:19:00 2004
      Applied patch from "Daniel L. Ashbrook" <anjiro [at] cc.gatech.edu>
      to fix a small memory allocation bug in infer()

      Added learn_rate() method to expose the network.learn_rate.  This 
      should help programmers who wish to fine-tune the network training.

0.01  Sun Oct  5 10:03:18 2003
	- original version; created by h2xs 1.22 with options

Simple.xs  view on Meta::CPAN

    double **hidden_to_output;
} SYNAPSE;

SYNAPSE weight;

typedef struct {
    double *hidden;
    double *output;
} ERROR;

ERROR error;

typedef struct {
    double *input;
    double *hidden;
    double *output;
    double *target;
} LAYER;

LAYER neuron;

Simple.xs  view on Meta::CPAN

    int input;
    int hidden;
    int output;
} NEURON_COUNT;

typedef struct {
    float        learn_rate;
    double       delta;
    int          use_bipolar;
    SYNAPSE      weight;
    ERROR        error;
    LAYER        neuron;
    NEURON_COUNT size;
    double       *tmp;
} NEURAL_NETWORK;

int networks = 0;
NEURAL_NETWORK **network = NULL;

AV*    get_array_from_aoa(SV* scalar, int index);
AV*    get_array(SV* aref);

Simple.xs  view on Meta::CPAN

    n->delta = 1.0;
    n->use_bipolar = 0;

    n->tmp = malloc(sizeof(double) * n->size.input);

    n->neuron.input  = malloc(sizeof(double) * n->size.input);
    n->neuron.hidden = malloc(sizeof(double) * n->size.hidden);
    n->neuron.output = malloc(sizeof(double) * n->size.output);
    n->neuron.target = malloc(sizeof(double) * n->size.output);

    n->error.hidden  = malloc(sizeof(double) * n->size.hidden);
    n->error.output  = malloc(sizeof(double) * n->size.output);
    
    /* one extra for sentinel */
    n->weight.input_to_hidden  
        = malloc(sizeof(void *) * (input_layer_with_bias + 1));
    n->weight.hidden_to_output 
        = malloc(sizeof(void *) * (hidden_layer_with_bias + 1));

    if(!n->weight.input_to_hidden || !n->weight.hidden_to_output) {
        printf("Initial malloc() failed\n");
        return 0;

Simple.xs  view on Meta::CPAN

    for(row = n->weight.hidden_to_output; *row != 0; row++) {
        free(*row);
    }
    free(n->weight.hidden_to_output);

    free(n->neuron.input);
    free(n->neuron.hidden);
    free(n->neuron.output);
    free(n->neuron.target);

    free(n->error.hidden);
    free(n->error.output);

    free(n->tmp);

    network[handle] = NULL;
}

/*
 * Build a Perl reference on array `av'.
 * This performs something like "$rv = \@av;" in Perl.
 */

Simple.xs  view on Meta::CPAN

 * Back-propogation algorithm.  This is where the learning gets done.
 */
void c_back_propagate(NEURAL_NETWORK *n)
{
    int inp, hid, out;
    double (*activation_derivative)(NEURAL_NETWORK *, double);

    activation_derivative = n->use_bipolar ?
        hyperbolic_tan_derivative : sigmoid_derivative;

    /* calculate the output layer error (step 3 for output cell) */
    for (out = 0; out < n->size.output; out++) {
        n->error.output[out] =
            (n->neuron.target[out] - n->neuron.output[out]) 
              * (*activation_derivative)(n, n->neuron.output[out]);
    }

    /* calculate the hidden layer error (step 3 for hidden cell) */
    for (hid = 0; hid < n->size.hidden; hid++) {

        n->error.hidden[hid] = 0.0;
        for (out = 0; out < n->size.output; out++) {
            n->error.hidden[hid] 
                += n->error.output[out] 
                 * n->weight.hidden_to_output[hid][out];
        }
        n->error.hidden[hid] 
            *= (*activation_derivative)(n, n->neuron.hidden[hid]);
    }

    /* update the weights for the output layer (step 4) */
    for (out = 0; out < n->size.output; out++) {
        for (hid = 0; hid < n->size.hidden; hid++) {
            n->weight.hidden_to_output[hid][out] 
                += (n->learn_rate 
                  * n->error.output[out] 
                  * n->neuron.hidden[hid]);
        }

        /* update the bias */
        n->weight.hidden_to_output[n->size.hidden][out] 
            += (n->learn_rate 
              * n->error.output[out]);
    }

    /* update the weights for the hidden layer (step 4) */
    for (hid = 0; hid < n->size.hidden; hid++) {

        for  (inp = 0; inp < n->size.input; inp++) {
            n->weight.input_to_hidden[inp][hid] 
                += (n->learn_rate 
                  * n->error.hidden[hid] 
                  * n->neuron.input[inp]);
        }

        /* update the bias */
        n->weight.input_to_hidden[n->size.input][hid] 
            += (n->learn_rate 
              * n->error.hidden[hid]);
    }
}

/*
 * Compute the Mean Square Error between the actual output and the
 * targeted output.
 */
double mean_square_error(NEURAL_NETWORK *n, double *target)
{
    double error = 0.0;
    int i;

    for (i = 0; i < n->size.output; i++)
        error += sqr(target[i] - n->neuron.output[i]);

    return 0.5 * error;
}

double c_train(int handle, SV* input, SV* output)
{
    NEURAL_NETWORK *n = c_get_network(handle);
    int i,length;
    AV *array;
    double *input_array  = malloc(sizeof(double) * n->size.input);
    double *output_array = malloc(sizeof(double) * n->size.output);
    double error;

    if (! is_array_ref(input) || ! is_array_ref(output)) {
        croak("train() takes two arrayrefs.");
    }
    
    array  = get_array(input);
    length = av_len(array)+ 1;
    
    if (length != n->size.input) {
        croak("Length of input array does not match network");

Simple.xs  view on Meta::CPAN

    length = av_len(array) + 1;
    
    if (length != n->size.output) {
        croak("Length of output array does not match network");
    }
    for (i = 0; i < length; i++) {
        output_array[i] = get_float_element(array, i);
    }

    c_feed(n, input_array, output_array, 1);
    error = mean_square_error(n, output_array);

    free(input_array);
    free(output_array);

    return error;
}

int c_new_network(int input, int hidden, int output)
{
    NEURAL_NETWORK *n;
    int handle;

    handle = c_new_handle();
    n = c_get_network(handle);

Simple.xs  view on Meta::CPAN

    c_assign_random_weights(n);

    return handle;
}

double c_train_set(int handle, SV* set, int iterations, double mse)
{
    NEURAL_NETWORK *n = c_get_network(handle);
    AV     *input_array, *output_array; /* perl arrays */
    double *input, *output; /* C arrays */
    double max_error = 0.0;

    int set_length=0;
    int i,j;
    int index;

    set_length = av_len(get_array(set))+1;

    if (!set_length)
        croak("_train_set() array ref has no data");
    if (set_length % 2)

Simple.xs  view on Meta::CPAN

        if (av_len(output_array)+1 != n->size.output)
            croak("Length of output data does not match");

        for (j = 0; j < n->size.output; j++) {
            index = (i/2*n->size.output)+j;
            output[index] = get_float_element(output_array, j); 
        }
    }

    for (i = 0; i < iterations; i++) {
        max_error = 0.0;

        for (j = 0; j < (set_length/2); j++) {
            double error;

            c_feed(n, &input[j*n->size.input], &output[j*n->size.output], 1);

            if (mse >= 0.0 || i == iterations - 1) {
                error = mean_square_error(n, &output[j*n->size.output]);
                if (error > max_error)
                    max_error = error;
            }
        }

        if (mse >= 0 && max_error <= mse)    /* Below their target! */
            break;
    }

    free(input);
    free(output);

    return max_error;
}

SV* c_infer(int handle, SV *array_ref)
{
    NEURAL_NETWORK *n = c_get_network(handle);
    int    i;
    AV     *perl_array, *result = newAV();

    /* feed the data */
    perl_array = get_array(array_ref);

lib/AI/NeuralNet/Simple.pm  view on Meta::CPAN

    return $self;
}

sub infer {
    my ( $self, $data ) = @_;
    c_infer( $self->handle, $data );
}

sub winner {

    # returns index of largest value in inferred answer
    my ( $self, $data ) = @_;
    my $arrayref = c_infer( $self->handle, $data );

    my $largest = 0;
    for ( 0 .. $#$arrayref ) {
        $largest = $_ if $arrayref->[$_] > $arrayref->[$largest];
    }
    return $largest;
}

lib/AI/NeuralNet/Simple.pm  view on Meta::CPAN


  This module is a simple neural net designed for those who have an interest
  in artificial intelligence but need a "gentle" introduction.  This is not
  intended to replace any of the neural net modules currently available on the
  CPAN.

=head1 DESCRIPTION

=head2 The Disclaimer

Please note that the following information is terribly incomplete.  That's
deliberate.  Anyone familiar with neural networks is going to laugh themselves
silly at how simplistic the following information is and the astute reader will
notice that I've raised far more questions than I've answered.

So why am I doing this?  Because I'm giving I<just enough> information for
someone new to neural networks to have enough of an idea of what's going on so
they can actually use this module and then move on to something more powerful,
if interested.

=head2 The Biology

lib/AI/NeuralNet/Simple.pm  view on Meta::CPAN

the expected results:

 input   output
 1   2   1    2
 -----   ------
 1   1   0    1
 1   0   0    1
 0   1   0    1
 0   0   1    0

The type of network we use is a forward-feed back error propagation network,
referred to as a back-propagation network, for short.  The way it works is
simple.  When we feed in our input, it travels from the input to hidden layers
and then to the output layers.  This is the "feed forward" part.  We then
compare the output to the expected results and measure how far off we are.  We
then adjust the weights on the "output to hidden" synapses, measure the error
on the hidden nodes and then adjust the weights on the "hidden to input"
synapses.  This is what is referred to as "back error propagation".

We continue this process until the amount of error is small enough that we are
satisfied.  In reality, we will rarely if ever get precise results from the
network, but we learn various strategies to interpret the results.  In the
example above, we use a "winner takes all" strategy.  Which ever of the output
nodes has the greatest value will be the "winner", and thus the answer.

In the examples directory, you will find a program named "logical_or.pl" which
demonstrates the above process.

=head2 Building a network

lib/AI/NeuralNet/Simple.pm  view on Meta::CPAN

=item 1 Designing

This is choosing the number of layers and the number of neurons per layer.  In
C<AI::NeuralNet::Simple>, the number of layers is fixed.

With more complete neural net packages, you can also pick which activation
functions you wish to use and the "learn rate" of the neurons.

=item 2 Training

This involves feeding the neural network enough data until the error rate is
low enough to be acceptable.  Often we have a large data set and merely keep
iterating until the desired error rate is achieved.

=item 3 Measuring results

One frequent mistake made with neural networks is failing to test the network
with different data from the training data.  It's quite possible for a
backpropagation network to hit what is known as a "local minimum" which is not
truly where it should be.  This will cause false results.  To check for this,
after training we often feed in other known good data for verification.  If the
results are not satisfactory, perhaps a different number of neurons per layer
should be tried or a different set of training data should be supplied.

lib/AI/NeuralNet/Simple.pm  view on Meta::CPAN

  $VAR1 = [
          '0.00993729281477686',
          '0.990100297418451'
        ];

That clearly has the second output item being close to 1, so as a helper method
for use with a winner take all strategy, we have ...

=head2 C<winner(\@input)>

This method returns the index of the highest value from inferred results:

  print $net->winner([1,1]); # will likely print "1"

For a more comprehensive example of how this is used, see the 
"examples/game_ai.pl" program.

=head1 EXPORT

None by default.

lib/AI/NeuralNet/Simple.pm  view on Meta::CPAN


The C code in this module is based heavily upon Mr. Jones backpropogation
network in the book.  The "game ai" example in the examples directory is based
upon an example he has graciously allowed me to use.  I I<had> to use it
because it's more fun than many of the dry examples out there :)

"Naturally Intelligent Systems", by Maureen Caudill and Charles Butler,
copyright (c) 1990 by Massachussetts Institute of Technology.

This book is a decent introduction to neural networks in general.  The forward
feed back error propogation is but one of many types.

=head1 AUTHORS

Curtis "Ovid" Poe, C<ovid [at] cpan [dot] org>

Multiple network support, persistence, export of MSE (mean squared error),
training until MSE below a given threshold and customization of the
activation function added by Raphael Manfredi C<Raphael_Manfredi@pobox.com>.

=head1 COPYRIGHT AND LICENSE

Copyright (c) 2003-2005 by Curtis "Ovid" Poe

Copyright (c) 2006 by Raphael Manfredi

This library is free software; you can redistribute it and/or modify



( run in 0.669 second using v1.01-cache-2.11-cpan-49f99fa48dc )