AI-NeuralNet-BackProp
view release on metacpan or search on metacpan
<PRE>
input
/ \
O O
|\ /|
| \/ |
| /\ |
|/ \|
O O
\ /
mapper</PRE>
<P>In this diagram, each neuron is connected to one input of every
neuron in the layer below it, but there are not connections
between neurons in the same layer. Weights of the connection
are controlled by the neuron it is connected to, not the connecting
neuron. (E.g. the connecting neuron has no idea how much weight
its output has when it sends it, it just sends its output and the
weighting is taken care of by the receiving neuron.) This is the
method used to connect cells in every network built by this package.</P>
<P>Input is fed into the network via a call like this:</P>
<PRE>
use AI;
my $net = new AI::NeuralNet::BackProp(2,2);
my @map = (0,1);
my $result = $net->run(\@map);</PRE>
<P>Now, this call would probably not give what you want, because
the network hasn't ``learned'' any patterns yet. But this
illustrates the call. Run now allows strings to be used as
input. See <A HREF="#item_run"><CODE>run()</CODE></A> for more information.</P>
<P>Run returns a refrence with $size elements (Remember $size? $size
is what you passed as the second argument to the network
constructor.) This array contains the results of the mapping. If
you ran the example exactly as shown above, $result would probably
contain (1,1) as its elements.</P>
<P>To make the network learn a new pattern, you simply call the learn
method with a sample input and the desired result, both array
refrences of $size length. Example:</P>
<PRE>
use AI;
my $net = new AI::NeuralNet::BackProp(2,2);
my @map = (0,1);
my @res = (1,0);
$net->learn(\@map,\@res);
my $result = $net->run(\@map);</PRE>
<P>Now $result will conain (1,0), effectivly flipping the input pattern
around. Obviously, the larger $size is, the longer it will take
to learn a pattern. <CODE>Learn()</CODE> returns a string in the form of</P>
<PRE>
Learning took X loops and X wallclock seconds (X.XXX usr + X.XXX sys = X.XXX CPU).</PRE>
<P>With the X's replaced by time or loop values for that loop call. So,
to view the learning stats for every learn call, you can just:
</P>
<PRE>
print $net->learn(\@map,\@res);</PRE>
<P>If you call ``$net->debug(4)'' with $net being the
refrence returned by the <CODE>new()</CODE> constructor, you will get benchmarking
information for the learn function, as well as plenty of other information output.
See notes on <A HREF="#item_debug"><CODE>debug()</CODE></A> in the METHODS section, below.</P>
<P>If you do call $net->debug(1), it is a good
idea to point STDIO of your script to a file, as a lot of information is output. I often
use this command line:</P>
<PRE>
$ perl some_script.pl > .out</PRE>
<P>Then I can simply go and use emacs or any other text editor and read the output at my leisure,
rather than have to wait or use some 'more' as it comes by on the screen.</P>
<P>
<H2><A NAME="methods">METHODS</A></H2>
<DL>
<DT><STRONG><A NAME="item_BackProp">new AI::NeuralNet::BackProp($layers, $size [, $outputs, $topology_flag])</A></STRONG><BR>
<DD>
Returns a newly created neural network from an <CODE>AI::NeuralNet::BackProp</CODE>
object. The network will have <CODE>$layers</CODE> number layers in it
and each layer will have <CODE>$size</CODE> number of neurons in that layer.
<P>There is an optional parameter of $outputs, which specifies the number
of output neurons to provide. If $outputs is not specified, $outputs
defaults to equal $size. $outputs may not exceed $size. If $outputs
exceeds $size, the <CODE>new()</CODE> constructor will return undef.</P>
<P>The optional parameter, $topology_flag, defaults to 0 when not used. There are
three valid topology flag values:</P>
<P><STRONG>0</STRONG> <EM>default</EM>
My feed-foward style: Each neuron in layer X is connected to one input of every
neuron in layer Y. The best and most proven flag style.</P>
<PRE>
^ ^ ^
O\ O\ /O Layer Y
^\\/^/\/^
| //|\/\|
|/ \|/ \|
O O O Layer X
^ ^ ^</PRE>
<P>(Sorry about the bad art...I am no ASCII artist! :-)</P>
<P><STRONG>1</STRONG>
In addition to flag 0, each neuron in layer X is connected to every input of
the neurons ahead of itself in layer X.</P>
<P><STRONG>2</STRONG> <EM>(``L-U Style'')</EM>
No, its not ``Learning-Unit'' style. It gets its name from this: In a 2 layer, 3
neuron network, the connections form a L-U pair, or a W, however you want to look
at it.</P>
<PRE>
^ ^ ^
| | |
O-->O-->O
^ ^ ^
| | |
| | |
O-->O-->O
^ ^ ^
| | |</PRE>
<P>As you can see, each neuron is connected to the next one in its layer, as well
as the neuron directly above itself.</P>
<P>Before you can really do anything useful with your new neural network
object, you need to teach it some patterns. See the <A HREF="#item_learn"><CODE>learn()</CODE></A> method, below.</P>
<P></P>
<DT><STRONG><A NAME="item_learn">$net->learn($input_map_ref, $desired_result_ref [, options ]);</A></STRONG><BR>
<DD>
This will 'teach' a network to associate an new input map with a desired resuly.
It will return a string containg benchmarking information. You can retrieve the
pattern index that the network stored the new input map in after <A HREF="#item_learn"><CODE>learn()</CODE></A> is complete
with the <CODE>pattern()</CODE> method, below.
the whole data set again after calling <A HREF="#item_range"><CODE>range()</CODE></A> on a network.</P>
<P>Subsequent calls to <A HREF="#item_range"><CODE>range()</CODE></A> invalidate any previous calls to <A HREF="#item_range"><CODE>range()</CODE></A></P>
<P>NOTE: It is recomended, you call <A HREF="#item_range"><CODE>range()</CODE></A> before you call <A HREF="#item_learn"><CODE>learn()</CODE></A> or else you will get unexpected
results from any <A HREF="#item_run"><CODE>run()</CODE></A> call after <A HREF="#item_range"><CODE>range()</CODE></A> .</P>
<P></P>
<DT><STRONG>$net->range($bottom..$top);</STRONG><BR>
<DD>
This is a common form often used in a <CODE>for my $x (0..20)</CODE> type of <CODE>for()</CODE> constructor. It works
the exact same way. It will allow all numbers from $bottom to $top, inclusive, to be given
as outputs of the network. No other values will be possible, other than those between $bottom
and $top, inclusive.
<P></P>
<DT><STRONG>$net->range(\@values);</STRONG><BR>
<DD>
This allows you to specify a range of values as an array refrence. As the ranges are stored internally
as a refrence, this is probably the most natural way. Any value specified by an element in @values
will be allows as an output, no other values will be allowed.
<P></P>
<DT><STRONG>$net->range(``string of values'');</STRONG><BR>
<DD>
With this construct you can specify a string of values to be allowed as the outputs. This string
is simply taken an <A HREF="#item_crunch"><CODE>crunch()</CODE></A> -ed internally and saved as an array ref. This has the same effect
as calling:
<PRE>
$net->range($net->crunch("string of values"));</PRE>
<P></P>
<DT><STRONG>$net->range(``first string'',``second string'');</STRONG><BR>
<DD>
This is the same as calling:
<PRE>
$net->range($net->crunch("first string"),$net->crunch("second string"));</PRE>
<P>Or:</P>
<PRE>
@range = ($net->crunch("first string"),
$net->crunch("second string"));
$net->range(\@range);</PRE>
<P></P>
<DT><STRONG>$net->range($value1,$value2);</STRONG><BR>
<DD>
This is the same as calling:
<PRE>
$net->range([$value1,$value2]);</PRE>
<P>Or:
</P>
<PRE>
@range = ($value1,$value2);
$net->range(\@range);</PRE>
<P>The second example is the same as the first example.</P>
<P></P>
<DT><STRONG><A NAME="item_benchmarked">$net->benchmarked();</A></STRONG><BR>
<DD>
<B>UPDATED:</B> <CODE>bencmarked()</CODE> now returns just the string from <CODE>timestr()</CODE> for the last <A HREF="#item_run"><CODE>run()</CODE></A> or
<A HREF="#item_learn"><CODE>learn()</CODE></A> call. Exception: If the last call was a loop the string will be prefixed with ``%d loops and ''.
<P>This returns a benchmark info string for the last <A HREF="#item_learn"><CODE>learn()</CODE></A> or the last <A HREF="#item_run"><CODE>run()</CODE></A> call,
whichever occured later. It is easily printed as a string,
as following:</P>
<PRE>
print $net->benchmarked() . "\n";</PRE>
<P></P>
<DT><STRONG><A NAME="item_debug">$net->debug($level)</A></STRONG><BR>
<DD>
Toggles debugging off if called with $level = 0 or no arguments. There are four levels
of debugging.
<P>Level 0 ($level = 0) : Default, no debugging information printed. All printing is
left to calling script.</P>
<P>Level 1 ($level = 1) : This causes ALL debugging information for the network to be dumped
as the network runs. In this mode, it is a good idea to pipe your STDIO to a file, especially
for large programs.</P>
<P>Level 2 ($level = 2) : A slightly-less verbose form of debugging, not as many internal
data dumps.</P>
<P>Level 3 ($level = 3) : JUST prints weight mapping as weights change.</P>
<P>Level 4 ($level = 4) : JUST prints the benchmark info for EACH learn loop iteteration, not just
learning as a whole. Also prints the percentage difference for each loop between current network
results and desired results, as well as learning gradient ('incremenet').</P>
<P>Level 4 is useful for seeing if you need to give a smaller learning incrememnt to <A HREF="#item_learn"><CODE>learn()</CODE></A> .
I used level 4 debugging quite often in creating the letters.pl example script and the small_1.pl
example script.</P>
<P>Toggles debuging off when called with no arguments.</P>
<P></P>
<DT><STRONG><A NAME="item_save">$net->save($filename);</A></STRONG><BR>
<DD>
This will save the complete state of the network to disk, including all weights and any
words crunched with <A HREF="#item_crunch"><CODE>crunch()</CODE></A> . Also saves any output ranges set with <A HREF="#item_range"><CODE>range()</CODE></A> .
<P>This has now been modified to use a simple flat-file text storage format, and it does not
depend on any external modules now.</P>
<P></P>
<DT><STRONG><A NAME="item_load">$net->load($filename);</A></STRONG><BR>
<DD>
This will load from disk any network saved by <A HREF="#item_save"><CODE>save()</CODE></A> and completly restore the internal
state at the point it was <A HREF="#item_save"><CODE>save()</CODE></A> was called at.
<P></P>
<DT><STRONG><A NAME="item_join_cols">$net->join_cols($array_ref,$row_length_in_elements,$high_state_character,$low_state_character);</A></STRONG><BR>
<DD>
This is more of a utility function than any real necessary function of the package.
Instead of joining all the elements of the array together in one long string, like <CODE>join()</CODE> ,
it prints the elements of $array_ref to STDIO, adding a newline (\n) after every $row_length_in_elements
number of elements has passed. Additionally, if you include a $high_state_character and a $low_state_character,
it will print the $high_state_character (can be more than one character) for every element that
has a true value, and the $low_state_character for every element that has a false value.
If you do not supply a $high_state_character, or the $high_state_character is a null or empty or
undefined string, it <A HREF="#item_join_cols"><CODE>join_cols()</CODE></A> will just print the numerical value of each element seperated
by a null character (\0). <A HREF="#item_join_cols"><CODE>join_cols()</CODE></A> defaults to the latter behaviour.
<P></P>
<DT><STRONG><A NAME="item_pdiff">$net->pdiff($array_ref_A, $array_ref_B);</A></STRONG><BR>
<DD>
This function is used VERY heavily internally to calculate the difference in percent
between elements of the two array refs passed. It returns a %.10f (sprintf-format)
percent sting.
<P></P>
<DT><STRONG><A NAME="item_p">$net->p($a,$b);</A></STRONG><BR>
<DD>
Returns a floating point number which represents $a as a percentage of $b.
<P></P>
<DT><STRONG><A NAME="item_intr">$net->intr($float);</A></STRONG><BR>
<DD>
Rounds a floating-point number rounded to an integer using <CODE>sprintf()</CODE> and <CODE>int()</CODE> , Provides
better rounding than just calling <CODE>int()</CODE> on the float. Also used very heavily internally.
<P></P>
<DT><STRONG><A NAME="item_high">$net->high($array_ref);</A></STRONG><BR>
<DD>
Returns the index of the element in array REF passed with the highest comparative value.
<P></P>
<DT><STRONG><A NAME="item_low">$net->low($array_ref);</A></STRONG><BR>
<DD>
Returns the index of the element in array REF passed with the lowest comparative value.
<P></P>
<DT><STRONG><A NAME="item_show">$net->show();</A></STRONG><BR>
<DD>
This will dump a simple listing of all the weights of all the connections of every neuron
in the network to STDIO.
<P></P>
<DT><STRONG><A NAME="item_crunch">$net->crunch($string);</A></STRONG><BR>
<DD>
<B>UPDATED:</B> Now you can use a variabled instead of using qw(). Strings will be split internally.
Do not use <CODE>qw()</CODE> to pass strings to crunch.
<P>This splits a string passed with /[\s\t]/ into an array ref containing unique indexes
to the words. The words are stored in an intenal array and preserved across <A HREF="#item_load"><CODE>load()</CODE></A> and <A HREF="#item_save"><CODE>save()</CODE></A>
calls. This is designed to be used to generate unique maps sutible for passing to <A HREF="#item_learn"><CODE>learn()</CODE></A> and
<A HREF="#item_run"><CODE>run()</CODE></A> directly. It returns an array ref.</P>
<P>The words are not duplicated internally. For example:</P>
<PRE>
$net->crunch("How are you?");</PRE>
<P>Will probably return an array ref containing 1,2,3. A subsequent call of:</P>
<PRE>
$net->crunch("How is Jane?");</PRE>
<P>Will probably return an array ref containing 1,4,5. Notice, the first element stayed
the same. That is because it already stored the word ``How''. So, each word is stored
only once internally and the returned array ref reflects that.</P>
<P></P>
<DT><STRONG><A NAME="item_uncrunch">$net->uncrunch($array_ref);</A></STRONG><BR>
<DD>
Uncrunches a map (array ref) into an scalar string of words seperated by ' ' and returns the
string. This is ment to be used as a counterpart to the <A HREF="#item_crunch"><CODE>crunch()</CODE></A> method, above, possibly to
<A HREF="#item_uncrunch"><CODE>uncrunch()</CODE></A> the output of a <A HREF="#item_run"><CODE>run()</CODE></A> call. Consider the below code (also in ./examples/ex_crunch.pl):
<PRE>
use AI::NeuralNet::BackProp;
my $net = AI::NeuralNet::BackProp->new(2,3);
for (0..3) { # Note: The four learn() statements below could
# be replaced with learn_set() to do the same thing,
# but use this form here for clarity.
$net->learn($net->crunch("I love chips."), $net->crunch("That's Junk Food!"));
$net->learn($net->crunch("I love apples."), $net->crunch("Good, Healthy Food."));
$net->learn($net->crunch("I love pop."), $net->crunch("That's Junk Food!"));
$net->learn($net->crunch("I love oranges."),$net->crunch("Good, Healthy Food."));
}
my $response = $net->run($net->crunch("I love corn."));
print $net->uncrunch($response),"\n";</PRE>
<P>On my system, this responds with, ``Good, Healthy Food.'' If you try to run <A HREF="#item_crunch"><CODE>crunch()</CODE></A> with
``I love pop.'', though, you will probably get ``Food! apples. apples.'' (At least it returns
that on my system.) As you can see, the associations are not yet perfect, but it can make
for some interesting demos!</P>
<P></P>
<DT><STRONG><A NAME="item_crunched">$net->crunched($word);</A></STRONG><BR>
<DD>
This will return undef if the word is not in the internal crunch list, or it will return the
index of the word if it exists in the crunch list.
<P></P>
<DT><STRONG><A NAME="item_col_width">$net->col_width($width);</A></STRONG><BR>
<DD>
This is useful for formating the debugging output of Level 4 if you are learning simple
bitmaps. This will set the debugger to automatically insert a line break after that many
elements in the map output when dumping the currently run map during a learn loop.
<P>It will return the current width when called with a 0 or undef value.</P>
<P></P>
<DT><STRONG><A NAME="item_random">$net->random($rand);</A></STRONG><BR>
<DD>
This will set the randomness factor from the network. Default is 0.001. When called
with no arguments, or an undef value, it will return current randomness value. When
called with a 0 value, it will disable randomness in the network. See NOTES on learning
a 0 value in the input map with randomness disabled.
<P></P>
<DT><STRONG><A NAME="item_load_pcx">$net->load_pcx($filename);</A></STRONG><BR>
<DD>
Oh heres a treat... this routine will load a PCX-format file (yah, I know ... ancient format ... but
it is the only one I could find specs for to write it in Perl. If anyone can get specs for
any other formats, or could write a loader for them, I would be very grateful!) Anyways, a PCX-format
file that is exactly 320x200 with 8 bits per pixel, with pure Perl. It returns a blessed refrence to
a AI::NeuralNet::BackProp::PCX object, which supports the following routinges/members. See example
files ex_pcxl.pl and ex_pcx.pl in the ./examples/ directory.
<P></P>
<DT><STRONG><A NAME="item_%24pcx%2D%3E%7Bimage%7D">$pcx->{image}</A></STRONG><BR>
<DD>
This is an array refrence to the entire image. The array containes exactly 64000 elements, each
element contains a number corresponding into an index of the palette array, details below.
<P></P>
<DT><STRONG><A NAME="item_%24pcx%2D%3E%7Bpalette%7D">$pcx->{palette}</A></STRONG><BR>
<DD>
This is an array ref to an AoH (array of hashes). Each element has the following three keys:
<PRE>
$pcx->{palette}->[0]->{red};
$pcx->{palette}->[0]->{green};
$pcx->{palette}->[0]->{blue};</PRE>
<P>Each is in the range of 0..63, corresponding to their named color component.</P>
<P></P>
<DT><STRONG><A NAME="item_get_block">$pcx->get_block($array_ref);</A></STRONG><BR>
<DD>
Returns a rectangular block defined by an array ref in the form of:
<PRE>
[$left,$top,$right,$bottom]</PRE>
<P>These must be in the range of 0..319 for $left and $right, and the range of 0..199 for
$top and $bottom. The block is returned as an array ref with horizontal lines in sequental order.
I.e. to get a pixel from [2,5] in the block, and $left-$right was 20, then the element in
the array ref containing the contents of coordinates [2,5] would be found by [5*20+2] ($y*$width+$x).
</P>
<PRE>
print (@{$pcx->get_block(0,0,20,50)})[5*20+2];</PRE>
<P>This would print the contents of the element at block coords [2,5].</P>
<P></P>
<DT><STRONG><A NAME="item_get">$pcx->get($x,$y);</A></STRONG><BR>
<DD>
Returns the value of pixel at image coordinates $x,$y.
$x must be in the range of 0..319 and $y must be in the range of 0..199.
<P></P>
<DT><STRONG><A NAME="item_rgb">$pcx->rgb($index);</A></STRONG><BR>
<DD>
Returns a 3-element array (not array ref) with each element corresponding to the red, green, or
blue color components, respecitvely.
<P></P>
( run in 1.605 second using v1.01-cache-2.11-cpan-0bd6704ced7 )