AI-Perceptron-Simple
view release on metacpan or search on metacpan
* The following features were implemented over the course of time (see also My::Perceptron v0.04 on github):
* create perceptron
* process data: &train method
* read csv - for training stage
* save and load the perceptron
* output algorithm for train
* read and calculate data line by line
* validate method
* read csv bulk
* write predicted values into original file
* write predicted values into new file
* test method
* read csv bulk
* write predicted values into original file
* write predicted values into new file
* confusion matrix
* read only expected and predicted columns, line by line
* return a hash of data
* TP, TN, FP, FN
* total entries
* accuracy
* sensitivity
* display confusion matrix data to console
(1) You are permitted to use the Standard Version and create and use
Modified Versions for any purpose without restriction, provided that
you do not Distribute the Modified Version.
Permissions for Redistribution of the Standard Version
(2) You may Distribute verbatim copies of the Source form of the
Standard Version of this Package in any medium without restriction,
either gratis or for a Distributor Fee, provided that you duplicate
all of the original copyright notices and associated disclaimers. At
your discretion, such verbatim copies may or may not include a
Compiled form of the Package.
(3) You may apply any bug fixes, portability changes, and other
modifications made available from the Copyright Holder. The resulting
Package will still be considered the Standard Version, and as such
will be subject to the Original License.
Distribution of Modified Versions of the Package as Source
docs/AI-Perceptron-Simple-1.04.html view on Meta::CPAN
<li><a href="#VERSION">VERSION</a></li>
<li><a href="#SYNOPSIS">SYNOPSIS</a></li>
<li><a href="#EXPORT">EXPORT</a></li>
<li><a href="#DESCRIPTION">DESCRIPTION</a></li>
<li><a href="#CONVENTIONS-USED">CONVENTIONS USED</a></li>
<li><a href="#DATASET-STRUCTURE">DATASET STRUCTURE</a></li>
<li><a href="#PERCEPTRON-DATA">PERCEPTRON DATA</a></li>
<li><a href="#DATA-PROCESSING-RELATED-SUBROUTINES">DATA PROCESSING RELATED SUBROUTINES</a>
<ul>
<li><a href="#shuffle_stimuli">shuffle_stimuli ( ... )</a></li>
<li><a href="#shuffle_data-original_data-shuffled_1-shuffled_2">shuffle_data ( $original_data => $shuffled_1, $shuffled_2, ... )</a></li>
<li><a href="#shuffle_data-ORIGINAL_DATA-shuffled_1-shuffled_2">shuffle_data ( ORIGINAL_DATA, $shuffled_1, $shuffled_2, ... )</a></li>
</ul>
</li>
<li><a href="#CREATION-RELATED-SUBROUTINES-METHODS">CREATION RELATED SUBROUTINES/METHODS</a>
<ul>
<li><a href="#new-options">new ( \%options )</a></li>
<li><a href="#get_attributes">get_attributes</a></li>
<li><a href="#learning_rate-value">learning_rate ( $value )</a></li>
<li><a href="#learning_rate">learning_rate</a></li>
<li><a href="#threshold-value">threshold ( $value )</a></li>
docs/AI-Perceptron-Simple-1.04.html view on Meta::CPAN
# or
$nerve->train(
$training_data_csv, $expected_column_name, $save_nerve_to,
$show_progress, $identifier); # these two parameters must go together
# validate
$nerve->take_lab_test( ... );
$nerve->take_mock_exam( ... );
# fill results to original file
$nerve->validate( {
stimuli_validate => $validation_data_csv,
predicted_column_index => 4,
} );
# or
# fill results to a new file
$nerve->validate( {
stimuli_validate => $validation_data_csv,
predicted_column_index => 4,
results_write_to => $new_csv
docs/AI-Perceptron-Simple-1.04.html view on Meta::CPAN
# load nerve data on the other computer
my $pearl_nerve = revive_from_yaml ( ... );
my $pearl_nerve = load_perceptron_yaml ( $yaml_nerve_file );
# processing data
use AI::Perceptron::Simple ":process_data";
shuffle_stimuli ( ... )
shuffle_data ( ORIGINAL_STIMULI, $new_file_1, $new_file_2, ... );
shuffle_data ( $original_stimuli => $new_file_1, $new_file_2, ... );</code></pre>
<h1 id="EXPORT">EXPORT</h1>
<p>None by default.</p>
<p>All the subroutines from <code>DATA PROCESSING RELATED SUBROUTINES</code>, <code>NERVE DATA RELATED SUBROUTINES</code> and <code>NERVE PORTABILITY RELATED SUBROUTINES</code> sections are importable through tags or manually specifying them.</p>
<p>The tags available include the following:</p>
<dl>
docs/AI-Perceptron-Simple-1.04.html view on Meta::CPAN
<p>Private methods/subroutines are prefixed with <code>_</code> or <code>&_</code> and they aren't meant to be called directly. You can if you want to. There are quite a number of them to be honest, just ignore them if you happen to see them ...
<p>Synonyms are placed before the actual ie. technical subroutines/methods. You will see <code>...</code> as the parameters if they are synonyms. Move to the next subroutine/method until you find something like <code>\%options</code> as the parameter...
<h1 id="DATASET-STRUCTURE">DATASET STRUCTURE</h1>
<p><i>This module can only process CSV files.</i></p>
<p>Any field ie columns that will be used for processing must be binary ie. <code>0</code> or <code>1</code> only. Your dataset can contain other columns with non-binary data as long as they are not one of the dendrites.</p>
<p>There are soem sample dataset which can be found in the <code>t</code> directory. The original dataset can also be found in <code>docs/book_list.csv</code>. The files can also be found <a href="https://github.com/Ellednera/AI-Perceptron-Simple">he...
<h1 id="PERCEPTRON-DATA">PERCEPTRON DATA</h1>
<p>The perceptron/neuron data is stored using the <code>Storable</code> module.</p>
<p>See <code>Portability of Nerve Data</code> section below for more info on some known issues.</p>
<h1 id="DATA-PROCESSING-RELATED-SUBROUTINES">DATA PROCESSING RELATED SUBROUTINES</h1>
<p>These subroutines can be imported using the tag <code>:process_data</code>.</p>
<p>These subroutines should be called in the procedural way.</p>
<h2 id="shuffle_stimuli">shuffle_stimuli ( ... )</h2>
<p>The parameters and usage are the same as <code>shuffled_data</code>. See the next two subroutines.</p>
<h2 id="shuffle_data-original_data-shuffled_1-shuffled_2">shuffle_data ( $original_data => $shuffled_1, $shuffled_2, ... )</h2>
<h2 id="shuffle_data-ORIGINAL_DATA-shuffled_1-shuffled_2">shuffle_data ( ORIGINAL_DATA, $shuffled_1, $shuffled_2, ... )</h2>
<p>Shuffles <code>$original_data</code> or <code>ORIGINAL_DATA</code> and saves them to other files.</p>
<h1 id="CREATION-RELATED-SUBROUTINES-METHODS">CREATION RELATED SUBROUTINES/METHODS</h1>
<h2 id="new-options">new ( \%options )</h2>
<p>Creates a brand new perceptron and initializes the value of each attribute / dendrite aka. weight. Think of it as the thickness or plasticity of the dendrites.</p>
<p>For <code>%options</code>, the followings are needed unless mentioned:</p>
<dl>
docs/AI-Perceptron-Simple-1.04.html view on Meta::CPAN
<dt id="tuning-status">tuning status</dt>
<dd>
<p>Indicates the nerve was tuned up, down or no tuning needed</p>
</dd>
<dt id="old-sum">old sum</dt>
<dd>
<p>The original sum of all <code>weightage * input</code> or <code>dendrite_size * binary_input</code></p>
</dd>
<dt id="threshold1">threshold</dt>
<dd>
<p>The threshold of the nerve</p>
</dd>
<dt id="new-sum">new sum</dt>
<dd>
docs/AI-Perceptron-Simple-1.04.html view on Meta::CPAN
<p>All the validation methods here have the same parameters as the actual <code>validate</code> method and they all do the same stuff. They are also used in the same way.</p>
<h2 id="take_mock_exam">take_mock_exam (...)</h2>
<h2 id="take_lab_test">take_lab_test (...)</h2>
<h2 id="validate-options">validate ( \%options )</h2>
<p>This method validates the perceptron against another set of data after it has undergone the training process.</p>
<p>This method calculates the output of each row of data and write the result into the predicted column. The data begin written into the new file or the original file will maintain it's sequence.</p>
<p>Please take note that this method will load all the data of the validation stimuli, so please split your stimuli into multiple files if possible and call this method a few more times.</p>
<p>For <code>%options</code>, the followings are needed unless mentioned:</p>
<dl>
<dt id="stimuli_validate-csv_file">stimuli_validate => $csv_file</dt>
<dd>
docs/AI-Perceptron-Simple-1.04.html view on Meta::CPAN
<p>This is the index of the column that contains the predicted output values. <code>$index</code> starts from <code>0</code>.</p>
<p>This column will be filled with binary numbers and the full new data will be saved to the file specified in the <code>results_write_to</code> key.</p>
</dd>
<dt id="results_write_to-new_csv_file">results_write_to => $new_csv_file</dt>
<dd>
<p>Optional.</p>
<p>The default behaviour will write the predicted output back into <code>stimuli_validate</code> ie the original data. The sequence of the data will be maintained.</p>
</dd>
</dl>
<p><i>*This method will call <code>_real_validate_or_test</code> to do the actual work.</i></p>
<h1 id="TESTING-RELATED-SUBROUTINES-METHODS">TESTING RELATED SUBROUTINES/METHODS</h1>
<p>All the testing methods here have the same parameters as the actual <code>test</code> method and they all do the same stuff. They are also used in the same way.</p>
docs/specifications.t view on Meta::CPAN
# $perceptron->validate( $stimuli_validate,
# $perceptron->train( $stimuli_train, $save_nerve_to_file )
# );
# and then check the confusion matrix, if not satisfied, run the loop again :)
$perceptron->validate( $stimuli_validate, $nerve_data_to_read );
$perceptron->test( $stimuli_test ); # loads nerve data from data file, turn into a object, then do the following:
# reads from csv :
# validation stimuli
# testing stimuli
# both will call the same subroutine to do calculation
# both will write predicted data into the original data file
# show results ie confusion matrix (TP-true positive, TN-true negative, FP-false positive, FN-false negative)
# this should only be done during validation and testing
$perceptron->generate_confusion_matrix( { 1 => $csv_header_true, 0 => $csv_header_false } );
# calculates the 4 thingy based on the current data on hand (RAM), don't read from file again, it shouldn't be a problem
# returns a hash
# ie it must be used together with validate() and test() to avoid problems
# ie validate() and test() must be in different scripts, which makes sense
# unless, create 3 similar objects to do the work in one go
lib/AI/Perceptron/Simple.pm view on Meta::CPAN
# or
$nerve->train(
$training_data_csv, $expected_column_name, $save_nerve_to,
$show_progress, $identifier); # these two parameters must go together
# validate
$nerve->take_lab_test( ... );
$nerve->take_mock_exam( ... );
# fill results to original file
$nerve->validate( {
stimuli_validate => $validation_data_csv,
predicted_column_index => 4,
} );
# or
# fill results to a new file
$nerve->validate( {
stimuli_validate => $validation_data_csv,
predicted_column_index => 4,
results_write_to => $new_csv
lib/AI/Perceptron/Simple.pm view on Meta::CPAN
# load nerve data on the other computer
my $pearl_nerve = revive_from_yaml ( ... );
my $pearl_nerve = load_perceptron_yaml ( $yaml_nerve_file );
# processing data
use AI::Perceptron::Simple ":process_data";
shuffle_stimuli ( ... )
shuffle_data ( ORIGINAL_STIMULI, $new_file_1, $new_file_2, ... );
shuffle_data ( $original_stimuli => $new_file_1, $new_file_2, ... );
=head1 EXPORT
None by default.
All the subroutines from C<DATA PROCESSING RELATED SUBROUTINES>, C<NERVE DATA RELATED SUBROUTINES> and C<NERVE PORTABILITY RELATED SUBROUTINES> sections are importable through tags or manually specifying them.
The tags available include the following:
=over 4
lib/AI/Perceptron/Simple.pm view on Meta::CPAN
Private methods/subroutines are prefixed with C<_> or C<&_> and they aren't meant to be called directly. You can if you want to. There are quite a number of them to be honest, just ignore them if you happen to see them :)
Synonyms are placed before the actual ie. technical subroutines/methods. You will see C<...> as the parameters if they are synonyms. Move to the next subroutine/method until you find something like C<\%options> as the parameter or anything that isn't...
=head1 DATASET STRUCTURE
I<This module can only process CSV files.>
Any field ie columns that will be used for processing must be binary ie. C<0> or C<1> only. Your dataset can contain other columns with non-binary data as long as they are not one of the dendrites.
There are soem sample dataset which can be found in the C<t> directory. The original dataset can also be found in C<docs/book_list.csv>. The files can also be found L<here|https://github.com/Ellednera/AI-Perceptron-Simple>.
=head1 PERCEPTRON DATA
The perceptron/neuron data is stored using the C<Storable> module.
See C<Portability of Nerve Data> section below for more info on some known issues.
=head1 DATA PROCESSING RELATED SUBROUTINES
These subroutines can be imported using the tag C<:process_data>.
These subroutines should be called in the procedural way.
=head2 shuffle_stimuli ( ... )
The parameters and usage are the same as C<shuffled_data>. See the next two subroutines.
=head2 shuffle_data ( $original_data => $shuffled_1, $shuffled_2, ... )
=head2 shuffle_data ( ORIGINAL_DATA, $shuffled_1, $shuffled_2, ... )
Shuffles C<$original_data> or C<ORIGINAL_DATA> and saves them to other files.
=cut
sub shuffle_stimuli {
shuffle_data( @_ );
}
sub shuffle_data {
my $stimuli = shift or croak "Please specify the original file name";
my @shuffled_stimuli_names = @_
or croak "Please specify the output files for the shuffled data";
my @aoa;
for ( @shuffled_stimuli_names ) {
# copied from _real_validate_or_test
# open for shuffling
my $aoa = csv (in => $stimuli, encoding => ":encoding(utf-8)");
my $attrib_array_ref = shift @$aoa; # 'remove' the header, it's annoying :)
@aoa = shuffle( @$aoa ); # this can only process actual array
lib/AI/Perceptron/Simple.pm view on Meta::CPAN
C<$display_stats> is B<optional> and the default is 0. It will display more output about the tuning process. It will show the followings:
=over 4
=item tuning status
Indicates the nerve was tuned up, down or no tuning needed
=item old sum
The original sum of all C<weightage * input> or C<dendrite_size * binary_input>
=item threshold
The threshold of the nerve
=item new sum
The new sum of all C<weightage * input> after fine-tuning the nerve
=back
lib/AI/Perceptron/Simple.pm view on Meta::CPAN
All the validation methods here have the same parameters as the actual C<validate> method and they all do the same stuff. They are also used in the same way.
=head2 take_mock_exam (...)
=head2 take_lab_test (...)
=head2 validate ( \%options )
This method validates the perceptron against another set of data after it has undergone the training process.
This method calculates the output of each row of data and write the result into the predicted column. The data begin written into the new file or the original file will maintain it's sequence.
Please take note that this method will load all the data of the validation stimuli, so please split your stimuli into multiple files if possible and call this method a few more times.
For C<%options>, the followings are needed unless mentioned:
=over 4
=item stimuli_validate => $csv_file
This is the CSV file containing the validation data, make sure that it contains a column with the predicted values as it is needed in the next key mentioned: C<predicted_column_index>
lib/AI/Perceptron/Simple.pm view on Meta::CPAN
=item predicted_column_index => $column_number
This is the index of the column that contains the predicted output values. C<$index> starts from C<0>.
This column will be filled with binary numbers and the full new data will be saved to the file specified in the C<results_write_to> key.
=item results_write_to => $new_csv_file
Optional.
The default behaviour will write the predicted output back into C<stimuli_validate> ie the original data. The sequence of the data will be maintained.
=back
I<*This method will call C<_real_validate_or_test> to do the actual work.>
=cut
sub take_mock_exam {
my ( $self, $data_hash_ref ) = @_;
$self->_real_validate_or_test( $data_hash_ref );
t/06-validate.t view on Meta::CPAN
my $nerve_file = $FindBin::Bin . "/perceptron_1.nerve";
for ( 0..5 ) {
print "Round $_\n";
$perceptron->train( TRAINING_DATA, "brand", $nerve_file, WANT_STATS, IDENTIFIER );
print "\n";
}
#print Dumper($perceptron), "\n";
# write ack to original file
my $ori_file_size = -s VALIDATION_DATA;
stdout_like {
ok ( $perceptron->validate( {
stimuli_validate => VALIDATION_DATA,
predicted_column_index => 4,
} ),
"Validate succedded!" );
} qr/book_list_validate\.csv/, "Correct output for validate when saving file";
# with new output file
t/06-validate_synonyms_lab.t view on Meta::CPAN
my $nerve_file = $FindBin::Bin . "/perceptron_1.nerve";
for ( 0..5 ) {
print "Round $_\n";
$perceptron->train( TRAINING_DATA, "brand", $nerve_file, WANT_STATS, IDENTIFIER );
print "\n";
}
#print Dumper($perceptron), "\n";
# write ack to original file
my $ori_file_size = -s VALIDATION_DATA;
stdout_like {
ok ( $perceptron->take_lab_test( {
stimuli_validate => VALIDATION_DATA,
predicted_column_index => 4,
} ),
"Validate succedded!" );
} qr/book_list_validate\.csv/, "Correct output for take_lab_test when saving file";
# with new output file
t/06-validate_synonyms_mock.t view on Meta::CPAN
my $nerve_file = $FindBin::Bin . "/perceptron_1.nerve";
for ( 0..5 ) {
print "Round $_\n";
$perceptron->train( TRAINING_DATA, "brand", $nerve_file, WANT_STATS, IDENTIFIER );
print "\n";
}
#print Dumper($perceptron), "\n";
# write ack to original file
my $ori_file_size = -s VALIDATION_DATA;
stdout_like {
ok ( $perceptron->take_mock_exam( {
stimuli_validate => VALIDATION_DATA,
predicted_column_index => 4,
} ),
"Validate succedded!" );
} qr/book_list_validate\.csv/, "Correct output for take_mock_exam when saving file";
# with new output file
t/10-test.t view on Meta::CPAN
use constant TEST_DATA_NEW_FILE => $FindBin::Bin . "/book_list_test-filled.csv";
use constant MODULE_NAME => "AI::Perceptron::Simple";
use constant WANT_STATS => 1;
use constant IDENTIFIER => "book_name";
my $nerve_file = $FindBin::Bin . "/perceptron_1.nerve";
ok( -s $nerve_file, "Found nerve file to load" );
my $mature_nerve = AI::Perceptron::Simple::load_perceptron( $nerve_file );
# write to original file
stdout_like {
ok ( $mature_nerve->test( {
stimuli_validate => TEST_DATA,
predicted_column_index => 4,
} ),
"Testing stage succedded!" );
} qr/book_list_test\.csv/, "Correct output for testing when saving back to original file";
# with new output file
stdout_like {
ok ( $mature_nerve->test( {
stimuli_validate => TEST_DATA,
predicted_column_index => 4,
results_write_to => TEST_DATA_NEW_FILE
} ),
"Testing stage succedded!" );
t/10-test_synonyms_exam.t view on Meta::CPAN
use constant TEST_DATA_NEW_FILE => $FindBin::Bin . "/book_list_test_exam-filled.csv";
use constant MODULE_NAME => "AI::Perceptron::Simple";
use constant WANT_STATS => 1;
use constant IDENTIFIER => "book_name";
my $nerve_file = $FindBin::Bin . "/perceptron_1.nerve";
ok( -s $nerve_file, "Found nerve file to load" );
my $mature_nerve = AI::Perceptron::Simple::load_perceptron( $nerve_file );
# write to original file
stdout_like {
ok ( $mature_nerve->take_real_exam( {
stimuli_validate => TEST_DATA,
predicted_column_index => 4,
} ),
"Testing stage succedded!" );
} qr/book_list_test\.csv/, "Correct output for testing when saving back to original file";
# with new output file
stdout_like {
ok ( $mature_nerve->take_real_exam( {
stimuli_validate => TEST_DATA,
predicted_column_index => 4,
results_write_to => TEST_DATA_NEW_FILE
} ),
"Testing stage succedded!" );
t/10-test_synonyms_work.t view on Meta::CPAN
use constant TEST_DATA_NEW_FILE => $FindBin::Bin . "/book_list_test_work-filled.csv";
use constant MODULE_NAME => "AI::Perceptron::Simple";
use constant WANT_STATS => 1;
use constant IDENTIFIER => "book_name";
my $nerve_file = $FindBin::Bin . "/perceptron_1.nerve";
ok( -s $nerve_file, "Found nerve file to load" );
my $mature_nerve = AI::Perceptron::Simple::load_perceptron( $nerve_file );
# write to original file
stdout_like {
ok ( $mature_nerve->work_in_real_world( {
stimuli_validate => TEST_DATA,
predicted_column_index => 4,
} ),
"Testing stage succedded!" );
} qr/book_list_test\.csv/, "Correct output for testing when saving back to original file";
# with new output file
stdout_like {
ok ( $mature_nerve->work_in_real_world( {
stimuli_validate => TEST_DATA,
predicted_column_index => 4,
results_write_to => TEST_DATA_NEW_FILE
} ),
"Testing stage succedded!" );
t/12-shuffle_data.t view on Meta::CPAN
use Test::More;
use Test::Output;
# use AI::Perceptron::Simple "shuffle_data";
use AI::Perceptron::Simple ":process_data";
use FindBin;
# this one will directly use "ORIGINAL_STIMULI" as the filename if use with "=>", strange
use constant ORIGINAL_STIMULI => $FindBin::Bin . "/book_list_to_shuffle.csv";
my $original_stimuli = $FindBin::Bin . "/book_list_to_shuffle.csv";
my $shuffled_data_1 = $FindBin::Bin . "/shuffled_1.csv";
my $shuffled_data_2 = $FindBin::Bin . "/shuffled_2.csv";
my $shuffled_data_3 = $FindBin::Bin . "/shuffled_3.csv";
ok( -e $original_stimuli, "Found the original file" );
{
local $@;
eval { shuffle_data };
like( $@, qr/^Please specify/, "Croaked at invocation with any arguements" )
}
{
local $@;
eval { shuffle_data($original_stimuli) };
like( $@, qr/output files/, "Croaked when new file names not present" )
}
shuffle_data( $original_stimuli => $shuffled_data_1, $shuffled_data_2, $shuffled_data_3 );
stdout_like {
shuffle_data( ORIGINAL_STIMULI, $shuffled_data_1, $shuffled_data_2, $shuffled_data_3 );
} qr/^Saved/, "Correct output after saving file";
ok( -e $shuffled_data_1, "Found the first shuffled file" );
ok( -e $shuffled_data_2, "Found the second shuffled file" );
ok( -e $shuffled_data_3, "Found the third shuffled file" );
t/12-shuffle_data_synonym.t view on Meta::CPAN
use Test::More;
use Test::Output;
use AI::Perceptron::Simple "shuffle_stimuli";
#use AI::Perceptron::Simple ":process_data";
use FindBin;
# this one will directly use "ORIGINAL_STIMULI" as the filename if use with "=>", strange
use constant ORIGINAL_STIMULI => $FindBin::Bin . "/book_list_to_shuffle.csv";
my $original_stimuli = $FindBin::Bin . "/book_list_to_shuffle.csv";
my $shuffled_data_1 = $FindBin::Bin . "/shuffled_1.csv";
my $shuffled_data_2 = $FindBin::Bin . "/shuffled_2.csv";
my $shuffled_data_3 = $FindBin::Bin . "/shuffled_3.csv";
ok( -e $original_stimuli, "Found the original file" );
{
local $@;
eval { shuffle_stimuli };
like( $@, qr/^Please specify/, "Croaked at invocation with any arguements" )
}
{
local $@;
eval { shuffle_stimuli($original_stimuli) };
like( $@, qr/output files/, "Croaked when new file names not present" )
}
shuffle_stimuli( $original_stimuli => $shuffled_data_1, $shuffled_data_2, $shuffled_data_3 );
stdout_like {
shuffle_stimuli( ORIGINAL_STIMULI, $shuffled_data_1, $shuffled_data_2, $shuffled_data_3 );
} qr/^Saved/, "Correct output after saving file";
ok( -e $shuffled_data_1, "Found the first shuffled file" );
ok( -e $shuffled_data_2, "Found the second shuffled file" );
ok( -e $shuffled_data_3, "Found the third shuffled file" );
( run in 0.319 second using v1.01-cache-2.11-cpan-f985c23238c )