view release on metacpan or search on metacpan
lib/AI/Perceptron/Simple.pm view on Meta::CPAN
The parameters and usage are the same as C<shuffled_data>. See the next two subroutines.
=head2 shuffle_data ( $original_data => $shuffled_1, $shuffled_2, ... )
=head2 shuffle_data ( ORIGINAL_DATA, $shuffled_1, $shuffled_2, ... )
Shuffles C<$original_data> or C<ORIGINAL_DATA> and saves them to other files.
=cut
sub shuffle_stimuli {
shuffle_data( @_ );
}
sub shuffle_data {
my $stimuli = shift or croak "Please specify the original file name";
my @shuffled_stimuli_names = @_
or croak "Please specify the output files for the shuffled data";
my @aoa;
for ( @shuffled_stimuli_names ) {
# copied from _real_validate_or_test
# open for shuffling
my $aoa = csv (in => $stimuli, encoding => ":encoding(utf-8)");
my $attrib_array_ref = shift @$aoa; # 'remove' the header, it's annoying :)
lib/AI/Perceptron/Simple.pm view on Meta::CPAN
Optional. The default is C<0.5>
This is the passing rate to determine the neuron output (C<0> or C<1>).
Generally speaking, this value is usually between C<0> and C<1>. However, it all depend on your combination of numbers for the other options.
=back
=cut
sub new {
my $class = shift;
my $data_ref = shift;
my %data = %{ $data_ref };
# check keys
$data{ learning_rate } = LEARNING_RATE if not exists $data{ learning_rate };
$data{ threshold } = THRESHOLD if not exists $data{ threshold };
#####
lib/AI/Perceptron/Simple.pm view on Meta::CPAN
bless \%processed_data, $class;
}
=head2 get_attributes
Obtains a hash of all the attributes of the perceptron
=cut
sub get_attributes {
my $self = shift;
%{ $self->{attributes_hash_ref} };
}
=head2 learning_rate ( $value )
=head2 learning_rate
If C<$value> is given, sets the learning rate to C<$value>. If not, then it returns the learning rate.
=cut
sub learning_rate {
my $self = shift;
if ( @_ ) {
$self->{learning_rate} = shift;
} else {
$self->{learning_rate}
}
}
=head2 threshold ( $value )
=head2 threshold
If C<$value> is given, sets the threshold / passing rate to C<$value>. If not, then it returns the passing rate.
=cut
sub threshold {
my $self = shift;
if ( @_ ) {
$self->{ threshold } = shift;
} else {
$self->{ threshold };
}
}
=head1 TRAINING RELATED SUBROUTINES/METHODS
lib/AI/Perceptron/Simple.pm view on Meta::CPAN
=item new sum
The new sum of all C<weightage * input> after fine-tuning the nerve
=back
If C<$display_stats> is specified ie. set to C<1>, then you B<MUST> specify the C<$identifier>. C<$identifier> is the column / header name that is used to identify a specific row of data in C<$stimuli_train_csv>.
=cut
sub tame {
train( @_ );
}
sub exercise {
train( @_ );
}
sub train {
my $self = shift;
my( $stimuli_train_csv, $expected_output_header, $save_nerve_to_file, $display_stats, $identifier ) = @_;
$display_stats = 0 if not defined $display_stats;
if ( $display_stats and not defined $identifier ) {
croak "Please specifiy a string for \$identifier if you are trying to display stats";
}
# CSV processing is all according to the documentation of Text::CSV
open my $data_fh, "<:encoding(UTF-8)", $stimuli_train_csv
lib/AI/Perceptron/Simple.pm view on Meta::CPAN
Calculates and returns the C<sum(weightage*input)> for each individual row of data. Actually, it justs add up all the existing weight since the C<input> is always 1 for now :)
C<%stimuli_hash> is the actual data to be used for training. It might contain useless columns.
This will get all the avaible dendrites using the C<get_attributes> method and then use all the keys ie. headers to access the corresponding values.
This subroutine should be called in the procedural way for now.
=cut
sub _calculate_output {
my $self = shift;
my $stimuli_hash_ref = shift;
my %dendrites = $self->get_attributes;
my $sum; # this is the output
for ( keys %dendrites ) {
# if input is 1 for a dendrite, then calculate it
if ( $stimuli_hash_ref->{ $_ } ) {
# $sum += $dendrites{ $_ } * 1; # no need, if 1 then it is always the value itself
lib/AI/Perceptron/Simple.pm view on Meta::CPAN
=item TUNE_DOWN
Value is C<0>
=back
This subroutine should be called in the procedural way for now.
=cut
sub _tune {
my $self = shift;
my ( $stimuli_hash_ref, $tuning_status ) = @_;
my %dendrites = $self->get_attributes;
for ( keys %dendrites ) {
if ( $tuning_status == TUNE_DOWN ) {
if ( $stimuli_hash_ref->{ $_ } ) { # must check this one, it must be 1 before we can alter the actual dendrite size in the nerve :)
$self->{ attributes_hash_ref }{ $_ } -= $self->learning_rate;
lib/AI/Perceptron/Simple.pm view on Meta::CPAN
Optional.
The default behaviour will write the predicted output back into C<stimuli_validate> ie the original data. The sequence of the data will be maintained.
=back
I<*This method will call C<_real_validate_or_test> to do the actual work.>
=cut
sub take_mock_exam {
my ( $self, $data_hash_ref ) = @_;
$self->_real_validate_or_test( $data_hash_ref );
}
sub take_lab_test {
my ( $self, $data_hash_ref ) = @_;
$self->_real_validate_or_test( $data_hash_ref );
}
sub validate {
my ( $self, $data_hash_ref ) = @_;
$self->_real_validate_or_test( $data_hash_ref );
}
=head1 TESTING RELATED SUBROUTINES/METHODS
All the testing methods here have the same parameters as the actual C<test> method and they all do the same stuff. They are also used in the same way.
=head2 take_real_exam (...)
lib/AI/Perceptron/Simple.pm view on Meta::CPAN
This method is used to put the trained nerve to the test. You can think of it as deploying the nerve for the actual work or maybe putting the nerve into an empty brain and see how
well the brain survives :)
This method works and behaves the same way as the C<validate> method. See C<validate> for the details.
I<*This method will call &_real_validate_or_test to do the actual work.>
=cut
# redirect to _real_validate_or_test
sub take_real_exam {
my ( $self, $data_hash_ref ) = @_;
$self->_real_validate_or_test( $data_hash_ref );
}
sub work_in_real_world {
my ( $self, $data_hash_ref ) = @_;
$self->_real_validate_or_test( $data_hash_ref );
}
sub test {
my ( $self, $data_hash_ref ) = @_;
$self->_real_validate_or_test( $data_hash_ref );
}
=head2 _real_validate_or_test ( $data_hash_ref )
This is where the actual validation or testing takes place.
C<$data_hash_ref> is the list of parameters passed into the C<validate> or C<test> methods.
This is a B<method>, so use the OO way. This is one of the exceptions to the rules where private subroutines are treated as methods :)
=cut
sub _real_validate_or_test {
my $self = shift; my $data_hash_ref = shift;
#####
my @missing_keys;
for ( qw( stimuli_validate predicted_column_index ) ) {
push @missing_keys, $_ unless exists $data_hash_ref->{ $_ };
}
croak "Missing keys: @missing_keys" if @missing_keys;
lib/AI/Perceptron/Simple.pm view on Meta::CPAN
}
=head2 &_fill_predicted_values ( $self, $stimuli_validate, $predicted_index, $aoa )
This is where the filling in of the predicted values takes place. Take note that the parameters naming are the same as the ones used in the C<validate> and C<test> method.
This subroutine should be called in the procedural way.
=cut
sub _fill_predicted_values {
my ( $self, $stimuli_validate, $predicted_index, $aoa ) = @_;
# CSV processing is all according to the documentation of Text::CSV
open my $data_fh, "<:encoding(UTF-8)", $stimuli_validate
or croak "Can't open $stimuli_validate: $!";
my $csv = Text::CSV->new( {auto_diag => 1, binary => 1} );
my $attrib = $csv->getline($data_fh);
lib/AI/Perceptron/Simple.pm view on Meta::CPAN
=item more_stats => 1
Optional.
Setting it to C<1> will process more stats that are usually not so important eg. C<precision>, C<specificity> and C<F1_Score>
=back
=cut
sub get_exam_results {
my ( $self, $info ) = @_;
$self->get_confusion_matrix( $info );
}
sub get_confusion_matrix {
my ( $self, $info ) = @_;
my %c_matrix = _collect_stats( $info ); # processes total_entries, accuracy, sensitivity etc
%c_matrix;
}
=head2 &_collect_stats ( \%options )
Generates a hash of confusion matrix based on C<%options> given in the C<get_confusion_matrix> method.
=cut
sub _collect_stats {
my $info = shift;
my $file = $info->{ full_data_file };
my $actual_header = $info->{ actual_output_header };
my $predicted_header = $info->{ predicted_output_header };
my $more_stats = defined ( $info->{ more_stats } ) ? 1 : 0;
my %c_matrix = (
true_positive => 0, true_negative => 0, false_positive => 0, false_negative => 0,
accuracy => 0, sensitivity => 0
);
lib/AI/Perceptron/Simple.pm view on Meta::CPAN
%c_matrix;
}
=head2 &_calculate_total_entries ( $c_matrix_ref )
Calculates and adds the data for the C<total_entries> key in the confusion matrix hash.
=cut
sub _calculate_total_entries {
my $c_matrix = shift;
my $total = $c_matrix->{ true_negative } + $c_matrix->{ false_positive };
$total += $c_matrix->{ false_negative } + $c_matrix->{ true_positive };
$c_matrix->{ total_entries } = $total;
}
=head2 &_calculate_accuracy ( $c_matrix_ref )
Calculates and adds the data for the C<accuracy> key in the confusion matrix hash.
=cut
sub _calculate_accuracy {
my $c_matrix = shift;
my $numerator = $c_matrix->{ true_positive } + $c_matrix->{ true_negative };
my $denominator = $numerator + $c_matrix->{ false_positive } + $c_matrix->{ false_negative };
$c_matrix->{ accuracy } = $numerator / $denominator * 100;
# no need to return anything, we're using ref
}
=head2 &_calculate_sensitivity ( $c_matrix_ref )
Calculates and adds the data for the C<sensitivity> key in the confusion matrix hash.
=cut
sub _calculate_sensitivity {
my $c_matrix = shift;
my $numerator = $c_matrix->{ true_positive };
my $denominator = $numerator + $c_matrix->{ false_negative };
$c_matrix->{ sensitivity } = $numerator / $denominator * 100;
# no need to return anything, we're using ref
}
=head2 &_calculate_precision ( $c_matrix_ref )
Calculates and adds the data for the C<precision> key in the confusion matrix hash.
=cut
sub _calculate_precision {
my $c_matrix = shift;
my $numerator = $c_matrix->{ true_positive };
my $denominator = $numerator + $c_matrix->{ false_positive };
$c_matrix->{ precision } = $numerator / $denominator * 100;
}
=head2 &_calculate_specificity ( $c_matrix_ref )
Calculates and adds the data for the C<specificity> key in the confusion matrix hash.
=cut
sub _calculate_specificity {
my $c_matrix = shift;
my $numerator = $c_matrix->{ true_negative };
my $denominator = $numerator + $c_matrix->{ false_positive };
$c_matrix->{ specificity } = $numerator / $denominator * 100;
}
=head2 &_calculate_f1_score ( $c_matrix_ref )
Calculates and adds the data for the C<F1_Score> key in the confusion matrix hash.
=cut
sub _calculate_f1_score {
my $c_matrix = shift;
my $numerator = 2 * $c_matrix->{ true_positive };
my $denominator = $numerator + $c_matrix->{ false_positive } + $c_matrix->{ false_negative };
$c_matrix->{ F1_Score } = $numerator / $denominator * 100;
}
=head2 &_calculate_negative_predicted_value( $c_matrix_ref )
Calculates and adds the data for the C<negative_predicted_value> key in the confusion matrix hash.
=cut
sub _calculate_negative_predicted_value {
my $c_matrix = shift;
my $numerator = $c_matrix->{ true_negative };
my $denominator = $numerator + $c_matrix->{ false_negative };
$c_matrix->{ negative_predicted_value } = $numerator / $denominator * 100;
}
=head2 &_calculate_false_negative_rate( $c_matrix_ref )
Calculates and adds the data for the C<false_negative_rate> key in the confusion matrix hash.
=cut
sub _calculate_false_negative_rate {
my $c_matrix = shift;
my $numerator = $c_matrix->{ false_negative };
my $denominator = $numerator + $c_matrix->{ true_positive };
$c_matrix->{ false_negative_rate } = $numerator / $denominator * 100;
}
=head2 &_calculate_false_positive_rate( $c_matrix_ref )
Calculates and adds the data for the C<false_positive_rate> key in the confusion matrix hash.
=cut
sub _calculate_false_positive_rate {
my $c_matrix = shift;
my $numerator = $c_matrix->{ false_positive };
my $denominator = $numerator + $c_matrix->{ true_negative };
$c_matrix->{ false_positive_rate } = $numerator / $denominator * 100;
}
=head2 &_calculate_false_discovery_rate( $c_matrix_ref )
Calculates and adds the data for the C<false_discovery_rate> key in the confusion matrix hash.
=cut
sub _calculate_false_discovery_rate {
my $c_matrix = shift;
my $numerator = $c_matrix->{ false_positive };
my $denominator = $numerator + $c_matrix->{ true_positive };
$c_matrix->{ false_discovery_rate } = $numerator / $denominator * 100;
}
=head2 &_calculate_false_omission_rate( $c_matrix_ref )
Calculates and adds the data for the C<false_omission_rate> key in the confusion matrix hash.
=cut
sub _calculate_false_omission_rate {
my $c_matrix = shift;
my $numerator = $c_matrix->{ false_negative };
my $denominator = $numerator + $c_matrix->{ true_negative };
$c_matrix->{ false_omission_rate } = $numerator / $denominator * 100;
}
=head2 &_calculate_balanced_accuracy( $c_matrix_ref )
Calculates and adds the data for the C<balanced_accuracy> key in the confusion matrix hash.
=cut
sub _calculate_balanced_accuracy {
my $c_matrix = shift;
my $numerator = $c_matrix->{ sensitivity } + $c_matrix->{ specificity };
my $denominator = 2;
$c_matrix->{ balanced_accuracy } = $numerator / $denominator; # numerator already in %
}
=head2 display_exam_results ( ... )
lib/AI/Perceptron/Simple.pm view on Meta::CPAN
=item one_as => $category_one_name
=back
Please take note that non-ascii characters ie. non-English alphabets B<might> cause the output to go off :)
For the C<%labels>, there is no need to enter "actual X", "predicted X" etc. It will be prefixed with C<A: > for actual and C<P: > for the predicted values by default.
=cut
sub display_exam_results {
my ( $self, $c_matrix, $labels ) = @_;
$self->display_confusion_matrix( $c_matrix, $labels );
}
sub display_confusion_matrix {
my ( $self, $c_matrix, $labels ) = @_;
#####
my @missing_keys;
for ( qw( zero_as one_as ) ) {
push @missing_keys, $_ unless exists $labels->{ $_ };
}
croak "Missing keys: @missing_keys" if @missing_keys;
#####
lib/AI/Perceptron/Simple.pm view on Meta::CPAN
=head2 &_build_matrix ( $c_matrix, $labels )
Builds the matrix using C<Text::Matrix> module.
C<$c_matrix> and C<$labels> are the same as the ones passed to C<display_exam_results> and C<>display_confusion_matrix.
Returns a list C<( $matrix, $c_matrix )> which can directly be passed to C<_print_extended_matrix>.
=cut
sub _build_matrix {
my ( $c_matrix, $labels ) = @_;
my $predicted_columns = [ "P: ".$labels->{ zero_as }, "P: ".$labels->{ one_as }, "Sum" ];
my $actual_rows = [ "A: ".$labels->{ zero_as }, "A: ".$labels->{ one_as }, "Sum"];
# row sum
my $actual_0_sum = $c_matrix->{ true_negative } + $c_matrix->{ false_positive };
my $actual_1_sum = $c_matrix->{ false_negative } + $c_matrix->{ true_positive };
# column sum
lib/AI/Perceptron/Simple.pm view on Meta::CPAN
}
=head2 &_print_extended_matrix ( $matrix, $c_matrix )
Extends and outputs the matrix on the screen.
C<$matrix> and C<$c_matrix> are the same as returned by C<&_build_matrix>.
=cut
sub _print_extended_matrix {
my ( $matrix, $c_matrix ) = @_;
print "~~" x24, "\n";
print "CONFUSION MATRIX (A:actual P:predicted)\n";
print "~~" x24, "\n";
print $matrix->matrix();
print "~~" x24, "\n";
lib/AI/Perceptron/Simple.pm view on Meta::CPAN
The parameters and usage are the same as C<save_perceptron>. See the next subroutine.
=head2 save_perceptron ( $nerve, $nerve_file )
Saves the C<AI::Perceptron::Simple> object into a C<Storable> file. There shouldn't be a need to call this method manually since after every training
process this will be called automatically.
=cut
sub preserve {
save_perceptron( @_ );
}
sub save_perceptron {
my $self = shift;
my $nerve_file = shift;
use Storable;
store $self, $nerve_file;
no Storable;
}
=head2 revive (...)
The parameters and usage are the same as C<load_perceptron>. See the next subroutine.
=head2 load_perceptron ( $nerve_file_to_load )
Loads the data and turns it into a C<AI::Perceptron::Simple> object as the return value.
=cut
sub revive {
load_perceptron( @_ );
}
sub load_perceptron {
my $nerve_file_to_load = shift;
use Storable;
my $loaded_nerve = retrieve( $nerve_file_to_load );
no Storable;
$loaded_nerve;
}
=head1 NERVE PORTABILITY RELATED SUBROUTINES
lib/AI/Perceptron/Simple.pm view on Meta::CPAN
=head2 preserve_as_yaml ( ... )
The parameters and usage are the same as C<save_perceptron_yaml>. See the next subroutine.
=head2 save_perceptron_yaml ( $nerve, $yaml_nerve_file )
Saves the C<AI::Perceptron::Simple> object into a C<YAML> file.
=cut
sub preserve_as_yaml {
save_perceptron_yaml( @_ );
}
sub save_perceptron_yaml {
my $self = shift;
my $nerve_file = shift;
use YAML;
YAML::DumpFile( $nerve_file, $self );
no YAML;
}
=head2 revive_from_yaml (...)
The parameters and usage are the same as C<load_perceptron>. See the next subroutine.
=head2 load_perceptron_yaml ( $yaml_nerve_file )
Loads the YAML data and turns it into a C<AI::Perceptron::Simple> object as the return value.
=cut
sub revive_from_yaml {
load_perceptron_yaml( @_ );
}
sub load_perceptron_yaml {
my $nerve_file_to_load = shift;
use YAML;
local $YAML::LoadBlessed = 1;
my $loaded_nerve = YAML::LoadFile( $nerve_file_to_load );
no YAML;
$loaded_nerve;
}
=head1 TO DO
t/02-creation.t view on Meta::CPAN
is( $perceptron->threshold, 0.85, "Correct custom passing rate -> ".$perceptron->threshold );
# get_attributes()
my %attributes = $perceptron->get_attributes;
for ( @attributes ) {
ok( $attributes{ $_ }, "Attribute \'$_\' present" );
is( $attributes{ $_ }, $initial_value, "Correct initial value (".$attributes{$_}.") for \'$_\'" );
}
# don't try to use Test::Carp, it won't work, it only tests for direct calling of carp and croak etc
subtest "Caught missing mandatory parameters" => sub {
eval {
my $no_attribs = AI::Perceptron::Simple->new( { initial_value => $initial_value} );
};
like( $@, qr/attribs/, "Caught missing attribs" );
eval {
my $perceptron = AI::Perceptron::Simple->new( { attribs => \@attributes} );
};
like($@, qr/initial_value/, "Caught missing initial_value");
t/02-state_portable.t view on Meta::CPAN
my @attributes = qw ( has_trees trees_coverage_more_than_half has_other_living_things );
my $total_headers = scalar @attributes;
my $perceptron = AI::Perceptron::Simple->new( {
initial_value => 0.01,
attribs => \@attributes
} );
subtest "All data related subroutines found" => sub {
# this only checks if the subroutines are contained in the package
ok( AI::Perceptron::Simple->can("preserve_as_yaml"), "&preserve_as_yaml is present" );
ok( AI::Perceptron::Simple->can("save_perceptron_yaml"), "&save_perceptron_yaml is persent" );
ok( AI::Perceptron::Simple->can("revive_from_yaml"), "&revive_from_yaml is present" );
ok( AI::Perceptron::Simple->can("load_perceptron_yaml"), "&load_perceptron_yaml is present" );
};
my $yaml_nerve_file = $FindBin::Bin . "/portable_nerve.yaml";
t/04-train.t view on Meta::CPAN
local $@ = "";
eval { $perceptron->train( TRAINING_DATA, "brand", $nerve_file, WANT_STATS, IDENTIFIER) };
is ( $@, "", "No problem with \'train\' method (verbose) so far" );
}
ok ( $perceptron->train( TRAINING_DATA, "brand", $nerve_file), "No problem with \'train\' method (non-verbose) so far" );
# no longer returns the file anymore since v0.03
# is ( $perceptron->train( TRAINING_DATA, "brand", $nerve_file), $nerve_file, "\'train\' method returns the correct value" );
subtest "Data related subroutine found" => sub {
ok( AI::Perceptron::Simple->can("save_perceptron"), "&save_perceptron is persent" );
ok( AI::Perceptron::Simple->can("load_perceptron"), "&loaded_perceptron is present" );
};
ok( save_perceptron( $perceptron, $nerve_file ), "save_perceptron is working good so far" );
ok( -e $nerve_file, "Found the perceptron file" );
ok( load_perceptron( $nerve_file ), "Perceptron loaded" );
my $loaded_perceptron = load_perceptron( $nerve_file );
t/08-confusion_matrix.t view on Meta::CPAN
eval {
$perceptron->display_confusion_matrix( \%c_matrix );
};
like ( $@, qr/zero_as one_as/, "Both keys not found" );
}
# more_stats enabled
subtest "More stats" => sub {
my %c_matrix_more_stats = $perceptron->get_confusion_matrix( {
full_data_file => TEST_FILE,
actual_output_header => "brand",
predicted_output_header => "predicted",
more_stats => 1,
} );
like ( $c_matrix_more_stats{ precision }, qr/66.66/, "Precision seems correct to me" );
is ( $c_matrix_more_stats{ specificity }, 80, "Specificity seems correct to me" );
t/08-confusion_matrix_synonyms.t view on Meta::CPAN
local $@;
eval {
$perceptron->display_exam_results( \%c_matrix );
};
like ( $@, qr/zero_as one_as/, "Both keys not found" );
}
# more_stats enabled
subtest "More stats" => sub {
my %c_matrix_more_stats = $perceptron->get_confusion_matrix( {
full_data_file => TEST_FILE,
actual_output_header => "brand",
predicted_output_header => "predicted",
more_stats => 1,
} );
like ( $c_matrix_more_stats{ precision }, qr/66.66/, "Precision seems correct to me" );
is ( $c_matrix_more_stats{ specificity }, 80, "Specificity seems correct to me" );
xt/boilerplate.t view on Meta::CPAN
#!perl
use 5.006;
use strict;
use warnings;
use Test::More;
plan tests => 3;
sub not_in_file_ok {
my ($filename, %regex) = @_;
open( my $fh, '<', $filename )
or die "couldn't open $filename for reading: $!";
my %violated;
while (my $line = <$fh>) {
while (my ($desc, $regex) = each %regex) {
if ($line =~ $regex) {
push @{$violated{$desc}||=[]}, $.;
xt/boilerplate.t view on Meta::CPAN
}
if (%violated) {
fail("$filename contains boilerplate text");
diag "$_ appears on lines @{$violated{$_}}" for keys %violated;
} else {
pass("$filename contains no boilerplate text");
}
}
sub module_boilerplate_ok {
my ($module) = @_;
not_in_file_ok($module =>
'the great new $MODULENAME' => qr/ - The great new /,
'boilerplate description' => qr/Quick summary of what the module/,
'stub function definition' => qr/function[12]/,
);
}
TODO: {
local $TODO = "Need to replace the boilerplate text";