AI-PSO
view release on metacpan or search on metacpan
0.85 Wed Nov 15 22:30:47 2006
- corrected the fitness function in the test
- added perceptron c++ code that I wrote a long time ago ;)
- added an example (pso_ann.pl) for training a simple feed-forward neural network
- updated POD
0.82 Sat Nov 11 22:20:31 2006
- fixed POD to correctly 'use AI::PSO'
- fixed fitness function in PSO.t
- added research paper to package
- moved into a subversion repository
- removed requirement for perl 5.8.8
- removed printing of solution array in test
0.80 Sat Nov 11 14:22:27 2006
- changed namespace to AI::PSO
- added a pso_get_solution_array function
0.70 Fri Nov 10 23:50:32 2006
- added user callback fitness function
- added POD
MPL-1.1.txt view on Meta::CPAN
as the Initial Developer in the Source Code notice required by Exhibit
A.
1.7. "Larger Work" means a work which combines Covered Code or
portions thereof with code not governed by the terms of this License.
1.8. "License" means this document.
1.8.1. "Licensable" means having the right to grant, to the maximum
extent possible, whether at the time of the initial grant or
subsequently acquired, any and all of the rights conveyed herein.
1.9. "Modifications" means any addition to or deletion from the
substance or structure of either the Original Code or any previous
Modifications. When Covered Code is released as a series of files, a
Modification is:
A. Any addition to or deletion from the contents of a file
containing Original Code or previous Modifications.
B. Any new file that contains any part of the Original Code or
previous Modifications.
1.10. "Original Code" means Source Code of computer software code
which is described in the Source Code notice required by Exhibit A as
MPL-1.1.txt view on Meta::CPAN
this definition, "control" means (a) the power, direct or indirect,
to cause the direction or management of such entity, whether by
contract or otherwise, or (b) ownership of more than fifty percent
(50%) of the outstanding shares or beneficial ownership of such
entity.
2. Source Code License.
2.1. The Initial Developer Grant.
The Initial Developer hereby grants You a world-wide, royalty-free,
non-exclusive license, subject to third party intellectual property
claims:
(a) under intellectual property rights (other than patent or
trademark) Licensable by Initial Developer to use, reproduce,
modify, display, perform, sublicense and distribute the Original
Code (or portions thereof) with or without Modifications, and/or
as part of a Larger Work; and
(b) under Patents Claims infringed by the making, using or
selling of Original Code, to make, have made, use, practice,
sell, and offer for sale, and/or otherwise dispose of the
Original Code (or portions thereof).
(c) the licenses granted in this Section 2.1(a) and (b) are
effective on the date Initial Developer first distributes
MPL-1.1.txt view on Meta::CPAN
separate from the Original Code; or 3) for infringements caused
by: i) the modification of the Original Code or ii) the
combination of the Original Code with other software or devices.
2.2. Contributor Grant.
Subject to third party intellectual property claims, each Contributor
hereby grants You a world-wide, royalty-free, non-exclusive license
(a) under intellectual property rights (other than patent or
trademark) Licensable by Contributor, to use, reproduce, modify,
display, perform, sublicense and distribute the Modifications
created by such Contributor (or portions thereof) either on an
unmodified basis, with other Modifications, as Covered Code
and/or as part of a Larger Work; and
(b) under Patent Claims infringed by the making, using, or
selling of Modifications made by that Contributor either alone
and/or in combination with its Contributor Version (or portions
of such combination), to make, use, sell, offer for sale, have
made, and/or otherwise dispose of: 1) Modifications made by that
Contributor (or portions thereof); and 2) the combination of
MPL-1.1.txt view on Meta::CPAN
Section 3.5.
3.2. Availability of Source Code.
Any Modification which You create or to which You contribute must be
made available in Source Code form under the terms of this License
either on the same media as an Executable version or via an accepted
Electronic Distribution Mechanism to anyone to whom you made an
Executable version available; and if made available via Electronic
Distribution Mechanism, must remain available for at least twelve (12)
months after the date it initially became available, or at least six
(6) months after a subsequent version of that particular Modification
has been made available to such recipients. You are responsible for
ensuring that the Source Code version remains available even if the
Electronic Distribution Mechanism is maintained by a third party.
3.3. Description of Modifications.
You must cause all Covered Code to which You contribute to contain a
file documenting the changes You made to create that Covered Code and
the date of any change. You must include a prominent statement that
the Modification is derived, directly or indirectly, from Original
Code provided by the Initial Developer and including the name of the
MPL-1.1.txt view on Meta::CPAN
6.1. New Versions.
Netscape Communications Corporation ("Netscape") may publish revised
and/or new versions of the License from time to time. Each version
will be given a distinguishing version number.
6.2. Effect of New Versions.
Once Covered Code has been published under a particular version of the
License, You may always continue to use it under the terms of that
version. You may also choose to use such Covered Code under the terms
of any subsequent version of the License published by Netscape. No one
other than Netscape has the right to modify the terms applicable to
Covered Code created under this License.
6.3. Derivative Works.
If You create or use a modified version of this License (which you may
only do in order to apply it to code which is not already Covered Code
governed by this License), You must (a) rename Your license so that
the phrases "Mozilla", "MOZILLAPL", "MOZPL", "Netscape",
"MPL", "NPL" or any confusingly similar phrase do not appear in your
license (except to note that your license differs from this License)
MPL-1.1.txt view on Meta::CPAN
YOU (NOT THE INITIAL DEVELOPER OR ANY OTHER CONTRIBUTOR) ASSUME THE
COST OF ANY NECESSARY SERVICING, REPAIR OR CORRECTION. THIS DISCLAIMER
OF WARRANTY CONSTITUTES AN ESSENTIAL PART OF THIS LICENSE. NO USE OF
ANY COVERED CODE IS AUTHORIZED HEREUNDER EXCEPT UNDER THIS DISCLAIMER.
8. TERMINATION.
8.1. This License and the rights granted hereunder will terminate
automatically if You fail to comply with terms herein and fail to cure
such breach within 30 days of becoming aware of the breach. All
sublicenses to the Covered Code which are properly granted shall
survive any termination of this License. Provisions which, by their
nature, must remain in effect beyond the termination of this License
shall survive.
8.2. If You initiate litigation by asserting a patent infringement
claim (excluding declatory judgment actions) against Initial Developer
or a Contributor (the Initial Developer or Contributor against whom
You file such action is referred to as "Participant") alleging that:
(a) such Participant's Contributor Version directly or indirectly
MPL-1.1.txt view on Meta::CPAN
The Covered Code is a "commercial item," as that term is defined in
48 C.F.R. 2.101 (Oct. 1995), consisting of "commercial computer
software" and "commercial computer software documentation," as such
terms are used in 48 C.F.R. 12.212 (Sept. 1995). Consistent with 48
C.F.R. 12.212 and 48 C.F.R. 227.7202-1 through 227.7202-4 (June 1995),
all U.S. Government End Users acquire Covered Code with only those
rights set forth herein.
11. MISCELLANEOUS.
This License represents the complete agreement concerning subject
matter hereof. If any provision of this License is held to be
unenforceable, such provision shall be reformed only to the extent
necessary to make it enforceable. This License shall be governed by
California law provisions (except to the extent applicable law, if
any, provides otherwise), excluding its conflict-of-law provisions.
With respect to disputes in which at least one party is a citizen of,
or an entity chartered or registered to do business in the United
States of America, any litigation relating to this License shall be
subject to the jurisdiction of the Federal Courts of the Northern
District of California, with venue lying in Santa Clara County,
California, with the losing party responsible for costs, including
without limitation, court costs and reasonable attorneys' fees and
expenses. The application of the United Nations Convention on
Contracts for the International Sale of Goods is expressly excluded.
Any law or regulation which provides that the language of a contract
shall be construed against the drafter shall not apply to this
License.
12. RESPONSIBILITY FOR CLAIMS.
MPL-1.1.txt view on Meta::CPAN
13. MULTIPLE-LICENSED CODE.
Initial Developer may designate portions of the Covered Code as
"Multiple-Licensed". "Multiple-Licensed" means that the Initial
Developer permits you to utilize portions of the Covered Code under
Your choice of the NPL or the alternative licenses, if any, specified
by the Initial Developer in the file described in Exhibit A.
EXHIBIT A -Mozilla Public License.
``The contents of this file are subject to the Mozilla Public License
Version 1.1 (the "License"); you may not use this file except in
compliance with the License. You may obtain a copy of the License at
http://www.mozilla.org/MPL/
Software distributed under the License is distributed on an "AS IS"
basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the
License for the specific language governing rights and limitations
under the License.
The Original Code is ______________________________________.
examples/NeuralNet/pso_ann.pl view on Meta::CPAN
my $numInputs = 3;
my $numHidden = 2;
my $xferFunc = "Logistic";
my $annConfig = "pso.ann";
my $annInputs = "pso.dat";
my $expectedValue = 3.5; # this is the value that we want to train the ANN to produce (just like the example in t/PTO.t)
sub test_fitness_function(@) {
my (@arr) = (@_);
&writeAnnConfig($annConfig, $numInputs, $numHidden, $xferFunc, @arr);
my $netValue = &runANN($annConfig, $annInputs);
print "network value = $netValue\n";
# the closer the network value gets to our desired value
# then we want to set the fitness closer to 1.
#
# This is a special case of the sigmoid, and looks an awful lot
# like the hyperbolic tangent ;)
examples/NeuralNet/pso_ann.pl view on Meta::CPAN
pso_set_params(\%test_params);
pso_register_fitness_function('test_fitness_function');
pso_optimize();
#my @solution = pso_get_solution_array();
##### io #########
sub writeAnnConfig() {
my ($configFile, $inputs, $hidden, $func, @weights) = (@_);
open(ANN, ">$configFile");
print ANN "$inputs $hidden\n";
print ANN "$func\n";
foreach my $weight (@weights) {
print ANN "$weight ";
}
print ANN "\n";
close(ANN);
}
sub runANN($$) {
my ($configFile, $dataFile) = @_;
my $networkValue = `ann_compute $configFile $dataFile`;
chomp($networkValue);
return $networkValue;
}
lib/AI/PSO.pm view on Meta::CPAN
my $themMax = 'null'; # 'social' maximum random weight (this should really be between 0, 1)
my $psoRandomRange = 'null'; # PSO::.86 new variable to support original unmodified algorithm
my $useModifiedAlgorithm = 'null';
#-#-# user/debug parameters #-#-#
my $verbose = 0; # This one defaults for obvious reasons...
#NOTE: $meWeight and $themWeight should really add up to a constant value.
# Swarm Intelligence defines a 'pso random range' constant and then computes two random numbers
# within this range by first getting a random number and then subtracting it from the range.
# e.g.
# $randomRange = 4.0
# $meWeight = random(0, $randomRange);
# $themWeight = $randomRange - $meWeight.
#
#
#---------- END GLOBAL PARAMETERS ------------
#---------- BEGIN GLOBAL DATA STRUCTURES --------
lib/AI/PSO.pm view on Meta::CPAN
my @solution = ();
#---------- END GLOBAL DATA STRUCTURES --------
#---------- BEGIN EXPORTED SUBROUTINES ----------
#
# pso_set_params
# - sets the global module parameters from the hash passed in
#
sub pso_set_params(%) {
my (%params) = %{$_[0]};
my $retval = 0;
#no strict 'refs';
#foreach my $key (keys(%params)) {
# $$key = $params{$key};
#}
#use strict 'refs';
$numParticles = defined($params{numParticles}) ? $params{numParticles} : 'null';
lib/AI/PSO.pm view on Meta::CPAN
$retval = 1 if($param_string =~ m/null/);
return $retval;
}
#
# pso_register_fitness_function
# - sets the user-defined callback fitness function
#
sub pso_register_fitness_function($) {
my ($func) = @_;
$user_fitness_function = new Callback(\&{"main::$func"});
return 0;
}
#
# pso_optimize
# - runs the particle swarm optimization algorithm
#
sub pso_optimize() {
&init();
return &swarm();
}
#
# pso_get_solution_array
# - returns the array of parameters corresponding to the best solution so far
sub pso_get_solution_array() {
return @solution;
}
#---------- END EXPORTED SUBROUTINES ----------
#--------- BEGIN INTERNAL SUBROUTINES -----------
#
# init
# - initializes global variables
# - initializes particle data structures
#
sub init() {
if($psoRandomRange =~ m/null/) {
$useModifiedAlgorithm = 1;
} else {
$useModifiedAlgorithm = 0;
}
&initialize_particles();
}
#
# initialize_particles
# - sets up internal data structures
# - initializes particle positions and velocities with an element of randomness
#
sub initialize_particles() {
for(my $p = 0; $p < $numParticles; $p++) {
$particles[$p] = {}; # each particle is a hash of arrays with the array sizes being the dimensionality of the problem space
$particles[$p]{nextPos} = []; # nextPos is the array of positions to move to on the next positional update
$particles[$p]{bestPos} = []; # bestPos is the position of that has yielded the best fitness for this particle (it gets updated when a better fitness is found)
$particles[$p]{currPos} = []; # currPos is the current position of this particle in the problem space
$particles[$p]{velocity} = []; # velocity ... come on ...
for(my $d = 0; $d < $dimensions; $d++) {
$particles[$p]{nextPos}[$d] = &random($deltaMin, $deltaMax);
$particles[$p]{currPos}[$d] = &random($deltaMin, $deltaMax);
$particles[$p]{bestPos}[$d] = &random($deltaMin, $deltaMax);
$particles[$p]{velocity}[$d] = &random($deltaMin, $deltaMax);
}
}
}
#
# initialize_neighbors
# NOTE: I made this a separate subroutine so that different topologies of neighbors can be created and used instead of this.
# NOTE: This subroutine is currently not used because we access neighbors by index to the particle array rather than storing their references
#
# - adds a neighbor array to the particle hash data structure
# - sets the neighbor based on the default neighbor hash function
#
sub initialize_neighbors() {
for(my $p = 0; $p < $numParticles; $p++) {
for(my $n = 0; $n < $numNeighbors; $n++) {
$particles[$p]{neighbor}[$n] = $particles[&get_index_of_neighbor($p, $n)];
}
}
}
sub dump_particle($) {
$| = 1;
my ($index) = @_;
print STDERR "[particle $index]\n";
print STDERR "\t[bestPos] ==> " . &compute_fitness(@{$particles[$index]{bestPos}}) . "\n";
foreach my $pos (@{$particles[$index]{bestPos}}) {
print STDERR "\t\t$pos\n";
}
print STDERR "\t[currPos] ==> " . &compute_fitness(@{$particles[$index]{currPos}}) . "\n";
foreach my $pos (@{$particles[$index]{currPos}}) {
print STDERR "\t\t$pos\n";
lib/AI/PSO.pm view on Meta::CPAN
print STDERR "\t[velocity]\n";
foreach my $pos (@{$particles[$index]{velocity}}) {
print STDERR "\t\t$pos\n";
}
}
#
# swarm
# - runs the particle swarm algorithm
#
sub swarm() {
for(my $iter = 0; $iter < $maxIterations; $iter++) {
for(my $p = 0; $p < $numParticles; $p++) {
## update position
for(my $d = 0; $d < $dimensions; $d++) {
$particles[$p]{currPos}[$d] = $particles[$p]{nextPos}[$d];
}
## test _current_ fitness of position
my $fitness = &compute_fitness(@{$particles[$p]{currPos}});
lib/AI/PSO.pm view on Meta::CPAN
}
&save_solution(@{$particles[$bestPartIndex]{bestPos}});
&dump_particle($bestPartIndex);
return 1;
}
#
# save solution
# - simply copies the given array into the global solution array
#
sub save_solution(@) {
@solution = @_;
}
#
# compute_fitness
# - computes the fitness of a particle by using the user-specified fitness function
#
# NOTE: I originally had a 'fitness cache' so that particles that stumbled upon the same
# position wouldn't have to recalculate their fitness (which is often expensive).
# However, this may be undesirable behavior for the user (if you come across the same position
# then you may be settling in on a local maxima so you might want to randomize things and
# keep searching. For this reason, I'm leaving the cache out. It would be trivial
# for users to implement their own cache since they are passed the same array of values.
#
sub compute_fitness(@) {
my (@values) = @_;
my $return_fitness = 0;
# no strict 'refs';
# if(defined(&{"main::$user_fitness_function"})) {
# $return_fitness = &$user_fitness_function(@values);
# } else {
# warn "error running user_fitness_function\n";
# exit 1;
# }
lib/AI/PSO.pm view on Meta::CPAN
$return_fitness = $user_fitness_function->call(@values);
return $return_fitness;
}
#
# random
# - returns a random number that is between the first and second arguments using the Math::Random module
#
sub random($$) {
my ($min, $max) = @_;
return random_uniform(1, $min, $max)
}
#
# get_index_of_neighbor
#
# - returns the index of Nth neighbor of the index for particle P
# ==> A neighbor is one of the next K particles following P where K is the neighborhood size.
# So, particle 1 has neighbors 2, 3, 4, 5 if K = 4. particle 4 has neighbors 5, 6, 7, 8
# ...
#
sub get_index_of_neighbor($$) {
my ($particleIndex, $neighborNum) = @_;
# TODO: insert error checking code / defensive programming
return ($particleIndex + $neighborNum) % $numParticles;
}
#
# get_index_of_best_fit_neighbor
# - returns the index of the neighbor with the best fitness (when given a particle index)...
#
sub get_index_of_best_fit_neighbor($) {
my ($particleIndex) = @_;
my $bestNeighborFitness = 0;
my $bestNeighborIndex = 0;
my $particleNeighborIndex = 0;
for(my $neighbor = 0; $neighbor < $numNeighbors; $neighbor++) {
$particleNeighborIndex = &get_index_of_neighbor($particleIndex, $neighbor);
if(&compute_fitness(@{$particles[$particleNeighborIndex]{bestPos}}) > $bestNeighborFitness) {
$bestNeighborFitness = &compute_fitness(@{$particles[$particleNeighborIndex]{bestPos}});
$bestNeighborIndex = $particleNeighborIndex;
}
}
# TODO: insert error checking code / defensive programming
return $particleNeighborIndex;
}
#
# clamp_velocity
# - restricts the change in velocity to be within a certain range (prevents large jumps in problem hyperspace)
#
sub clamp_velocity($) {
my ($dx) = @_;
if($dx < $deltaMin) {
$dx = $deltaMin;
} elsif($dx > $deltaMax) {
$dx = $deltaMax;
}
return $dx;
}
#--------- END INTERNAL SUBROUTINES -----------
lib/AI/PSO.pm view on Meta::CPAN
meMin => 0.0, # 'individuality' minimum random weight
meMax => 1.0, # 'individuality' maximum random weight
themWeight => 2.0, # 'social' weighting constant (higher means trust group more)
themMin => 0.0, # 'social' minimum random weight
themMax => 1.0, # 'social' maximum random weight
exitFitness => 0.9, # minimum fitness to achieve before exiting
verbose => 0, # 0 prints solution
# 1 prints (Y|N):particle:fitness at each iteration
# 2 dumps each particle (+1)
psoRandomRange => 4.0, # setting this enables the original PSO algorithm and
# also subsequently ignores the me*/them* parameters
);
sub custom_fitness_function(@input) {
# this is a callback function.
# @input will be passed to this, you do not need to worry about setting it...
# ... do something with @input which is an array of floats
# return a value in [0,1] with 0 being the worst and 1 being the best
}
pso_set_params(\%params);
pso_register_fitness_function('custom_fitness_function');
pso_optimize();
my @solutionArray = pso_get_solution_array();
themMax => 1.0,
exitFitness => 0.99,
verbose => 1,
);
my %test_params2 = %test_params;
$test_params2{psoRandomRange} = 4.0;
# simple test function to sum the position values up to 3.5
my $testValue = 3.5;
sub test_fitness_function(@) {
my (@arr) = (@_);
my $sum = 0;
my $ret = 0;
foreach my $val (@arr) {
$sum += $val;
}
# sigmoid-like ==> squash the result to [0,1] and get as close to 3.5 as we can
return 2 / (1 + exp(abs($testValue - $sum)));
return $ret;
( run in 2.158 seconds using v1.01-cache-2.11-cpan-88abd93f124 )