AI-MXNet
view release on metacpan or search on metacpan
lib/AI/MXNet/Module/Base.pm view on Meta::CPAN
if("@$data_names" ne "@actual")
{
my $msg = sprintf(
"Data provided by %s_shapes don't match names specified by %s_names (%s vs. %s)",
$name, $name, "@$data_shapes", "@$data_names"
);
if($throw)
{
confess($msg);
}
else
{
AI::MXNet::Logging->warning($msg);
}
}
}
method _parse_data_desc(
ArrayRef[Str] $data_names,
Maybe[ArrayRef[Str]] $label_names,
ArrayRef[NameShapeOrDataDesc] $data_shapes,
Maybe[ArrayRef[NameShapeOrDataDesc]] $label_shapes
)
{
$data_shapes = [map { blessed $_ ? $_ : AI::MXNet::DataDesc->new(@$_) } @$data_shapes];
$self->_check_names_match($data_names, $data_shapes, 'data', 1);
if($label_shapes)
{
$label_shapes = [map { blessed $_ ? $_ : AI::MXNet::DataDesc->new(@$_) } @$label_shapes];
$self->_check_names_match($label_names, $label_shapes, 'label', 0);
}
else
{
$self->_check_names_match($label_names, [], 'label', 0);
}
return ($data_shapes, $label_shapes);
}
=head1 DESCRIPTION
The base class of a modules. A module represents a computation component. The design
purpose of a module is that it abstract a computation "machine", that one can run forward,
backward, update parameters, etc. We aim to make the APIs easy to use, especially in the
case when we need to use imperative API to work with multiple modules (e.g. stochastic
depth network).
A module has several states:
- Initial state. Memory is not allocated yet, not ready for computation yet.
- Binded. Shapes for inputs, outputs, and parameters are all known, memory allocated,
ready for computation.
- Parameter initialized. For modules with parameters, doing computation before initializing
the parameters might result in undefined outputs.
- Optimizer installed. An optimizer can be installed to a module. After this, the parameters
of the module can be updated according to the optimizer after gradients are computed
(forward-backward).
In order for a module to interact with others, a module should be able to report the
following information in its raw stage (before binded)
- data_names: array ref of string indicating the names of required data.
- output_names: array ref of string indicating the names of required outputs.
And also the following richer information after binded:
- state information
- binded: bool, indicating whether the memory buffers needed for computation
has been allocated.
- for_training: whether the module is binded for training (if binded).
- params_initialized: bool, indicating whether the parameters of this modules
has been initialized.
- optimizer_initialized: bool, indicating whether an optimizer is defined
and initialized.
- inputs_need_grad: bool, indicating whether gradients with respect to the
input data is needed. Might be useful when implementing composition of modules.
- input/output information
- data_shapes: am array ref of [name, shape]. In theory, since the memory is allocated,
we could directly provide the data arrays. But in the case of data parallelization,
the data arrays might not be of the same shape as viewed from the external world.
- label_shapes: an array ref of [name, shape]. This might be [] if the module does
not need labels (e.g. it does not contains a loss function at the top), or a module
is not binded for training.
- output_shapes: an array ref of [name, shape] for outputs of the module.
- parameters (for modules with parameters)
- get_params(): return an array ($arg_params, $aux_params). Each of those
is a hash ref of name to NDArray mapping. Those NDArrays always on
CPU. The actual parameters used for computing might be on other devices (GPUs),
this function will retrieve (a copy of) the latest parameters. Therefore, modifying
- get_params($arg_params, $aux_params): assign parameters to the devices
doing the computation.
- init_params(...): a more flexible interface to assign or initialize the parameters.
- setup
- bind(): prepare environment for computation.
- init_optimizer(): install optimizer for parameter updating.
- computation
- forward(data_batch): forward operation.
- backward(out_grads=): backward operation.
- update(): update parameters according to installed optimizer.
- get_outputs(): get outputs of the previous forward operation.
- get_input_grads(): get the gradients with respect to the inputs computed
in the previous backward operation.
- update_metric(metric, labels): update performance metric for the previous forward
computed results.
- other properties (mostly for backward compatability)
- symbol: the underlying symbolic graph for this module (if any)
This property is not necessarily constant. For example, for AI::MXNet::Module::Bucketing,
this property is simply the *current* symbol being used. For other modules,
this value might not be well defined.
When those intermediate-level API are implemented properly, the following
high-level API will be automatically available for a module:
- fit: train the module parameters on a data set
- predict: run prediction on a data set and collect outputs
- score: run prediction on a data set and evaluate performance
=cut
lib/AI/MXNet/Module/Base.pm view on Meta::CPAN
{
&{$callback}($batch_end_params);
}
}
$nbatch++;
}
# one epoch of training is finished
my $name_value = $eval_metric->get_name_value;
while(my ($name, $val) = each %{ $name_value })
{
$self->logger->info('Epoch[%d] Train-%s=%f', $epoch, $name, $val);
}
my $toc = time;
$self->logger->info('Epoch[%d] Time cost=%.3f', $epoch, ($toc-$tic));
# sync aux params across devices
my ($arg_params, $aux_params) = $self->get_params;
$self->set_params($arg_params, $aux_params);
if($epoch_end_callback)
{
for my $callback (@{ _as_list($epoch_end_callback) })
{
&{$callback}($epoch, $self->get_symbol, $arg_params, $aux_params);
}
}
#----------------------------------------
# evaluation on validation set
if(defined $eval_data)
{
my $res = $self->score(
$eval_data,
$validation_metric,
score_end_callback => $eval_end_callback,
batch_end_callback => $eval_batch_end_callback,
epoch => $epoch
);
#TODO: pull this into default
while(my ($name, $val) = each %{ $res })
{
$self->logger->info('Epoch[%d] Validation-%s=%f', $epoch, $name, $val);
}
}
# end of 1 epoch, reset the data-iter for another epoch
$train_data->reset;
}
}
################################################################################
# Symbol information
################################################################################
=head2 get_symbol
The symbol used by this module.
=cut
method get_symbol() { $self->symbol }
=head2 data_names
An array ref of names for data required by this module.
=cut
method data_names() { confess("NotImplemented") }
=head2 output_names
An array ref of names for the outputs of this module.
=cut
method output_names() { confess("NotImplemented") }
################################################################################
# Input/Output information
################################################################################
=head2 data_shapes
An array ref of AI::MXNet::DataDesc objects specifying the data inputs to this module.
=cut
method data_shapes() { confess("NotImplemented") }
=head2 label_shapes
A array ref of AI::MXNet::DataDesc objects specifying the label inputs to this module.
If this module does not accept labels -- either it is a module without a loss
function, or it is not binded for training, then this should return an empty
array ref.
=cut
method label_shapes() { confess("NotImplemented") }
=head2 output_shapes
An array ref of (name, shape) array refs specifying the outputs of this module.
=cut
method output_shapes() { confess("NotImplemented") }
################################################################################
# Parameters of a module
################################################################################
=head2 get_params
The parameters, these are potentially a copies of the the actual parameters used
to do computation on the device.
Returns
-------
($arg_params, $aux_params), a pair of hash refs of name to value mapping.
=cut
method get_params() { confess("NotImplemented") }
=head2 init_params
Initialize the parameters and auxiliary states.
Parameters
----------
:$initializer : Maybe[AI::MXNet::Initializer]
Called to initialize parameters if needed.
:$arg_params= : Maybe[HashRef[AI::MXNet::NDArray]]
If not undef, should be a hash ref of existing arg_params.
lib/AI/MXNet/Module/Base.pm view on Meta::CPAN
{
assert($self->binded and $self->params_initialized);
assert(not $merge_multi_context);
return [];
}
=head2 set_states
Set value for states. You can specify either $states or $value, not both.
Parameters
----------
$states= : Maybe[ArrayRef[ArrayRef[AI::MXNet::NDArray]]]
source states arrays formatted like [[$state1_dev1, $state1_dev2],
[$state2_dev1, $state2_dev2]].
$value= : Maybe[Num]
a single scalar value for all state arrays.
=cut
method set_states(Maybe[ArrayRef[ArrayRef[AI::MXNet::NDArray]]] $states=, Maybe[Num] $value=)
{
assert($self->binded and $self->params_initialized);
assert(not $states and not $value);
}
=head2 install_monitor
Install monitor on all executors
Parameters
----------
$mon : AI::MXNet::Monitor
=cut
method install_monitor(AI::MXNet::Monitor $mon) { confess("NotImplemented") }
=head2 prepare
Prepare the module for processing a data batch.
Usually involves switching a bucket and reshaping.
Parameters
----------
$data_batch : AI::MXNet::DataBatch
=cut
method prepare(AI::MXNet::DataBatch $data_batch){}
################################################################################
# Computations
################################################################################
=head2 forward
Forward computation. It supports data batches with different shapes, such as
different batch sizes or different image sizes.
If reshaping of data batch relates to modification of symbol or module, such as
changing image layout ordering or switching from training to predicting, module
rebinding is required.
Parameters
----------
$data_batch : DataBatch
Could be anything with similar API implemented.
:$is_train= : Bool
Default is undef, which means is_train takes the value of $self->for_training.
=cut
method forward(AI::MXNet::DataBatch $data_batch, Bool :$is_train=) { confess("NotImplemented") }
=head2 backward
Backward computation.
Parameters
----------
$out_grads : Maybe[AI::MXNet::NDArray|ArrayRef[AI::MXNet::NDArray]], optional
Gradient on the outputs to be propagated back.
This parameter is only needed when bind is called
on outputs that are not a loss function.
=cut
method backward(Maybe[AI::MXNet::NDArray|ArrayRef[AI::MXNet::NDArray]] $out_grads=)
{
confess("NotImplemented")
}
=head2 get_outputs
The outputs of the previous forward computation.
Parameters
----------
$merge_multi_context=1 : Bool
=cut
method get_outputs(Bool $merge_multi_context=1) { confess("NotImplemented") }
=head2 get_input_grads
The gradients to the inputs, computed in the previous backward computation.
Parameters
----------
$merge_multi_context=1 : Bool
=cut
method get_input_grads(Bool $merge_multi_context=1) { confess("NotImplemented") }
=head2 update
Update parameters according to the installed optimizer and the gradients computed
in the previous forward-backward batch.
=cut
method update() { confess("NotImplemented") }
=head2 update_metric
( run in 0.844 second using v1.01-cache-2.11-cpan-39bf76dae61 )