view release on metacpan or  search on metacpan

MP.pm  view on Meta::CPAN

some messages. Messages send to ports will not be queued, regardless of
anything was listening for them or not.

Ports are represented by (printable) strings called "port IDs".

=item port ID - C<nodeid#portname>

A port ID is the concatenation of a node ID, a hash-mark (C<#>)
as separator, and a port name (a printable string of unspecified
format created by AnyEvent::MP).

=item node

A node is a single process containing at least one port - the node port,
which enables nodes to manage each other remotely, and to create new
ports.

Nodes are either public (have one or more listening ports) or private
(no listening ports). Private nodes cannot talk to other private nodes
currently, but all nodes can talk to public nodes.

Nodes is represented by (printable) strings called "node IDs".

=item node ID - C<[A-Za-z0-9_\-.:]*>

A node ID is a string that uniquely identifies the node within a
network. Depending on the configuration used, node IDs can look like a
hostname, a hostname and a port, or a random string. AnyEvent::MP itself
doesn't interpret node IDs in any way except to uniquely identify a node.

=item binds - C<ip:port>

Nodes can only talk to each other by creating some kind of connection to
each other. To do this, nodes should listen on one or more local transport
endpoints - binds.

Currently, only standard C<ip:port> specifications can be used, which
specify TCP ports to listen on. So a bind is basically just a tcp socket
in listening mode that accepts connections from other nodes.

=item seed nodes

When a node starts, it knows nothing about the network it is in - it
needs to connect to at least one other node that is already in the
network. These other nodes are called "seed nodes".

Seed nodes themselves are not special - they are seed nodes only because
some other node I<uses> them as such, but any node can be used as seed
node for other nodes, and eahc node can use a different set of seed nodes.

In addition to discovering the network, seed nodes are also used to
maintain the network - all nodes using the same seed node are part of the
same network. If a network is split into multiple subnets because e.g. the
network link between the parts goes down, then using the same seed nodes
for all nodes ensures that eventually the subnets get merged again.

Seed nodes are expected to be long-running, and at least one seed node
should always be available. They should also be relatively responsive - a
seed node that blocks for long periods will slow down everybody else.

For small networks, it's best if every node uses the same set of seed
nodes. For large networks, it can be useful to specify "regional" seed
nodes for most nodes in an area, and use all seed nodes as seed nodes for
each other. What's important is that all seed nodes connections form a
complete graph, so that the network cannot split into separate subnets
forever.

Seed nodes are represented by seed IDs.

=item seed IDs - C<host:port>

Seed IDs are transport endpoint(s) (usually a hostname/IP address and a
TCP port) of nodes that should be used as seed nodes.

=item global nodes

An AEMP network needs a discovery service - nodes need to know how to
connect to other nodes they only know by name. In addition, AEMP offers a
distributed "group database", which maps group names to a list of strings
- for example, to register worker ports.

A network needs at least one global node to work, and allows every node to
be a global node.

Any node that loads the L<AnyEvent::MP::Global> module becomes a global
node and tries to keep connections to all other nodes. So while it can
make sense to make every node "global" in small networks, it usually makes
sense to only make seed nodes into global nodes in large networks (nodes
keep connections to seed nodes and global nodes, so making them the same
reduces overhead).

=back

=head1 VARIABLES/FUNCTIONS

=over 4

=cut

package AnyEvent::MP;

use AnyEvent::MP::Config ();
use AnyEvent::MP::Kernel;
use AnyEvent::MP::Kernel qw(
   %NODE %PORT %PORT_DATA $UNIQ $RUNIQ $ID
   add_node load_func

   NODE $NODE
   configure
   node_of port_is_local
   snd kil
   db_set db_del
   db_mon db_family db_keys db_values
);

use common::sense;

use Carp ();

use AnyEvent ();
use Guard ();

MP.pm  view on Meta::CPAN

without problems, and in fact, this is the common method to create worker
pools. For example, a worker port for image scaling might do this:

   db_set my_image_scalers => $port;

And clients looking for an image scaler will want to get the
C<my_image_scalers> keys from time to time:

   db_keys my_image_scalers => sub {
      @ports = @{ $_[0] };
   };

Or better yet, they want to monitor the database family, so they always
have a reasonable up-to-date copy:

   db_mon my_image_scalers => sub {
      @ports = keys %{ $_[0] };
   };

In general, you can set or delete single subkeys, but query and monitor
whole families only.

If you feel the need to monitor or query a single subkey, try giving it
it's own family.

=over

=item $guard = db_set $family => $subkey [=> $value]

Sets (or replaces) a key to the database - if C<$value> is omitted,
C<undef> is used instead.

When called in non-void context, C<db_set> returns a guard that
automatically calls C<db_del> when it is destroyed.

=item db_del $family => $subkey...

Deletes one or more subkeys from the database family.

=item $guard = db_reg $family => $port => $value

=item $guard = db_reg $family => $port

=item $guard = db_reg $family

Registers a port in the given family and optionally returns a guard to
remove it.

This function basically does the same as:

   db_set $family => $port => $value

Except that the port is monitored and automatically removed from the
database family when it is kil'ed.

If C<$value> is missing, C<undef> is used. If C<$port> is missing, then
C<$SELF> is used.

This function is most useful to register a port in some port group (which
is just another name for a database family), and have it removed when the
port is gone. This works best when the port is a local port.

=cut

sub db_reg($$;$) {
   my $family = shift;
   my $port = @_ ? shift : $SELF;

   my $clr = sub { db_del $family => $port };
   mon $port, $clr;

   db_set $family => $port => $_[0];

   defined wantarray
      and &Guard::guard ($clr)
}

=item db_family $family => $cb->(\%familyhash)

Queries the named database C<$family> and call the callback with the
family represented as a hash. You can keep and freely modify the hash.

=item db_keys $family => $cb->(\@keys)

Same as C<db_family>, except it only queries the family I<subkeys> and passes
them as array reference to the callback.

=item db_values $family => $cb->(\@values)

Same as C<db_family>, except it only queries the family I<values> and passes them
as array reference to the callback.

=item $guard = db_mon $family => $cb->(\%familyhash, \@added, \@changed, \@deleted)

Creates a monitor on the given database family. Each time a key is
set or is deleted the callback is called with a hash containing the
database family and three lists of added, changed and deleted subkeys,
respectively. If no keys have changed then the array reference might be
C<undef> or even missing.

If not called in void context, a guard object is returned that, when
destroyed, stops the monitor.

The family hash reference and the key arrays belong to AnyEvent::MP and
B<must not be modified or stored> by the callback. When in doubt, make a
copy.

As soon as possible after the monitoring starts, the callback will be
called with the intiial contents of the family, even if it is empty,
i.e. there will always be a timely call to the callback with the current
contents.

It is possible that the callback is called with a change event even though
the subkey is already present and the value has not changed.

The monitoring stops when the guard object is destroyed.

Example: on every change to the family "mygroup", print out all keys.

   my $guard = db_mon mygroup => sub {
      my ($family, $a, $c, $d) = @_;