AnyEvent-MP
view release on metacpan or search on metacpan
=head1 NAME
AnyEvent::MP - erlang-style multi-processing/message-passing framework
=head1 SYNOPSIS
use AnyEvent::MP;
$NODE # contains this node's node ID
NODE # returns this node's node ID
$SELF # receiving/own port id in rcv callbacks
# initialise the node so it can send/receive messages
configure;
# ports are message destinations
# sending messages
snd $port, type => data...;
snd $port, @msg;
snd @msg_with_first_element_being_a_port;
# creating/using ports, the simple way
my $simple_port = port { my @msg = @_ };
# creating/using ports, tagged message matching
my $port = port;
rcv $port, ping => sub { snd $_[0], "pong" };
rcv $port, pong => sub { warn "pong received\n" };
# create a port on another node
my $port = spawn $node, $initfunc, @initdata;
# destroy a port again
kil $port; # "normal" kill
kil $port, my_error => "everything is broken"; # error kill
# monitoring
mon $port, $cb->(@msg) # callback is invoked on death
mon $port, $localport # kill localport on abnormal death
mon $port, $localport, @msg # send message on death
# temporarily execute code in port context
peval $port, sub { die "kill the port!" };
# execute callbacks in $SELF port context
my $timer = AE::timer 1, 0, psub {
die "kill the port, delayed";
};
# distributed database - modification
db_set $family => $subkey [=> $value] # add a subkey
db_del $family => $subkey... # delete one or more subkeys
db_reg $family => $port [=> $value] # register a port
# distributed database - queries
db_family $family => $cb->(\%familyhash)
db_keys $family => $cb->(\@keys)
db_values $family => $cb->(\@values)
# distributed database - monitoring a family
db_mon $family => $cb->(\%familyhash, \@added, \@changed, \@deleted)
=head1 DESCRIPTION
This module (-family) implements a simple message passing framework.
Despite its simplicity, you can securely message other processes running
on the same or other hosts, and you can supervise entities remotely.
For an introduction to this module family, see the L<AnyEvent::MP::Intro>
manual page and the examples under F<eg/>.
=head1 CONCEPTS
=over 4
=item port
Not to be confused with a TCP port, a "port" is something you can send
messages to (with the C<snd> function).
Ports allow you to register C<rcv> handlers that can match all or just
some messages. Messages send to ports will not be queued, regardless of
anything was listening for them or not.
Ports are represented by (printable) strings called "port IDs".
=item port ID - C<nodeid#portname>
A port ID is the concatenation of a node ID, a hash-mark (C<#>)
as separator, and a port name (a printable string of unspecified
format created by AnyEvent::MP).
=item node
A node is a single process containing at least one port - the node port,
which enables nodes to manage each other remotely, and to create new
ports.
Nodes are either public (have one or more listening ports) or private
(no listening ports). Private nodes cannot talk to other private nodes
currently, but all nodes can talk to public nodes.
Nodes is represented by (printable) strings called "node IDs".
=item node ID - C<[A-Za-z0-9_\-.:]*>
A node ID is a string that uniquely identifies the node within a
network. Depending on the configuration used, node IDs can look like a
hostname, a hostname and a port, or a random string. AnyEvent::MP itself
doesn't interpret node IDs in any way except to uniquely identify a node.
=item binds - C<ip:port>
Nodes can only talk to each other by creating some kind of connection to
each other. To do this, nodes should listen on one or more local transport
endpoints - binds.
Currently, only standard C<ip:port> specifications can be used, which
specify TCP ports to listen on. So a bind is basically just a tcp socket
in listening mode that accepts connections from other nodes.
=item seed nodes
When a node starts, it knows nothing about the network it is in - it
set or is deleted the callback is called with a hash containing the
database family and three lists of added, changed and deleted subkeys,
respectively. If no keys have changed then the array reference might be
C<undef> or even missing.
If not called in void context, a guard object is returned that, when
destroyed, stops the monitor.
The family hash reference and the key arrays belong to AnyEvent::MP and
B<must not be modified or stored> by the callback. When in doubt, make a
copy.
As soon as possible after the monitoring starts, the callback will be
called with the intiial contents of the family, even if it is empty,
i.e. there will always be a timely call to the callback with the current
contents.
It is possible that the callback is called with a change event even though
the subkey is already present and the value has not changed.
The monitoring stops when the guard object is destroyed.
Example: on every change to the family "mygroup", print out all keys.
my $guard = db_mon mygroup => sub {
my ($family, $a, $c, $d) = @_;
print "mygroup members: ", (join " ", keys %$family), "\n";
};
Exmaple: wait until the family "My::Module::workers" is non-empty.
my $guard; $guard = db_mon My::Module::workers => sub {
my ($family, $a, $c, $d) = @_;
return unless %$family;
undef $guard;
print "My::Module::workers now nonempty\n";
};
Example: print all changes to the family "AnyEvent::Fantasy::Module".
my $guard = db_mon AnyEvent::Fantasy::Module => sub {
my ($family, $a, $c, $d) = @_;
print "+$_=$family->{$_}\n" for @$a;
print "*$_=$family->{$_}\n" for @$c;
print "-$_=$family->{$_}\n" for @$d;
};
=cut
=back
=head1 AnyEvent::MP vs. Distributed Erlang
AnyEvent::MP got lots of its ideas from distributed Erlang (Erlang node
== aemp node, Erlang process == aemp port), so many of the documents and
programming techniques employed by Erlang apply to AnyEvent::MP. Here is a
sample:
http://www.erlang.se/doc/programming_rules.shtml
http://erlang.org/doc/getting_started/part_frame.html # chapters 3 and 4
http://erlang.org/download/erlang-book-part1.pdf # chapters 5 and 6
http://erlang.org/download/armstrong_thesis_2003.pdf # chapters 4 and 5
Despite the similarities, there are also some important differences:
=over 4
=item * Node IDs are arbitrary strings in AEMP.
Erlang relies on special naming and DNS to work everywhere in the same
way. AEMP relies on each node somehow knowing its own address(es) (e.g. by
configuration or DNS), and possibly the addresses of some seed nodes, but
will otherwise discover other nodes (and their IDs) itself.
=item * Erlang has a "remote ports are like local ports" philosophy, AEMP
uses "local ports are like remote ports".
The failure modes for local ports are quite different (runtime errors
only) then for remote ports - when a local port dies, you I<know> it dies,
when a connection to another node dies, you know nothing about the other
port.
Erlang pretends remote ports are as reliable as local ports, even when
they are not.
AEMP encourages a "treat remote ports differently" philosophy, with local
ports being the special case/exception, where transport errors cannot
occur.
=item * Erlang uses processes and a mailbox, AEMP does not queue.
Erlang uses processes that selectively receive messages out of order, and
therefore needs a queue. AEMP is event based, queuing messages would serve
no useful purpose. For the same reason the pattern-matching abilities
of AnyEvent::MP are more limited, as there is little need to be able to
filter messages without dequeuing them.
This is not a philosophical difference, but simply stems from AnyEvent::MP
being event-based, while Erlang is process-based.
You can have a look at L<Coro::MP> for a more Erlang-like process model on
top of AEMP and Coro threads.
=item * Erlang sends are synchronous, AEMP sends are asynchronous.
Sending messages in Erlang is synchronous and blocks the process until
a connection has been established and the message sent (and so does not
need a queue that can overflow). AEMP sends return immediately, connection
establishment is handled in the background.
=item * Erlang suffers from silent message loss, AEMP does not.
Erlang implements few guarantees on messages delivery - messages can get
lost without any of the processes realising it (i.e. you send messages a,
b, and c, and the other side only receives messages a and c).
AEMP guarantees (modulo hardware errors) correct ordering, and the
guarantee that after one message is lost, all following ones sent to the
same port are lost as well, until monitoring raises an error, so there are
no silent "holes" in the message sequence.
my $local_group_copy;
db_mon $group => sub { $local_group_copy = $_[0] };
# now "keys %$local_group_copy" always returns the most up-to-date
# list of ports in the group.
C<grp_mon> can be replaced by C<db_mon> with minor changes - C<db_mon>
passes a hash as first argument, and an extra C<$chg> argument that can be
ignored:
db_mon $group => sub {
my ($ports, $add, $chg, $del) = @_;
$ports = [keys %$ports];
# now $ports, $add and $del are the same as
# were originally passed by grp_mon.
...
};
=item Nodes not longer connect to all other nodes.
In AEMP 1.x, every node automatically loads the L<AnyEvent::MP::Global>
module, which in turn would create connections to all other nodes in the
network (helped by the seed nodes).
In version 2.x, global nodes still connect to all other global nodes, but
other nodes don't - now every node either is a global node itself, or
attaches itself to another global node.
If a node isn't a global node itself, then it attaches itself to one
of its seed nodes. If that seed node isn't a global node yet, it will
automatically be upgraded to a global node.
So in many cases, nothing needs to be changed - one just has to make sure
that all seed nodes are meshed together with the other seed nodes (as with
AEMP 1.x), and other nodes specify them as seed nodes. This is most easily
achieved by specifying the same set of seed nodes for all nodes in the
network.
Not opening a connection to every other node is usually an advantage,
except when you need the lower latency of an already established
connection. To ensure a node establishes a connection to another node,
you can monitor the node port (C<mon $node, ...>), which will attempt to
create the connection (and notify you when the connection fails).
=item Listener-less nodes (nodes without binds) are gone.
And are not coming back, at least not in their old form. If no C<binds>
are specified for a node, AnyEvent::MP assumes a default of C<*:*>.
There are vague plans to implement some form of routing domains, which
might or might not bring back listener-less nodes, but don't count on it.
The fact that most connections are now optional somewhat mitigates this,
as a node can be effectively unreachable from the outside without any
problems, as long as it isn't a global node and only reaches out to other
nodes (as opposed to being contacted from other nodes).
=item $AnyEvent::MP::Kernel::WARN has gone.
AnyEvent has acquired a logging framework (L<AnyEvent::Log>), and AEMP now
uses this, and so should your programs.
Every module now documents what kinds of messages it generates, with
AnyEvent::MP acting as a catch all.
On the positive side, this means that instead of setting
C<PERL_ANYEVENT_MP_WARNLEVEL>, you can get away by setting C<AE_VERBOSE> -
much less to type.
=back
=head1 LOGGING
AnyEvent::MP does not normally log anything by itself, but since it is the
root of the context hierarchy for AnyEvent::MP modules, it will receive
all log messages by submodules.
=head1 SEE ALSO
L<AnyEvent::MP::Intro> - a gentle introduction.
L<AnyEvent::MP::Kernel> - more, lower-level, stuff.
L<AnyEvent::MP::Global> - network maintenance and port groups, to find
your applications.
L<AnyEvent::MP::DataConn> - establish data connections between nodes.
L<AnyEvent::MP::LogCatcher> - simple service to display log messages from
all nodes.
L<AnyEvent>.
=head1 AUTHOR
Marc Lehmann <schmorp@schmorp.de>
http://home.schmorp.de/
=cut
1
( run in 2.337 seconds using v1.01-cache-2.11-cpan-df04353d9ac )