AnyEvent-MP
view release on metacpan or search on metacpan
NAME
AnyEvent::MP - erlang-style multi-processing/message-passing framework
SYNOPSIS
use AnyEvent::MP;
$NODE # contains this node's node ID
NODE # returns this node's node ID
$SELF # receiving/own port id in rcv callbacks
# initialise the node so it can send/receive messages
configure;
# ports are message destinations
# sending messages
snd $port, type => data...;
snd $port, @msg;
snd @msg_with_first_element_being_a_port;
# creating/using ports, the simple way
my $simple_port = port { my @msg = @_ };
# creating/using ports, tagged message matching
my $port = port;
rcv $port, ping => sub { snd $_[0], "pong" };
rcv $port, pong => sub { warn "pong received\n" };
# create a port on another node
my $port = spawn $node, $initfunc, @initdata;
# destroy a port again
kil $port; # "normal" kill
kil $port, my_error => "everything is broken"; # error kill
# monitoring
mon $port, $cb->(@msg) # callback is invoked on death
mon $port, $localport # kill localport on abnormal death
mon $port, $localport, @msg # send message on death
# temporarily execute code in port context
peval $port, sub { die "kill the port!" };
# execute callbacks in $SELF port context
my $timer = AE::timer 1, 0, psub {
die "kill the port, delayed";
};
# distributed database - modification
db_set $family => $subkey [=> $value] # add a subkey
db_del $family => $subkey... # delete one or more subkeys
db_reg $family => $port [=> $value] # register a port
# distributed database - queries
db_family $family => $cb->(\%familyhash)
db_keys $family => $cb->(\@keys)
db_values $family => $cb->(\@values)
# distributed database - monitoring a family
db_mon $family => $cb->(\%familyhash, \@added, \@changed, \@deleted)
DESCRIPTION
This module (-family) implements a simple message passing framework.
Despite its simplicity, you can securely message other processes running
on the same or other hosts, and you can supervise entities remotely.
For an introduction to this module family, see the AnyEvent::MP::Intro
manual page and the examples under eg/.
CONCEPTS
port
Not to be confused with a TCP port, a "port" is something you can
send messages to (with the "snd" function).
Ports allow you to register "rcv" handlers that can match all or
just some messages. Messages send to ports will not be queued,
regardless of anything was listening for them or not.
Ports are represented by (printable) strings called "port IDs".
port ID - "nodeid#portname"
A port ID is the concatenation of a node ID, a hash-mark ("#") as
separator, and a port name (a printable string of unspecified format
created by AnyEvent::MP).
node
A node is a single process containing at least one port - the node
port, which enables nodes to manage each other remotely, and to
create new ports.
Nodes are either public (have one or more listening ports) or
private (no listening ports). Private nodes cannot talk to other
private nodes currently, but all nodes can talk to public nodes.
Nodes is represented by (printable) strings called "node IDs".
node ID - "[A-Za-z0-9_\-.:]*"
A node ID is a string that uniquely identifies the node within a
network. Depending on the configuration used, node IDs can look like
a hostname, a hostname and a port, or a random string. AnyEvent::MP
itself doesn't interpret node IDs in any way except to uniquely
identify a node.
binds - "ip:port"
Nodes can only talk to each other by creating some kind of
connection to each other. To do this, nodes should listen on one or
more local transport endpoints - binds.
Currently, only standard "ip:port" specifications can be used, which
specify TCP ports to listen on. So a bind is basically just a tcp
socket in listening mode that accepts connections from other nodes.
seed nodes
When a node starts, it knows nothing about the network it is in - it
needs to connect to at least one other node that is already in the
network. These other nodes are called "seed nodes".
Seed nodes themselves are not special - they are seed nodes only
because some other node *uses* them as such, but any node can be
used as seed node for other nodes, and eahc node can use a different
set of seed nodes.
$guard = db_mon $family => $cb->(\%familyhash, \@added, \@changed,
\@deleted)
Creates a monitor on the given database family. Each time a key is
set or is deleted the callback is called with a hash containing the
database family and three lists of added, changed and deleted
subkeys, respectively. If no keys have changed then the array
reference might be "undef" or even missing.
If not called in void context, a guard object is returned that, when
destroyed, stops the monitor.
The family hash reference and the key arrays belong to AnyEvent::MP
and must not be modified or stored by the callback. When in doubt,
make a copy.
As soon as possible after the monitoring starts, the callback will
be called with the intiial contents of the family, even if it is
empty, i.e. there will always be a timely call to the callback with
the current contents.
It is possible that the callback is called with a change event even
though the subkey is already present and the value has not changed.
The monitoring stops when the guard object is destroyed.
Example: on every change to the family "mygroup", print out all
keys.
my $guard = db_mon mygroup => sub {
my ($family, $a, $c, $d) = @_;
print "mygroup members: ", (join " ", keys %$family), "\n";
};
Exmaple: wait until the family "My::Module::workers" is non-empty.
my $guard; $guard = db_mon My::Module::workers => sub {
my ($family, $a, $c, $d) = @_;
return unless %$family;
undef $guard;
print "My::Module::workers now nonempty\n";
};
Example: print all changes to the family
"AnyEvent::Fantasy::Module".
my $guard = db_mon AnyEvent::Fantasy::Module => sub {
my ($family, $a, $c, $d) = @_;
print "+$_=$family->{$_}\n" for @$a;
print "*$_=$family->{$_}\n" for @$c;
print "-$_=$family->{$_}\n" for @$d;
};
AnyEvent::MP vs. Distributed Erlang
AnyEvent::MP got lots of its ideas from distributed Erlang (Erlang node
== aemp node, Erlang process == aemp port), so many of the documents and
programming techniques employed by Erlang apply to AnyEvent::MP. Here is
a sample:
http://www.erlang.se/doc/programming_rules.shtml
http://erlang.org/doc/getting_started/part_frame.html # chapters 3 and 4
http://erlang.org/download/erlang-book-part1.pdf # chapters 5 and 6
http://erlang.org/download/armstrong_thesis_2003.pdf # chapters 4 and 5
Despite the similarities, there are also some important differences:
* Node IDs are arbitrary strings in AEMP.
Erlang relies on special naming and DNS to work everywhere in the
same way. AEMP relies on each node somehow knowing its own
address(es) (e.g. by configuration or DNS), and possibly the
addresses of some seed nodes, but will otherwise discover other
nodes (and their IDs) itself.
* Erlang has a "remote ports are like local ports" philosophy, AEMP
uses "local ports are like remote ports".
The failure modes for local ports are quite different (runtime
errors only) then for remote ports - when a local port dies, you
*know* it dies, when a connection to another node dies, you know
nothing about the other port.
Erlang pretends remote ports are as reliable as local ports, even
when they are not.
AEMP encourages a "treat remote ports differently" philosophy, with
local ports being the special case/exception, where transport errors
cannot occur.
* Erlang uses processes and a mailbox, AEMP does not queue.
Erlang uses processes that selectively receive messages out of
order, and therefore needs a queue. AEMP is event based, queuing
messages would serve no useful purpose. For the same reason the
pattern-matching abilities of AnyEvent::MP are more limited, as
there is little need to be able to filter messages without dequeuing
them.
This is not a philosophical difference, but simply stems from
AnyEvent::MP being event-based, while Erlang is process-based.
You can have a look at Coro::MP for a more Erlang-like process model
on top of AEMP and Coro threads.
* Erlang sends are synchronous, AEMP sends are asynchronous.
Sending messages in Erlang is synchronous and blocks the process
until a connection has been established and the message sent (and so
does not need a queue that can overflow). AEMP sends return
immediately, connection establishment is handled in the background.
* Erlang suffers from silent message loss, AEMP does not.
Erlang implements few guarantees on messages delivery - messages can
get lost without any of the processes realising it (i.e. you send
messages a, b, and c, and the other side only receives messages a
and c).
AEMP guarantees (modulo hardware errors) correct ordering, and the
guarantee that after one message is lost, all following ones sent to
the same port are lost as well, until monitoring raises an error, so
db_mon $group => sub { $local_group_copy = $_[0] };
# now "keys %$local_group_copy" always returns the most up-to-date
# list of ports in the group.
"grp_mon" can be replaced by "db_mon" with minor changes - "db_mon"
passes a hash as first argument, and an extra $chg argument that can
be ignored:
db_mon $group => sub {
my ($ports, $add, $chg, $del) = @_;
$ports = [keys %$ports];
# now $ports, $add and $del are the same as
# were originally passed by grp_mon.
...
};
Nodes not longer connect to all other nodes.
In AEMP 1.x, every node automatically loads the AnyEvent::MP::Global
module, which in turn would create connections to all other nodes in
the network (helped by the seed nodes).
In version 2.x, global nodes still connect to all other global
nodes, but other nodes don't - now every node either is a global
node itself, or attaches itself to another global node.
If a node isn't a global node itself, then it attaches itself to one
of its seed nodes. If that seed node isn't a global node yet, it
will automatically be upgraded to a global node.
So in many cases, nothing needs to be changed - one just has to make
sure that all seed nodes are meshed together with the other seed
nodes (as with AEMP 1.x), and other nodes specify them as seed
nodes. This is most easily achieved by specifying the same set of
seed nodes for all nodes in the network.
Not opening a connection to every other node is usually an
advantage, except when you need the lower latency of an already
established connection. To ensure a node establishes a connection to
another node, you can monitor the node port ("mon $node, ..."),
which will attempt to create the connection (and notify you when the
connection fails).
Listener-less nodes (nodes without binds) are gone.
And are not coming back, at least not in their old form. If no
"binds" are specified for a node, AnyEvent::MP assumes a default of
"*:*".
There are vague plans to implement some form of routing domains,
which might or might not bring back listener-less nodes, but don't
count on it.
The fact that most connections are now optional somewhat mitigates
this, as a node can be effectively unreachable from the outside
without any problems, as long as it isn't a global node and only
reaches out to other nodes (as opposed to being contacted from other
nodes).
$AnyEvent::MP::Kernel::WARN has gone.
AnyEvent has acquired a logging framework (AnyEvent::Log), and AEMP
now uses this, and so should your programs.
Every module now documents what kinds of messages it generates, with
AnyEvent::MP acting as a catch all.
On the positive side, this means that instead of setting
"PERL_ANYEVENT_MP_WARNLEVEL", you can get away by setting
"AE_VERBOSE" - much less to type.
LOGGING
AnyEvent::MP does not normally log anything by itself, but since it is
the root of the context hierarchy for AnyEvent::MP modules, it will
receive all log messages by submodules.
SEE ALSO
AnyEvent::MP::Intro - a gentle introduction.
AnyEvent::MP::Kernel - more, lower-level, stuff.
AnyEvent::MP::Global - network maintenance and port groups, to find your
applications.
AnyEvent::MP::DataConn - establish data connections between nodes.
AnyEvent::MP::LogCatcher - simple service to display log messages from
all nodes.
AnyEvent.
AUTHOR
Marc Lehmann <schmorp@schmorp.de>
http://home.schmorp.de/
( run in 0.475 second using v1.01-cache-2.11-cpan-df04353d9ac )