view release on metacpan or  search on metacpan

AIO.pm  view on Meta::CPAN

While request submission and execution is fully asynchronous, result
processing is not and relies on the perl interpreter calling C<poll_cb>
(or another function with the same effect).

=item result

The request results are processed synchronously by C<poll_cb>.

The C<poll_cb> function will process all outstanding aio requests by
calling their callbacks, freeing memory associated with them and managing
any groups they are contained in.

=item done

Request has reached the end of its lifetime and holds no resources anymore
(except possibly for the Perl object, but its connection to the actual
aio request is severed and calling its methods will either do nothing or
result in a runtime error).

=back

=cut

package IO::AIO;

use Carp ();

use common::sense;

use base 'Exporter';

BEGIN {
   our $VERSION = 4.81;

   our @AIO_REQ = qw(aio_sendfile aio_seek aio_read aio_write aio_open aio_close
                     aio_stat aio_lstat aio_unlink aio_rmdir aio_readdir aio_readdirx
                     aio_scandir aio_symlink aio_readlink aio_realpath aio_fcntl aio_ioctl
                     aio_sync aio_fsync aio_syncfs aio_fdatasync aio_sync_file_range
                     aio_pathsync aio_readahead aio_fiemap aio_allocate
                     aio_rename aio_rename2 aio_link aio_move aio_copy aio_group
                     aio_nop aio_mknod aio_load aio_rmtree aio_mkdir aio_chown
                     aio_chmod aio_utime aio_truncate
                     aio_msync aio_mtouch aio_mlock aio_mlockall
                     aio_statvfs
                     aio_slurp
                     aio_wd);

   our @EXPORT = (@AIO_REQ, qw(aioreq_pri aioreq_nice));
   our @EXPORT_OK = qw(poll_fileno poll_cb poll_wait flush
                       min_parallel max_parallel max_idle idle_timeout
                       nreqs nready npending nthreads
                       max_poll_time max_poll_reqs
                       sendfile fadvise madvise
                       mmap munmap mremap munlock munlockall

                       accept4 tee splice pipe2 pipesize
                       fexecve mount umount memfd_create eventfd
                       timerfd_create timerfd_settime timerfd_gettime
                       pidfd_open pidfd_send_signal pidfd_getfd);

   push @AIO_REQ, qw(aio_busy); # not exported

   @IO::AIO::GRP::ISA = 'IO::AIO::REQ';

   require XSLoader;
   XSLoader::load ("IO::AIO", $VERSION);
}

=head1 FUNCTIONS

=head2 QUICK OVERVIEW

This section simply lists the prototypes most of the functions for
quick reference. See the following sections for function-by-function
documentation.

   aio_wd $pathname, $callback->($wd)
   aio_open $pathname, $flags, $mode, $callback->($fh)
   aio_close $fh, $callback->($status)
   aio_seek  $fh,$offset,$whence, $callback->($offs)
   aio_read  $fh,$offset,$length, $data,$dataoffset, $callback->($retval)
   aio_write $fh,$offset,$length, $data,$dataoffset, $callback->($retval)
   aio_sendfile $out_fh, $in_fh, $in_offset, $length, $callback->($retval)
   aio_readahead $fh,$offset,$length, $callback->($retval)
   aio_stat  $fh_or_path, $callback->($status)
   aio_lstat $fh, $callback->($status)
   aio_statvfs $fh_or_path, $callback->($statvfs)
   aio_utime $fh_or_path, $atime, $mtime, $callback->($status)
   aio_chown $fh_or_path, $uid, $gid, $callback->($status)
   aio_chmod $fh_or_path, $mode, $callback->($status)
   aio_truncate $fh_or_path, $offset, $callback->($status)
   aio_allocate $fh, $mode, $offset, $len, $callback->($status)
   aio_fiemap $fh, $start, $length, $flags, $count, $cb->(\@extents)
   aio_unlink $pathname, $callback->($status)
   aio_mknod $pathname, $mode, $dev, $callback->($status)
   aio_link $srcpath, $dstpath, $callback->($status)
   aio_symlink $srcpath, $dstpath, $callback->($status)
   aio_readlink $pathname, $callback->($link)
   aio_realpath $pathname, $callback->($path)
   aio_rename $srcpath, $dstpath, $callback->($status)
   aio_rename2 $srcpath, $dstpath, $flags, $callback->($status)
   aio_mkdir $pathname, $mode, $callback->($status)
   aio_rmdir $pathname, $callback->($status)
   aio_readdir $pathname, $callback->($entries)
   aio_readdirx $pathname, $flags, $callback->($entries, $flags)
      IO::AIO::READDIR_DENTS IO::AIO::READDIR_DIRS_FIRST
      IO::AIO::READDIR_STAT_ORDER IO::AIO::READDIR_FOUND_UNKNOWN
   aio_scandir $pathname, $maxreq, $callback->($dirs, $nondirs)
   aio_load $pathname, $data, $callback->($status)
   aio_copy $srcpath, $dstpath, $callback->($status)
   aio_move $srcpath, $dstpath, $callback->($status)
   aio_rmtree $pathname, $callback->($status)
   aio_fcntl $fh, $cmd, $arg, $callback->($status)
   aio_ioctl $fh, $request, $buf, $callback->($status)
   aio_sync $callback->($status)
   aio_syncfs $fh, $callback->($status)
   aio_fsync $fh, $callback->($status)
   aio_fdatasync $fh, $callback->($status)
   aio_sync_file_range $fh, $offset, $nbytes, $flags, $callback->($status)
   aio_pathsync $pathname, $callback->($status)
   aio_msync $scalar, $offset = 0, $length = undef, flags = MS_SYNC, $callback->($status)

AIO.pm  view on Meta::CPAN

If an error occurs, the callback receives no arguments. The special
C<errno> value C<IO::AIO::EBADR> is available to test for flag errors.

Otherwise, the callback receives an array reference with extent
structures. Each extent structure is an array reference itself, with the
following members:

   [$logical, $physical, $length, $flags]

Flags is any combination of the following flag values (typically either C<0>
or C<IO::AIO::FIEMAP_EXTENT_LAST> (1)):

C<IO::AIO::FIEMAP_EXTENT_LAST>, C<IO::AIO::FIEMAP_EXTENT_UNKNOWN>,
C<IO::AIO::FIEMAP_EXTENT_DELALLOC>, C<IO::AIO::FIEMAP_EXTENT_ENCODED>,
C<IO::AIO::FIEMAP_EXTENT_DATA_ENCRYPTED>, C<IO::AIO::FIEMAP_EXTENT_NOT_ALIGNED>,
C<IO::AIO::FIEMAP_EXTENT_DATA_INLINE>, C<IO::AIO::FIEMAP_EXTENT_DATA_TAIL>,
C<IO::AIO::FIEMAP_EXTENT_UNWRITTEN>, C<IO::AIO::FIEMAP_EXTENT_MERGED> or
C<IO::AIO::FIEMAP_EXTENT_SHARED>.

At the time of this writing (Linux 3.2), this request is unreliable unless
C<$count> is C<undef>, as the kernel has all sorts of bugs preventing
it to return all extents of a range for files with a large number of
extents. The code (only) works around all these issues if C<$count> is
C<undef>.

=item aio_group $callback->(...)

This is a very special aio request: Instead of doing something, it is a
container for other aio requests, which is useful if you want to bundle
many requests into a single, composite, request with a definite callback
and the ability to cancel the whole request with its subrequests.

Returns an object of class L<IO::AIO::GRP>. See its documentation below
for more info.

Example:

   my $grp = aio_group sub {
      print "all stats done\n";
   };

   add $grp
      (aio_stat ...),
      (aio_stat ...),
      ...;

=item aio_nop $callback->()

This is a special request - it does nothing in itself and is only used for
side effects, such as when you want to add a dummy request to a group so
that finishing the requests in the group depends on executing the given
code.

While this request does nothing, it still goes through the execution
phase and still requires a worker thread. Thus, the callback will not
be executed immediately but only after other requests in the queue have
entered their execution phase. This can be used to measure request
latency.

=item IO::AIO::aio_busy $fractional_seconds, $callback->()  *NOT EXPORTED*

Mainly used for debugging and benchmarking, this aio request puts one of
the request workers to sleep for the given time.

While it is theoretically handy to have simple I/O scheduling requests
like sleep and file handle readable/writable, the overhead this creates is
immense (it blocks a thread for a long time) so do not use this function
except to put your application under artificial I/O pressure.

=back


=head2 IO::AIO::WD - multiple working directories

Your process only has one current working directory, which is used by all
threads. This makes it hard to use relative paths (some other component
could call C<chdir> at any time, and it is hard to control when the path
will be used by IO::AIO).

One solution for this is to always use absolute paths. This usually works,
but can be quite slow (the kernel has to walk the whole path on every
access), and can also be a hassle to implement.

Newer POSIX systems have a number of functions (openat, fdopendir,
futimensat and so on) that make it possible to specify working directories
per operation.

For portability, and because the clowns who "designed", or shall I write,
perpetrated this new interface were obviously half-drunk, this abstraction
cannot be perfect, though.

IO::AIO allows you to convert directory paths into a so-called IO::AIO::WD
object. This object stores the canonicalised, absolute version of the
path, and on systems that allow it, also a directory file descriptor.

Everywhere where a pathname is accepted by IO::AIO (e.g. in C<aio_stat>
or C<aio_unlink>), one can specify an array reference with an IO::AIO::WD
object and a pathname instead (or the IO::AIO::WD object alone, which
gets interpreted as C<[$wd, "."]>). If the pathname is absolute, the
IO::AIO::WD object is ignored, otherwise the pathname is resolved relative
to that IO::AIO::WD object.

For example, to get a wd object for F</etc> and then stat F<passwd>
inside, you would write:

   aio_wd "/etc", sub {
      my $etcdir = shift;

      # although $etcdir can be undef on error, there is generally no reason
      # to check for errors here, as aio_stat will fail with ENOENT
      # when $etcdir is undef.

      aio_stat [$etcdir, "passwd"], sub {
         # yay
      };
   };

The fact that C<aio_wd> is a request and not a normal function shows that
creating an IO::AIO::WD object is itself a potentially blocking operation,
which is why it is done asynchronously.