view release on metacpan or  search on metacpan

bin/cron.loghack  view on Meta::CPAN

  foreach my $file (@files) {
    $file = File::Fu->file($file);
    my $server = $file->dirname->part(-1);
    (my $base = $file->file) =~ s/.*\.(\d{4}-\d{2}-\d{2})\./$server.$1./
      or die "$file is a strange filename\n";
    $base =~ s/\.gz$/\.bz2/;
    my $dest = $self->archive / $server + $base;
    $self->stdout("archive $file to $dest");
    $dest->e and die "$dest already exists";
    if($file =~ m/\.bz2$/) {
      $file->rename($dest);
    }
    else {
      IPC::Run::run(
        [qw(gunzip -c)], '<', "$file", '|',
        [qw(bzip2 -c)], '>', "$dest"
      ) or die "archiving $dest failed";
      $file->unlink;
    }
  }
}

bin/loghack  view on Meta::CPAN

will not appear until 10min later.  If there are any large downloads,
they could possibly even span a couple of logrotates.

This also means that tomorrow or the next day could concievably hold a
bit of data from a big download that started 24+ hours ago.  In
practice, logrotate is actually just disposing of this data when it runs
gzip.  That is, a request always goes in the logfile that was open when
the apache process spawned?

Still need to figure out the cleanup pass.  Add the skiplists together
(and/or rename them), figure out where to tie-off the last item, etc.
Probably need some tracking of sources and/or chunks.  Chunks can
probably be treated as closed until further notice as long as a
chunkcount file is maintained somewhere.

=end notes

=begin tznotes

Probably going to just leave the date string unprocessed (but we will
definitely slot it into files according to the adjusted zone.)  Of

bin/loghack  view on Meta::CPAN

    else {
      record_source($opt, $file, "$outpath.sources/", $checksum);

      # TODO this could stand to be more atomic
      { # record results
        want_dir("$outpath.loaded");
        my $tag = ($ENV{HOSTNAME} || '') . '.' . $$;
        open($ch, '>', "$checkfile.$tag") or
          die "cannot write '$checkfile.$tag' $!";
        # TODO chmod
        rename("$checkfile.$tag", $checkfile) or
          die "cannot make $checkfile $!";
      }

      # TODO a replayable pipe would be nice
      {local $SIG{CHLD}; close($fh);} # stupid macs
      $fh = open_file($file);
    }

    $opt->{quiet} or print "$nicename -- ",
      sprintf("%02d:%02d:%02d", (localtime)[2,1,0]), "\n";