App-MBUtiny
17 results

 view release on metacpan or  search on metacpan

META.json  view on Meta::CPAN 

{
   "abstract" : "Websites and any file system elements backup tool",
   "author" : [
      "Serz Minus (Sergey Lepenkov) <abalama@cpan.org>"
   ],
   "dynamic_config" : 1,
   "generated_by" : "ExtUtils::MakeMaker version 7.0401, CPAN::Meta::Converter version 2.150001",
   "license" : [
      "perl_5"
   ],
   "meta-spec" : {
      "url" : "http://search.cpan.org/perldoc?CPAN::Meta::Spec",

META.yml  view on Meta::CPAN 

---
abstract: 'Websites and any file system elements backup tool'
author:
  - 'Serz Minus (Sergey Lepenkov) <abalama@cpan.org>'
build_requires:
  ExtUtils::MakeMaker: '6.6'
  Test::More: '0.94'
configure_requires:
  ExtUtils::MakeMaker: '0'
dynamic_config: 1
generated_by: 'ExtUtils::MakeMaker version 7.0401, CPAN::Meta::Converter version 2.150001'
license: perl

README  view on Meta::CPAN 

NAME

    App::MBUtiny - Websites and any file system elements backup tool

VERSION

    Version 1.12

SYNOPSIS

        # mbutiny test

        # mbutiny backup

        # mbutiny restore

        # mbutiny report

DESCRIPTION

    Websites and any file system elements backup tool

 FEATURES

    Backup Files and Folders

    Backup small databases

    Run external utilities for object preparation

    Supported storage of backups on local drives

    Supported storage of backups on remote SFTP storages

    Supported storage of backups on remote FTP storages

    Supported storage of backups on remote HTTP storages

    Easy configuration

    Monitoring feature enabled

 SYSTEM REQUIREMENTS

    Perl v5.16+

    SSH client

README  view on Meta::CPAN 


    ...and then:

        # sudo mbutiny configure

 CONFIGURATION

    By default configuration file located in /etc/mbutiny directory

    Every configuration directive detailed described in mbutiny.conf file,
    also see hosts/foo.conf.sample file for MBUtiny backup hosts
    configuration

 CRONTAB

    To automatically launch the program, we recommend using standard
    scheduling tools, such as crontab

        0 2 * * * mbutiny -l backup >/dev/null 2>>/var/log/mbutiny-error.log

    Or for selected hosts only:

        0 2 * * * mbutiny -l backup foo bar >/dev/null 2>>/var/log/mbutiny-error.log
        15 2 * * * mbutiny -l backup baz >/dev/null 2>>/var/log/mbutiny-error.log

    For daily reporting:

        0 9 * * * mbutiny -l report >/dev/null 2>>/var/log/mbutiny-error.log

 COLLECTOR

    Collector is a monitoring server that allows you to collect data on the
    status of performs backups. The collector allows you to build reports
    on the collected data from various servers.

    How it work?

        +------------+
        | Monitoring |<--http/https-+
        +------------+              |
                                    |
        +----------+          +-----+-----+        +----------+
        | Server 1 |--local-->| COLLECTOR |--DBI-->| DataBase |

README  view on Meta::CPAN 

        +----------+                |
        | Server 2 |---http/https---+
        +----------+

    For installation of the collector Your need Apache 2.2/2.4 web server
    and CGI/FastCGI script. See collector.cgi.sample in /etc/mbutiny
    directory

 HTTP SERVER

    If you want to use the HTTP server as a storage for backups, you need
    to install the CGI/FastCGI script on Apache 2.2/2.4 web server.

    See server.cgi

INTERNAL METHODS

    again

      The CTK method for classes extension. For internal use only!

README  view on Meta::CPAN 

    objdir

          my $objdir = $app->objdir;

      Returns path to processed objects

    rstdir

          my $rstdir = $app->rstdir;

      Returns path to restored backups

HISTORY

    See Changes file

DEPENDENCIES

    CTK

TO DO

bin/mbutiny  view on Meta::CPAN 


mbutiny - easily interact with App::MBUtiny from the command line

=head1 SYNOPSIS

    mbutiny [options] [commands [args]] 

    mbutiny [-dlv]

    mbutiny [--debug] [--log] [--config=CONFIG_FILE] [--datadir=DATADIR]
            [ configure | test [HOSTs] | backup [HOSTs] | 
            restore [HOSTs] [YYYY.MM.DD] | report ]

    mbutiny configure

    mbutiny test

    mbutiny backup

    mbutiny restore

    mbutiny report

=head1 OPTIONS

=over 4

=item B<-c CONFIG_FILE, --config=CONFIG_FILE>

bin/mbutiny  view on Meta::CPAN 

=item B<-H, --longhelp>

Show long help information and quit

=item B<-l, --log>

Enabling write debug information to syslog or user log file.

Do not confuse the debug logging from regular logging to a file mbutiny.log.
Regular logging allows you to store information in mbutiny.log on the progress of the processes 
module (test, backup, restore), whereas debug logging for debugging of the internal components 
of the module.

To control the level of debugging mbutiny.log see parameter LogEnable and LogLevel.

=item B<-v, --verbose>

Enabling at which displays information about the progress on the screen

=item B<-V, --version>

bin/mbutiny  view on Meta::CPAN 

=head1 COMMANDS

=over 4

=item B<configure>

Configure (initializing) the MBUtiny

=item B<test [HOSTs]>

Testing of HOSTs after configuration and before backup performing.

By default will used all enabled in configuration HOSTs.

=item B<backup [HOSTs]>

Compressing all the objects for the specified HOSTs and then send the resulting archive to 
the storage.

By default will used all enabled in configuration HOSTs.

=item B<restore [HOSTs] [DATE]>

Downloading the file from the storages previously created backup by name.
And then, each downloaded file is unpacked to the restore directory

By default will used all enabled in configuration HOSTs.

DATE - is date of backup in format: DD.MM.YYYY or YYYY.MM.DD

=item B<report [HOSTs]>

Checking the last backup-file on each available collector for all defined backup names

By default will used all enabled in configuration HOSTs.

=back

=head1 DESCRIPTION

Websites and any file system elements backup tool

See L<WWW::MLite> for details

=head1 AUTHOR

Serz Minus (Sergey Lepenkov) L<http://www.serzik.com> E<lt>abalama@cpan.orgE<gt>

=head1 COPYRIGHT

Copyright (C) 1998-2019 D&D Corporation. All Rights Reserved

bin/mbutiny  view on Meta::CPAN 


use Cwd qw/getcwd/;
use File::Spec;

use CTK::FilePid;

use App::MBUtiny;

use constant {
    PIDMASK     => '%s.%s.pid',
    PIDREQS     => [qw/backup restore/],
};

$| = 1;  # autoflush

my $options = {};
Getopt::Long::Configure ("bundling");
GetOptions($options,
    # NoUsed keys map:
    #
    # a A b B   C     e E

lib/App/MBUtiny.pm  view on Meta::CPAN 

package App::MBUtiny; # $Id: MBUtiny.pm 131 2019-07-16 18:45:44Z abalama $
use strict;
use utf8;

=encoding utf-8

=head1 NAME

App::MBUtiny - Websites and any file system elements backup tool

=head1 VERSION

Version 1.13

=head1 SYNOPSIS

    # mbutiny test

    # mbutiny backup

    # mbutiny restore

    # mbutiny report

=head1 DESCRIPTION

Websites and any file system elements backup tool

=head2 FEATURES

=over 4

=item Backup Files and Folders

=item Backup small databases

=item Run external utilities for object preparation

=item Supported storage of backups on local drives

=item Supported storage of backups on remote SFTP storages

=item Supported storage of backups on remote FTP storages

=item Supported storage of backups on remote HTTP storages

=item Easy configuration

=item Monitoring feature enabled

=back

=head2 SYSTEM REQUIREMENTS

=over 4

lib/App/MBUtiny.pm  view on Meta::CPAN 


...and then:

    # sudo mbutiny configure

=head2 CONFIGURATION

By default configuration file located in C</etc/mbutiny> directory

Every configuration directive detailed described in C<mbutiny.conf> file, also
see C<hosts/foo.conf.sample> file for MBUtiny backup hosts configuration

=head2 CRONTAB

To automatically launch the program, we recommend using standard scheduling tools, such as crontab

    0 2 * * * mbutiny -l backup >/dev/null 2>>/var/log/mbutiny-error.log

Or for selected hosts only:

    0 2 * * * mbutiny -l backup foo bar >/dev/null 2>>/var/log/mbutiny-error.log
    15 2 * * * mbutiny -l backup baz >/dev/null 2>>/var/log/mbutiny-error.log

For daily reporting:

    0 9 * * * mbutiny -l report >/dev/null 2>>/var/log/mbutiny-error.log

=head2 COLLECTOR

Collector is a monitoring server that allows you to collect data on the status of performs backups.
The collector allows you to build reports on the collected data from various servers.

How it work?

    +------------+
    | Monitoring |<--http/https-+
    +------------+              |
                                |
    +----------+          +-----+-----+        +----------+
    | Server 1 |--local-->| COLLECTOR |--DBI-->| DataBase |

lib/App/MBUtiny.pm  view on Meta::CPAN 

                                ^
    +----------+                |
    | Server 2 |---http/https---+
    +----------+

For installation of the collector Your need Apache 2.2/2.4 web server and CGI/FastCGI script.
See C<collector.cgi.sample> in C</etc/mbutiny> directory

=head2 HTTP SERVER

If you want to use the HTTP server as a storage for backups, you need to install the CGI/FastCGI
script on Apache 2.2/2.4 web server.

See C<server.cgi>

=head1 INTERNAL METHODS

=over 4

=item B<again>

lib/App/MBUtiny.pm  view on Meta::CPAN 

=item B<objdir>

    my $objdir = $app->objdir;

Returns path to processed objects

=item B<rstdir>

    my $rstdir = $app->rstdir;

Returns path to restored backups

=back

=head1 HISTORY

See C<Changes> file

=head1 DEPENDENCIES

L<CTK>

lib/App/MBUtiny.pm  view on Meta::CPAN 



        #
        # Init
        #
        my $name = _getName($pair); # Backup name
        my $host = node($pair, $name); # Config section
        my $hostskip = (!@arguments || grep {lc($name) eq lc($_)} @arguments) ? 0 : 1;
        my $enabled = value($host, 'enable') ? 1 : 0;
        if ($hostskip || !$enabled) {
            $self->log_info("Skip testing for \"%s\" backup host section", $name);
            next;
        }
        my $tbl = Text::SimpleTable->new(@{(TEST_HEADERS)});
        $self->log_info("Start testing for \"%s\" backup host section", $name);
        push @header, ["Backup name", $name];
        push @errors, $self->getdbi->dsn, $self->getdbi->error, "" if $self->getdbi->error;


        #
        # Loading backup data
        #
        my $buday   = (value($host, 'buday')   // $self->config('buday'))   || 0;
        my $buweek  = (value($host, 'buweek')  // $self->config('buweek'))  || 0;
        my $bumonth = (value($host, 'bumonth') // $self->config('bumonth')) || 0;
        push @header, (
                ["Daily backups", $buday],
                ["Weekly backups", $buweek],
                ["Monthly backups", $bumonth],
            );

        # Get mask vars
        my $arc = $self->_getArc($host);
        my $arcmask = value($host, 'arcmask') || ARC_MASK;
           $arcmask =~ s/\[DEFAULT\]/ARC_MASK()/gie;
        my %maskfmt = (
                HOST  => $name,
                YEAR  => '',
                MONTH => '',

lib/App/MBUtiny.pm  view on Meta::CPAN 

                EXT   => value($arc, 'ext') || '',
            );
        push @header, ["Backup mask", $arcmask];

        # Get saved dates
        my @dates = $self->_getDates($buday, $buweek, $bumonth);

        # Get paths
        push @header, (
                ["Work directory", $self->datadir],
                ["Directory for backups", $self->objdir],
                ["Directory for restores", $self->rstdir],
            );

        # Regular objects
        my $objects = array($host, 'object');
        my $regular_objects = 0;
        {
            my $i = 0;
            foreach my $o (@$objects) {
                next unless $o;

lib/App/MBUtiny.pm  view on Meta::CPAN 

            $files_number ? sprintf("%d files found", $files_number) : "No files found",
            $storage->error ? 'FAIL' : $files_number ? 'PASS' : 'SKIP',
        );
        if ($storage->error) {
            $self->log_error($storage->error);
            push @errors, $storage->error, "";
            $ostat = 0;
        };
        my $last_file = (sort {$b cmp $a} @filelist)[0];
        if ($files_number && $last_file) {
            push @header, ["Last backup file", $last_file];
            my $list = hash($storage->{list});
            foreach my $k (keys %$list) {
                my $l = array($list, $k);
                my $st = (grep {$_ eq $last_file} @$l) ? 1 : 0;
                $tbl->row(sprintf("%s storage", $k),
                    $st ? sprintf("File %s is available", $last_file) : sprintf("File %s missing", $last_file),
                    $st ? 'PASS' : 'SKIP',
                );
            }
            #say(explain($storage->{list}));

lib/App/MBUtiny.pm  view on Meta::CPAN 

        );
        push @header, ["Summary status", $ostat ? 'PASS' : 'FAIL'];
        my @report;
        my $report_name = $ostat ? "report" : "error report";
        push @report, $self->_report_common(@header); # Common information
        push @report, $self->_report_summary($ostat ? "All tests successful" : "Errors occurred while testing"); # Summary table
        push @report, $tbl->draw() || ''; # Table
        push @report, $self->_report_errors(@errors); # List of occurred errors
        if ($TTY || $self->verbosemode) { # Draw to TTY
            printf("%s\n\n", "~" x 94);
            printf("The %s for %s backup host\n\n", $report_name, $name);
            print join("\n", @report, "");
        }


        #
        # SendMail (Send report)
        #
        if ($send_report) {
            unshift @report, $self->_report_title($report_name, $name);
            push @report, $self->_report_footer();

lib/App/MBUtiny.pm  view on Meta::CPAN 

            $ma{"-subject"} = sprintf("%s %s (%s on %s)", PROJECTNAME, $report_name, $name, $hostname);
            $ma{"-message"} = join("\n", @report);

            # Send!
            my $sent = sendmail(%ma);
            if ($sent) { $self->debug(sprintf("Mail has been sent to: %s", $to)) }
            else { $self->error(sprintf("Mail was not sent to: %s", $to)) }
        }

        # Finish testing
        $self->log_info("Finish testing for \"%s\" backup host section", $name);

        # General status
        $status = 0 unless $ostat;
    }

    return $status;
});

__PACKAGE__->register_handler(
    handler     => "backup",
    description => "Backup hosts",
    code => sub {
### CODE:
    my ($self, $meta, @arguments) = @_;
    $self->configure or return 0;
    my $status = 1;

    # Get host-list
    my @hosts = $self->_getHosts();
    unless (scalar(@hosts)) {

lib/App/MBUtiny.pm  view on Meta::CPAN 



        #
        # Init
        #
        my $name = _getName($pair); # Backup name
        my $host = node($pair, $name); # Config section
        my $hostskip = (!@arguments || grep {lc($name) eq lc($_)} @arguments) ? 0 : 1;
        my $enabled = value($host, 'enable') ? 1 : 0;
        if ($hostskip || !$enabled) {
            $self->log_info("Skip backup process for \"%s\" backup host section", $name);
            next;
        }
        my $tbl = Text::SimpleTable->new(@{(TABLE_HEADERS)});
        $self->log_info("Start backup process for \"%s\" backup host section", $name);
        push @header, ["Backup name", $name];
        push @errors, $self->getdbi->dsn, $self->getdbi->error, "" if $self->getdbi->error;


        #
        # Loading backup data
        #
        my $buday   = (value($host, 'buday')   // $self->config('buday'))   || 0;
        my $buweek  = (value($host, 'buweek')  // $self->config('buweek'))  || 0;
        my $bumonth = (value($host, 'bumonth') // $self->config('bumonth')) || 0;
        push @header, (
                ["Daily backups", $buday],
                ["Weekly backups", $buweek],
                ["Monthly backups", $bumonth],
            );

        # Get mask vars
        my $arc = $self->_getArc($host);
        my $arcmask = value($host, 'arcmask') || ARC_MASK;
           $arcmask =~ s/\[DEFAULT\]/ARC_MASK()/gie;
        my %maskfmt = (
                HOST  => $name,
                YEAR  => '',
                MONTH => '',

lib/App/MBUtiny.pm  view on Meta::CPAN 



        #
        # Testing storages
        #
        $step = "Storages testing";
        $self->debug($step);
        my $storage = new App::MBUtiny::Storage(
                name => $name, # Backup name
                host => $host, # Host config section
                path => $self->objdir, # Where is located backup archive
                fixup => sub {
                    my $strg = shift; # Storage object
                    my $oper = shift // 'noop'; # Operation name
                    my $colret;
                    if ($oper =~ /^(del)|(rem)/i) {
                        my $f = shift;
                        $colret = $collector->fixup(
                                operation => $oper,
                                name    => $name,
                                file    => $f,

lib/App/MBUtiny.pm  view on Meta::CPAN 

        #
        # Report generate
        #
        $tbl->hr;
        $tbl->row(dtf(DATE_FORMAT), 'RESULT',
            $ostat ? 'All processes successful' : 'Errors have occurred!',
            $ostat ? 'PASS' : 'FAIL'
        );
        push @header, ["Summary status", $ostat ? 'PASS' : 'FAIL'];
        my @report;
        my $report_name = $ostat ? "backup report" : "backup error report";
        push @report, $self->_report_common(@header); # Common information
        push @report, $self->_report_summary($ostat ? "Backup is done" : "Errors occurred while performing backup"); # Summary table
        push @report, $tbl->draw() || ''; # Table
        push @report, $self->_report_errors(@errors); # List of occurred errors
        if ($TTY || $self->verbosemode) { # Draw to TTY
            printf("%s\n\n", "~" x 114);
            printf("The %s for %s backup host\n\n", $report_name, $name);
            print join("\n", @report, "");
        }


        #
        # SendMail (Send report)
        #
        if ($send_report) {
            unshift @report, $self->_report_title($report_name, $name);
            push @report, $self->_report_footer();
            my %ma = (); foreach my $k (keys %$sm) { $ma{"-".$k} = $sm->{$k} };
            $ma{"-subject"} = sprintf("%s %s (%s on %s)", PROJECTNAME, $report_name, $name, $hostname);
            $ma{"-message"} = join("\n", @report);

            # Send!
            my $sent = sendmail(%ma);
            if ($sent) { $self->debug(sprintf("Mail has been sent to: %s", $to)) }
            else { $self->error(sprintf("Mail was not sent to: %s", $to)) }
        }

        # Finish backup
        $self->log_info("Finish backup process for \"%s\" backup host section", $name);

        # General status
        $status = 0 unless $ostat;
    }

    return $status;
});

__PACKAGE__->register_handler(
    handler     => "restore",

lib/App/MBUtiny.pm  view on Meta::CPAN 



        #
        # Init
        #
        my $name = _getName($pair); # Backup name
        my $host = node($pair, $name); # Config section
        my $hostskip = (!@arguments || grep {lc($name) eq lc($_)} @arguments) ? 0 : 1;
        my $enabled = value($host, 'enable') ? 1 : 0;
        if ($hostskip || !$enabled) {
            $self->log_info("Skip restore process for \"%s\" backup host section", $name);
            next;
        }
        my $tbl = Text::SimpleTable->new(@{(TABLE_HEADERS)});
        $self->log_info("Start restore process for \"%s\" backup host section", $name);
        push @header, ["Backup name", $name];
        push @errors, $self->getdbi->dsn, $self->getdbi->error, "" if $self->getdbi->error;

        # Get mask vars
        my $arc = $self->_getArc($host);
        my $arcmask = value($host, 'arcmask') || ARC_MASK;
           $arcmask =~ s/\[DEFAULT\]/ARC_MASK()/gie;
        my %maskfmt = (
                HOST  => $name,
                YEAR  => sprintf("%04d", $ymd[0]),

lib/App/MBUtiny.pm  view on Meta::CPAN 



        #
        # Testing storages
        #
        $step = "Storages testing";
        $self->debug($step);
        my $storage = new App::MBUtiny::Storage(
                name => $name, # Backup name
                host => $host, # Host config section
                path => $self->rstdir, # Where is located restored backup archive
                validate => sub {
                    my $strg = shift; # storage object
                    my $file = shift; # fetched file
                    if ($info{size}) { # Valid sizes
                        my $size = filesize($file) // 0;
                        unless ($size == $info{size}) {
                            $strg->error(sprintf("File size incorrect: got=%d; expected=%d", $size, $info{size}));
                            return 0;
                        }
                    }

lib/App/MBUtiny.pm  view on Meta::CPAN 

        my $restore_dir = File::Spec->catdir($self->rstdir, $name,
            sprintf("%04d-%02d-%02d", $ymd[0], $ymd[1], $ymd[2]));
        if ($is_downloaded) {
            preparedir($restore_dir);
            my $st = $self->_extract(
                arcdef => $arc,
                archive=> $archive_file,
                dirdst => $restore_dir,
            );
            if ($st) {
                push @header, ["Location of restored backup", $restore_dir];
                $self->log_info("Downloaded backup archive: %s", $archive_file);
                $self->log_info("Location of restored backup: %s", $restore_dir);
            } else {
                my $msg = sprintf("Extracting archive \"%s\" failed: %s", $archive_file, $self->error);
                $self->log_error($msg);
                push @errors, $msg, "";
                $ostat = 0;
            }
            $tbl->row(dtf(DATE_FORMAT), $step, $archive_name, $st ? 'PASS' : 'FAIL');
        } else {
            $tbl->row(dtf(DATE_FORMAT), $step, $archive_name, 'SKIP');
        }

lib/App/MBUtiny.pm  view on Meta::CPAN 

        );
        push @header, ["Summary status", $ostat ? 'PASS' : 'FAIL'];
        my @report;
        my $report_name = $ostat ? "restore report" : "restore error report";
        push @report, $self->_report_common(@header); # Common information
        push @report, $self->_report_summary($ostat ? "Restore is done" : "Errors occurred while performing restore"); # Summary table
        push @report, $tbl->draw() || ''; # Table
        push @report, $self->_report_errors(@errors); # List of occurred errors
        if ($TTY || $self->verbosemode) { # Draw to TTY
            printf("%s\n\n", "~" x 114);
            printf("The %s for %s backup host\n\n", $report_name, $name);
            print join("\n", @report, "");
        }


        # Finish restore
        $self->log_info("Finish restore process for \"%s\" backup host section", $name);

        # General status
        $status = 0 unless $ostat;
    }

    return $status;
});

__PACKAGE__->register_handler(
    handler     => "report",

lib/App/MBUtiny.pm  view on Meta::CPAN 

    # Hosts processing
    #
    my @collectors = ();
    foreach my $pair (sort {(keys(%$a))[0] cmp (keys(%$b))[0]} @hosts) {
        my $name = _getName($pair); # Backup name
        my $host = node($pair, $name); # Config section
        my $hostskip = (!@arguments || grep {lc($name) eq lc($_)} @arguments) ? 0 : 1;
        my $enabled = value($host, 'enable') ? 1 : 0;
        $tbl_hosts->row($name, ($hostskip || !$enabled) ? 'SKIP' : 'PASS');
        if ($hostskip || !$enabled) {
            $self->log_info("Skip reporting for \"%s\" backup host section", $name);
            next;
        }
        my $lcols = $self->_getCollector($host);
        push @collectors, @$lcols;
    }
    push @collectors, {} unless @collectors; # Default support
    #say(explain(\@collectors));

    #
    # Select collectors

lib/App/MBUtiny.pm  view on Meta::CPAN 

            push @errors, $collector->error, "";
            next;
        }
        next unless $colret;
        push @ok_collectors, $col
    }

    #
    # Collectors processing
    #
    my @backups;
    if (@ok_collectors) {
        my $collector = new App::MBUtiny::Collector(
                collector_config => [@ok_collectors],
                dbi => $self->getdbi, # For local storage only
            );
        @backups = $collector->report(); # start => 1561799600;
        if ($collector->error) {
            $self->log_error(sprintf("Collector error: %s", $collector->error));
            push @errors, $collector->error, "";
        }
    }

    #
    # Get report data about LAST backups on collector for each available host
    #
    my %requires;
    foreach (@req_hosts) {$requires{$_} = 0};
    foreach my $rec (@backups) {
        push @comments, sprintf("%s: %s", uv2null($rec->{file}), $rec->{comment}), "" if $rec->{comment};
        push @errors, uv2null($rec->{file}), $rec->{error}, "" if $rec->{error};
        my $nm = $rec->{name} || 'virtual';
        $tbl_report->row(
            sprintf("%s\n%s", $nm, uv2null($rec->{addr})),
            sprintf("%s\n%s (%s bytes)",
                    variant_stf(uv2null($rec->{file}), 32),
                    _fbytes(uv2zero($rec->{size})),
                    correct_number(uv2zero($rec->{size}))
                ),

lib/App/MBUtiny.pm  view on Meta::CPAN 

    my $report_name = $status ? "report" : "error report";
    push @report, $self->_report_common(@header); # Common information
    push @report, "Hosts:", $tbl_hosts->draw(); # Hosts table
    push @report, "Collectors:", $tbl_collectors->draw(); # Hosts table
    push @report, $self->_report_summary($status ? "All tests successful" : "Errors occurred while testing"); # Summary table
    push @report, $tbl_report->draw(); # Report table
    push @report, "Comments:", "", @comments, "" if @comments;
    push @report, $self->_report_errors(@errors); # List of occurred errors
    if ($TTY || $self->verbosemode) { # Draw to TTY
        printf("%s\n\n", "~" x 106);
        printf("The %s for all backup hosts on %s\n\n", $report_name, $hostname);
        print join("\n", @report, "");
    }


    #
    # SendMail (Send report)
    #
    if ($send_report) {
        unshift @report, $self->_report_title($report_name, "last backups");
        push @report, $self->_report_footer();
        my %ma = (); foreach my $k (keys %$sm) { $ma{"-".$k} = $sm->{$k} };
        $ma{"-subject"} = sprintf("%s %s (%s on %s)", PROJECTNAME, $report_name, "last backups", $hostname);
        $ma{"-message"} = join("\n", @report);

        # Send!
        my $sent = sendmail(%ma);
        if ($sent) { $self->debug(sprintf("Mail has been sent to: %s", $to)) }
        else { $self->error(sprintf("Mail was not sent to: %s", $to)) }
    }

    # Finish reporting
    $self->log_info("Finish reporting for \"%s\"", $hostname);

lib/App/MBUtiny.pm  view on Meta::CPAN 


    # Set period as maximum days to "back"
    my $period = 7 * $buweek > $buday ? 7 * $buweek : $buday;
    $period = 30 * $bumonth if 30 * $bumonth > $period;

    for (my $i=0; $i<$period; $i++) {
        my ( $y, $m, $d, $wd ) = (localtime( time - $i * 86400 ))[5,4,3,6];
        my $date = sprintf( "%04d%02d%02d", ($y+1900), ($m+1), $d );

        if (($i < $buday)
                || (($i < $buweek * 7) && $wd == 0) # do weekly backups on sunday
                || (($i < $bumonth * 30) && $d == 1)) # do monthly backups on 1-st day of month
        {
            $dates{ $date } = 1;
        } else {
            $dates{ $date } = 0;
        }

        if (($i < $buday) || (($wd == 0) && (($wcnt++) < $buweek)) || (($d == 1) && (($mcnt++) < $bumonth))) {
            $dates{$date} ++;
        }

lib/App/MBUtiny.pm  view on Meta::CPAN 

    return $exe_stt ? 0 : 1;
}

# Report internal methods
sub _report_title {
    my $self = shift;
    my $title = shift || "report";
    my $name = shift || "virtual";
    return (
        sprintf("Dear %s user,", PROJECTNAME),"",
        sprintf("This is a automatic-generated %s for %s backup\non %s, created by %s/%s",
            $title, $name, $hostname, __PACKAGE__, $VERSION),"",
        "Sections of this report:","",
        " * Common information",
        " * Summary",
        " * List of occurred errors","",
    );
}
sub _report_common {
    my $self = shift;
    my @hdr = @_;

lib/App/MBUtiny/Collector.pm  view on Meta::CPAN 

Scope: put

=item B<error>

Error message of the performed operation

Scope: put

=item B<file>

Name of backup file. Required argument

Scope: put, del

=item B<md5>, B<sha1>

MD5 and SHA1 checksums of backup file

Scope: put

=item B<name>

Name of backup. Required argument

Scope: put, del

=item B<operation>

Name of operation: del/put

Default: put

=item B<size>

Size of backup file

Scope: put

=item B<status>

Status of backup operation: 0 or 1

Default: 0 (operation failed)

Scope: put

=back

=head2 info

    my %info = $collector->info(

lib/App/MBUtiny/Collector.pm  view on Meta::CPAN 

        file => "foo-2019-06-25.tar.gz",
    );

Gets information about specified file name

Returns hash of values in "AS IN DATABASE DEFINED" format,
see L<App::MBUtiny::Collector::DBI>

=head2 report

    my @last_backup_files = $collector->report( start => 123456789 );

Returns list of last backups from all collectors as array of info-hashes.

See L</info> method

=head1 PUBLIC FUNCTIONS

=head2 int2type

    my $type = int2type(0); # internal

Returns name of specified type

lib/App/MBUtiny/Collector/Client.pm  view on Meta::CPAN 

        name => "foo",
        file => "foo-2019-06-25.tar.gz",
        size => 123456,
        md5  => "3a5fb8a1e0564eed5a6f5c4389ec5fa0",
        sha1 => "22d12324fa2256e275761b55d5c063b8d9fc3b95",
        status => 1,
        error => "",
        comment => "Test external fixup"
    ) or die $client->error;

Request for fixupping of backup on collector by name and others parameters.

The method returns status of operation: 0 - Error; 1 - Ok

=head2 check

    my $check = $client->check;

Performs the checking of MBUtiny collector server and returns structure in format:

    {

lib/App/MBUtiny/Collector/Client.pm  view on Meta::CPAN 


Request for getting list of files on collector by name.

The method returns array of info-structures.
See L<App::MBUtiny::Collector::DBI/list>

=head2 report

    my @list = $client->report(start => 123456789);

Request for getting report of backup on collector by name.
See L<App::MBUtiny::Collector::DBI/report>

=head2 request

    my $struct = $client->request();

Performs request to collector server over L<WWW::MLite::Client>

=head1 HISTORY

lib/App/MBUtiny/Collector/DBI.pm  view on Meta::CPAN 

=item B<file>

Backup filename

=item B<id>

Record ID. Autoincremented value!

=item B<md5>

MD5-checksum of backup file

=item B<name>

Name of mbutiny host

=item B<sha1>

SHA1-checksum of backup file

=item B<size>

 Size of backup file

=item B<status>

Backup status: 0=false, 1=true

Default: 0

=item B<time>

Time of record insert

lib/App/MBUtiny/Collector/DBI.pm  view on Meta::CPAN 

Returns list of files by specified the name

Record format of return result: see L</get>

=head2 report

    my @files = $dbi->report(
        start => 123456789
    );

Returns list of all last backup files, starting at the specified the "start" value

Record format of return result: see L</get>

=head1 SEE ALSO

L<App::MBUtiny>, L<CTK::DBI>

=head1 AUTHOR

Serż Minus (Sergey Lepenkov) L<http://www.serzik.com> E<lt>abalama@cpan.orgE<gt>

lib/App/MBUtiny/Collector/DBI.pm  view on Meta::CPAN 

        },
};

use constant COLLECTOR_DDL  => <<'DDL';
CREATE TABLE IF NOT EXISTS mbutiny (
    `id` INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
    `type` INTEGER DEFAULT 0,         -- 0=internal/1=external
    `time` NUMERIC DEFAULT 0,         -- time()
    `name` CHAR(255) DEFAULT NULL,    -- name of mbutiny host
    `addr` CHAR(45) DEFAULT NULL,     -- client ip addr
    `status` INTEGER DEFAULT 0,       -- backup status
    `file` CHAR(255) DEFAULT NULL,    -- backup filename
    `size` INTEGER DEFAULT 0,         -- size of backup file
    `md5` CHAR(32) DEFAULT NULL,      -- md5-checksum of backup file
    `sha1` CHAR(40) DEFAULT NULL,     -- sha1-checksum of backup file
    `error` TEXT DEFAULT NULL,        -- error message
    `comment` TEXT DEFAULT NULL       -- comment
)
DDL

use constant COLLECTOR_INSERT  => <<'DML';
INSERT INTO mbutiny
    (`type`, `time`, `name`, `addr`, `status`, `file`, `size`, `md5`, `sha1`, `error`, `comment`)
VALUES
    (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)

lib/App/MBUtiny/Collector/Server.pm  view on Meta::CPAN 

=cut

__PACKAGE__->register_method( # GET /mbutiny/report
    name    => "report",
    method  => "GET",
    path    => "$BASE_URL_PATH_PREFIX/report",
    deep    => 0,
    attrs   => {
            serialize   => 1,
        },
    description => "Get backup report by start time",
    code    => sub {
### CODE:
    my $self = shift;
    my $q = shift;
    my $dbi = $self->dbi;
    my $start = $q->param("start") || 0;
    unless (is_int($start)) {
        $self->error("The start attribute is not integer value type!");
        $self->{status} = 0;
        return HTTP_BAD_REQUEST;

lib/App/MBUtiny/Collector/Server.pm  view on Meta::CPAN 

        serialize => 1,
    },
    description => "Delete file",
    code    => sub {
### CODE:
    my $self = shift;
    my $q = shift;
    my $dbi = $self->dbi;
    my $name = _get_name_from_path($self->{request_uri} // "");
    unless ($name) {
        $self->error("Incorrect path! Check backup host name");
        $self->status(0);
        return HTTP_BAD_REQUEST;
    }
    my $file = $q->param("file");
    unless ($file) {
        $self->error("Incorrect file name for delete");
        $self->status(0);
        return HTTP_BAD_REQUEST;
    }
    my $type = $q->param("type") // 0;

lib/App/MBUtiny/ConfigSkel.pm  view on Meta::CPAN 

# Before using the collector-server, please check your DataBase and create the mbutiny table
#

#-- For SQLite DB
#CREATE TABLE IF NOT EXISTS mbutiny (
#  `id` INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
#  `type` INTEGER DEFAULT 0,         -- 0=internal/1=external
#  `time` NUMERIC DEFAULT 0,         -- unix time()
#  `name` CHAR(255) DEFAULT NULL,    -- name of mbutiny host
#  `addr` CHAR(45) DEFAULT NULL,     -- client ip addr
#  `status` INTEGER DEFAULT 0,       -- backup status
#  `file` CHAR(255) DEFAULT NULL,    -- backup filename
#  `size` INTEGER DEFAULT 0,         -- size of backup file
#  `md5` CHAR(32) DEFAULT NULL,      -- md5-checksum of backup file
#  `sha1` CHAR(40) DEFAULT NULL,     -- sha1-checksum of backup file
#  `error` TEXT DEFAULT NULL,        -- error message
#  `comment` TEXT DEFAULT NULL       -- comment
#);

#-- For MySQL DB
#CREATE TABLE `mbutiny` (
#  `id` int(11) NOT NULL auto_increment,
#  `type` int(2) default '0' COMMENT '0=internal/1=external',
#  `time` int(11) default '0' COMMENT 'unix time()',
#  `name` varchar(255) default NULL COMMENT 'name of mbutiny host',
#  `addr` varchar(45) default NULL COMMENT 'client ip addr',
#  `status` int(2) default '0' COMMENT 'backup status: 0=error/1=ok',
#  `file` varchar(255) default NULL COMMENT 'backup filename',
#  `size` int(11) default '0' COMMENT 'size of backup file',
#  `md5` varchar(32) default NULL COMMENT 'md5-checksum of backup file',
#  `sha1` varchar(40) default NULL COMMENT 'sha1-checksum of backup file',
#  `error` text default NULL COMMENT 'error message',
#  `comment` text default NULL COMMENT 'comment',
#  PRIMARY KEY  (`id`),
#  UNIQUE KEY `id` (`id`)
#) ENGINE=MyISAM DEFAULT CHARSET=utf8 COLLATE=utf8_bin;

# Section for connection with Your database.
# Recommended for use follow databases: SQLite, MySQL, Oracle or PostgreSQL
# Default: SQLite

lib/App/MBUtiny/ConfigSkel.pm  view on Meta::CPAN 

        Localdir    ./test/mbutimy-local1
        Localdir    ./test/mbutimy-local2
        Comment     Local storage said blah-blah-blah # Optional for collector
    </Local>

    #
    # SFTP storage (multiple blocks can be specified)
    #
    #<SFTP>
    #    FixUP       on
    #    URL         sftp://user@example.com:22/path/to/backup/dir1
    #    URL         sftp://user@example.com:22/path/to/backup/dir2
    #    Set         timeout  180
    #    Set         key_path  /path/to/private/file.key
    #    Comment     SFTP storage said blah-blah-blah # Optional for collector
    #</SFTP>

    #
    # FTP storage (multiple blocks can be specified)
    #
    #<FTP>
    #    FixUP       on
    #    URL         ftp://user:password@example.com:21/path/to/backup/dir1
    #    URL         ftp://user:password@example.com:21/path/to/backup/dir2
    #    Set         Passive 1
    #    Set         Debug 1
    #    Comment     FTP storage said blah-blah-blah # Optional for collector
    #</FTP>

    #
    # HTTP storage (multiple blocks can be specified)
    # See eg/server.cgi example file on CPAN web site
    #
    #<HTTP>
    #    FixUP       on
    #    URL         https://user:password@example.com/mbuserver/path/to/backup/dir1
    #    URL         https://user:password@example.com/mbuserver/to/backup/dir2
    #    Set         User-Agent TestServer/1.00
    #    Set         X-Test Foo Bar Baz
    #    Comment     HTTP storage said blah-blah-blah # Optional for collector
    #</HTTP>

    #
    # Command storage (multiple blocks can be specified)
    #
    #<Command>
    #    FixUP       on

lib/App/MBUtiny/Storage.pm  view on Meta::CPAN 


Version 1.00

=head1 SYNOPSIS

    use App::MBUtiny::Storage;

    my $storage = new App::MBUtiny::Storage(
        name => $name, # Backup name
        host => $host, # Host config section
        path => "/tmp/mbutiny/files", # Where is located backup archive
    );

    print $storage->error unless $storage->status;

=head1 DESCRIPTION

App::MBUtiny storage class

Storage - is a directory on disk, a remote FTP/SFTP/ HTTP server
or CLI process that simulates storage functional.

=head2 new

    my $storage = new App::MBUtiny::Storage(
        name => $name, # Backup name
        host => $host, # Host config section
        path => "/tmp/mbutiny/files", # Where is located backup archive
        fixup => sub {
            my $strg = shift; # Storage object
            my $oper = shift // 'noop'; # Operation name
            my @args = @_;

            return 1;
        },
        validate => sub {
            my $strg = shift; # storage object
            my $file = shift; # fetched file name

lib/App/MBUtiny/Storage.pm  view on Meta::CPAN 

Callback the "fixup" method. This method called automatically
when the put method performs

=head2 get

    $st = $storage->get(
        name => "foo-2019-06-25.tar.gz",
        file => "/full/path/to/foo-2019-06-25.tar.gz",
    );

Fetching backup file to specified file path from each storage until first successful result

Returns summary status. See L</summary>

=head2 init

Performs the "init" method in all storage subclasses and returns self object instance

For internal use only

=head2 list

    my @filelist = $storage->list;

Returns summary list of backup files from all available storages

=head2 put

    $st = $storage->put(
        name => "foo-2019-06-25.tar.gz",
        file => "/full/path/to/foo-2019-06-25.tar.gz",
        size => 123456,
    );

Sending backup file to each available storage

Returns summary status. See L</summary>

=head2 status

    my $new_status = $storage->status(0);

Sets new status value and returns it

    my $status = $storage->status;

lib/App/MBUtiny/Storage/Command.pm  view on Meta::CPAN 


=head2 del

Removes the specified file.
This is backend method of L<App::MBUtiny::Storage/del>

Variables: B<PATH>, B<HOST>, B<NAME>

=head2 get

Gets the backup file from storage and saves it to specified path.
This is backend method of L<App::MBUtiny::Storage/get>

Variables: B<PATH>, B<HOST>, B<NAME>, B<FILE>

=head2 init

The method performs initialization of storage.
This is backend method of L<App::MBUtiny::Storage/init>

=head2 list

Gets backup file list on storage.
This is backend method of L<App::MBUtiny::Storage/list>

Variables: B<PATH>, B<HOST>

=head2 cmd_storages

    my @list = $storage->cmd_storages;

Returns list of command storage nodes

=head2 put

Sends backup file to storage.
This is backend method of L<App::MBUtiny::Storage/put>

Variables: B<PATH>, B<HOST>, B<SIZE>, B<NAME>, B<FILE>

=head2 test

Storage testing.
This is backend method of L<App::MBUtiny::Storage/test>

Variables: B<PATH>, B<HOST>

=head1 VARIABLES

=over 4

=item B<FILE>

Full file path of backup file

For example:

    /tmp/mbutiny/files/foo-2019-06-20.tar.gz

=item B<HOST>

MBUtiny host name

For example:

    foo

=item B<NAME>

File name of backup file

For example:

    foo-2019-06-20.tar.gz

=item B<PATH>

Path to backup files

For example:

    /tmp/mbutiny/files

=item B<SIZE>

Size of backup file (bytes)

For example:

    32423

=back

=head1 HISTORY

See C<Changes> file

lib/App/MBUtiny/Storage/FTP.pm  view on Meta::CPAN 


=head1 VIRSION

Version 1.00

=head1 SYNOPSIS

  <Host "foo">
    <FTP>
        #FixUP   on
        URL     ftp://user:password@example.com:21/path/to/backup/dir1
        URL     ftp://user:password@example.com:21/path/to/backup/dir2
        Set     Passive 1
        Set     Debug 1
        Comment FTP storage said blah-blah-blah # Optional for collector
    </FTP>

    # . . .

  </Host>

=head1 DESCRIPTION

App::MBUtiny::Storage subclass for FTP storage support

=head2 del

Removes the specified file.
This is backend method of L<App::MBUtiny::Storage/del>

=head2 get

Gets the backup file from storage and saves it to specified path.
This is backend method of L<App::MBUtiny::Storage/get>

=head2 init

The method performs initialization of storage.
This is backend method of L<App::MBUtiny::Storage/init>

=head2 list

Gets backup file list on storage.
This is backend method of L<App::MBUtiny::Storage/list>

=head2 ftp_storages

    my @list = $storage->ftp_storages;

Returns list of FTP storage nodes

=head2 put

Sends backup file to storage.
This is backend method of L<App::MBUtiny::Storage/put>

=head2 test

Storage testing.
This is backend method of L<App::MBUtiny::Storage/test>

=head1 HISTORY

See C<Changes> file

lib/App/MBUtiny/Storage/HTTP.pm  view on Meta::CPAN 


App::MBUtiny::Storage subclass for HTTP storage support

=head2 del

Removes the specified file.
This is backend method of L<App::MBUtiny::Storage/del>

=head2 get

Gets the backup file from storage and saves it to specified path.
This is backend method of L<App::MBUtiny::Storage/get>

=head2 init

The method performs initialization of storage.
This is backend method of L<App::MBUtiny::Storage/init>

=head2 list

Gets backup file list on storage.
This is backend method of L<App::MBUtiny::Storage/list>

=head2 http_storages

    my @list = $storage->http_storages;

Returns list of HTTP storage nodes

=head2 put

Sends backup file to storage.
This is backend method of L<App::MBUtiny::Storage/put>

=head2 test

Storage testing.
This is backend method of L<App::MBUtiny::Storage/test>

=head1 HISTORY

See C<Changes> file

lib/App/MBUtiny/Storage/Local.pm  view on Meta::CPAN 


App::MBUtiny::Storage subclass for local storage support

=head2 del

Removes the specified file.
This is backend method of L<App::MBUtiny::Storage/del>

=head2 get

Gets the backup file from storage and saves it to specified path.
This is backend method of L<App::MBUtiny::Storage/get>

=head2 init

The method performs initialization of storage.
This is backend method of L<App::MBUtiny::Storage/init>

=head2 list

Gets backup file list on storage.
This is backend method of L<App::MBUtiny::Storage/list>

=head2 local_storages

    my @list = $storage->local_storages;

Returns list of local storage nodes

=head2 put

Sends backup file to storage.
This is backend method of L<App::MBUtiny::Storage/put>

=head2 test

Storage testing.
This is backend method of L<App::MBUtiny::Storage/test>

=head1 HISTORY

See C<Changes> file

lib/App/MBUtiny/Storage/SFTP.pm  view on Meta::CPAN 


=head1 VIRSION

Version 1.01

=head1 SYNOPSIS

  <Host "foo">
    <SFTP>
        FixUP       on
        URL         sftp://user@example.com:22/path/to/backup/dir1
        URL         sftp://user@example.com:22/path/to/backup/dir2
        Set         timeout  180
        Set         key_path  /path/to/private/file.key
        Comment     SFTP storage said blah-blah-blah # Optional for collector
    </SFTP>

    # . . .

  </Host>

=head1 DESCRIPTION

lib/App/MBUtiny/Storage/SFTP.pm  view on Meta::CPAN 

    ssh-keygen -t rsa
    ssh-copy-id -i /path/to/private/file.pub user@example.com

=head2 del

Removes the specified file.
This is backend method of L<App::MBUtiny::Storage/del>

=head2 get

Gets the backup file from storage and saves it to specified path.
This is backend method of L<App::MBUtiny::Storage/get>

=head2 init

The method performs initialization of storage.
This is backend method of L<App::MBUtiny::Storage/init>

=head2 list

Gets backup file list on storage.
This is backend method of L<App::MBUtiny::Storage/list>

=head2 sftp_storages

    my @list = $storage->sftp_storages;

Returns list of SFTP storage nodes

=head2 put

Sends backup file to storage.
This is backend method of L<App::MBUtiny::Storage/put>

=head2 test

Storage testing.
This is backend method of L<App::MBUtiny::Storage/test>

=head1 HISTORY

See C<Changes> file

share/manual_ru.pod  view on Meta::CPAN 

(резервные копии). Данных директив Localdir может быть несколько, тогда произойдёт копирование
бэкапа в различные локальные хранилища.

FixUP - Указывает выполнять фиксацию результата работы на коллекторе. может быть значение on или off

Comment - Комментарий для коллектора. Полезен для мониторинга и отладки. Настоятельно рекомендуем
не оставлять данную директиву пустой.

    <FTP>
        FixUP       on
        URL         ftp://user:password@example.com:21/path/to/backup/dir1
        URL         ftp://user:password@example.com:21/path/to/backup/dir2
        Set         Passive 1
        Set         Debug 1
        Comment     FTP storage said blah-blah-blah # Optional for collector
    </FTP>

Секция <FTP>...</FTP> определяет параметры удаленного FTP-хранилища. <FTP> секций может быть
несколько в рамках данного хоста. В этом случае выполнится работа над каждым из указанных
FTP-хранилищ. Директивы URL - определяют расположения хранилищ, когда как директивы FixUP и Comment
аналогичны по значению секции <Local>...</Local>. Директива Set устанавливает атрибуты соединения FTP

share/manual_ru.pod  view on Meta::CPAN 

коллектор регистрирует результат заливки. Хотя как правило и коллектор и хранилище HTTP находятся на
одном физическом сервере. URL - Адрес до хранилища (как и коллектора). FixUP - Указывает
выполнять фиксацию результата работы на коллекторе. Может аналогично FTP и Local секциям принимать
значения on или off. Следует учитывать, что параметры для коллектора фиксапа задаются отдельной
секцией самого коллектора, т.к. файлы могут хранится на одном коллекторе а данные о статусе на
другом. TimeOut - таймаут. По умолчанию 180 секунд. Следует задавать если размер бэкапа существенно
большой, и файл не успевает пройти за дефолтное значение параметра.

    <SFTP>
        FixUP       on
        URL         sftp://user@example.com:22/path/to/backup/dir1
        URL         sftp://user@example.com:22/path/to/backup/dir2
        Set         key_path  /path/to/private/file.key
        Comment     SFTP storage said blah-blah-blah # Optional for collector
    </SFTP>

Секция <SFTP>...</SFTP> определяет параметры удаленного SFTP-хранилища. Секций <SFTP> может быть
несколько. В этом случае выполнится работа над каждым SFTP-хранилищем. Значения директив секции
аналогичны директивам секции <FTP>

    <Command>
        FixUP       on



( run in 1.052 second using v1.01-cache-2.11-cpan-49f99fa48dc )