App-Yabsm
view release on metacpan or search on metacpan
#!/usr/bin/env perl
# Author: Nicholas Hubbard
# WWW: https://github.com/NicholasBHubbard/yabsm
# License: MIT
# The toplevel script of Yabsm.
use strict;
use warnings;
use v5.16.3;
use App::Yabsm;
App::Yabsm::main(@ARGV);
__END__
=pod
=head1 Name
Yabsm - yet another btrfs snapshot manager
=head1 What is Yabsm?
Yabsm is a btrfs snapshot and backup management system that provides the
following features:
=over 4
=item *
Takes read only snapshots and performs both remote and local incremental backups.
=item *
Separates snapshots and backups into 5minute, hourly, daily, weekly, and monthly
timeframe categories.
=item *
Provides a simple query language for locating snapshots and backups.
=back
=head1 Usage
Yabsm provides 3 commands: L<config|/"Configuration Querying">,
L<find|/"Finding Snapshots">, and L<daemon|/"The Yabsm Daemon">
usage: yabsm [--help] [--version] [<COMMAND> <ARGS>]
commands:
<config|c> [--help] [check ?file] [ssh-check <SSH_BACKUP>] [ssh-key]
[yabsm-user-home] [yabsm_dir] [subvols] [snaps] [ssh_backups]
[local_backups] [backups]
<find|f> [--help] [<SNAP|SSH_BACKUP|LOCAL_BACKUP> <QUERY>]
<daemon|d> [--help] [start] [stop] [restart] [status] [init]
=head1 Dependencies
=over 4
=item *
L<Perl|https://perldoc.perl.org/>
=item *
L<OpenSSH|https://www.openssh.com/>
=item *
L<Sudo|https://www.sudo.ws/>
=item *
L<btrfs-progs|https://github.com/kdave/btrfs-progs>
=back
=head1 Installation
In the near future Yabsm should be available in all major Linux distribution
repositories.
Until then Yabsm can be installed with L<cpanminus|https://metacpan.org/pod/App::cpanminus>.
# apt install cpanminus
# cpanm App::Yabsm
=head1 The Yabsm Daemon
usage: yabsm <daemon|d> [--help] [start] [stop] [restart] [status] [init]
Snapshots and backups are performed by the Yabsm daemon. The Yabsm daemon must be
started as root so it can initialize its runtime environment, which includes
creating a locked user named I<yabsm> (and a group named I<yabsm>) that the
daemon will run as. You can initialize the daemon's runtime environment without
actually starting the daemon by running C<yabsm daemon init>.
When the daemon starts, it reads the C</etc/yabsm.conf> file that specifies its
L<configuration|/"Configuration"> to determine when to schedule the snapshots and
backups and how to perform them. If the Yabsm daemon is already running and you
make a configuration change, you must run C<yabsm daemon restart> to apply the
changes.
=head3 Initialize Daemon Runtime Environment
You can use the command C<yabsm daemon init> to initialize the daemon's runtime
environment without actually starting the daemon. Running this command creates
the I<yabsm> user and group, gives the I<yabsm> user sudo access to btrfs-progs,
creates I<yabsms> SSH keys, and creates the directories needed for performing all
the I<snaps>, I<ssh_backups>, and I<local_backups> defined in C</etc/yabsm.conf>.
=head3 Daemon Logging
The Yabsm daemon logs all of its errors to C</var/log/yabsm>. If, for example,
you have an I<ssh_backup> that is not being performed, the first thing you should
do is check the logs.
=head1 Configuration
The Yabsm daemon is configured via the C</etc/yabsm.conf> file.
You can run the command C<yabsm config check> that will check your config and
output useful error messages if there are any problems.
=head3 Configuration Grammar
First things first: you must specify a C<yabsm_dir> that Yabsm will use for
storing snapshots and as a cache for holding data needed for performing snapshots
and backups. Most commonly this directory is set to C</.snapshots/yabsm>. Yabsm
will take this directory literally so you almost certainly want the path to end
in C</yabsm>. If this directory does not exist, the Yabsm daemon will create it
automatically when it starts.
There are 4 different configuration objects: I<subvols>, I<snaps>,
I<ssh_backups>, and I<local_backups>. The general form of each configuration
object is:
type name {
key=val
...
}
All configuration objects share a namespace, so you must make sure they all have
unique names. You can define as many configuration objects as you want.
=head4 Subvols
A subvol is the simplest configuration object and is used to give a name to a
L<btrfs subvolume|https://btrfs.wiki.kernel.org/index.php/SysadminGuide#Subvolumes>
on your system. A subvol definition accepts one field named C<mountpoint> which
takes a value that is a path to a subvolume.
subvol home_subvol {
mountpoint=/home
}
=head4 Timeframes
We need to understand timeframes before we can understand I<snaps>,
I<ssh_backups>, and I<local_backups>. There are 5 timeframes: 5minute, hourly,
daily, weekly, and monthly.
I<Snaps>, I<ssh_backups>, and I<local_backups> are performed in one or more
timeframes. For example, a I<ssh_backup> may be configured to take backups in the
I<hourly> and I<weekly> categories, which means that we want to backup every hour
and once a week.
The following table describes in plain English what each timeframe means:
5minute -> Every 5 minutes.
hourly -> At the beginning of every hour.
daily -> Every day at one or more times of the day.
weekly -> Once a week on a specific weekday at a specific time.
monthly -> Once a month on a specific day at a specific time.
To specify the timeframes you want, you set the value of C<timeframes> to a comma
separated list of timeframe values. For example, this is how you specify that you
want every timeframe:
timeframes=5minute,hourly,daily,weekly,monthly
Each timeframe you specify adds new required settings for the configuration
object. Here is a table that shows the timeframe settings:
5minute -> 5minute_keep
hourly -> hourly_keep
daily -> daily_keep, daily_times
weekly -> weekly_keep, weekly_time, weekly_day
monthly -> monthly_keep, monthly_time, monthly_day
Any C<*_keep> setting defines how many snapshots/backups you want to keep at one
time for the configuration object. A common configuration is to keep 48 hourly
snapshots so you can go back 2 days in one-hour increments.
The C<daily_times> setting for daily snapshots takes a comma separated list of
I<hh:mm> times. Yabsm will perform the snapshot/backup every day at all the given
times.
The weekly timeframe requires a C<weekly_day> setting that takes a day of week
string such as I<monday>, I<thursday>, or I<saturday> and a I<weekly_time>
setting that takes a I<hh:mm> time. The weekly snapshot/backup will be performed
on the given day of the week at the given time.
The monthly timeframe requires a C<monthly_day> setting that takes an integer
between 1-31 and a C<monthly_time> setting that takes a I<hh:mm> time. The
monthly snapshot/backup will be performed on the given day of the month at the
given time.
=head4 Snaps
A I<snap> represents a snapshot configuration for some I<subvol>. Here is an
example of a I<snap> that snapshots I<home_subvol> twice a day.
snap home_subvol_snap {
subvol=home_subvol
timeframes=daily
daily_keep=62 # two months
daily_times=13:40,23:59
}
=head4 SSH Backups
A I<ssh_backup> represents a backup configuration that sends snapshots over a
network via SSH. See this example of a I<ssh_backup> that backs up I<home_subvol>
to C<larry@192.168.1.73:/backups/yabsm/laptop_home> every night at midnight:
ssh_backup home_subvol_larry_server {
subvol=home_subvol
ssh_dest=larry@192.168.1.73
dir=/backups/yabsm/laptop_home
timeframes=daily
daily_keep=31
daily_times=23:59
}
The difficult part of configuring a I<ssh_backup> is making sure the SSH server
is properly configured. You can test that a I<ssh_backup> is able to be performed
by running C<yabsm config ssh-check E<lt>SSH_BACKUPE<gt>>. For a I<ssh_backup> to
be able to be performed the following conditions must be satisfied:
=over 4
=item *
The host's I<yabsm> user can sign into the SSH destination (I<ssh_dest>) using
key based authentication. To achieve this you must add the I<yabsm> users SSH key
(available via C<# yabsm ssh print-key>) to the server user's
C<$HOME/.ssh/authorized_keys> file.
=item *
The remote backup directory (I<dir>) is an existing directory residing on a btrfs
filesystem that the remote user has read and write permissions to.
=item *
The SSH user has root access to btrfs-progs via sudo. To do this you can add a
file containing a string like C<larry ALL=(root) NOPASSWD: /sbin/btrfs> to
a file in C</etc/sudoers.d/>.
=back
=head4 Local Backups
A I<local_backup> represents a backup configuration that sends snapshots to a
partition mounted on the host OS. This is useful for sending snapshots to an
external hard drive plugged into your computer.
Here is an example I<local_backup> that backs up C<home_subvol> every hour, and
once a week.
local_backup home_subvol_easystore {
subvol=home_subvol
dir=/mnt/easystore/backups/yabsm/home_subvol
timeframes=hourly,weekly
hourly_keep=48
weekly_keep=56
weekly_day=sunday
weekly_time=23:59
}
The backup directory (C<dir>) must be an existing directory residing on a btrfs
filesystem that the I<yabsm> user has read permission on.
=head1 Configuration Querying
Yabsm comes with a C<config> command that allows you to check and query your
configuration.
usage: yabsm <config|c> [--help] [check ?file] [ssh-check <SSH_BACKUP>]
[ssh-key] [yabsm-user-home] [yabsm_dir] [subvols]
[snaps] [ssh_backups] [local_backups] [backups]
The C<check ?file> subcommand checks that C<?file> is a valid Yabsm configuration
file and if not prints useful error messages. If the C<?file> argument is omitted
it defaults to C</etc/yabsm.conf>.
The C<ssh-check E<lt>SSH_BACKUPE<gt>> subcommand checks that C<E<lt>SSH_BACKUPE<gt>> can be
performed and if not prints useful error messages. See the section
L<SSH Backups|/"SSH Backups"> for an explanation on the configuration required
for performing an I<ssh_backup>.
The C<ssh-key> subcommand prints the I<yabsm> user's public SSH key.
All of the other subcommands query for information derived from your
C</etc/yabsm.conf>:
subvols -> The names of all subvols.
snaps -> The names of all snaps.
ssh_backups -> The names of all ssh_backups.
local_backups -> The names of all local_backups.
backups -> The names of all ssh_backups and local_backups.
yabsm_dir -> The directory used as the yabsm_dir.
yabsm_user_home -> The 'yabsm' users home directory.
=head1 Finding Snapshots
Now that we know how to configure Yabsm to take snapshots, we are going to want
to locate those snapshots. Yabsm comes with a command C<find> that allows you to
locate snapshots and backups using a simple query language. Here is the usage
string for the C<find> command.
usage: yabsm <find|f> [--help] [<SNAP|SSH_BACKUP|LOCAL_BACKUP> <QUERY>]
Here are a few examples:
$ yabsm find home_snap back-2-mins
$ yabsm f root_ssh_backup 'after b-2-m'
$ yabsm f home_local_backup 10:45
The first argument is the name of any I<snap>, I<ssh_backup>, or I<local_backup>.
Because these configuration entities share the same namespace there is no risk of
ambiguity.
The second argument is a snapshot location query. There are 7 types of queries:
all -> Every snapshot sorted newest to oldest
newest -> The most recent snapshot/backup.
oldest -> The oldest snapshot/backup.
after TIME -> All the snapshot/backups that are newer than TIME.
before TIME -> All the snapshot/backups that are older than TIME.
between TIME1 TIME2 -> All the snapshot/backups that were taken between TIME1 and TIME2.
TIME -> The snapshot/backup that was taken closest to TIME.
=head2 Time Abbreviations
In the list above the C<TIME> variables stand for a I<time abbreviation>.
There are two different kinds of I<time abbreviations>: I<relative times> and
I<immediate times>.
=head3 Relative Times
A relative time comes in the form C<back-AMOUNT-UNIT>, where C<back> can be
abbreviated to C<b>, C<AMOUNT> is a positive integer, and C<UNIT> is either
C<minutes>, C<hours>, or C<days>. Each C<UNIT> can be abbreviated:
minutes -> mins, m
hours -> hrs, h
days -> d
Here are some English descriptions of I<relative times>.
back-5-h -> 5 hours ago
b-10-m -> 10 minutes ago
b-24-days -> 24 days ago
=head3 Immediate Times
An I<immediate_time> is an abbreviation for a time/date denoted by C<yr_mon_day_hr:min>.
There are 7 I<immediate_time> forms, the following table gives an example of each
form:
yr_mon_day_hr:min -> 2020_5_13_23:59
yr_mon_day -> 2020_12_25
mon_day_hr:min -> 12_25_8:30
mon_day_hr -> 12_25_8
mon_day -> 12_25
hr:min -> 23:59
The I<immediate_time> abbreviation rules are simple. If the C<yr>, C<mon>, or
C<day> is omitted then the current year, month, or day is assumed. If the C<hr>
or C<min> is omitted then they are assumed to be 0. Therefore C<2020_12_25> is
always the same as C<2020_12_25_00:00>. If the current day is I<2020/12/25>, then
C<23:59> is the same as C<2020_12_25_23:59>.
=head1 Getting Support
Do not hesitate to open an issue at
L<https://github.com/NicholasBHubbard/Yabsm/issues>! To help get support, you may
want to include the output of the following commands in your issue:
yabsm config check
yabsm config ssh-check <SSH_BACKUP>
cat /var/log/yabsm
=head1 Packaging
An example systemd service and sysvinit script are provided in the C<init/>
directory.
An example config file is located at C<examples/yabsm.conf.example>. It
( run in 1.261 second using v1.01-cache-2.11-cpan-ceb78f64989 )