view release on metacpan or  search on metacpan

README.md  view on Meta::CPAN

It's possible that in the future `mtglacier` will support some other metadata things.

## Specification for some commands

### `sync`

Propagates current local filesystem state to Amazon Glacier server.

`sync` accepts one or several of the following mode options: `--new`, `--replace-modified`, `--delete-removed`

If none of three above mode options provided, `--new` is implied (basically for backward compatibility).

1. `--new`

	Uploads files, which exist in local filesystem (and have non-zero size), but not exist in Amazon Glacier (i.e. in Journal)

2. `--replace-modified`

	Uploads modified files (i.e. which exist in local filesystem and in Amazon Glacier). After file gets successfully uploaded,
	previous version of file is deleted. Logic of detection of modified files controlled by `--detect` option.

3. `--delete-removed`

	Deletes files, which exist in Amazon Glacier, but missing in local filesystem (or have zero size) , from Amazon Glacier.

4. `--detect`

	Controls how `--replace-modified` detect modified files. Possible values are: `treehash`, `mtime`, `mtime-or-treehash`, `mtime-and-treehash`,
	`always-positive`, `size-only`.
	Default value is `mtime-and-treehash`

	File is always considered modified if its *size changed* (but not zero)

	 1. `treehash` - calculates TreeHash checksum for file and compares with one in Journal. If checksum does not match - file is modified.

	 2. `mtime` - compares file last modification time in local filesystem and in journal, if it differs - file is modified.

	 3. `mtime-or-treehash` - compares file last modification time, if it differs - file is modified. If it matches - compares TreeHash.

	 4. `mtime-and-treehash` - compares file last modification time, if it differs - compares TreeHash. If modification time is not changed, file
	 treated as not-modified, treehash not checked.

	 5. `always-positive` - always treat files as modified, Modification time and TreeHash are ignored. Probably makes some sense only with `--filter` options.

	 6. `size-only` - treat files as modified only if size differs

	NOTE: default mode for detect is `mtime-and-treehash`, it's more performance wise (treehash checked only for files with modification time changed),
	but `mtime-or-treehash` and `treehash` are more safe in case you're not sure which programs change your files and how.

	NOTE: `mtime-or-treehash` is mnemonic for *File is modified if mtime differs OR treehash differs*
	`mtime-and-treehash`  is mnemonic for  *File is modified if mtime differs AND treehash differs*. Words
	*AND* and *OR* means here logical operators with [short-circuit evaluation](http://en.wikipedia.org/wiki/Short-circuit_evaluation)
	i.e. with `mtime-and-treehash` treehash never checked if mtime not differs. And with `mtime-or-treehash` treehash never checked if mtime differs.

NOTE: files with zero sizes are not supported by Amazon Glacier API, thus considered non-existing for consistency, for all `sync` modes.

NOTE: `sync` does not upload empty directories, there is no such thing as directory in Amazon Glacier.

NOTE: With `--dry-run` option TreeHash will not be calculated, instead *Will VERIFY treehash and upload...* message will be displayed.

NOTE: TreeHash calculation performed in parallel, so some of workers (defined with `--concurrency`) might be busy calculating treehash instead
of network IO.

### `restore`

Initiate Amazon Glacier RETRIEVE oparation for files listed in Journal, which don't *exist* on local filesystem and for
which RETRIEVE was not initiated during last 24 hours (that information obtained from *Journal* too - each retrieval logged
into journal together with timestamp)

### `restore-completed`

Donwloads files, listed in Journal, which don't *exist* on local filesystem, and which were previously
RETRIEVED (using `restore` command) and now available for download (i.e. in a ~4hours after retrieve).
Unlike `restore` command, list of retrieved files is requested from Amazon Glacier servers at runtime using API, not from
journal.

Data downloaded to unique temporary files (created in same directory as destination file). Temp files renamed to real files
only when download successfully finished. In case program terminated with error or after Ctrl-C, temp files with unfinished
downloads removed.

If `segment-size` specified (greater than 0) and particular file size in megabytes is larger than `segment-size`,
download for this file performed in multiple segments, i.e. using HTTP `Range:` header (each of size `segment-size` MiB, except last,
which can be smaller). Segments are downloaded in parallel (and different segments from different files can
be downloaded at same time).

Only values that are power of two supported for `segment-size` now.

Currenly if download breaks due to network problem, no resumption is performed, download of file or of current segment
started from beginning.

In case multi-segment downloads, TreeHash reported by Amazon Glacier for each segment is compared with actual TreeHash, calculated for segment at runtime.
In case of mismatch error is thrown and process stopped. Final TreeHash for whole file not checked yet.

In case full-file downloads, TreeHash reported by Amazon Glacier for whole file is compared with one calculated runtime and with one found in Journal file,
in case of mismatch, error is thrown and process stopped.

Unlike `partsize` option, `segment-size` does not allocate buffers in memory of the size specified, so you can use large `segment-size`.

### `upload-file`

Uploads a single file into Amazon Glacier. File will be tracked with Journal (just like when using `sync` command).

There are several possible combinations of options for `upload-file`:

1. **--filename** and **--dir**

	_Uploads what_: a file, pointed by `filename`.

	_Filename in Journal and Amazon Glacier metadata_: A relative path from `dir` to `filename`

		./mtglacier upload-file --config glacier.cfg --vault myvault --journal journal.log --dir /data/backup --filename /data/backup/dir1/myfile

	(this will upload content of `/data/backup/dir1/myfile` to Amazon Glacier and use `dir1/myfile` as filename for Journal )

		./mtglacier upload-file --config glacier.cfg --vault myvault --journal journal.log --dir data/backup --filename data/backup/dir1/myfile

	(Let's assume current directory is `/home`. Then this will upload content of `/home/data/backup/dir1/myfile` to Amazon Glacier and use `dir1/myfile` as filename for Journal)

	NOTE: file `filename` should be inside directory `dir`

	NOTE: both `-filename` and `--dir` resolved to full paths, before determining relative path from `--dir` to `--filename`. Thus yo'll get an error