App-MtAws

 view release on metacpan or  search on metacpan

README.md  view on Meta::CPAN


		--max-number-of-files 100

5. `key/secret/region/vault/protocol` - you can override any option from config

6. `dry-run` (with `sync`, `purge-vault`, `restore`, `restore-completed ` and even `check-local-hash` commands) - do not perform actual work, print what will happen instead.

		--dry-run

6. `leaf-optimization` (only `sync` command). `0` - disable. `1` - enable (default).
Similar to [find][find] (coreutils tools) `-noleaf` option and [File::Find][File::Find] `$dont_use_nlink` option.
When disabled number of hardlinks to directory is ignored during file tree traversal. This slow down file search, but more
compatible with (some) CIFS/CD-ROM filesystems.
For more information see [find][find] and [File::Find][File::Find] manuals.

7. `token` (all commands which connect Amazon Glacier API) - a STS/IAM security token, described in [Amazon STS/IAM Using Temporary Security Credentials to Access AWS]

[Amazon STS/IAM Using Temporary Security Credentials to Access AWS]:http://docs.aws.amazon.com/STS/latest/UsingSTS/UsingTokens.html
[find]:http://unixhelp.ed.ac.uk/CGI/man-cgi?find
[File::Find]:http://search.cpan.org/perldoc?File%3A%3AFind

8. `timeout` (all commands which connect Amazon Glacier API)

	Sets the timeout value in seconds, default value is 180 seconds. Request to Amazon Glacier is retried, if if no activity
	on the connection to the server is observed for `timeout` seconds. This means that the time it takes for the complete whole
	request might be longer.

9. `follow` (only `sync` command)

	Follow symbolic links during directory traversal. This option hits performance and increases memory usage. Similar to `find -L`

## Configuring Character Encodings

Autodetection of locale/encodings not implemented yet, but currently there is ability to tune encodings manually.

Below 4 options, that can be used in config file and in command line.

1. `terminal-encoding` - Encoding of your terminal (STDOUT/STDERR for system messages)

2. `filenames-encoding` - Encoding of filenames in filesystem.

	Under most *nix filesystems filenames stored as byte sequences, not characters. So in theory application is responsible for managing encodings.

3. `config-encoding` - Encoding of your config file (`glacier.cfg` in examples above)

4. `journal-encoding` - Encoding to be used for Journal file (when reading and writing journal specified with `--journal` and `--new-journal` options)


Default value for all options is 'UTF-8'. Under Linux and Mac OS X you usually don't need to change encodings.
Under *BSD systems often single-byte encodings are used. Most likely yo'll need to change `terminal-encoding` and `filenames-encoding`. Optionaly you can also
change `config-encoding` and `journal-encoding`.

Notes:

* Before switching `config-encoding` and `journal-encoding` you are responsible for transcoding file content of config and journal files manually.

* You are responsible for encoding compatibility. For example Don't try to work with UTF-8 journal with non-Cyrilic characters and KOI8-R (Cyrilic) filesystem.

* Don't try to use UTF-16 for *nix filesystem. It's not ASCII compatible and contains \x00 bytes, which can't be stored in filesystem.

* Don't use `UTF8` - it does not validate data, use `UTF-8` (one with a dash) instead.

* To get list of encodings installed with your Perl run:

		perl -MEncode -e 'print join qq{\n}, Encode->encodings(q{:all})'

* Config file name (specified with `--config`) can be in any encoding (it's used as is) Of course it will work only if your terminal encoding match your
filesystem encoding or if your config file name consists of ASCII-7bit characters only.

* Additional information about encoding support in Perl programming language: [CPAN module Encode::Supported](http://search.cpan.org/perldoc?Encode%3A%3ASupported)

* Amazon Glacier metadata (on Amazon servers) is always stored in UTF-8. No way to override it. You can use Journal in any encoding with same
metdata without problems and you can dump metadata to journals with different encodings (using `download-inventory` command)

* See also [convmv tool](http://www.j3e.de/linux/convmv/man/)

## Limitations

* Only support filenames, which consist of octets, that can be mapped to a valid character sequence in desired encoding (i.e. filename
which are made of random bytes/garbage is not supported. usually it's not a problem).

* Filenames with CR (Carriage return, code 0x0D) LF (Line feed, code 0x0A) and TAB (0x09) are not supported (usually not a problem too).

* Length of relative filenames. Currently limit is about 700 ASCII characters or 350 2-byte UTF-8 character (.. or 230 3-byte characters).

* File modification time should be in range from year 1000 to year 9999.

(NOTE: if above requirements are not met, error will be thrown)

* If you uploaded files with file modifications dates past Y2038 on system which supports it, and then restored on system
which does not (like Linux 32bit), resulting file timestamp (of course) wrong and also
unpredictible (undefined behaviour). The only thing is guaranteed that if you restore journal from Amazon servers on affected (i.e. 32bit)
machine - journal will contain correct timestamp (same as on 64bit).

* Memory usage (for 'sync') formula is ~ min(NUMBER_OF_FILES_TO_SYNC, max-number-of-files) + partsize*concurrency

* With high partsize*concurrency there is a risk of getting network timeouts HTTP 408/500.


## Test/Play with it

1. create empty dir MYDIR
2. Set vault name inside `cycletest.sh`
3. Run

		./cycletest.sh init MYDIR
		./cycletest.sh retrieve MYDIR
		./cycletest.sh restore MYDIR

* OR

		./cycletest.sh init MYDIR
		./cycletest.sh purge MYDIR

## See also

* Amazon Glacier Perl library on CPAN - see [Net::Amazon::Glacier][Amazon Glacier API CPAN module - Net::Amazon::Glacier] by *Tim Nordenfur*
* Amazon Glacier TreeHash CPAN module [Net::Amazon::TreeHash][Amazon Glacier TreeHash CPAN module - Net::Amazon::TreeHash] (copied from `mtglacier` code)
* [Amazon Glacier development forum][Amazon Glacier development forum]

[Amazon Glacier API CPAN module - Net::Amazon::Glacier]:https://metacpan.org/module/Net::Amazon::Glacier



( run in 0.893 second using v1.01-cache-2.11-cpan-437f7b0c052 )