App-Fetchware
view release on metacpan or search on metacpan
lib/App/Fetchware.pm view on Meta::CPAN
you would like to edit the Fetchwarefile, fetchware has created for you in an
editor. If you say yes, fetchware will open a editor for you, but if you say
no, fetchware will skip the custom editing. Next, fetchware will create a test
Fetchwarefile for you, and ask you if you would like to test it by trying to
install it now. If you say yes, fetchware will install it, and if you say no,
then fetchware will print the location of the Fetchwarefile it created for
you to later use to install your application.
EOM
opening_message($opening_message);
# Ask user for name of program unless the user provided one at command
# line such as fetchware new <programname>.
$program_name = fetchwarefile_name(program => $program_name);
vmsg "Determined name of your program to be [$program_name]";
$fetchwarefile->config_options(program => $program_name);
vmsg "Appended program [$program_name] configuration option to Fetchwarefile";
my $lookup_url = get_lookup_url($term);
vmsg "Asked user for lookup_url [$lookup_url] from user.";
$fetchwarefile->config_options(lookup_url => $lookup_url);
vmsg "Appended lookup_url [$lookup_url] configuration option to Fetchwarefile";
vmsg "Downloaded lookup_url [$lookup_url]";
my $filename_listing = download_lookup_url($term, $lookup_url);
vmsg "Downloaded lookup_url's directory listing";
vmsg Dumper($filename_listing);
my $mirrors_hashref = get_mirrors($term, $filename_listing);
vmsg "Added mirrors to your Fetchwarefile.";
vmsg Dumper($mirrors_hashref);
my $verify_hashref = get_verification($term, $filename_listing, $lookup_url);
vmsg "Added verification settings to Fetchwarefile.";
vmsg Dumper($verify_hashref);
my $filter_hashref = get_filter_option($term, $filename_listing);
vmsg "Added [$filter_hashref->{filter}] filter setting to Fetchwarefile.";
$fetchwarefile->config_options(
%$mirrors_hashref,
%$verify_hashref,
%$filter_hashref
);
###BUGALERT### Ask to parrallelize make with make_options???
###BUGALERT### Verify prefix is writable by current user, who will
#presumably be the user who will install the package now and later.
###BUGALERT### Ask user for a prefix if their running nonroot???
vmsg 'Prompting for other options that may be needed.';
my $other_options_hashref = prompt_for_other_options($term,
temp_dir => {
prompt => <<EOP,
What temp_dir configuration option would you like?
EOP
print_me => <<EOP
temp_dir is the directory where fetchware creates a temporary directory that
stores all of the temporary files it creates while it is building your software.
The default directory is /tmp on Unix systems and C:\\temp on Windows systems.
EOP
},
user => {
prompt => <<EOP,
What user configuration option would you like?
EOP
print_me => <<EOP
user specifies what user fetchware will drop priveleges to on Unix systems
capable of doing so. This allows fetchware to download files from the internet
with user priveleges, and not do anything as the administrative root user until
after the downloaded software package has been verified as exactly the same as
the author of the package intended it to be. If you use this option, the only
thing that is run as root is 'make install' or whatever this package's
install_commands configuratio option is.
EOP
},
prefix => {
prompt => <<EOP,
What prefix configuration option would you like?
EOP
print_me => <<EOP
prefix specifies the base path that will be used to install this software. The
default is /usr/local, which is acceptable for most unix users. Please note that
this difective only works for software packages that use GNU AutoTools, software
that uses ./configure --prefix=<your prefix will go here> to change the prefix.
EOP
},
configure_options => {
prompt => <<EOP,
What configure_options configuration option would you like?
EOP
print_me => <<EOP
configure_options specifies what options fetchware should add when it configures
this software package for you. A list of possible options can be obtained by
running unarchiving the software package that corresponds to this Fetchwarefile,
and running the command './configure --help'. These options vary from software
package to software package. Please note that this option only works for GNU
AutoTools based software distributions, ones that use ./configure to configure
the software.
EOP
},
make_options => {
prompt => <<EOP,
What make_options configuration option would you like?
EOP
print_me => <<EOP
make_options specifies what options fetchware will pass to make when make is run
to compile, perhaps test, and install your software package. They are simpley
added after make is called. An example is '-j 4', which will cause make to
execute 4 jobs simultaneously. A reasonable rule of thumb is to set make's -j
argument to two times as many cpu cores your computer has as compiling programs
is sometimes IO bound instead of CPU bound, so you can get away with running
more jobs then you have cores.
EOP
},
###BUGALERT### Create a config sub called build_system that takes args like
#AutoTools, cmake, MakeMaker, Module::Build, and so on that will use the default
#build commands of whatever system this option specifies.
build_commands => {
prompt => <<EOP,
What build_commands configuration option would you like?
EOP
print_me => <<EOP
build_commands specifies what commands fetchware will run to compile your
software package. Fetchware's default is simply 'make', which is good for most
programs. If you're software package uses something other than fetchware's
default of GNU AutoTools, then you may need to change this configuration option
to specify what you would like instead. Specify multiple build commands in
single quotes with a comma between them:
'./configure', 'make'
EOP
},
install_commands => {
prompt => <<EOP,
What install_commands configuration option would you like?
EOP
print_me => <<EOP
install_commands specifies what commands fetchware will run to install your
software package. Fetchware's default is simply 'make install', which is good
for most programs. If you're software package uses something other than
fetchware's default of GNU AutoTools, then you may need to change this
configuration option to specify what you would like instead. Specify multiple
build commands in single quotes with a comma between them:
lib/App/Fetchware.pm view on Meta::CPAN
installed programs that fetchware installed.
If you would like C<fetchware upgrade-all> to be run every night automatically
by cron, then just create a file say fetchware with the contents below in it,
and add it to /etc/cron.daily.
#!/bin/sh
# Update all already installed fetchware packages.
fetchware upgrade-all
And if you don't want to run it system wide as root, you can add it to your user
crontab by pasting the snippet below in to your crontab by executing C<crontab -e>.
# Check for updates using fetchware every night at 2:30AM.
# Minute Hour Day of Month Month Day of Week Command
# (0-59) (0-23) (1-31) (1-12 or Jan-Dec) (0-6 or Sun-Sat)
30 2 * * * fetchware upgrade-all
=back
=head1 App::Fetchware'S FETCHWAREFILE CONFIGURATION OPTIONS
App::Fetchware has many configuration options. Most were briefly described in
the section L<MANUALLY CREATING A App::Fetchware FETCHWAREFILE>. All of them are
detailed below.
=head2 program 'Program Name';
C<program> simply gives this Fetchwarefile a name. It is availabe to fetchware
after parsing your Fetchwarefile, and is used to name your Fetchwarefile when
using C<fetchware new>. It is required just like C<lookup_url>, C<mirror>,
perhaps C<filter>, and some method to verify downloads are.
=head2 filter 'perl regex here';
Specifies a Perl regular expression that fetchware uses when it determines what
the latest version of a program is. It simply compares each file in the
directory listing specified in your C<lookup_url> to this regular expression,
and only matching files are allowed to pass through to the next part of
fetchware that looks for source code archives to download.
See L<perlretut> for details on how to use and create Perl regular expressions;
however, actual regex know how is not really needed just paste verbatim text
between the single quotes C<'>. For example, C<filter 'httpd-2.2';> will cause
fetchware to only download Apache 2.2 instead of the version for Windows or
whatever is in the weird httpd-deps-* package.
=head2 temp_dir '/tmp';
C<temp_dir> tells fetchware where to store fetchware's temporary working
directory that it uses to download, verify, unarchive, build, and install your
software. By default it uses your system temp directory, which is whatever
directory L<File::Temp's> tempdir() decides to use, which is whatever
L<File::Spec>'s tmpdir() decides to use.
=head2 fetchware_db_path '~/.fetchwaredb';
C<fetchware_db_path> tells fetchware to use a different directory other
than its default directory to store the installed fetchware package for the
particular fetchware package that this option is specified in your
Fetchwarefile. Fetchware's default is C</var/log/fetchware> on Unix when run as
root, and something like C</home/[username]/.local/share/Perl/dist/fetchware/>
when run nonroot.
This option is B<not> recommended unless you only want to change it for just one
fetchware package, because fetchware also consults the
C<FETCHWARE_DATABASE_PATH> environment variable that you should set in your
shell startup files if you want to change this globally for all of your
fetchware packages. For sh/bash like shells use:
export FETCHWARE_DATABASE_PATH='/your/path/here'
=head2 user 'nobody';
Tells fetchware what user it should drop privileges to. The default is
C<nobody>, but you can specify a different username with this configuration
option if you would like to.
Dropping privileges allows fetchware to avoid downloading files and executing
anything inside the downloaded archive as root. Except of course the commands
needed to install the software, which will still need root to able to write
to system directories. This improves security, because the downloaded software
won't have sytem privileges until after it is verified, providing that what you
downloaded is exactly what the author uploaded.
Note this only works for unix like systems, and is not used on Windows and
other non-unix systems.
Also note, that if you are running fetchware on Unix even if you do not specify
the C<user> configuration option to configure what user you will drop privileges
to, fetchware will still drop privileges using the ubiquitous C<nobody> user.
If you do B<not> want to drop privileges, then you must use the C<stay_root>
configuration option as described below.
=head2 stay_root 'On';
Tells fetchware to B<not> drop privileges. Dropping privileges when run as root
is fetchware's default behavior. It improves security, and allows fetchware to
avoid exposing the root account by downloading files as root.
Do B<not> use this feature unless you are absolutely sure you need it.
=over
=item SECURITY NOTICE
stay_root, when turned on, causes fetchware to not drop privileges when
fetchware looks up, downloads, verifies, and builds your program. Instead,
fetchware will stay root through the entire build cycle, which needlessly
exposes the root account when downloading files from the internet. These files
may come from trusted mirrors, but mirrors can, and do get cracked:
L<http://www.itworld.com/security/322169/piwik-software-installer-rigged-back-door-following-website-compromise?page=0,0>
L<http://www.networkworld.com/news/2012/092612-compromised-sourceforge-mirror-distributes-backdoored-262815.html>
L<http://www.csoonline.com/article/685037/wordpress-warns-server-admins-of-trojans>
L<http://www.computerworld.com/s/article/9233822/Hackers_break_into_two_FreeBSD_Project_servers_using_stolen_SSH_keys>
=back
=head2 lookup_url 'ftp://somedomain.com/some/path
This configuration option specifies a url of a FTP or HTTP or local (file://)
directory listing that fetchware can download, and use to determine what actual
file to download and perhaps also what version of that program to download if
more than one version is available as some mirrors delete old versions and only
keep the latest one.
This url is used for:
=over
=item 1. To determine what the actual download url is for the latest version of this program
=item 2. As the base url to also download a cryptographic signature (ends in .asc) or a SHA-1 or MD5 signature to verify the contents match what the SHA-1 or MD5 checksum is.
=back
You can use the C<mirror> configuration option to specify additional mirrors.
However, those mirrors will only be used to download the large software
archives. Only the lookup_url will be used to download directory listings to
check for new versions, and to download digital signatures or checksums to
verify downloads.
=head2 lookup_method 'timestamp';
Fetchware has two algorithms it uses to determine what version of your program
lib/App/Fetchware.pm view on Meta::CPAN
Parses $filename_listing to determine what type of verification is available.
Prefering gpg, but falling back on sha1, and then md5 if gpg is not available.
If the type is gpg, then get_verification() will ask the user to specify a
C<gpg_keys_url>, which is required for gpg, because fetchware needs to be able
to import the needed keys to be able to use those keys to verify package
downloads. If this URL is not provided by the author, then get_verification()
will ask the user if they would like to import the author's key into their own
gpg public keyring. If they would, then get_verification() will use the
C<user_keyring> C<'On'> option to use the user's public keyring instead of
fetchware's own keyring. And if the user does not want to use their own gpg
public keyring, then get_verification will fall back to sha1 or md5 setting
C<verify_method> to sha1 or md5 as needed.
Also, adds a gpg_keys_url option if a C<KEYS> file is found in
$filename_listing.
If no verification methods are available, fetchware will print a big nasty
warning message, and offer to use C<verify_failure_ok> to make such a failure
cause fetchware to continue installing your software.
Returns a hashref of options for the user's Fetchwarefile. You're responsible
for calling append_options_to_fetchwarefile() to add them to the user's
Fetchwarefile, or perhaps the caller could analyze them in some way, before
adding them if needed. The keys are the names of the configuration options, and
the values are their values.
=head3 get_filter_option()
$filter_hashref = get_filter_option($term, $filename_listing);
# $filter_hashref = (
# filter => 'user specfied filter option',
# );
Analyzes $filename_listing and asks the user whatever questions are needed by
fetchware to determine if a C<filter> configuration option is needed, and if it
is what it should be. C<filter> is simply a perl regex that the list of files
that fetchware downloads is checked against, and only files that match this
regex will fetchware consider to be the latest version of the software package
that you want to install. The C<filter> option is needed, because some mirrors
will have multiple software packages in the same directory or multitple
different versions of one piece of software in the same directory. An example
would be Apache, which has Apache versions 2.0, 2.2, and 2.4 all in the same
directory. The C<filter> option is how you differentiate between them.
If a filter was provided by the user than it is returned as a hashref with
C<filter> as the key for use with append_options_to_fetchwarefile(), or for
further analysis by extension authors.
=head3 prompt_for_other_options()
prompt_for_other_options($term,
temp_dir => {
prompt => <<EOP,
What temp_dir configuration option would you like?
EOP
print_me => <<EOP
temp_dir is the directory where fetchware creates a temporary directory that
stores all of the temporary files it creates while it is building your software.
The default directory is /tmp on Unix systems and C:\\temp on Windows systems.
EOP
},
...
);
Accepts a Term::Readline/Term::UI object as an argument to use to ask the user
questions, and a gigantic hash of hashes in list form. The hash of hashes,
%option_description, argument incluedes the C<prompt> and C<print_me> options
that are then passed through to Term::UI to ask the user what argument they want
for each specified option in the %option_description hash.
The user's answers are tallied up an returned as a hash reference.
=head3 edit_manually()
$fetchwarefile = edit_manually($term, $fetchwarefile);
edit_manually() asks the user if they would like to edit the specified
$fetchwarefile manually. If the user answers no, then nothing is done. But if
the user answers yes, then fetchware will open their favorit editor either using
the C<$ENV{EDITOR}> environment variable, or fetchware will ask the user what
editor they would like to use. Then this editor, and a temporary fetchwarefile
are opened, and the user can edit their Fetchwarefile as they please. If they
are not satisfied with their edits, and wan to undo them, they can delete the
entire file, and write a size 0 file, which will cause fetchware to ignore the
file they edited. If the write a file with a size greater than 0, then the file
the user wrote, will be used as their Fetchwarefile.
=head2 new_install()
my $fetchware_package_path = new_install($program_name, $fetchwarefile);
=over
=item Configuration subroutines used:
=over
=item All of them, because it calls bin/fetchware's cmd_install(),
which in turn calls all of the API subroutines that fetchware's install command
does.
=back
=back
Exists separate from new(), because new() drops privileges like most other
fetchware commands do. But the new command includes the ability to ask the user
if they want to install the associated program from their newly created
Fetchwarefile, which requires root privileges. Therefore, we must also have a
API subroutine that runs in the root privileged parent to that the install
commands will run with proper permissions when fetchware is run as root.
=over
=item drop_privs() NOTES
This section notes whatever problems you might come accross implementing and
debugging your Fetchware extension due to fetchware's drop_privs mechanism.
lib/App/Fetchware.pm view on Meta::CPAN
Yes and no. I intend to support Windows, but right now I'm running Linux, and my
Windows virtual machine is broken, so I can't easily test it on Windows. The
codebase makes heavy use of File::Spec and Path::Class, so all of its file
operations should work on Windows.
I currently have not tested fetchware on Windows. There are probably some test
failures on Windows, but Windows support should be just a few patches away.
So the real answer is no, but I intend to support Windows in a future release.
=head2 How do I configure a HTTP proxy for use with Fetchware?
Fetchware uses Perl's L<HTTP::Tiny> module to download files over HTTP. So, all
you need to do is configure HTTP::Tiny for use with proxies. This is done with
environment variables that can easily be set from your shell.
Set C<http_proxy> in the format C<http://host:port>. You can do this permanantly
for this session with:
export http_proxy='http://example.com:8080'
Or once just for this invocation of fetchware:
http_proxy='http://example.com:8080 fetchware new
See your OS's shell for more details regarding using and exporting environment
variables.
=head2 How do I get Fetchware to I<not> delete its temporary directory, so I can troubleshoot a fetchware failure.
Use fetchware's C<-v> (verbose) and C<--keep-temp> command-line options. C<-v>
will turn on extra verbose messages that fetchware prints out detailing what its
doing and why. These messages will also list where fetchware's temporary
directory is. C<--keep-temp> tell fetchware to keep its temporary files and
directories and I<not> delete them. This way you can examine them to help
troubleshoot the problem, and see what's going on.
C<--keep-temp> should not be needed when the installation or upgrade of a
fetchware package or Fetchwarefile fails, because the temp file deletion code
will not run after an error occurs before fetchware exits failure.
Use C<fetchware clean> to delete this and any other left over fetchware
temporary directories or files.
=head1 SECURITY
App::Fetchware is inherently insecure, because its Fetchwarefile's are
executable Perl code that actually is executed by Perl. These Fetchwarefiles are
capable of doing everything someone can program Perl to do. This is why
App::Fetchware will refuse to execute any Fetchwarefile's that are writable by
anyone other than the owner who is executing them. It will also exit with a
fatal error if you try to use a Fetchwarefile that is not the same user as the
one who is running fetchware. These saftey measures help prevent fetchware being
abused to get unauthorized code executed on your computer.
App::Fetchware also features the C<user> configuration option that tells
fetchware what user you want fetchware to drop privileges to when it does
everything but install (install()) and clean up (end()). The configuration
option does B<not> tell fetchware to turn on the drop privelege code; that code
is B<always> on, but just uses the fairly ubuiquitous C<nobody> user by default.
This feature requires the OS to be some version of Unix, because Windows and
other OSes do not support the same fork()ing method of limiting what processes
can do. On non-Unix OSes, fetchware won't fork() or try to use some other way of
dropping privileges. It only does it on Unix. If you use some version of Unix,
and do not want fetchware to drop privileges, then specify the C<stay_root>
configuration option.
=head1 ERRORS
App::Fetchware does not return any error codes; instead, all errors are die()'d
if it's App::Fetchware::Config's error, or croak()'d if its the caller's fault.
These exceptions are short paragraphs that give full details about the error
instead of the vague one liner that perl's own errors give.
=head1 BUGS
The official bug tracker for fetchware is its
L<github issues page.|https://github.com/deeelwy/Fetchware/issues>
=head1 AUTHOR
David Yingling <deeelwy@gmail.com>
=head1 COPYRIGHT AND LICENSE
This software is copyright (c) 2016 by David Yingling.
This is free software; you can redistribute it and/or modify it under
the same terms as the Perl 5 programming language system itself.
=cut
( run in 2.031 seconds using v1.01-cache-2.11-cpan-64ef6c95b5d )