Firefox-Marionette
view release on metacpan or search on metacpan
my $firefox = Firefox::Marionette->new()->delete_header( 'Accept' )->add_header( 'Accept' => 'text/perl' )->go('https://metacpan.org/');
will only send out an Accept
<https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Accept>
header that looks like Accept: text/perl.
use Firefox::Marionette();
my $firefox = Firefox::Marionette->new()->add_header( 'Accept' => 'text/perl' )->go('https://metacpan.org/');
by itself, will send out an Accept
<https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Accept>
header that may resemble Accept:
text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8,
text/perl. This method returns itself to aid in chaining methods.
add_login
accepts a hash of the following keys;
* host - The scheme + hostname of the page where the login applies,
for example 'https://www.example.org'.
* user - The username for the login.
* password - The password for the login.
* origin - The scheme + hostname that the form-based login was
submitted to
<https://developer.mozilla.org/en-US/docs/Web/HTML/Element/form#attr-action>.
Forms with no action attribute default to submitting to the URL of
the page containing the login form, so that is stored here. This
field should be omitted (it will be set to undef) for http auth type
authentications and "" means to match against any form action.
* realm - The HTTP Realm for which the login was requested. When an
HTTP server sends a 401 result, the WWW-Authenticate header includes
a realm. See RFC 2617
<https://datatracker.ietf.org/doc/html/rfc2617>. If the realm is not
specified, or it was blank, the hostname is used instead. For HTML
form logins, this field should not be specified.
* user_field - The name attribute for the username input in a form.
Non-form logins should not specify this field.
* password_field - The name attribute for the password input in a
form. Non-form logins should not specify this field.
or a Firefox::Marionette::Login object as the first parameter and adds
the login to the Firefox login database.
use Firefox::Marionette();
use UUID();
my $firefox = Firefox::Marionette->new();
# for http auth logins
my $http_auth_login = Firefox::Marionette::Login->new(host => 'https://pause.perl.org', user => 'AUSER', password => 'qwerty', realm => 'PAUSE');
$firefox->add_login($http_auth_login);
$firefox->go('https://pause.perl.org/pause/authenquery')->accept_alert(); # this goes to the page and submits the http auth popup
# for form based login
my $form_login = Firefox::Marionette::Login(host => 'https://github.com', user => 'me2@example.org', password => 'uiop[]', user_field => 'login', password_field => 'password');
$firefox->add_login($form_login);
# or just directly
$firefox->add_login(host => 'https://github.com', user => 'me2@example.org', password => 'uiop[]', user_field => 'login', password_field => 'password');
Note for HTTP Authentication, the realm
<https://datatracker.ietf.org/doc/html/rfc2617#section-2> must
perfectly match the correct realm supplied by the server.
This method returns itself to aid in chaining methods.
add_site_header
accepts a host name and a hash of HTTP headers to include in every
future HTTP Request that is being sent to that particular host.
use Firefox::Marionette();
use UUID();
my $firefox = Firefox::Marionette->new();
my $uuid = UUID::uuid();
$firefox->add_site_header( 'metacpan.org', 'Track-my-automated-tests' => $uuid );
$firefox->go('https://metacpan.org/');
these headers are added to any existing headers going to the
metacpan.org site, but no other site. To clear site headers, see the
delete_site_header method
add_webauthn_authenticator
accepts a hash of the following keys;
* has_resident_key - boolean value to indicate if the authenticator
will support client side discoverable credentials
<https://www.w3.org/TR/webauthn-2/#client-side-discoverable-credential>
* has_user_verification - boolean value to determine if the
authenticator
<https://www.w3.org/TR/webauthn-2/#virtual-authenticators> supports
user verification
<https://www.w3.org/TR/webauthn-2/#user-verification>.
* is_user_consenting - boolean value to determine the result of all
user consent <https://www.w3.org/TR/webauthn-2/#user-consent>
authorization gestures
<https://www.w3.org/TR/webauthn-2/#authorization-gesture>, and by
extension, any test of user presence
<https://www.w3.org/TR/webauthn-2/#test-of-user-presence> performed
on the Virtual Authenticator
<https://www.w3.org/TR/webauthn-2/#virtual-authenticators>. If set to
true, a user consent will always be granted. If set to false, it will
not be granted.
* is_user_verified - boolean value to determine the result of User
Verification <https://www.w3.org/TR/webauthn-2/#user-verification>
<https://firefox-source-docs.mozilla.org/networking/http/logging.html>
option on the command line of the firefox binary to allow extra
levels of debug.
* developer - only allow a developer edition
<https://www.mozilla.org/en-US/firefox/developer/> to be launched.
This defaults to "0" (off).
* devtools - begin the session with the devtools
<https://developer.mozilla.org/en-US/docs/Tools> window opened in a
separate window.
* geo - setup the browser preferences
<http://kb.mozillazine.org/About:config> to allow the Geolocation API
<https://developer.mozilla.org/en-US/docs/Web/API/Geolocation_API> to
work. If the value for this key is a URI object or a string beginning
with '^(?:data|http)', this object will be retrieved using the json
method and the response will used to build a GeoLocation object,
which will be sent to the geo method. If the value for this key is a
hash, the hash will be used to build a GeoLocation object, which will
be sent to the geo method.
* height - set the height
<http://kb.mozillazine.org/Command_line_arguments#List_of_command_line_arguments_.28incomplete.29>
of the initial firefox window
* har - begin the session with the devtools
<https://developer.mozilla.org/en-US/docs/Tools> window opened in a
separate window. The HAR Export Trigger
<https://addons.mozilla.org/en-US/firefox/addon/har-export-trigger/>
addon will be loaded into the new session automatically, which means
that -safe-mode
<http://kb.mozillazine.org/Command_line_arguments#List_of_command_line_arguments_.28incomplete.29>
will not be activated for this session AND this functionality will
only be available for Firefox 61+.
* host - use ssh <https://man.openbsd.org/ssh.1> to create and
automate firefox on the specified host. See REMOTE AUTOMATION OF
FIREFOX VIA SSH and NETWORK ARCHITECTURE. The user will default to
the current user name (see the user parameter to change this).
Authentication should be via public keys loaded into the local
ssh-agent <https://man.openbsd.org/ssh-agent>.
* implicit - a shortcut to allow directly providing the implicit
timeout, instead of needing to use timeouts from the capabilities
parameter. Overrides all longer ways.
* index - a parameter to allow the user to specify a specific firefox
instance to survive and reconnect to. It does not do anything else at
the moment. See the survive parameter.
* insecure - this is a shortcut method for setting the
accept_insecure_certs option in the capabilities parameter above.
* kiosk - start the browser in kiosk
<https://support.mozilla.org/en-US/kb/firefox-enterprise-kiosk-mode>
mode.
* mime_types - any MIME types that Firefox will encounter during this
session. MIME types that are not specified will result in a hung
browser (the File Download popup will appear).
* nightly - only allow a nightly release
<https://www.mozilla.org/en-US/firefox/channel/desktop/#nightly> to
be launched. This defaults to "0" (off).
* port - if the "host" parameter is also set, use ssh
<https://man.openbsd.org/ssh.1> to create and automate firefox via
the specified port. See REMOTE AUTOMATION OF FIREFOX VIA SSH and
NETWORK ARCHITECTURE.
* page_load - a shortcut to allow directly providing the page_load
timeout, instead of needing to use timeouts from the capabilities
parameter. Overrides all longer ways.
* profile - create a new profile based on the supplied profile. NOTE:
firefox ignores any changes made to the profile on the disk while it
is running, instead, use the set_pref and clear_pref methods to make
changes while firefox is running.
* profile_name - pick a specific existing profile to automate, rather
than creating a new profile. Firefox <https://firefox.com> refuses to
allow more than one instance of a profile to run at the same time.
Profile names can be obtained by using the
Firefox::Marionette::Profile::names() method. The following
conditions are required to use existing profiles;
* the preference security.webauth.webauthn_enable_softtoken must be
set to true in the profile OR
* the webauth parameter to this method must be set to 0
NOTE: firefox ignores any changes made to the profile on the disk
while it is running, instead, use the set_pref and clear_pref methods
to make changes while firefox is running.
* proxy - this is a shortcut method for setting a proxy using the
capabilities parameter above. It accepts a proxy URL, with the
following allowable schemes, 'http' and 'https'. It also allows a
reference to a list of proxy URLs which will function as list of
proxies that Firefox will try in left to right order
<https://developer.mozilla.org/en-US/docs/Web/HTTP/Proxy_servers_and_tunneling/Proxy_Auto-Configuration_PAC_file#description>
until a working proxy is found. See REMOTE AUTOMATION OF FIREFOX VIA
SSH, NETWORK ARCHITECTURE and SETTING UP SOCKS SERVERS USING SSH.
* reconnect - an experimental parameter to allow a reconnection to
firefox that a connection has been discontinued. See the survive
parameter.
* scp - force the scp protocol when transferring files to remote
hosts via ssh. See REMOTE AUTOMATION OF FIREFOX VIA SSH and the
--scp-only option in the ssh-auth-cmd-marionette
<https://metacpan.org/pod/ssh-auth-cmd-marionette> script in this
distribution.
* script - a shortcut to allow directly providing the script timeout,
instead of needing to use timeouts from the capabilities parameter.
Overrides all longer ways.
* seer - this option is switched off "0" by default. When it is
switched on "1", it will activate the various speculative and
| Site | | Server |
---------- ----------
In addition, the proxy parameter can be used to specify multiple
proxies using a reference to a list.
my $firefox = Firefox::Marionette->new(
host => 'Firefox.runs.here'
trust => '/path/to/ca-for-squid-proxy-server.crt',
proxy => [ 'https://Squid1.Proxy.Server:3128', 'https://Squid2.Proxy.Server:3128' ]
)->go('https://Target.Web.Site');
When firefox gets a list of proxies, it will use the first one that
works. In addition, it will perform a basic form of proxy failover,
which may involve a failed network request before it fails over to the
next proxy. In the diagram below, Squid1.Proxy.Server is the first
proxy in the list and will be used exclusively, unless it is
unavailable, in which case Squid2.Proxy.Server will be used.
----------
TLS | Squid1 |
------>| Proxy |-----
| | Server | |
--------- ----------- | ---------- | -----------
| Perl | SSH | Firefox | | | HTTPS | Target |
| runs |----->| runs |----| ------->| Web |
| here | | here | | | | Site |
--------- ----------- | ---------- | -----------
| TLS | Squid2 | |
------>| Proxy |-----
| Server |
----------
See the REMOTE AUTOMATION OF FIREFOX VIA SSH section for more options.
See SETTING UP SOCKS SERVERS USING SSH for easy proxying via ssh
<https://man.openbsd.org/ssh>
See GEO LOCATION section for how to combine this with providing
appropriate browser settings for the end point.
AUTOMATING THE FIREFOX PASSWORD MANAGER
This module allows you to login to a website without ever directly
handling usernames and password details. The Password Manager may be
preloaded with appropriate passwords and locked, like so;
use Firefox::Marionette();
my $firefox = Firefox::Marionette->new( profile_name => 'locked' ); # using a pre-built profile called 'locked'
if ($firefox->pwd_mgr_needs_login()) {
my $new_password = IO::Prompt::prompt(-echo => q[*], 'Enter the password for the locked profile:');
$firefox->pwd_mgr_login($password);
} else {
my $new_password = IO::Prompt::prompt(-echo => q[*], 'Enter the new password for the locked profile:');
$firefox->pwd_mgr_lock($password);
}
...
$firefox->pwd_mgr_logout();
Usernames and passwords (for both HTTP Authentication popups and HTML
Form based logins) may be added, viewed and deleted.
use WebService::HIBP();
my $hibp = WebService::HIBP->new();
$firefox->add_login(host => 'https://github.com', user => 'me@example.org', password => 'qwerty', user_field => 'login', password_field => 'password');
$firefox->add_login(host => 'https://pause.perl.org', user => 'AUSER', password => 'qwerty', realm => 'PAUSE');
...
foreach my $login ($firefox->logins()) {
if ($hibp->password($login->password())) { # does NOT send the password to the HIBP webservice
warn "HIBP reports that your password for the " . $login->user() " account at " . $login->host() . " has been found in a data breach";
$firefox->delete_login($login); # how could this possibly help?
}
}
And used to fill in login prompts without explicitly knowing the
account details.
$firefox->go('https://pause.perl.org/pause/authenquery')->accept_alert(); # this goes to the page and submits the http auth popup
$firefox->go('https://github.com/login')->fill_login(); # fill the login and password fields without needing to see them
GEO LOCATION
The firefox Geolocation API
<https://developer.mozilla.org/en-US/docs/Web/API/Geolocation_API> can
be used by supplying the geo parameter to the new method and then
calling the geo method (from a secure context
<https://developer.mozilla.org/en-US/docs/Web/Security/Secure_Contexts>).
The geo method can accept various specific latitude and longitude
parameters as a list, such as;
$firefox->geo(latitude => -37.82896, longitude => 144.9811);
OR
$firefox->geo(lat => -37.82896, long => 144.9811);
OR
$firefox->geo(lat => -37.82896, lng => 144.9811);
OR
$firefox->geo(lat => -37.82896, lon => 144.9811);
or it can be passed in as a reference, such as;
$firefox->geo({ latitude => -37.82896, longitude => 144.9811 });
the combination of a variety of parameter names and the ability to pass
parameters in as a reference means it can be deal with various geo
location websites, such as;
$firefox->geo($firefox->json('https://freeipapi.com/api/json/')); # get geo location from current IP address
$firefox->geo($firefox->json('https://geocode.maps.co/search?street=101+Collins+St&city=Melbourne&state=VIC&postalcode=3000&country=AU&format=json')->[0]); # get geo location of street address
$firefox->geo($firefox->json('http://api.positionstack.com/v1/forward?access_key=' . $access_key . '&query=101+Collins+St,Melbourne,VIC+3000')->{data}->[0]); # get geo location of street address using api key
$firefox->geo($firefox->json('https://api.ipgeolocation.io/ipgeo?apiKey=' . $api_key)); # get geo location from current IP address
$firefox->geo($firefox->json('http://api.ipstack.com/142.250.70.206?access_key=' . $api_key)); # get geo location from specific IP address (http access only for free)
These sites were active at the time this documentation was written, but
mainly function as an illustration of the flexibility of geo and json
methods in providing the desired location to the Geolocation API
<https://developer.mozilla.org/en-US/docs/Web/API/Geolocation_API>.
As mentioned in the geo method documentation, the ipgeolocation API
<https://ipgeolocation.io/documentation/ip-geolocation-api.html> is the
only API that currently providing geolocation data and matching
timezone data in one API call. If this url is used, the tz method will
be automatically called to set the timezone to the matching timezone
for the geographic location.
CONSOLE LOGGING
( run in 1.162 second using v1.01-cache-2.11-cpan-364913b4093 )