Android-ElectricSheep-Automator
view release on metacpan or search on metacpan
# NAME
Android::ElectricSheep::Automator - Do Androids Dream of Electric Sheep? Smartphone control from your desktop.
# VERSION
Version 0.09
# WARNING
Current distribution is extremely alpha. API may change.
# SYNOPSIS
The present package fascilitates the control
of a USB-debugging-enabled
Android device, e.g. a real smartphone,
or an emulated (virtual) Android device,
from your desktop computer using Perl.
It's basically a thickishly-thin wrapper
to the omnipotent Android Debug Bridge (adb)
program.
**Note that absolutely nothing is
installed on the connected device,
neither any of its settings will be modified by this package**.
See ["WILL ANYTHING BE INSTALLED ON THE DEVICE?"](#will-anything-be-installed-on-the-device).
use Android::ElectricSheep::Automator;
my $mother = Android::ElectricSheep::Automator->new({
# optional as there is a default, but you may have
# problems with the location of the adb executable
'configfile' => $configfile,
'verbosity' => 1,
# we already have a device connected and ready to control
'device-is-connected' => 1,
});
# find the devices connected to desktop and set one.
my @devices = $mother->adb->devices;
$mother->connect_device({'serial' => $devices->[0]->serial})
or die;
# no device identification is required for the method call
# if there is only one connected device:
$mother->connect_device() if scalar(@devices)==0;
# Go Home
$mother->home_screen() or die;
# swipe up/down/left/right
$mother->swipe({'direction'=>up}) or die;
# dt is the time to swipe in millis,
# the shorter the faster the swipe
$mother->swipe({'direction'=>left, 'dt'=>100}) or die;
# tap
$mother->tap({'position'=>[100,200]});
# uses swipe() to move in screens (horizontally):
$mother->next_screen() or die;
$mother->previous_screen() or die;
# bottom navigation:
# the "triangle" back button
$mother->navigation_menu_back_button() or die;
# the "circle" home button
$mother->navigation_menu_home_button() or die;
# the "square" overview button
$mother->navigation_menu_overview_button() or die;
# open/close apps
$mother->open_app({'package'=>qr/calendar$/i}) or die;
$mother->close_app({'package'=>qr/calendar$/i}) or die;
# push pull files
$mother->adb->pull($deviceFile, $localFile);
$mother->adb->push($localFile, $deviceFileOrDir);
# guess what!
my $xmlstr = $mother->dump_current_screen_ui();
# Pull the apk(s) for an app from device and save locally
my $res = $mother->pull_app_apk_from_device({
package => 'com.google.android.calendar'
# or qr/calendar/i
'output-dir' => '/tmp/apks-of-calendar-app',
});
print $res->{'com.google.android.calendar'}->[0]->['local-path'};
# Install apk(s) for an app onto the device
$mother->install_app({
'apk-filename' => ['/tmp/apks/base.apk', '/tmp/apks/config.apk'],
# or just a string scalar '/tmp/apks/1.apk'
# optional params to the adb install command
'install-parameters' => ['-r', '-g']
});
# CONSTRUCTOR
## **`new($params)`**
Creates a new `Android::ElectricSheep::Automator` object. `$params`
is a hash reference used to pass initialization options which may
or should include the following:
- **`confighash`** or **`configfile`**
the configuration
file holds
configuration parameters. Its format is "enhanced" JSON
(see [Config::JSON::Enhanced](https://metacpan.org/pod/Config%3A%3AJSON%3A%3AEnhanced)) which is basically JSON
which allows comments between ` </* ` and ` */> `.
Here is an example configuration file to get you started:
{
"adb" : {
"path-to-executable" : "/usr/local/android-sdk/platform-tools/adb"
},
"debug" : {
"verbosity" : 0,
</* cleanup temp files on exit */>
"cleanup" : 1
},
"logger" : {
</* log to file if you uncomment this, else console */>
"filename" : "my.log"
}
}
All sections in the configuration are mandatory.
Setting `"adb"` to the wrong path will yield problems.
`confighash` is a hash of configuration options with
structure as above and can be supplied to the constructor
instead of the configuration file.
If no configuration is specified, then a default
configuration will be used. In this case please
specify **`adb-path-to-executable`** to point
to the location of `adb`. Most likely
the default path will not work for you.
- **`adb-path-to-executable`**
optionally specify the path to the `adb` executable in
your desktop system. This will override the setting
` 'adb'->'path-to-executable' ` in the configuration,
if it was provided. Use this option if you are not
providing any configuration and so the default configuration
will be used. But it will most likely fail because of this
path not being correct for your system. So, if you are going
to omit providing a configuration and the default configuration
will be used do specify the `adb` path via this option (but you
don't have to and your mileage may vary).
- **`device-serial`** or **`device-object`**
optionally specify the serial
of a device to connect to on instantiation,
or a [Android::ElectricSheep::Automator::DeviceProperties](https://metacpan.org/pod/Android%3A%3AElectricSheep%3A%3AAutomator%3A%3ADeviceProperties)
object you already have handy. Alternatively,
use ["connect\_device($params)"](#connect_device-params) to set the connected device at a later
time. Note that there is no need to specify a
device if there is exactly one connected device.
- **`adb`**
optionally specify an already created [Android::ADB](https://metacpan.org/pod/Android%3A%3AADB) object.
Otherwise, a fresh object will be created based
on the configuration under the `adb` section of the configuration.
- **`device-is-connected`**
optionally set it to 1
in order to communicate with the device
and get some information about it like
screen size, resolution, orientation, etc.
And also allow use of
functionality which needs communicating with a device
- **`device-object`** should be
an already existing [Android::ElectricSheep::Automator::DeviceProperties](https://metacpan.org/pod/Android%3A%3AElectricSheep%3A%3AAutomator%3A%3ADeviceProperties)
object.
It returns `0` on success, `1` on failure.
## **`dump_current_screen_ui($params)`**
It dumps the current screen as XML and returns that as
a string, optionally saving it to the specified file.
`$params` is a HASH\_REF which may or should contain:
- **`filename`**
optionally save the returned XML string to the specified file.
It returns `undef` on failure or the UI XML dump, as a string, on success.
## **`dump_current_screen_shot($params)`**
It dumps the current screen as a PNG image and returns that as
a [Image::PNG](https://metacpan.org/pod/Image%3A%3APNG) object, optionally saving it to the specified file.
`$params` is a HASH\_REF which may or should contain:
- **`filename`**
optionally save the returned XML string to the specified file.
It returns `undef` on failure or a [Image::PNG](https://metacpan.org/pod/Image%3A%3APNG) image, on success.
## **`dump_current_screen_video($params)`**
It dumps the current screen as MP4 video and saves that
in specified file.
`$params` is a HASH\_REF which may or should contain:
- **`filename`**
save the recorded video to the specified file in MP4 format. This
is required.
- **`time-limit`**
optionally specify the duration of the recorded video, in seconds. Default is 10 seconds.
- **`bit-rate`**
optionally specify the bit rate of the recorded video in bits per second. Default is 20Mbps.
- **`size`**
optionally specify the size (geometry) of the recorded video as a
HASH\_REF with keys `width` and `height`, in pixels. Default is "_the
device's main display resolution_".
- **`bugreport`**
optionally set this flag to 1 to have Android overlay debug information
on the recorded video, e.g. timestamp.
- **`display-id`**
for a device set up with multiple physical displays, optionally
specify which one to record -- if not the main display -- by providing the
display id. You can find display ids with ["list\_physical\_displays()"](#list_physical_displays)
or, from the CLI, by `adb shell dumpsys SurfaceFlinger --display-id`
`adb shell screenrecord --help` contains some more documentation.
## **`list_physical_displays()`**
It lists the IDs of all the physical displays connected to the
device, including the main one and returns these back as a HASH\_REF
keyed on display ID. It needs that connect\_device() to have been called prior to this call
It returns `undef` on failure or the results as a HASH\_REF keyed on display ID.
## **`list_running_processes($params)`**
It finds the running processes on device (using a \`ps\`),
optionally can save the (parsed) \`ps\`
results as JSON to the specified 'filename'.
It returns `undef` on failure or the results as a hash of hashes on success.
`$params` is a HASH\_REF which may or should contain:
- **`extra-fields`**
optionally add more fields (columns) to the report by `ps`, as an ARRAY\_REF.
For example, `['TTY','TIME']`.
It needs that connect\_device() to have been called prior to this call
It returns `undef` on failure or a hash with these keys on success:
- **`raw`** : contains the raw \`ps\` output as a string.
- **`perl`** : contains the parsed raw output as a Perl hash with
each item corresponding to one process, keyed on process command and arguments
(as reported by `ps`, verbatim), as a hash keyed on each field (column)
of the `ps` output.
- **`json`** : the above data converted into a JSON string.
## **`pidof($params)`**
It returns the PID of the specified command name.
The specified command name must match the app or command
name exactly. **Use L/pgrep()** if you want to match command
names with a regular expression>.
`$params` is a HASH\_REF which should contain:
- **`name`**
the name of the process. It can be a command name,
e.g. `audioserver` or an app name e.g. `android.hardware.vibrator-service.example`.
It returns `undef` on failure or the PID of the matched command on success.
Close an app by its exact name or a keyword matching it (uniquely):
electric-sheep-close-app.pl --configfile config/myapp.conf --name com.android.settings
electric-sheep-close-app.pl --configfile config/myapp.conf --keyword 'clock'
Note that it constructs a regular expression from escaped user input.
## **`electric-sheep-dump-ui.pl`**
Dump the current screen UI as XML to STDOUT or to a file:
electric-sheep-dump-ui.pl --configfile config/myapp.conf --output ui.xml
Note that it constructs a regular expression from escaped user input.
## **`electric-sheep-dump-current-location.pl`**
Dump the GPS / geo-location position for the device from its various providers, if enabled.
electric-sheep-dump-current-location.pl --configfile config/myapp.conf --output geolocation.json
## **`electric-sheep-emulator-geofix.pl`**
Set the GPS / geo-location position to the specified coordinates.
electric-sheep-dump-ui.pl --configfile config/myapp.conf --latitude 12.3 --longitude 45.6
## **`electric-sheep-dump-screen-shot.pl`**
Take a screenshot of the device (current screen) and save to a PNG file.
electric-sheep-dump-screen-shot.pl --configfile config/myapp.conf --output screenshot.png
## **`electric-sheep-dump-screen-video.pl`**
Record a video of the device's current screen and save to an MP4 file.
electric-sheep-dump-screen-video.pl --configfile config/myapp.conf --output video.mp4 --time-limit 30
## **`electric-sheep-pull-app-apk.pl`**
Extract the APK file (java bytecode) for an app installed on the device and save locally, perhaps, for disassembly and/or modification and/or re-installation.
electric-sheep-pull-app-apk.pl --package calendar2 --wildcard --output anoutdir --configfile config/myapp.conf --device Pixel_2_API_30_x86_
## **`electric-sheep-install-app`**
Install an APK file onto the device, passing extra installation
parameters `-r` (for re-install) and `-g` (for granting permissions),
electric-sheep-install-app --apk-filename test.apk -p '-r' -p '-g' --configfile config/myapp.conf --device Pixel_2_API_30_x86_
## **`electric-sheep-viber-send-message.pl`**
Send a message using the Viber app.
electric-sheep-viber-send-message.pl --message 'hello%sthere' --recipient 'george' --configfile config/myapp.conf --device Pixel_2_API_30_x86_
This one saves a lot of debugging information to `debug` which can be used to
deal with special cases or different versions of Viber:
electric-sheep-viber-send-message.pl --outbase debug --verbosity 1 --message 'hello%sthere' --recipient 'george' --configfile config/myapp.conf --device Pixel_2_API_30_x86_
# TESTING
The normal tests under the `t/` directory, initiated with `make test` command,
are quite limited in scope because they do not assume
a connected device. That is, they do not check any
functions which require interaction with a connected
device.
The _live tests_ under the `xt/live` directory, initiated with
`make livetest` command, require
an Android emulator or real device (the latter **is not recommended**)
connected to your desktop computer on which you are doing the testing.
Note that testing
with your smartphone is not a good idea, please do not do this,
unless it is some phone which you do not store important data.
It is very easy to get an emulated Android device running on any OS.
So, prior to `make livetest` make sure you have an android
emulator up and running with, for example,
`emulator -avd Pixel_2_API_30_x86_` . See section
["Android Emulators"](#android-emulators) for how to install, list and run them
buggers.
At least one of the _author tests_ under the `xt/author` directory,
initiated with
`make authortest` command, require an APK file (to be installed on the connected
device) which is quite large and it is not included in the distribution
bundle of this module. Anyway, it is not a good idea to install
an unknown APK to your device. But if you want to make this test
then pull an APK of an existing app on your connected device
with [electric-sheep-pull-app-apk.pl](https://metacpan.org/pod/electric-sheep-pull-app-apk.pl) and point the test file
to this APK.
Testing will not send any messages via the device's apps.
E.g. the plugin [Android::ElectricSheep::Automator::Plugins::Apps::Viber](https://metacpan.org/pod/Android%3A%3AElectricSheep%3A%3AAutomator%3A%3APlugins%3A%3AApps%3A%3AViber)
will not send a message via Viber but it will mock it.
The live tests will sometimes fail because, so far,
something unexpected happened in the device. For example,
in testing sending input text to a text-edit widget,
the calendar will be opened and a new entry will be added
and its text-edit widget will be targeted. Well, sometimes
the calendar app will give you some notification
on startup and this messes up with the focus.
Other times, the OS will detect that some app is taking too
long to launch and pops up a notification about
"_something is not responding, shall I close it_".
This steals the focus and sometimes it causes
the tests to fail.
# PREREQUISITES
## Android Studio
This is not a prerequisite but it is
highly recommended to install it
(from [https://developer.android.com/studio](https://developer.android.com/studio))
on your desktop computer because it contains
all the executables you will need,
( run in 2.173 seconds using v1.01-cache-2.11-cpan-9581c071862 )