Android-ElectricSheep-Automator
view release on metacpan or search on metacpan
It returns 0 on success, 1 on failure.
input_text($params)
It "types" the specified text into the specified position, where a
text-input widget is expected to exist. At first it taps at the
widget's location in order to get the focus. And then it enters the
text. You need to find the position of the desired text-input widget
by first getting the current screen UI (using dump_current_screen_ui)
and then using an XPath selector to identify the desired widget by
name/id/attributes. See the source code of method send_message() in
file lib/Android/ElectricSheep/Automator/Plugins/Apps/Viber.pm for
how this is done for the message-sending text-input widget of the
Viber app.
$params is a HASH_REF which must contain text and one of the two
position (of the text-edit widget) specifiers position or bounds:
text
- input\_text($params)
It "`types`" the specified text into the specified position,
where a text-input widget is expected to exist.
At first it taps at the widget's
location in order to get the focus. And then it enters
the text. You need to find the position of the desired
text-input widget by first getting the current screen UI
(using [dump\_current\_screen\_ui](https://metacpan.org/pod/dump_current_screen_ui)) and then using an XPath
selector to identify the desired widget by name/id/attributes.
See the source code of method `send_message()` in file
`lib/Android/ElectricSheep/Automator/Plugins/Apps/Viber.pm`
for how this is done for the message-sending text-input widget
of the Viber app.
`$params` is a HASH\_REF which must contain `text`
and one of the two position (of the text-edit widget)
specifiers `position` or `bounds`:
- **`text`**
lib/Android/ElectricSheep/Automator.pm view on Meta::CPAN
=item input_text($params)
It "C<types>" the specified text into the specified position,
where a text-input widget is expected to exist.
At first it taps at the widget's
location in order to get the focus. And then it enters
the text. You need to find the position of the desired
text-input widget by first getting the current screen UI
(using L<dump_current_screen_ui>) and then using an XPath
selector to identify the desired widget by name/id/attributes.
See the source code of method C<send_message()> in file
C<lib/Android/ElectricSheep/Automator/Plugins/Apps/Viber.pm>
for how this is done for the message-sending text-input widget
of the Viber app.
C<$params> is a HASH_REF which must contain C<text>
and one of the two position (of the text-edit widget)
specifiers C<position> or C<bounds>:
=over 4
lib/Android/ElectricSheep/Automator/Plugins/Apps/Viber.pm view on Meta::CPAN
. '['
. ' matches(@content-desc,\'navigate\s+up\',"i")'
. ' and matches(@class,\'ImageButton\',"i")'
. ' and @package="com.viber.voip"'
. ' and matches(@bounds,\'^\[\',"i")'
. ']'
;
@nodes = $xc->findnodes($asel);
$N = scalar @nodes;
if( $N == 0 ){
if( $verbosity > 0 ){ $log->info("${whoami} (via $parent), line ".__LINE__." : no nodes matching this XPath selector (for getting the 'back-arrow' icon, hopefully, this means we reached home-page of the app): ${asel}") }
last ONBACKARROW;
} elsif( $N > 1 ){ $log->error("--begin matched nodes:\n".join("\n", @nodes)."\n--end nodes matched.\n\n${whoami} (via $parent), line ".__LINE__." : error, matched more than one node (see above) with this XPath selector (for getting the 'Chats' ico...
$node = $nodes[0];
# click the 'back-arrow' at the top
$boundstr = $node->getAttribute('bounds');
if( ! defined $boundstr ){ $log->error("${node}\n\n${whoami} (via $parent), line ".__LINE__." : error, above node does not have attribute 'bounds'."); return 1 }
if( $boundstr !~ /\[\s*(\d+)\s*,\s*(\d+)\]\s*\[\s*(\d+)\s*,\s*(\d+)\]/ ){ $log->error("${whoami} (via $parent), line ".__LINE__." : error, failed to parse bounds string: '$boundstr'."); return 1 }
$bounds = [[$1,$2],[$3,$4]];
# click it
if( $verbosity > 0 ){ $log->info("${whoami} (via $parent), line ".__LINE__." : clicking the back-arrow ...") }
if( $self->mother->tap({'bounds' => $bounds}) ){ $log->error("${whoami} (via $parent), line ".__LINE__." : error, failed to tap on $bounds."); return 1 }
lib/Android/ElectricSheep/Automator/Plugins/Apps/Viber.pm view on Meta::CPAN
if( $verbosity > 0 ){ $log->info("${whoami} (via $parent), line ".__LINE__." : calling ".'dump_current_screen_ui()'." for at repeat $repeatsUI ...") }
$ui = $self->mother->dump_current_screen_ui({'filename'=>$outfile});
usleep(0.75);
} while( ($repeatsUI-- > 0) && (! defined($ui)) );
if( ! defined $ui ){ $log->error("${whoami} (via $parent), line ".__LINE__." : error, failed to dump the UI, call to ".'dump_current_screen_ui()'." has failed after a number of repeats. I am not sure what the problem is, most likely a race condition...
$dom = $ui->{'XML::LibXML'};
$asel = '//node[@text="Chats" and @resource-id="com.viber.voip:id/bottomBarItemTitle"]';
@nodes = $dom->findnodes($asel);
$N = scalar @nodes;
if( $N == 0 ){ $log->error("${whoami} (via $parent), line ".__LINE__." : error, failed to find any nodes matching this XPath selector (for getting the 'Chats' icon): ${asel}"); return undef }
elsif( $N > 1 ){ $log->error("--begin matched nodes:\n".join("\n", @nodes)."\n--end nodes matched.\n\n${whoami} (via $parent), line ".__LINE__." : error, matched more than one node (see above) with this XPath selector (for getting the 'Chats' icon):...
$node = $nodes[0];
# click the 'Chats' at the bottom
$boundstr = $node->getAttribute('bounds');
if( ! defined $boundstr ){ $log->error("${node}\n\n${whoami} (via $parent), line ".__LINE__." : error, above node does not have attribute 'bounds'."); return undef }
if( $boundstr !~ /\[\s*(\d+)\s*,\s*(\d+)\]\s*\[\s*(\d+)\s*,\s*(\d+)\]/ ){ $log->error("${whoami} (via $parent), line ".__LINE__." : error, failed to parse bounds string: '$boundstr'."); return undef }
$bounds = [[$1,$2],[$3,$4]];
# click it
if( $self->mother->tap({'bounds' => $bounds}) ){ $log->error("${whoami} (via $parent), line ".__LINE__." : error, failed to tap on $bounds."); return undef }
sleep(1);
lib/Android/ElectricSheep/Automator/Plugins/Apps/Viber.pm view on Meta::CPAN
$xc = $ui->{'XML::LibXML::XPathContext'};
$asel = '//node'
. '['
. '@text'
. ' and matches(@text,\''.$params->{'recipient'}.'\',"i")'
. ' and @resource-id="com.viber.voip:id/from"'
. ']'
;
@nodes = $xc->findnodes($asel);
$N = scalar @nodes;
if( $N == 0 ){ $log->error("${whoami} (via $parent), line ".__LINE__." : error, failed to find any nodes matching this XPath selector (for getting the recipient (".$params->{'recipient'}.") from the contacts on the central pane: ${asel}"); return un...
elsif( $N > 1 ){ $log->error("--begin matched nodes:\n".join("\n", @nodes)."\n--end nodes matched.\n\n${whoami} (via $parent), line ".__LINE__." : error, matched more than one node (see above) with this XPath selector (for getting the recipient (".$...
$node = $nodes[0];
# click the Recipient contact name at the bottom
$boundstr = $node->getAttribute('bounds');
if( ! defined $boundstr ){ $log->error("${node}\n\n${whoami} (via $parent), line ".__LINE__." : error, above node does not have attribute 'bounds'."); return undef }
if( $boundstr !~ /\[\s*(\d+)\s*,\s*(\d+)\]\s*\[\s*(\d+)\s*,\s*(\d+)\]/ ){ $log->error("${whoami} (via $parent), line ".__LINE__." : error, failed to parse bounds string: '$boundstr'."); return undef }
$bounds = [[$1,$2],[$3,$4]];
# click it
if( $self->mother->tap({'bounds' => $bounds}) ){ $log->error("${whoami} (via $parent), line ".__LINE__." : error, failed to tap on $bounds."); return undef }
usleep(1.5);
lib/Android/ElectricSheep/Automator/Plugins/Apps/Viber.pm view on Meta::CPAN
$xc = $ui->{'XML::LibXML::XPathContext'};
$asel = '//node'
. '['
. ' matches(@class,\'EditText$\',"i")'
. ' and @resource-id="com.viber.voip:id/send_text"'
. ' and @package="com.viber.voip"'
. ']'
;
@nodes = $xc->findnodes($asel);
$N = scalar @nodes;
if( $N == 0 ){ $log->error("${whoami} (via $parent), line ".__LINE__." : error, failed to find any nodes matching this XPath selector (for getting the specified recipient (".$params->{'recipient'}."): ${asel}"); return undef }
elsif( $N > 1 ){ $log->error("--begin matched nodes:\n".join("\n", @nodes)."\n--end nodes matched.\n\n${whoami} (via $parent), line ".__LINE__." : error, matched more than one node (see above) with this XPath selector (for getting the specified reci...
$node = $nodes[0];
$boundstr = $node->getAttribute('bounds');
if( ! defined $boundstr ){ $log->error("${node}\n\n${whoami} (via $parent), line ".__LINE__." : error, above node does not have attribute 'bounds'."); return undef }
if( $boundstr !~ /\[\s*(\d+)\s*,\s*(\d+)\]\s*\[\s*(\d+)\s*,\s*(\d+)\]/ ){ $log->error("${whoami} (via $parent), line ".__LINE__." : error, failed to parse bounds string: '$boundstr'."); return undef }
$bounds = [[$1,$2],[$3,$4]];
# add the text in the node
# the message will be sanitised in input_text(), spaces will be replaced with %s
# unicode is not supported
# TODO: perhaps warn about unicode in message?
lib/Android/ElectricSheep/Automator/Plugins/Apps/Viber.pm view on Meta::CPAN
# on the same XML we search for the send button
$asel = '//node'
. '['
. ' matches(@class,\'FrameLayout$\',"i")'
. ' and @resource-id="com.viber.voip:id/btn_send"'
. ' and @package="com.viber.voip"'
. ']'
;
@nodes = $xc->findnodes($asel);
$N = scalar @nodes;
if( $N == 0 ){ $log->error("${whoami} (via $parent), line ".__LINE__." : error, failed to find any nodes matching this XPath selector (for getting the recipient (".$params->{'recipient'}.") from the contacts on the central pane: ${asel}"); return un...
elsif( $N > 1 ){ $log->error("--begin matched nodes:\n".join("\n", @nodes)."\n--end nodes matched.\n\n${whoami} (via $parent), line ".__LINE__." : error, matched more than one node (see above) with this XPath selector (for getting the recipient (".$...
$node = $nodes[0];
# click the Send button
$boundstr = $node->getAttribute('bounds');
if( ! defined $boundstr ){ $log->error("${node}\n\n${whoami} (via $parent), line ".__LINE__." : error, above node does not have attribute 'bounds'."); return undef }
if( $boundstr !~ /\[\s*(\d+)\s*,\s*(\d+)\]\s*\[\s*(\d+)\s*,\s*(\d+)\]/ ){ $log->error("${whoami} (via $parent), line ".__LINE__." : error, failed to parse bounds string: '$boundstr'."); return undef }
$bounds = [[$1,$2],[$3,$4]];
# I know it does not support sending unicode to a textbox but ...
# TODO: check with a unicode recipient handle to see if this works
if( $verbosity > 0 ){ $log->info("To: '".Encode::decode_utf8($recipient)."'\nMessage:\n".Encode::decode_utf8($message)."\n--end message.\n${whoami} (via $parent), line ".__LINE__." : about to send the above message ...") }
if( $mock == 0 ){
t/t-data/example-screen-layout.xml view on Meta::CPAN
<?xml version='1.0' encoding='UTF-8' standalone='yes' ?><hierarchy rotation="0"><node index="0" text="" resource-id="" class="android.widget.FrameLayout" package="com.huawei.android.launcher" content-desc="" checkable="false" checked="false" clickabl...
xt/live/400-input_text-clear_input_field.t view on Meta::CPAN
my $xc = $ui->{'XML::LibXML::XPathContext'};
my $asel =
'//node[@resource-id="com.google.android.deskclock:id/fab"'
.' and matches(@class, \'Button\',"i")'
.' and matches(@content-desc, \'city\',"i")'
.']'
;
my @nodes = $xc->findnodes($asel);
#for my $anode (@nodes){ print $anode; } die 123;
my $N = scalar @nodes;
is($N, 1, "app clock : found + button from UI of the app with this selector: ${asel}") or BAIL_OUT(${ui}->{'raw'}."\nno it failed with above XML dump");
my $addbutton = $nodes[0];
# click the add-button found
my $boundstr = $addbutton->getAttribute('bounds');
ok(defined($boundstr), "bounds of the widget found: ${boundstr}") or BAIL_OUT;
ok($boundstr =~ /\[\s*(\d+)\s*,\s*(\d+)\]\s*\[\s*(\d+)\s*,\s*(\d+)\]/, "bounds of the widget parsed: ${boundstr}") or BAIL_OUT;
my $bounds = [[$1,$2],[$3,$4]];
is($mother->tap({'bounds'=>$bounds}), 0, 'add (+) button of the app has been clicked.') or BAIL_OUT(perl2dump($bounds)."no, failed to tap the add button with above bounds");
sleep(1);
xt/live/400-input_text-clear_input_field.t view on Meta::CPAN
$xc = $ui->{'XML::LibXML::XPathContext'};
$asel =
'//node[@resource-id="com.google.android.deskclock:id/open_search_view_edit_text"'
.' and matches(@class, \'Edit\',"i")'
.' and matches(@text, \'city\',"i")'
.']'
;
@nodes = $xc->findnodes($asel);
#for my $anode (@nodes){ print $anode; } die 123;
$N = scalar @nodes;
is($N, 1, "app clock : found text-edit widget from UI of the app with this selector: ${asel}") or BAIL_OUT(${ui}->{'raw'}."\nno it failed with above XML dump");
my $texteditbox = $nodes[0];
# click the add-button found
$boundstr = $texteditbox->getAttribute('bounds');
ok(defined($boundstr), "bounds of the widget found: ${boundstr}") or BAIL_OUT;
ok($boundstr =~ /\[\s*(\d+)\s*,\s*(\d+)\]\s*\[\s*(\d+)\s*,\s*(\d+)\]/, "bounds of the widget parsed: ${boundstr}") or BAIL_OUT;
$bounds = [ [$1,$2], [$3,$4] ];
my $text = 'hello%sworld';
is($mother->input_text({'bounds'=>$bounds, 'text'=>$text}), 0, "text ($text) has been entered in the text-edit widget.") or BAIL_OUT(perl2dump($bounds)."no, failed to input text ($text) in the text-edit widget with above bounds");
( run in 0.722 second using v1.01-cache-2.11-cpan-2b1a40005be )