Net-Async-WebSearch

 view release on metacpan or  search on metacpan

.env.example  view on Meta::CPAN

# TEST_WEBSEARCH_GOOGLE_CX=

# Yandex Search API
#   Signup: https://console.yandex.cloud/link/search-api/
#   Docs:   https://yandex.cloud/en/docs/search-api/
# Requires a Yandex Cloud account + folder + service account with the
# role search-api.executor. There is a free trial (Cloud credits).
# TEST_WEBSEARCH_YANDEX_API_KEY=
# TEST_WEBSEARCH_YANDEX_FOLDERID=

# Reddit OAuth — https://www.reddit.com/prefs/apps
# Free. Create an app (type "script" works). 100 QPM per client_id.
# TEST_WEBSEARCH_REDDIT_CLIENT_ID=
# TEST_WEBSEARCH_REDDIT_CLIENT_SECRET=
# TEST_WEBSEARCH_REDDIT_USER_AGENT="net-async-websearch-tests/0.001 by /u/YOURNAME"

lib/Net/Async/WebSearch.pm  view on Meta::CPAN

Cloud credits; a free trial exists via the standard Cloud welcome
credits. API key: create a service account in the Cloud Console,
grant it the C<search-api.executor> role, then generate an API key
(C<apikey>) or IAM token — that's your C<api_key>.

=item * B<Reddit> (public JSON) — no key

Works out of the box but rate-limited aggressively with generic UAs.
Fine for low-volume use; for anything serious use OAuth (below).

=item * B<Reddit OAuth> — L<https://www.reddit.com/prefs/apps>

Free. You need a Reddit account and a working User-Agent string
(Reddit insists on the form C<< app/1.0 by /u/yourname >>). At the
bottom of L<https://www.reddit.com/prefs/apps> click I<create app>,
pick type B<script> (for C<client_credentials>/C<password>) or
B<installed> (for C<installed>) or B<web> (for the full
C<authorization_code> consent flow). The short string under the app
name is C<client_id>; C<secret> is shown once on creation. Rate limit
is 100 QPM per OAuth identity. See
L<Net::Async::WebSearch::Provider::Reddit::OAuth/SETUP> for the full
walkthrough.

=back

lib/Net/Async/WebSearch.pm  view on Meta::CPAN

  Provider         Free tier                        CC?   Key source
  ---------------- -------------------------------- ---- --------------------------------------
  DuckDuckGo       unlimited (HTML scrape)          no   (no key)
  SearxNG          self-hosted, unlimited           no   (self-host; see ex/docker-compose.*)
  Brave            $5/month credits (~1000 q)       yes  api.search.brave.com/app/dashboard
  Serper           2500 / signup                    no   serper.dev (dashboard after signup)
  Google CSE       100 / day                        no   Cloud Console + programmablesearchengine.google.com
  Yandex           Cloud trial credits              no   console.yandex.cloud/link/search-api/
  Reddit           keyless (rate-limited)           no   (no key)
  Reddit OAuth     100 QPM per client_id            no   reddit.com/prefs/apps

=head2 Fetching result bodies

Pass C<fetch =E<gt> N> to any of the search modes to additionally GET the top
N result URLs and attach the response to each C<Result> under C<< $r->fetched >>
(see L<Net::Async::WebSearch::Result/fetched> for the hash shape). You still
get the full search result list — fetch is I<additive>.

Semantics per mode:

lib/Net/Async/WebSearch/Provider/Reddit/OAuth.pm  view on Meta::CPAN

=head2 search

Same contract as the parent provider — plus a transparent token fetch/refresh
round-trip before the first search (and again when the cached token is about
to expire). For C<authorization_code> grants without a seeded refresh_token,
the caller must drive L</authorize_url> / L</complete_authorization> first;
otherwise the first search will fail.

=head1 SETUP

Reddit apps are created at L<https://www.reddit.com/prefs/apps>. The flow:

=over 4

=item 1.

Log in to Reddit with the account that should own the app. For app-only /
installed grants this can be any account; for C<grant_type=password> it must
be the account whose credentials you're going to use.

=item 2.

Scroll to the bottom of L<https://www.reddit.com/prefs/apps> and click
B<"create app"> (or B<"create another app">).

=item 3.

Pick the app type:

=over 4

=item * B<script> — when you plan to use C<grant_type=password> or the default
C<client_credentials>. This is the right choice for personal/CLI tools and

lib/Net/Async/WebSearch/Provider/Reddit/OAuth.pm  view on Meta::CPAN

=back

After setup, the provider handles the token dance for you: it POSTs to
C<https://www.reddit.com/api/v1/access_token>, caches the bearer token
until C<token_margin> seconds before expiry, and attaches it to every
search request against C<https://oauth.reddit.com>.

=head1 SEE ALSO

L<Net::Async::WebSearch::Provider::Reddit>,
L<https://www.reddit.com/prefs/apps>,
L<https://github.com/reddit-archive/reddit/wiki/OAuth2>,
L<https://support.reddithelp.com/hc/en-us/articles/16160319875092-Reddit-Data-API-Wiki>

=head1 SUPPORT

=head2 Issues

Please report bugs and feature requests on GitHub at
L<https://github.com/Getty/p5-net-async-websearch/issues>.