API 101

The API is currently in its first version and very naive / limited, but it exists... the API is not versionned yet, but if a number should be picked '0.0' will be the best match, this gives you an idea of how far you can go with it.

Basic concepts

The API is built around these simple concepts:

  • simple http GET queries
  • the output format is JSON
  • just add the '/api' prefix to any of your search, and you should be to go !
Here is a simple example: if you are browsing grep.metacpan.org, just copy paste your URL which should looks like this 
	https://grep.metacpan.org/api/search?q=test&qd=&qft=
You just want to add the /api/ prefix to your URI 
	curl -X GET 'https://grep.metacpan.org/api/search?q=test&qd=&qft' | json

The Output Format

The format is not definitive and still in an early stage development, but here's what it should looks like

Keys used at the main level of the output format

  • results: array which contains all the matching informations
  • is_incomplete: boolean which tell you if the request was truncated (after 2000 files), or if it contains all result from CPAN
  • search_in_progress: boolean to tell you if you need to query it a little later, as the query might still be running in background
  • time_elapsed: time in seconds to render the request
  • is_a_known_distro: boolean to tell you if the distro filter matches a known CPAN distribution
  • match: quick sumup metrics for your query

The results values

The match of your query are stored in the results array at the main level. This is a list hashes of all matching results. Where each hash describes the results for a specific distribution. The format of a distribution result is the following:
  • files: list of all files for this distribution matching your query
  • matches: list of array representing all matching codeblocks for a file
  • prefix: distribution filepath on disk
  • distro: distribution name

Codeblock structure

A codeblock represent the output of git grep with some context, so we need to indicate which line is the first one in the code extract and which lines are the one matching your query.
  • matchlines: array of integers listing all the file line number matching the query.
  • code: code extract for the match with some context
  • start_at: integer indicating the first line number in the code extract

The sumup statistics

Basic metrics about your query, you can know how many distributions and files match your query.
  • distros: integer value with the number of total distributions matching your query
  • files: integer value with the number of total files matching your query

Sample output format

{
  "results": [
    {
      "files": [
        "MANIFEST",
        "t/test.t"
      ],
      "matches": [
        {
          "blocks": [
            {
              "matchlines": [
                "6",
                "7",
                "8",
                "9",
                "10"
              ],
              "code": "Changes\nMANIFEST\nMakefile.PL\nREADME\nlib/abbreviation.pm\nt/test.t\ntestlib/CAPS/On.pm\ntestlib/Foo.pm\ntestlib/FooBar/Baz.pm\ntestlib/FooBar/Baz/Doh.pm\n",
              "start_at": "1"
            }
          ],
          "file": "MANIFEST"
        }
      ],
      "prefix": "distros/a/abbreviation",
      "distro": "abbreviation"
    },
    ... --- additional results where cut from there ---
  ],
  "is_incomplete": 0,
  "search_in_progress": 0,
  "time_elapsed": 0.016502,
  "is_a_known_distro": "",
  "match": {
    "files": "64",
    "distros": "7"
  }
}

Adding a filter to your query

Here are the valid parameters for your http query:
  • q: string for the query pattern (required)
  • qci: boolean 0 or 1 for a case insensitive search (optional - default 0)
  • qd: string for the distribution filter pattern (optional)
  • qft: string for the file type filter pattern (optional)
  • f: string for a specific file name filter pattern (optional)
  • p: integer for the page number. As the WebUI this is using pagination (optional - default 1)
Some API queries samples:
- search for 'test' among all CPAN distributions: /api/search?q=test
- case insensitive search for 'test' among all CPAN distributions: /api/search?q=test&qci=1
- search for 'test' among all distributions matching '*snap*': /api/search?q=test&qd=*snap*

Thumb Rules

Use it, but do not abuse it for now, or at your own risks :-) Try to be a good citizen. This is not designed on the same architecture than fastapi.metacpan.org...