chmod results from the CPAN

App-Unliner
However, you notice that report.cgi is chewing up lots of system resources. Who is responsible? Let's find out the IP addresses that are hitting this URL the most so we can track them down.

The first step is to extract out the requests for report.cgi so we'd probably do something like this:

    $ grep "GET /report.cgi" access.log

Now we'll extract the IP address:

    $ grep "GET /report.cgi" access.log | awk '{print $1}'

Next we add the standard C<sort | uniq -c | sort -rn> tallying pipeline:

    $ grep "GET /report.cgi" access.log | awk '{print $1}' | sort | uniq -c | sort -rn

Oops, the important bit scrolled off the screen. Let's add a C<head> process to limit the output:

    $ grep "GET /report.cgi" access.log | awk '{print $1}' | sort | uniq -c | sort -rn | head -n 5

And we finally get our nice report:

    3271039 10.3.0.29
        912 10.9.2.7
        897 10.9.2.1
        292 10.9.2.3
        101 10.9.2.4

Looks like we've found our culprit.



=head2 Installing unliner

If you want to follow along with this tutorial, or start coding right away, the easiest way to install unliner is with cpanminus:

    curl -sL https://raw.github.com/miyagawa/cpanminus/master/cpanm | sudo perl - App::Unliner



=head2 You want it to do I<what>?

Usually one-liners entered in your shell are thrown away after they are used because it's so easy to re-create them as necessary. That's one reason why unix pipes are so cool.

Besides, as soon as your pipelines reach a full line or two of text they start to become very hard to work with (though I confess I've gotten a lot of use out of crazy long pipelines before). At this point, usually the one-liner is re-written as a "r...

The point of unliner is to provide an intermediate stage between a one-liner and a real program. And you might even find that there is no need to make it a real program after all.

To turn your one-liner into an unliner just wrap a C<def main { }> around it like this:

    def main {
      grep "GET /report.cgi" access.log | awk '{print $1}' | sort | uniq -c | sort -rn | head -n 5
    }

If you save this in the file C<log-report> then your unliner program can be invoked with this command:

    $ unliner log-report < input.txt

You could also put a L<shebang line|https://en.wikipedia.org/wiki/Shebang_(Unix)> at the top of your script:

    #!/usr/bin/env unliner

Now if you C<chmod +x log-report> you can run it directly:

    $ ./log-report < input.txt



=head2 Defs

The C<def main { }> isn't a special type of def except that it happens to be what is called when your program is invoked. You can create other defs and they can be invoked by your main def and other defs, kind of like subroutines.

For example, we could move the C<awk> command into a C<ip-extractor> def, and the tallying logic into a C<tally> def:

    def main {
      grep "GET /report.cgi" access.log | ip-extractor | tally | head -n 5
    }

    def ip-extractor {
      awk '{print $1}'
    }

    def tally {
      sort | uniq -c | sort -rn
    }

The same sequences of processes will be created with this program as with the previous. However, defs let you organize and re-use pipeline components better.





=head2 Arguments

The unliner program shown so far is not very flexible. For instance, the C<access.log> filename is hard-coded.

To fix this the arguments passed in to our log-report program are available in the variable C<$@>, just like in a shell script:

    def main {
      grep "GET /report.cgi" $@ | ip-extractor | tally | head -n 5
    }

Now we can pass in a log file argument to our program (otherwise it will read input from standard input):

    $ unliner log-report access.log

Note that $@ escapes whitespace like bourne shell's C<"$@">. Actually it just passes the argument array untouched through to the process (grep in this case) so the arguments can contain any characters. The bourne equivalent of unquoted C<$@> and C<$*...

We can parameterise other aspects of the unliner program too. For example, suppose you wanted to control the number of lines that are included in the report. To do this add a "prototype":

    def main(head|h=i, junkarg=s) {
      grep "GET /report.cgi" $@ | ip-extractor | tally | head -n $head
    }

The prototype indicates that the main def requires arguments. Since the main def is the entry-point, these arguments must come from the command line:

    $ unliner log-report access.log --head 5

C<head|h=i> is a L<Getopt::Long> argument definition. It means that the official name of this argument is C<head>, that there is a single-dash alias C<h>, and that the argument's "type" is required to be an integer. Because C<h> is an alias we could ...

    $ unliner log-report access.log -h 5

However, if you forget to add one of these arguments, the head process will die with an error like C<head: : invalid number of lines>.
( run in 0.521 second using v1.01-cache-2.11-cpan-d8267643d1d )