streaming results from the CPAN

Perl500503Syntax-OrDie

=head1 PSGI COMPATIBILITY NOTES

HTTP::Handy implements a strict I<subset> of the PSGI/1.1 specification.
The following keys defined by the PSGI spec are B<not> set in C<$env>:

  psgi.version        (PSGI requires [1,1]; not set)
  psgi.multithread    (not set; effectively false)
  psgi.multiprocess   (not set; effectively false)
  psgi.run_once       (not set; effectively false)
  psgi.nonblocking    (not set; always blocking)
  psgi.streaming      (not set; not supported)

Applications that check for these keys must treat their absence as false.
For full PSGI/1.1 compliance use L<Plack> (requires Perl 5.8+).

=head1 SECURITY

HTTP::Handy is designed for B<personal use and local development only>.
It is not hardened for production or internet-facing deployment.

=over 4

t/corpus-stack/HTTP-Handy/lib/HTTP/Handy.pm view on Meta::CPAN

=item * HTTP/1.0 only -- no Keep-Alive, no HTTP/1.1, no HTTP/2

=item * GET and POST only -- HEAD, PUT, DELETE, etc. return 405

=item * Single process, single thread -- requests are handled one at a time

=item * No HTTPS (see above)

=item * No chunked transfer encoding

=item * No streaming -- POST body and response body are fully buffered in memory

=item * Maximum POST body size: 10 MB by default (configurable via C<max_post_size>)

=item * No cookie or session management (implement in the application layer)

=back

=head1 DEMO

Run directly to start a self-contained demo server:

t/corpus-stack/PSGI-Handy/lib/PSGI/Handy.pm view on Meta::CPAN

=item C<after: a code reference is required>

The argument to C<after> was not a CODE reference.

=back

=head1 LIMITATIONS

The C<$app> returned by C<to_app> always produces the buffered,
three-element PSGI response C<[ $status, \@headers, \@body ]>. The PSGI
delayed-response form (the streaming "responder" callback) is not
generated; this is what "PSGI-subset" means throughout this distribution.

Concurrency and the HTTP version depend on the PSGI server you choose.
No multipart uploads or WebSocket in this version. C<HEAD> requests are
served by the matching C<GET> route with the body removed.

=head1 SEE ALSO

L<HTTP::Handy>, L<HP::Handy>, L<PSGI::Handy::Router>,
L<PSGI::Handy::Request>, L<PSGI::Handy::Response>,

t/corpus/JSON-LINQ/lib/JSON/LINQ.pm view on Meta::CPAN

        print {*{$fhn}} _json_encode($record);
    });
    { no strict 'refs'; print {*{$fhn}} "\n]\n" }

    { no strict 'refs'; close($fhn) }
    return 1;
}

# ToJSONL - write sequence as a JSONL (JSON Lines) file
# Each element is encoded as one line of JSON.
# This is streaming-friendly and memory-efficient.
sub ToJSONL {
    my($self, $file) = @_;

    my $fhn = _open_fh('>', $file, 1);

    $self->ForEach(sub {
        my $record = shift;
        no strict 'refs';
        print {*{$fhn}} _json_encode($record), "\n";
    });

t/corpus/JSON-LINQ/lib/JSON/LINQ.pm view on Meta::CPAN


JSON::LINQ provides a LINQ-style query interface for JSON, JSONL
(JSON Lines), and LTSV (Labeled Tab-Separated Values) files. It is
the JSON counterpart of L<LTSV::LINQ>, sharing the same LINQ API and
adding JSON-specific I/O methods.

Key features:

=over 4

=item * B<Lazy evaluation> - O(1) memory for JSONL and LTSV streaming;
JSON arrays are loaded once then iterated lazily

=item * B<Method chaining> - Fluent, readable query composition

=item * B<DSL syntax> - Simple key-value filtering

=item * B<67 LINQ methods> - including JSON I/O (FromJSON, FromJSONL,
FromJSONString, ToJSON, ToJSONL), LTSV I/O (FromLTSV, ToLTSV),
CSV I/O (FromCSV, ToCSV), and all 60 methods from L<LTSV::LINQ>

t/corpus/JSON-LINQ/lib/JSON/LINQ.pm view on Meta::CPAN

=item * B<Empty()> - Empty sequence

=item * B<Repeat($element, $count)> - Repeated element

=back

=head2 What is JSONL?

JSONL (JSON Lines, also known as ndjson - newline-delimited JSON) is a
text format where each line is a valid JSON value (typically an object).
It is particularly suited for log files and streaming data because:

=over 4

=item * One record per line enables streaming with O(1) memory usage

=item * Compatible with standard Unix tools (grep, sed, awk)

=item * Easily appendable without rewriting the whole file

=item * Each line is independently parseable

=back

B<Format example:>

t/corpus/JSON-LINQ/lib/JSON/LINQ.pm view on Meta::CPAN


LINQ (Language Integrated Query) is the Microsoft .NET query API.
This module brings the same LINQ interface to JSON data in Perl.
See L<LTSV::LINQ> for a detailed description of the LINQ design philosophy.

=head1 INCLUDED DOCUMENTATION

The C<eg/> directory contains sample programs:

  eg/01_json_query.pl       FromJSON/Where/Select/OrderByDescending/Distinct/ToLookup
  eg/02_jsonl_query.pl      FromJSONL streaming, GroupBy, aggregation, ToJSONL
  eg/03_grouping.pl         GroupBy, ToLookup, GroupJoin, SelectMany, Join
  eg/04_sorting.pl          OrderBy/ThenBy multi-key sort, OrderByNum vs OrderByStr
  eg/05_json_ltsv_join.pl   JOIN main JSON x sub-table LTSV
  eg/06_ltsv_json_join.pl   JOIN main LTSV x sub-table JSON
  eg/07_csv_query.pl        FromCSV/Where/Select/GroupBy/OrderByNum/ToCSV
  eg/08_csv_json_join.pl    JOIN main CSV x sub-table JSON, CSV to JSON conversion

The C<doc/> directory contains JSON::LINQ cheat sheets in 21 languages:

  doc/json_linq_cheatsheet.EN.txt   English

t/corpus/JSON-LINQ/lib/JSON/LINQ.pm view on Meta::CPAN

as a one-element sequence.

B<File format:>

  [
    {"name": "Alice", "age": 30},
    {"name": "Bob",   "age": 25}
  ]

The entire file is read into memory and parsed once. For large files,
consider JSONL format with C<FromJSONL> for streaming access.

B<Concurrent use (e.g. Join/GroupJoin):> On Perl 5.006 and later,
each call to C<FromJSON> uses a distinct numbered filehandle slot, so
multiple iterators may be open simultaneously without interference.
On Perl 5.005_03, a unique numbered package glob is used per call
(JSON::LINQ::FH::H1, JSON::LINQ::FH::H2, ...) to achieve the same safety.

=item B<FromJSONL($filename)>

Read a JSONL (JSON Lines) file. Each non-empty line is parsed as a

t/corpus/JSON-LINQ/lib/JSON/LINQ.pm view on Meta::CPAN

  [
  {"age":30,"name":"Alice"},
  {"age":25,"name":"Bob"}
  ]

Hash keys are sorted alphabetically for deterministic output.

=item B<ToJSONL($filename)>

Write the sequence as a JSONL file. Each element is written as one line
of JSON. This is the streaming counterpart of C<ToJSON>.

  $query->ToJSONL("output.jsonl");

B<Output format:>

  {"age":30,"name":"Alice"}
  {"age":25,"name":"Bob"}

=back

t/corpus/JSON-LINQ/lib/JSON/LINQ.pm view on Meta::CPAN


JSON::LINQ and LTSV::LINQ are parallel modules sharing the same LINQ API.

  LTSV::LINQ  - LINQ for LTSV (Labeled Tab-Separated Values) files
  JSON::LINQ  - LINQ for JSON and JSONL files

Both share the same LINQ API. JSON::LINQ adds the following I/O methods
on top of LTSV::LINQ's interface:

  FromJSON($file)         - read JSON array file
  FromJSONL($file)        - read JSONL file (streaming)
  FromJSONString($json)   - read JSON string
  FromLTSV($file)         - read LTSV file (streaming)
  FromCSV($file)          - read CSV file (streaming, RFC 4180)
  ToJSON($file)           - write JSON array file
  ToJSONL($file)          - write JSONL file
  ToLTSV($file)           - write LTSV file (streaming)
  ToCSV($file)            - write CSV file

C<FromLTSV>, C<ToLTSV>, C<FromCSV>, and C<ToCSV> are provided so a
JSON::LINQ pipeline can JOIN against (or emit into) LTSV and CSV files
without requiring LTSV::LINQ or CSV::LINQ to be installed.

The internal iterator architecture is identical: each operator returns a
new query object wrapping a closure.

=head2 Memory Characteristics

  FromJSONL  - O(1) per record: one line at a time
  FromJSON   - O(n): entire file loaded once, then lazy iteration
  FromLTSV   - O(1) per record: one line at a time
  FromCSV    - O(1) per record: one line at a time
  ToJSON     - O(n): entire sequence collected for array output
  ToJSONL    - O(1) per record: streaming write
  ToLTSV     - O(1) per record: streaming write
  ToCSV      - O(n): entire sequence collected before writing header

=head1 COMPATIBILITY

=head2 Perl Version Support

Compatible with B<Perl 5.00503 and later>. See L<LTSV::LINQ> for the
full compatibility rationale (Universal Consensus 1998 / Perl 5.005_03).

=head2 Pure Perl Implementation

t/corpus/LTSV-LINQ/lib/LTSV/LINQ.pm view on Meta::CPAN


  time:2026-02-13T10:00:00	host:192.0.2.1	status:200	url:/index.html	bytes:1024

=head3 LTSV Characteristics

=over 4

=item * B<One record per line>

A complete record is always a single newline-terminated line. This makes
streaming processing trivial: read a line, parse it, process it, discard it.
There is no multi-line quoting problem, no block parser required.

=item * B<Tab as field delimiter>

Fields are separated by a single horizontal tab character (C<0x09>).
The tab is a C0 control character in the ASCII range (C<0x00>-C<0x7F>),
which has an important consequence for multibyte character encodings.

=item * B<Colon as label-value separator>

t/corpus/LTSV-LINQ/lib/LTSV/LINQ.pm view on Meta::CPAN

=item * B<Self-describing fields>

Every field carries its own label. A record is human-readable without a
separate schema or header line. Fields can appear in any order, and
optional fields can simply be omitted. Adding a new field to some records
does not break parsers that do not know about it.

=item * B<Streaming-friendly>

Because each record is one line, LTSV files can be processed with line-by-line
streaming. Memory usage is proportional to the longest single record, not
the total file size. This is why C<FromLTSV> in this module uses a lazy
iterator rather than loading the whole file.

=item * B<Grep- and awk-friendly>

Standard Unix text tools (C<grep>, C<awk>, C<sed>, C<sort>, C<cut>) work
naturally on LTSV files. A field can be located with a pattern like
C<status:5[0-9][0-9]> without any special parser. This makes ad-hoc
analysis and shell scripting straightforward.

t/corpus/LTSV-LINQ/lib/LTSV/LINQ.pm view on Meta::CPAN

rules differ between implementations (RFC 4180 vs. Microsoft Excel vs. others).
LTSV has no quoting: the tab delimiter and the colon separator do not appear
inside values in any of the supported encodings (by the multibyte-safety
argument above), so no escaping mechanism is needed.

=item * B<Wide adoption in server logging>

LTSV originated in the Japanese web industry as a structured log format for
HTTP access logs. Many web servers (Apache, Nginx) and log aggregation tools
support LTSV output or parsing. The format is particularly popular for
application and infrastructure logging where grep-ability and streaming
analysis matter.

=back

For the formal LTSV specification, see L<https://ltsv.org/>.

=head2 What is LINQ?

LINQ (Language Integrated Query) is a set of query capabilities introduced
in the .NET Framework 3.5 (C# 3.0, 2007) by Microsoft. It defines a

t/corpus/LTSV-LINQ/lib/LTSV/LINQ.pm view on Meta::CPAN

A: OrderBy must load all elements into memory to sort them.

  # Slow on 1GB file - loads everything
  ->OrderBy(sub { $_[0]{timestamp} })->Take(10)

  # Faster - limit before sorting (if possible)
  ->Where(status => '500')->OrderBy(...)->Take(10)

=item B<Q: How do I process files larger than memory?>

A: Use ForEach or streaming terminal operations:

  # Process 100GB file with 1KB memory
  my $error_count = 0;
  LTSV::LINQ->FromLTSV("100gb.log")
      ->Where(sub { $_[0]{level} eq 'ERROR' })
      ->ForEach(sub { $error_count++ });

  print "Errors: $error_count\n";

=back

( run in 0.563 second using v1.01-cache-2.11-cpan-140bd7fdf52 )