Deboilerplating

I agree with Damian that envy is clearly a virtue. We should add being boastful to that list. What good does it that we can make easy things easy without much efford, if we never tell anybody? Hence this blog.

While working on Shell::Piping I realised that many languages use operators or operator overloading to get rid of plenty of boilerplate. Please allow me to boastfully illustrate.

my $find = Proc::Async.new('/usr/bin/find', '/tmp');
my $sort = Proc::Async.new('/usr/bin/sort', :w);

$find.stdout.lines.tap: -> $l {
    once await $sort.ready;
    $sort.write: „$l\n“.encode if $l ~~ /a/;
}
my $handle-sort-output = start { put $sort.stdout.lines.join(„\n“); }
my $sort-started = $sort.start;
{
    await $find.start;
    CATCH { default { } }
}
$sort.close-stdin;

await $handle-sort-output;

So we basically find /tmp | grep "a" | sort. Sort is a bit unusual as it waits for its STDIN to be closed before it actually starts to do anything. We don’t use grep but do the filtering ourselves. If we wouldn’t we could just shell-out and save us the bother. I found a way to do the same with a little less code.

$find |> -> $l { $l ~~ /a/ ?? $l !! Nil } |> $sort :quiet;

Looking at the operators of Raku that pattern can be found all over the place. Especially the hyper operators replace a loop and/or a chain of method calls with a single expression. The same goes for subscripts.

my %a = flat (1..12) Z <January February March April May June July August September October November December>;
say %a<3 5 7>;
# OUTPUT: (March May July)

Here the postcircumfix operator iterates over 3 5 7 and for each element calls .AT-KEY on %a. The result is returned as a list of the produced values. If we would do that by hand, we would be fairly quickly by 6 lines of code.

An instance of Proc::Async can only run once. When shell scripting in Raku that might become a burden. I need a way to declare a routine (or something that behaves like one) that will create objects from the same set of arguments. My goal was the following.

Shell::<&ls> = px<ls -l>;
Shell::<&grep> = px<grep>;
Shell::ls |> Shell::grep('a');

This would be quite easy to achieve if I didn’t step on a bug. So for now there is one extra space needed in px <grep>. While I was on it I added some error checking in px. It makes little sense to try to start a shell command if the file in question is not there nor executable.

Simple error handling however was easy to add because I rely on an infix to build the pipe. In contrast to postcircumfix they are well supported by Rakudo.

multi sub handle-stderr(|) { };
multi sub handle-stderr(0, $line) { say „ERR stream find: $line“ };
$find |> $errorer |> $sort :done({ say .exitcode if .exitcode }) :stderr(&handle-stderr);

The adverb :stderr registers a callback with the Shell::Pipe that is called with lines of any STDERR of the members of the pipe. As a first argument that callback receives the position of the command that produced that line. By using a multi we can offload the selection of the correct handler to the compiler. A single | in a signature declares a default candidate that will catch all otherwise unhandled outputs to STDERR. The operator doesn’t really do much here.

my multi infix:«|>»(Shell::Pipe:D $pipe where $pipe.pipees.tail ~~ Shell::Pipe::BlockContainer, Proc::Async:D $in, :&done? = Code, :$stderr? = CodeOrChannel, Bool :$quiet    ?) {
     ...
     $pipe.done = &done;
     $pipe.stderr = $stder;
     $pipe.quiet = $quiet;
     ...
}

It just sets a public attribute of the Shell::Pipe object to the provided callback. So the pattern that I’m using here is actually quite simple. Use an infix to turn two operands into an intermediary type. If more infix are used on that type, add to its state. Adverbs are used to trigger optional behaviour. The compiler will then call .sink on that intermediary to set anything in motion. As the first example in this blog post shows that motion can actually be quite elaborate. Yet we can hide that behind very cursory syntax by defining a custom operator.

I managed to implement anything I need to start turning the whole thing into a module. Automatic testing of that module will be a bit challenging. Luckily Raku is well suited to test crashing as jnthn kindly pointed out.

Judging by the weeklies we are producing more blog posts then ever. I welcome that move. We have no reason to be modest about Raku. Go forth and show off!