Concurrent dogfood

I’m using zef --verbose test . in a Makefile triggered by the F2 key to run test in Raku projects. While zef is very nice it’s also very slow not fast yet. It could be a good bit faster if it could run tests in parallel. Since I split up tests in individual files running them concurrently isn’t all that hard. The biggest hurdle is to collect all the outputs and show them without mixing up lines. With Shell::Piping that is very easy indeed.

constant NL = $?NL;
my &RED = { "\e[31m$_\e[0m" };
my &BOLD = { "\e[1m$_\e[0m" };

&RED = &BOLD = { $_ } unless $*OUT.t;

sub run-test(IO::Path $file where { .e & .f }) {
    my @out;
    my @err;
    my $failed;
    px«raku -Ilib $file» |» @out :stderr(@err) :done({$failed = .exitcodes.so});

    („Testing: {$file}“.&BOLD, @out, @err».&RED).flat.join(NL);
}

I shell out to raku and let it run a single test file. The streams of STDOUT and STDERR end up in Arrays. These are then merged in the right order with some colouring for good measure. Now I have to get list of files and run run-test in parallel.

.put for dir(‚t/‘).grep(*.extension eq ‚t‘).sort.hyper(:batch(1), :degree(12)).map(*.&run-test);

The outputs are .put out in the right order thanks to the magic of .hyper. With a single raku process the tests need 11.3s. With 12 threads it’s down to 3s. I shall change the binding to F2 in vim at once!

The whole script can be found here and Shell::Piping here. The latter will land in the ecosystem shortly.