Home > Raku > Concurrent dogfood

Concurrent dogfood

I’m using zef --verbose test . in a Makefile triggered by the F2 key to run test in Raku projects. While zef is very nice it’s also very slow not fast yet. It could be a good bit faster if it could run tests in parallel. Since I split up tests in individual files running them concurrently isn’t all that hard. The biggest hurdle is to collect all the outputs and show them without mixing up lines. With Shell::Piping that is very easy indeed.

constant NL = $?NL;
my &RED = { "\e[31m$_\e[0m" };
my &BOLD = { "\e[1m$_\e[0m" };

&RED = &BOLD = { $_ } unless $*OUT.t;

sub run-test(IO::Path $file where { .e & .f }) {
    my @out;
    my @err;
    my $failed;
    px«raku -Ilib $file» |» @out :stderr(@err) :done({$failed = .exitcodes.so});

    („Testing: {$file}“.&BOLD, @out, @err».&RED).flat.join(NL);
}

I shell out to raku and let it run a single test file. The streams of STDOUT and STDERR end up in Arrays. These are then merged in the right order with some colouring for good measure. Now I have to get list of files and run run-test in parallel.

.put for dir(‚t/‘).grep(*.extension eq ‚t‘).sort.hyper(:batch(1), :degree(12)).map(*.&run-test);

The outputs are .put out in the right order thanks to the magic of .hyper. With a single raku process the tests need 11.3s. With 12 threads it’s down to 3s. I shall change the binding to F2 in vim at once!

The whole script can be found here and Shell::Piping here. The latter will land in the ecosystem shortly.

Categories: Raku
  1. No comments yet.
  1. August 3, 2020 at 15:17

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: