Internal indirection

With writing more and more shell scripts in Raku, I realised that I call a MAIN by a MAIN in a very indirect manner. I wondered if I can find a way to reduce the indirection to get rid of the extra process and at least some of the overhead of Proc::Async. First we need a script to call.

#! /usr/bin/env raku

constant \frame = gather while my $frame = callframe($++) {
    take $frame
}

sub MAIN {
    say frame[0];

    .note for lines;

    exit 42;

    say 'alive';
}

This script prints its caller. Then reads blocking indirectly on $*IN via lines. It would print alive if the process would make it past the exit. Please note, that MAIN is the last statement in this compunit. That allows us to slurp the script and EVAL it.

my &main := $path.IO.slurp.&EVAL;

And that’s it! We can call a MAIN from another script. There are a few considerations though. We can pipe data to from one script to another because they share $*IN and $*OUT. Also, STDERR might get confusing. The semantics of exit may be wrong. It is likely that we want to keep the outer script running. Capturing the return-value of the inner script is easy, the exitcode is not.

Since we are calling functions lexicals can help with solving most of those problems. In Raku we can’t easily trap c-land exit. But we can prevent it from being called.

my &*EXIT = sub ($exitcode) {
    CapturedExitcode.new(:$exitcode).throw;
}

Here the exitcode is packaged in an exception so we can extract it from MAIN. This sub might exit with a return so we have to capture that one too.

        $out.exitcode = main();

        CATCH {
            when CapturedExitcode {
                $out.exitcode = .exitcode;
            }

            default {
                say .^name, ': ', .message;
                put .backtrace».Str
            }
        }

Now we need to deal with $*IN and $*OUT. Since the target script just calles lines and that forwards to $*ARGFILES.lines we can use a Channel. One of the Channels is a good place to store the exitcode.

    my $out = Channel.new but role :: { has $.exitcode is rw; };
    my $in = Channel.new but role :: { method put(Any:D \value){ self.send: value ~ $?NL } };

Since lines requires a Str we provide the familiar put-method. Other methods like say would go into the anonymous role too. When the inner MAIN terminates, we want to close the $out channel. We can do so in a LEAVE block. The whole thing can be wrapped into a sub which provides the outer script with a nice interface.

my ($out, $in) = embed-script('./script.raku');

start {
    for ^10 {
        $in.put: $++;
    }
    $in.close;
}

.print for $out.list;

say $out.exitcode;

Dealing with a multi sub MAIN is tricky. If the last statement in the script is &MAIN, it will refer to the dispatcher. With any multi candidate at the end, we can only get hold of the proto by descending into nqp-land.

my &main := $path.IO.slurp.&EVAL;

use nqp;
my &disp = &main.multi ?? nqp::getattr(&main, Routine, '$!dispatcher') !! &main;

We can then call disp() to dispatch to the correct MAIN candidate. I’m not sure if this is fragile. Every routine that is a multi got a dispatcher. Yet Routine.dispatcher is not exposed by CORE.

The whole example can be found here.

Avoiding Proc::Async does speed things up quite a bit. Since we can use a Channel, we don’t have to stringify and parse output when moving data around. The called MAIN needs to cooperate in this case and thus needs to know it is not called by the runtime. We could introduce a new lexical or check $*IN against Channel. There is also the option to check the callframe for EVAL.

constant \frame = gather while my $frame = callframe($++) {
    take $frame
}

say 'indirect' if any frame».gist».starts-with('EVAL');

Quite in contrast to Perl 5, for Raku I never used EVAL much. Not because I’m scared — there was little reason for code generation. After all, between the two of us I have always been the more evil twin.

UPDATE:

As jnthn pointed out, Routine.dispatcher is exposed. We can therefore keep well clear of nqp with:

my &disp = &main.multi ?? &main.dispatcher() !! &main;

As of now, it is unclear if this should be specced and thus documented.