Chain calling

When working with IO::Path we have to take platform dependent directory separator into account. To alleviate the problem .add was added. Surprisingly there is no candidate that takes a list. So we have to chain method calls as if we would use an inferior language.

'/home/dex'.IO.chain.add('tmp').add('foo.txt').say;
"/home/dex/tmp/foo.txt".IO

Since it is rather unlikely that we ever do calculations with IO::Path objects we can repurpose infix:</>. Doing so gives us metaoperators for free.

multi sub infix:</>(IO::Path:D \p is raw, Str:D \s) {
    p.add(s).resolve()
}

multi sub infix:</>(IO::Path:D \p is copy, List:D \l) {
    for l {
        p = p.add: .Str
    }

    p
}

my $p = '/home/dex/'.IO;
$p /= 'bar';
dd $p;
# OUTPUT: Path $p = IO::Path.new("/home/dex/bar", :SPEC(IO::Spec::Unix), :CWD("/"))

Having to chain method calls because the implementer was lazynot overly imaginative is a common theme. Since the default parent class is Any, any method added to that class should show up everywhere.

use MONKEY-TYPING;

augment class Any {
    method chain {
        class Chainer {
            has $.the-object;
            method FALLBACK($name, *@a) {
                my $o = $.the-object;
                for @a -> $e {
                    $o = $o."$name"($e, |%_);
                }

                $o
            }
        }

        Chainer.new(the-object => self)
    }
}

IO::Path.HOW.compose(IO::Path);

'/home/dex'.IO.chain.add(<tmp foo.txt>).say;
# OUTPUT: "/home/dex/tmp/foo.txt".IO

The method chain actually breaks the chain by returning an instance of a private class. This object knows the original object and will change any method call to a loop over the first positional. Named arguments are forwarded via the implicit argument %_.

You likely spotted both “should” and .HOW.compose. This is a long-standing issue. The MOP does keep a list of parent classes but not for children. So neither the compiler nor we can easily walk all type objects to recompose them. It’s a bit of a shame. There is likely much more that could be done in raku.land with a properly working augment.

Anonymous slurpers

I have a script where I’m only interested in the last two lines of its output. That’s easy to handle with Shell::Piping.

px«script.sh ./foo.txt» |» my @a;
my ($second-last-line, $last-line) = @a.tail(2);

That works but is wasteful because it stores lines of text in @a and keeps them around until the Array goes out of scope. Also, any temporary variable is a clear indicator of boilerplate. And we don’t do boilerplate in raku.land.

Declarators are quite powerful because they can take a list and immediately hand it over to infix:<=>. We can even skip values by using anonymous scalars.

my ($, $a, $b) = sub-that-returns-a-list();

I want to do something quite similar. Looking for the last two elements means to skip over any but the last two elements. In subscripts and signatures we use the Whatever * to indicate multiplicity. (Sometimes I don’t get English. Why is “manyness” not a word? o.O)

px«script.sh ./foo.txt» |» my (*, $second-last, $last);

That doesn’t quite work because Rakudo doesn’t expect the whatever star in a declarator list. In fact it doesn’t expect any term in that spot. We can work around that by being explicit. While we are on it we may add an anonymous scalar to the mix.

px«script.sh ./foo.txt» |» my (Whatever, $third-last, $, $last);

Declarators return a List of containers and values. We can use introspection to dissect it.

my \l := my (Whatever, $second-last, $, $last);
say l».&{ .WHAT, .VAR.WHAT, .VAR.name };
# OUTPUT: (((Whatever) Whatever anon) ((Any) Any $second-last) ((Any) Any anon) ((Any) Any $last))

By checking both for type objects and for type object and names in .VAR we can tell *, $ and normal containers apart.

sub infix:<|»>(\l, \r) {
    sub is-whatever($_ is raw) { (.VAR.name eq 'anon' && .WHAT === Whatever) but role :: { method gist { self ?? '*' !! '' } } }
    sub is-anon($_ is raw) { (.VAR.name eq 'anon' && .WHAT === Any) but role :: { method gist { self ?? 'anon' !! '' } } }
    sub is-scalar($_ is raw) { (.VAR ~~ Scalar && .WHAT !=== Whatever && .VAR.name ne 'anon') but role :: { method gist { self ?? 'Scalar' !! '' } } }

    sub is-left-slurpy(\l) { l.head.&is-whatever but role :: { method gist { self ?? 'left-slurpy: yes' !! 'left-slurpy: no' } } }
    sub is-right-slurpy(\l) { l.tail.&is-whatever but role :: { method gist { self ?? 'right-slurpy: yes' !! 'left-slurpy: no' } } }

    say r».&{ .&is-whatever, .&is-anon, .&is-scalar }
    say r.&is-left-slurpy;
    say r.&is-right-slurpy;
}

42 |» my (W, $, $c);
42 |» my ($d, $e, W);

# OUTPUT: ((*  ) ( anon ) (  Scalar))
          left-slurpy: yes
          left-slurpy: no
          ((  Scalar) (  Scalar) (*  ))
          left-slurpy: no
          right-slurpy: yes

Now I have everything I need to teach Shell::Piping to skip over values.

That Rakudo doesn’t allow the Whatever star in declarator lists feels like a bug.

my @List = my (42, "", Cool, $, $a);
dd @List;
Array @List = [Mu, Mu, Cool, Any, Any]

It is perfectly fine to skip over literals, take type objects and keep containers but for whatever reason it doesn’t like stars. However, what does work, works really well. It does provide us with containers that we can reason about via .VAR. Looks like Santa was onto something.

Introspective labeling

IRC is a good place to find answers. Often I find the questions found there to be even more enlightening.

12:26 < notandinus> do other languages have this Label thing? apart from Perl and Raku, i don't seem to find it for python, nim
12:28 < gfldex> notandinus: C/C++ does, but with different semantics (read: you can mess up the stack).
12:30 < sortiz> Javascript also, with semantics similar to Raku.
12:32 < El_Che> isn't label extremely commom to break out of the correct loop?
12:34 < El_Che> Besides python, all other languages probably have it
12:34 < sortiz> But only in Raku they are also objects, like almost everything.
12:35 < notandinus> is it being object better?

So are objects better? That depends on the needs and the attributes or methods provided. Let’s have a look what Label can do for us.

LABEL: Nil;
say LABEL.^attributes;
# OUTPUT: (Str $!name Str $!file Int $!line Str $!prematch Str $!postmatch)
          (new name goto leave Int next redo last gist BUILDALL)

So we get the label name, what means we can label things with them. There are also attributes with the line number and file providing a location for human consumption. With .Str and .gist we get some strings.

Label<94051339353632>
Label<LABEL>(at /home/dex/tmp/tmp2.raku:99, ' ~ self.line } });

⏏LABEL: Nil;

say LABEL.^a')

We do get the line number and file but in a piece of text that we don’t want to parse. Stringification is not helpful and there are no accessors for $!line and $!file. Not to worry, Lord Hanuman is with us.

use MONKEY-TYPING;
augment class Label {
    method Str { $!file ~ ':' ~ $!line }
}

sub documented {
    BEWARE: sub hax { 0x1ee7 }

    warn „Beware of the leet hax at {BEWARE} !“;
}

documented;
# OUTPUT: Beware of the leet hax at /home/dex/tmp/tmp2.raku:113 !
            in sub documented at /home/dex/tmp/tmp2.raku line 115

The class Label feels unfinished to me. With better .Str or access to all attributes it could be a nice tool to refer to spots in source code. This in turn could lead to better error messages and allow folk more creative then me to come up with something clever. I doubt it would be a hard change. I shall query Rakudo’s source code the coming week.

A hard simple thing

Bugs that originate in reliable subsystems are hard to spot because we don’t expect them to fail. A filesystem path usually just works. This has bit me a few times before. While renovating my backup script I found that I provide quite a few status and debug messages that basically call .gist on an IO::Path instance. Since one might want to use a hypothetical path, Raku can’t complain on non-existing paths until they are used. Further, it can be quite useful to know that a path is a symlink. Bash and friends help here with colour coding. Since I have plenty of say-calls, it would be nice to change them all in one go. So wraping it is!

my &RED = sub (*@s) { "\e[31m{@s.join('')}\e[0m" }
my &GREEN = sub (*@s) { "\e[32m{@s.join('')}\e[0m" }
my &CYAN = sub (*@s) { "\e[36m{@s.join('')}\e[0m" }

my &io-path-gist = IO::Path.^can('gist')[0];

&io-path-gist = &io-path-gist.wrap(my method gist (Mu: --> Str ) {
    nextsame unless self ~~ IO::Path;

    my $s = self.Str;
    $s = self.e && self.r ?? $s !! RED($s);
    $s = self.l ?? CYAN($s) !! $s;
    $s = self.x && !self.d ?? GREEN($s) !! $s;
    $s = self.d ?? $s ~ ‚/‘ !! $s;

    "⟨$s⟩"
}).&{ -> { .restore } };

We get a handle on the proto of .gist because .wrap never operates on multi candidates (this might change, see here). I’m using a method with Mu as the invocant for clarity. A sub with a single positional would do as well. Since the wrapper is called by the proto, we need to redispatch on anything but IO::Path. The returned WrapHandle isn’t really useful and neither is the unwrapped handle to .gist. By storing a closure we can restore the default by calling io-path-gist(). The rest is some colouring and adding a / to mark directories. I like to put a path into fancy quotes to make it easy to spot leading and trailing white spaces.

Declaring class-methods and calling them is a hard thing that is easy because the implementer was sufficiently tormented. Getting hold of methods at runtime is a simple thing that is hard. Huffmanizing applies. We call methods much more often then wrapping them. Sometimes it is good to break a design principle.

Internal indirection

With writing more and more shell scripts in Raku, I realised that I call a MAIN by a MAIN in a very indirect manner. I wondered if I can find a way to reduce the indirection to get rid of the extra process and at least some of the overhead of Proc::Async. First we need a script to call.

#! /usr/bin/env raku

constant \frame = gather while my $frame = callframe($++) {
    take $frame
}

sub MAIN {
    say frame[0];

    .note for lines;

    exit 42;

    say 'alive';
}

This script prints its caller. Then reads blocking indirectly on $*IN via lines. It would print alive if the process would make it past the exit. Please note, that MAIN is the last statement in this compunit. That allows us to slurp the script and EVAL it.

my &main := $path.IO.slurp.&EVAL;

And that’s it! We can call a MAIN from another script. There are a few considerations though. We can pipe data to from one script to another because they share $*IN and $*OUT. Also, STDERR might get confusing. The semantics of exit may be wrong. It is likely that we want to keep the outer script running. Capturing the return-value of the inner script is easy, the exitcode is not.

Since we are calling functions lexicals can help with solving most of those problems. In Raku we can’t easily trap c-land exit. But we can prevent it from being called.

my &*EXIT = sub ($exitcode) {
    CapturedExitcode.new(:$exitcode).throw;
}

Here the exitcode is packaged in an exception so we can extract it from MAIN. This sub might exit with a return so we have to capture that one too.

        $out.exitcode = main();

        CATCH {
            when CapturedExitcode {
                $out.exitcode = .exitcode;
            }

            default {
                say .^name, ': ', .message;
                put .backtrace».Str
            }
        }

Now we need to deal with $*IN and $*OUT. Since the target script just calles lines and that forwards to $*ARGFILES.lines we can use a Channel. One of the Channels is a good place to store the exitcode.

    my $out = Channel.new but role :: { has $.exitcode is rw; };
    my $in = Channel.new but role :: { method put(Any:D \value){ self.send: value ~ $?NL } };

Since lines requires a Str we provide the familiar put-method. Other methods like say would go into the anonymous role too. When the inner MAIN terminates, we want to close the $out channel. We can do so in a LEAVE block. The whole thing can be wrapped into a sub which provides the outer script with a nice interface.

my ($out, $in) = embed-script('./script.raku');

start {
    for ^10 {
        $in.put: $++;
    }
    $in.close;
}

.print for $out.list;

say $out.exitcode;

Dealing with a multi sub MAIN is tricky. If the last statement in the script is &MAIN, it will refer to the dispatcher. With any multi candidate at the end, we can only get hold of the proto by descending into nqp-land.

my &main := $path.IO.slurp.&EVAL;

use nqp;
my &disp = &main.multi ?? nqp::getattr(&main, Routine, '$!dispatcher') !! &main;

We can then call disp() to dispatch to the correct MAIN candidate. I’m not sure if this is fragile. Every routine that is a multi got a dispatcher. Yet Routine.dispatcher is not exposed by CORE.

The whole example can be found here.

Avoiding Proc::Async does speed things up quite a bit. Since we can use a Channel, we don’t have to stringify and parse output when moving data around. The called MAIN needs to cooperate in this case and thus needs to know it is not called by the runtime. We could introduce a new lexical or check $*IN against Channel. There is also the option to check the callframe for EVAL.

constant \frame = gather while my $frame = callframe($++) {
    take $frame
}

say 'indirect' if any frame».gist».starts-with('EVAL');

Quite in contrast to Perl 5, for Raku I never used EVAL much. Not because I’m scared — there was little reason for code generation. After all, between the two of us I have always been the more evil twin.

UPDATE:

As jnthn pointed out, Routine.dispatcher is exposed. We can therefore keep well clear of nqp with:

my &disp = &main.multi ?? &main.dispatcher() !! &main;

As of now, it is unclear if this should be specced and thus documented.