Calling by name

While looking for something completely different, I found that Roast really likes named callable placeholder variables.

dex@dexhome:~/projects/raku/roast$ ack -l  '&\:'
S02-names/SETTING-6c.t
S02-names/SETTING-6e.t
S02-names/pseudo-6c.t
S02-names/pseudo-6d.t
S02-names/pseudo-6e.t
S02-names/symbolic-deref.t
S06-operator-overloading/infix.t
S06-signature/closure-parameters.t
S32-num/rounders.t
integration/advent2013-day10.t
integration/weird-errors.t

That is hardly a surprise, given that the specs are composed of short snippets of code. Placeholder variables fit right in!

sub foo { "Good &:greeting(now.DateTime.hour) $:name!" };
say foo :name<Paul>, :greeting{$_ < 12 ?? 'morning' !! 'day'};
say &foo.signature;

# OUTPUT: Good day Paul!
          (:&greeting!, :$name!)

It’s really nice how interpolation and placeholders work together here. I have reported the ENODOC.

Even after years and years of being a Raku beginner, I still find new stuff when digging deep enough. Maybe it’s a good thing that we don’t have a Raku book. It would surely be backbreaking.

Dynamic declaration

Shortly after my last blog post, Stashes raised a question. Coincidence? Conspiracy? You decide! Anyway, the EVAL caught my eye, because with it we can dynamically create compile time constructs such as a package.

our package EXPORTHOW {
}

sub EXPORT($declarator-name = 'registered') {
    use MONKEY-SEE-NO-EVAL;
    OUR::EXPORTHOW::DECLARE := EVAL q:s:to /EOH/;
        package DECLARE {
            constant $declarator-name = MetamodelX::RegisteredHOW;
        }
        EOH

    Map.new
}

Thanks to be our-scoped, the package EXPORTHOW can be modified at runtime. The EVAL allows to define a constant at runtime. Runtime in this context, is when the use statement of a consuming module is executed.

use Registered 'remembered', :recall-types;

remembered Foo {
    method answer { 42 }
    method common { self.^name }
}

With this technique, we allow the user of a module to decide what symbols are being used for a declarator. Pretty handy, if a module is added late to a project, which might have occupied a given symbol already. Quite some time ago, I lamented a little lizmats decision to modify class with InterceptAllMethods. This is now a solvable problem.

I once believed Raku to be less dynamic then Perl. Looks like I have to reconsider.

Most fancy

On Discord (yes, we are that cool) MrDispatch wished for a way to collect a bunch of classes right after their declaration. I believe, with the power of the MOP, we can go a step further and register the type-object even before the definition is finished.

class MetamodelX::RegisteredHOW is Metamodel::ClassHOW {
    our @registered;

    method new_type(|) {
        my \type = callsame;
        type.HOW.remember_type(type);

        type
    }

    method remember_type(Mu \type) {
        @registered.push: type
    }

    method recall-types {
        @registered;
    }
}

sub recall-types is export(:recall-types) {
    MetamodelX::RegisteredHOW.recall-types;
}

my package EXPORTHOW {
    package DECLARE {
        constant registered = MetamodelX::RegisteredHOW;
    }
}

We introduce a new declarator registered that hooks up a new meta-class. It would be possible to overload class to register all classes in a compilation unit. For now, doing so would not be good conduct, because playing with EXPORTHOW is not quite lexical yet. The list of declared type objects will reside as a class-attribute inside the meta-class. We can access it through any type object or instance via .HOW or with the exported sub recall-types.

use v6.*;

use Registered :recall-types;

registered Foo {
    method answer { 42 }
    method common { self.^name }
}

registered Bar {
    method ohai { ‚Hello Universe!‘ }
    method common { self.^name }
}

Foo.HOW.recall-types.say;
say Foo.new.HOW.recall-types;
say recall-types; # requires the import adverb :recall-types;
say recall-types».common;

# OUTPUT: [(Foo) (Bar)]
#         [(Foo) (Bar)]
#         [(Foo) (Bar)]
#         [Foo Bar]

There are plenty of other ways to implement a registry of types. Using a custom meta-class is the most fancy and flexible way. We could hook into many other things here too. Rakudo is a dynamic compiler for a dynamic language. It uses introspection much more then you do. Sometimes it introspects a package called EXPORTHOW for a key called DECLARE to create a keyword that is more classy then class.

Inequality

As stated before, I like to read code I didn’t write myself. Flavio had trouble with triples. One line stood out to me.

take @triple if $N == @triple.any;

This looks like a set-operation to me. But using $N ∈ @triple dropped one result. After some debugging I found the culprit.

.map({($_, $n / $_)});  # take it and its counterpart

This might look like a division but is actually a type cast to Rat.

say 1 ∈ (1/1, );
# OUTPUT: False

Rakudo implements set-operations as equivalence checks, not numerical equality. Quite in contrast to ==, eqv does a type check and Int aint’t Rat. This might explain that the mathematical inclined don’t use Set as much as I did expect them to. The type mismatch simply produces a result that is not useful to mathematicians.

As implemented right now, we can’t tell the set operators what we consider equality . With meta-operators and junctions, we can specify what operator we actually want to use. The set-operators are not meta-operators and at least for now, we can’t user define new meta-operators. However, we can lexically redefine ordinary operators.

proto sub infix:<(elem)>($, $, *% --> Bool:D) is pure {*}
multi sub infix:<(elem)>(Numeric:D \a, Iterable:D \listy --> Bool:D) {
    Any.fail-iterator-cannot-be-lazy('∈', '') if listy.is-lazy;

    for listy -> \b {
        return True if a == b;
    }

    False
}

constant &infix:<∈> := &infix:<(elem)>;

proto sub infix:<(&)>(|) is pure {*}
multi sub infix:<(&)>(Iterable:D \lhs, Iterable:D \rhs) {
    Any.fail-iterator-cannot-be-lazy('∩', '') if lhs.is-lazy || rhs.is-lazy;

    my @result;

    for lhs -> \l {
        for rhs -> \r {
            @result.push: r if l == r;
        }
    }

    +@result ?? @result !! ∅
}

constant &infix:<∩> := &infix:<(&)>;

say 1 ∈ (1/1, );
say (42, 42/2, 42/3) ∩ (1, 21, 3);
# OUTPUT: True
#         Set(21)

The proto is needed, because we need to get rid of multi-candidates that are already present. A more general approach might be to have a &*SET-COMPARATOR that defaults to &infix:<cmp>. This would slow down a fairly simple operator by a factor of 13. I may still be able to write a module that takes a comparator and returns a host of operators. With proper macros this would be easy. Maybe next year.

For now I shall be evil and report back with success.

They returned an empty package

I don’t like to solve maths-puzzles. I do like to read other folks solutions thought. You never know where to spot a new idiom. A good way to find them is to look for code that feels unusual.

method normalize (Numeric:D $sum = 1) {
  my $total = self.total or return;
  my $factor = $sum / $total;
  %!pmf.values »*=» $factor;
  self;
}

I have never seen the construct in the first line, at least not in a method. The return is triggered when self.total returns something falsesy, like 0. It protects the 2nd line from a division by zero by returning Nil. Let’s see if that actually works.

$cookie.multiply('Bowl 1', 0);
$cookie.multiply('Bowl 2', 0);
say 'probability it came from Bowl 1: ', $cookie.P('Bowl 1');

# OUTPUT: Attempt to divide by zero when coercing Rational to Str
            in sub MAIN at tbr-pmf.rakumod line 73
            in block <unit> at tbr-pmf.rakumod line 3

Well, it does blow up some place else. This is not an unreasonable scenario either. When I’m around, the likelihood of a cookie to come from bowl1 or bowl2 is indeed 0. Returning to normalize we can check what happens when a method that should return self returns Nil.

class C {
    method foo { Nil }
}

dd C.new.foo.bar;
# OUTPUT: Nil

The cause of this mis-dispatch can be found in src/core.c/Nil.pm6:16:

method FALLBACK(| --> Nil) { }

Nil is the baseclass of Failure and as such will only throw when assigned to or used to gain values from a list. It’s purpose is to revert containers to their default value. In a numeric context it will warn and turn into 0. Eventually it might end up in a division and cause much grief. Dealing with a depressing lack of cookies could be done with a multi-method.

   multi method P ($key where { self.total == 0 } ) {
        0
   }
   multi method P ($key) {
      die "no key '$key' in PMF" unless %!pmf{$key}:exists;
      return %!pmf{$key} / self.total;
   }

This works because self is an implicit part of the signature of a method (sans an explicit invocant).

Another thing I never seen before is the following gist method.

method gist () {
  return gather {
    take '---';
    for %!pmf.keys.sort -> $key {
      take "  «$key» {%!pmf{$key}}";
    }
  }.join("\n");
}

Using take to prefix the resulting list is quite neat. Using a gather-block avoids any nesting that might need flattening afterwards. In a gist-method it’s a bit wasteful though. We like to truncate lists after 100 elements in say. This could be done with a .head(100) after .keys. Or we skip the rather slow gather-block altogether (each take fires a control exception and Rakudo can not optimise those away yet).

method gist () {
  ( |'---', |%!pmf.keys.sort.map: { "  «$_» %!pmf{$_}" } ).head(100).join($?NL)
}

I like to avoid explicit flattening by using Slips. If the map-block gets complicated, this allows to sneak a .hyper in to gain some speed on large lists.

Please keep in mind that cookies are nice and Nil is shifty.

Only infinite elements

Flavio solved a puzzle with an implementation that puzzled me. The following code appears not as idiomatic as such a simple problem should require.

sub simulation-round () {
   return [+] gather {
      loop {
         my $value = roll-die();
         take $value;
         last if $value < 3;
      }
   }
}

We tend to do our dice rolls with a simpler construct.

sub simulation-round () {
    [+] (1..6).roll(*).map({ last .Int if .Int < 3; .Int });
}

# OUTPUT: Cannot .sum a lazy list onto a Seq
#           in sub simulation-round at ETOOBUSY-2.raku line 4
#           in block <unit> at ETOOBUSY-2.raku line 7

Rakudo assumes that a lazy list must be infinite. This would catch many bugs but is not what I want in this case. Sadly we don’t have a build-in to say .lazy-but-finite. Neither do we got roles we could mixin. This might be the reason why it took me 10 minutes of staring‑at‑the‑code to find a solution.

[+] (1..6).roll(*).map({ last if .Int < 3; .Int })[^∞]

So we are asking for up to infinite elements of this infinite list to defuse the laziness. This smells of bad design and may warrant a problem solving issue.

The rest of my version is quite different because I wanted to shoehorn a .hyper in.

use v6.*;

unit sub MAIN($rounds = 1_000_000);

sub term:<🎲>() { (1..6).roll(*) };

sub simulation-round () {
    [+] 🎲.map({ last .Int if .Int < 3; .Int })[^∞];
}

my $total = (0..^$rounds).hyper(:degree(12), :batch(10000)).map({ simulation-round }).sum;

put ‚average gain: ‘, $total / $rounds;

I believe there is a life‑lesson to be learned. Laziness can be problematic if the compiler is less then virtuos.

UPDATE:

As lizmat pointed out, last .Int requires v6.e.PREVIEW (v6.* will work too). Instead of [^∞] a simple .eager will do (but will break the title of the blog post). We can also use ^$rounds instead of (0..^$rounds) if we disambiguate with a space before the method call.

my $total = ^$rounds .hyper(:degree(12), :batch(10000)).map({ simulation-round }).sum;

Contextual addition

I took the absence of complaint as silent consent and set forth to implement HyperWhatever in associative subscripts. To do so I setup a little bit of tooling to lower cognitive load. When writing code I like to hit F1 in Vim and have it do the right thing depending on context. Since Vim is not self aware yet, we have to tell it what to do to help us. To specify context we can add a line to a source file to define a pseudo filetype.

use v6.*;
use Test;

# tests for the changes to Rakudo go here

# vim: ft=rakudotest

We can then define a filetype based mapping in ~/.vimrc.

autocmd FileType rakudotest nmap <F1> :w<CR>:!./testcase %<CR>

Since I change Rakudo I need to run a local copy by forking on Github and cloning into a fresh directory. There I can place a small shellscript.

#! /bin/sh

test 0 -lt $(find src/ -newer install/bin/raku -iname '*.pm6' | wc -l)\
&& make clean all test install

install/bin/raku $1

With this little chain I can edit Rakudo files and then hit F1 in the file with the tests. Rakudo will be rebuild and the test file executed with the local Rakudo instance. The latter needs to be prepared with perl ./Configure.pl --gen-moar --gen-nqp.

Since I wanted to put the new operators into v6.e I had to hunt down the spots where Rakudo needs the information what to put where. The new operator candidate goes into its own file in src/core.e/ which then has to be made known in tools/templates/6.e/core_sources. There are sanity test to avoid features to bleed from one version into another in t/02-rakudo/03-corekeys.t. This provided me with some bafflement, as institutional knowledge tends to do. I was mostly guided by error messages, which shows how important LTA is.

A few days ago we had a discussion about the one argument rule, where I claimed to have little difficulties with it. While implementing %your-hash{**}:deepkv I had to change my mind. The reason why the rule doesn’t bite me in practical code, is actually rooted in good testing.

    my $seen-J;
    my $seen-E;
    for %hash{**}:deepkv -> @deepkey, $value {
        $seen-J++ if @deepkey ~~ <J> && $value == 7;
        $seen-E++ if @deepkey ~~ <A D E> && $value == 3;
    }
    is $seen-J, 1, 'seen leaf in {**}:deepkv';
    is $seen-E, 1, 'seen deep leaf in {**}:deepkv';

Here the destructuring of the return value of %hash{**}:deepkv only works when the operator returns exactly the right thing.

multi sub postcircumfix:<{ }>( \SELF, HyperWhatever, :$deepkv!, *%other ) is raw {
    sub recurse(\v, @keys){
        if v ~~ Associative {
            for v.kv -> \k, \v {
                recurse v, [@keys.Slip, slip k]
            }
        }else{
            take slip(@keys, v)
        }
    }

    gather for SELF.kv -> \k, \v {
        if v ~~ Associative {
            recurse(v, [k])
        } else {
            take slip([k], v)
        }
    };
}

Getting the right amount of Slip into the right places took half an hour (building Rakudo takes about 60s) and plenty of cursing. I hope this is a case of torturing the implementer on behalf of the user. When using test driven code that seems to happen automatically. I’m to lazy too write a fancy test, so I have the testee to be clever enough to satisfy it.

Since I couldn’t find any documentation on how to add features in specific language versions, I hope this to be helpful to those who seek the same.

The next fast thing

A few commits ago lizmat taught next to take an argument. I started to play with this and found that not all loops are created equal.

sub prefix:<♥>(&c) {
    LEAVE say (now - ENTER now) ~ 's';  # Don't you ♥ Raku?
    c
}

♥ { say sum gather for ^1_000_000 { .take if .is-prime; } } # 1.399831818s
♥ { say sum eager for ^1_000_000 { .&next if .is-prime; } } # 1.131352526s
♥ { say sum do for ^1_000_000 { .&next if .is-prime; } }    # 1.60557427s
♥ { say sum (^1_000_000).grep: *.is-prime; }             # 0.778440528s

I’m surprised that the do for-form is the slowest. That gather is slower leaves the question if a for-loop is the better way to create a lazy list.

my \a = lazy gather for ^100_000_000 { .take if .is-prime; }
my \b = lazy for ^100_000_000 { .&next if .is-prime; }
♥ { say sum a[^1_000_000]; } # 25.91494395s
♥ { say sum b[^1_000_000]; } # 26.521749639s

Optimising is an art of doing the same thing over and over again. There seems to be room for optimising things that do things over and over again.

Typed filters

The Discord Raku bot is now also an IRC -> Discord bridge. To handle the streams of messages I use a react-block with a few whenevers. I would like to handle filtering of debug output from API::Discord in there as well, without disrupting something simple like printing to the terminal.

In my last post I showed how I can control the behaviour of a module with the use statement. The next step is to divert writes to $*ERR to a Supply.

my $active = False;

sub debug-print(|c) {
    $*ERR.print: |c if $active;
}

sub debug-print-supply(Supply $in? --> Supply:D) {
    my $result = $in // Supplier::Preserving.new;

    &debug-print.wrap(-> |c {
        $result.emit: |c
    });

    $result.Supply
}

multi sub EXPORT('FROM-MODULE') {
    %(
        '&debug-print' => &debug-print,
    )
}

multi sub EXPORT() {
    $active = True;

    %(
        '&debug-print' => &debug-print-supply,
    )
}

As long as the user of the module doesn’t call debug-print, we have the simple case of writing to $*ERR. When debug-print is called, we divert the stream into a Supply. If no Supply is supplied, we create one. To be able to filter output a role is created in API::Discord::Debug.

role LogEventType is export {}

This role is indeed empty and so are the roles that are created to provide filters.

unit module Testmodule;

use API::Discord::Debug <FROM-MODULE>;

role HEARTBEAT does LogEventType is export {}
role PING does HEARTBEAT does LogEventType is export {}

start loop {
    sleep ¼;
    debug-print("ping" but PING);
    debug-print("pong" but PONG) unless 10.rand < 2;
}

So we create a Str and mixin a role, what is a type object we can check against.

use API::Discord::Debug;
use Testmodule;

react {
    my  $pings = 0;

    whenever debug-say().merge(debug-print()) {
        when PING { $pings++; }
        when PONG { $pings--; }
        default { .note }
    }

    whenever Supply.interval(5) {
        say „$pings heartbeats are lost“ if $pings > 1;
    }
}

Mixing in an empty role wont upset any unsuspecting bystander and type checks compose well. Here I use them with when/default, a multi dispatch would work as well. Introspection can be done with .WHAT.say.

On CPAN we can find many “complete” modules that kill a specific problem once and for all. For many of those, there tends to be another module suffixed with ::Simple. Simple stuff should be simple because we can’t buy any more days in this life. We also don’t take anything with us. I hope I can leave something simple behind instead.

Being pragmat-ish

The question was raised if one can provide a custom pragma. As it happens, today I needed just that. API::Discord is a bit chatty. It is outputting debug and status info right to the terminal. Since I got the Discord bot now also being an IRC bot, the clutter got a bit much. My first thought was to wrap note and warn to filter messages out I don’t want. But there is also $*ERR.print in the mix. So I went and meta6 --fork-module=API::Discord.

The goal is to use API::Discord::Debug; to change the behaviour of the module. That is what a pragma does. It changes the way how the compiler or the runtime work. I also want two subs to make the whole thing .wrap-able, to allow feeding the debug output into a Supply.

use v6.d;

my $active = False;

sub debug-print(|c) {
    $*ERR.print: |c if $active;
}

sub debug-say(|c) {
    $*ERR.say: |c if $active;
}

multi sub EXPORT('FROM-MODULE') {
    %(
        '&debug-print' => &debug-print,
        '&debug-say' => &debug-say,
    )
}

multi sub EXPORT() {
    $active = True;
    %()
}

The trick is to have to multi sub EXPORT so we can do use API::Discord::Debug to switch debugging output on and use API::Discord::Debug <FROM-MODULE> to get two functions exported to be called in the various .rakumod-files of the distribution.

We can’t really define custom pragmas. But use is kindly calling a function we can define, to get at least some of the behaviour we need.