my $*RAKU++ for -> **@ {};

As the title states, I made Raku bigger because lol context (that’s how the Synopsis is calling **@) makes supporting feed operators fairly easy. I wonder if Larry added this syntax to Signature with that goal in mind. With PR#5532 the following becomes possible.

<abc bbc cbc> ==> trans('a' => 'x', 'b' => 'i') ==> say();
# OUTPUT: (xic iic cic)

Armed with this I can make a script of mine a little simpler.

use MONKEY-TYPING;

augment class IO::Path {
    method trans(|c) {
        my $from = self.Str;
        my $to = self.Str.trans(|c);

        self.rename($to) unless $from eq $to
    }
}

sub rename-whitespace(IO::Path $dir where *.d){
    dir($dir).grep({ .d || .f && .rw })
        ==> trans("\c[space]" => "\c[no-break space]", "\c[apostrophe]" => "\c[prime]")
        ==> sub (*@a) { print '.' for @a}();

    dir($dir).grep({ .d && .rw })».&?ROUTINE;
}

rename-whitespace('.'.IO);

put '';

I don’t like spaces in filenames, as they are often found with audio or video files. Having auto-complete friendly names makes using a CLI less bumpy. By teaching IO::Path to rename files by providing rules, as they are understood by Str.trans, I can use a feed operator to get the job done. (I wouldn’t be surprised to learn, that anonymous subs DWIM here to be emergent behaviour in Raku.)

Having another PR that adds .trans to IO::Path is tempting but requires more thought.

Autoarraying

Over on Reddit zeekar wasn’t too happy about Raku’s love of Seq. It’s immutability can be hindering indeed.

my @nums = [ [1..10], ];
@nums[0] .= grep: * % 2;
@nums[0].push(11); # We can't push to a Seq.

I provided a solution I wasn’t happy with. It doesn’t DWIM and is anything but elegant. So while heavily digesting on my sofa (it is this time of the year), the problem kept rolling around in my head. At first I wanted to wrap Array.grep(), but that would be rather intrusive and likely break Rakudo itself. After quite a bit of thinking, I ended up with the question. How can I have indexable container (aka Array) that will turn each value on assignment into an (sub-)Array?

my Array() @foo = [ 1..10, ];
dd @foo;
# Array[Array(Any)] @foo = Array[Array(Any)].new($[1, 2, 3, 4, 5, 6, 7, 8, 9, 10])
@foo[0] .= grep: * % 2;
@foo[1] = 42;
dd @foo;
# Array[Array(Any)] @foo = Array[Array(Any)].new($[1, 3, 5, 7, 9], $[42])

The answer is obvious. By telling the compiler what I want! Coersion-types have become really hard to distinguish from magic.

I wish you all a Merry Christmas and the very best questions for 2024.

Missing Virtuousness

According to Larry, laziness is a programmers virtue. The best way to be lazy is having somebody else do it. By my request, SmokeMachine kindly did so. This is not fair. We both should have been lazy and offload the burden to the CORE-team.

Please consider the following code.

my @many-things = (1..10).List;
sub doing-one-thing-at-a-time($foo) { ... }
say doing-one-thing-at-a-time(@many-things.all);

Rakudo goes out of it’s way to create the illusion that sub doing-one-thing-at-a-time can deal with a Junction. It can’t, the dispatcher does all the work of running code in parallel. There are tricks we can play to untangle a Junction, but there is no guarantee that all values are produced. Junctions are allowed to short-circuit.

This was bouncing around in my head for quite some time, until it collided with my thoughts about Range. We may be handling HyperSeq and RaceSeq wrong.

my @many-things = (1..10).List;
sub doing-one-thing-at-a-time($foo) { ... }
say doing-one-thing-at-a-time(@many-tings.hyper(:degree<10>));

As with Junctions doing dispatch-magic to make hyper/race just work, moving the handling to the dispatcher would move the decision from the callee to the caller and, as such, from the author of a module to the user. We can do that by hand already with .hyper.grep(*.foo) or other forms of boilerplate. In Raku-land we should be able to do better and provide a generalisation of transforming calls with the help of the dispatcher.

I now know what to ask Santa for this year.

Incomplete Ranges

Lately, some unhappiness has popped up about Range and it’s incomplete numericaliness. Having just one blogpost about it is clearly not enough, given how big Ranges can be.

say (-∞..∞).elems;
# Cannot .elems a lazy list
  in block <unit> at tmp/2021-03-08.raku line 2629

I don’t quite agree with Rakudo here. There are clearly ∞ elements in that lazy list. This could very well be special-cased.

The argument has been made, that many operators in Raku tell you what type the returned value will have. Is that so? (This question is always silly or unnecessary.)

say (1 + 2&3).WHAT;
# (Junction)

Granted, Junction is quite special. But so are Ranges. Yet, Raku covers the former everywhere but the latter feels uncompleted. Please consider the following code.

multi sub infix:<±>(Numeric \n, Numeric \variance --> Range) {
    (n - variance) .. (n + variance)
}

say 2.6 > 2 ± 0.5;
# True

my @heavy-or-light = 25.6, 50.3, 75.4, 88.8;

@heavy-or-light.map({ $_ ≤ 75 ± 0.5 ?? „$_ is light“ !! „$_ is heavy“ }).say;
# (25.6 is heavy 50.3 is heavy 75.4 is heavy 88.8 is heavy)

To me that looks like it should DWIM. It doesn’t, because &infix:«≤» defaults to coercing to Real and then comparing numerically.

This could easily be fixed by adding a few more multis and I don’t think it would break any production code. We already provide quite a few good tools for scientists. And those scientists do love their error bars — which are ranges. I would love for them to have another reason to use Raku over … that other language.

Spawnratelimiting

IRC keeps insisting on being a good source of thought. A few days ago rcmlz wished to parallelise quicksort. Raku doesn’t sport a facility to make recursion “just work” with oodles of cores. Quicksort is a little special in that regard, as it has a double tail-call.

quicksort(@before), $pivot, quicksort(@after)

Just wrapping the two calls into start-block will spawn way to many threads. Ergo, I need a function that takes another function and it’s arguments and limits the use of start-blocks.

my atomicint $spawn-limit = 8;
sub spawn(&c, |args) {
    # dd ⚛$spawn-limit;
    ⚛$spawn-limit > 0
        ?? ($spawn-limit⚛-- ;start { LEAVE $spawn-limit⚛++; c(|args) })
        !! c(|args)
}

This Routine will either return whatever c returns or a Promise. I need a way to convert the latter to values.

sub collect(*@things) {
   slip @things.map: -> $thing { $thing ~~ Promise ?? await $thing !! $thing }
}

Now I can change quicksort without getting into trouble with oversharing objects.

multi quicksort([]) { () }
multi quicksort([$pivot, *@rest]) {
    my @before = @rest.grep(* before $pivot);
    my @after  = @rest.grep(* !before $pivot);

    flat collect spawn(&?ROUTINE.dispatcher, @before), $pivot, spawn(&?ROUTINE.dispatcher, @after)
}

my @a = ('a'..'z').roll(12).join xx 2**16;
say quicksort @a;

# OUTPUT:
# Flattened array has 65536 elements, but argument lists are limited to 65535
#   in sub quicksort at tmp/2021-03-08.raku line 2601
#   in block <unit> at tmp/2021-03-08.raku line 2611

Well, I could if Rakudo would be production ready. This bug is old and awful. Test driven programmers like to program test-driven but don’t fancy to wait for hours to see tests complete. As a result very few test with large datasets. (And 65536 elements is not much these days.) It’s fairly easy to start with a project in production that slowly grows it’s database and eventually fails with a runtime error.

At least for now, destructuring is best done by hand.

Dodging segfaults

While fighting with NativeCall to get github-flavoured-markdown to work, I stepped onto a nice idiom. As many C-libs, cmark-gfm does enjoy enums that hide bitmasks. They have to do that because C doesn’t sport named arguments. Raku does and hence a nice interface would not ask for binary operations on the caller side.

markdown-gfm-to-html('# Heading 1', :UNSAFE, :HARDBREAKS, :FULL_INFO_STRING);

Now I need a way to turn the named arguments into a list so I can use that as keys in of enum. A Capture in the signature of the sub will do the trick.

multi sub markdown-gfm-to-html(Str:D $str, |opts ( :$DEFAULT, :$SOURCEPOS, :$HARDBREAKS, :$SAFE, :$UNSAFE, :$NOBREAKS, :$NORMALIZE, :$VALIDATE_UTF8, :$SMART, :$GITHUB_PRE_LANG, :$LIBERAL_HTML_TAG, :$FOOTNOTES, :$STRIKETHROUGH_DOUBLE_TILDE, :$TABLE_PREFER_STYLE_ATTRIBUTES, :$FULL_INFO_STRING ) --> Str:D) {
    enum CMARK_OPTIONS (
        DEFAULT => 0,
        SOURCEPOS =>  1 +< 1,
        HARDBREAKS  => 1 +< 2,
        SAFE => 1 +< 3,
        UNSAFE => 1 +< 17,
        NOBREAKS => 1 +< 4,
        NORMALIZE => 1 +< 8,
        VALIDATE_UTF8 => 1 +< 9 ,
        SMART => 1 +< 10,
        GITHUB_PRE_LANG => 1 +< 11,
        LIBERAL_HTML_TAG => 1 +< 12,
        FOOTNOTES => 1 +< 13,
        STRIKETHROUGH_DOUBLE_TILDE => 1 +< 14,
        TABLE_PREFER_STYLE_ATTRIBUTES => 1 +< 15,
        FULL_INFO_STRING => 1 +< 16
    );

    my $opts = [+|] CMARK_OPTIONS::{opts.hash.keys}».Numeric;

    cmark_markdown_to_html($str, $str.encode('utf-8').bytes, $opts);
}

Raku allows us to store part of a Signature in a Capture. The latter will be transparent for the caller of the Callable. Having to use a hyper-method-call-operator isn’t that great, so the enum could become a Map to simplify things a bit.

If you wish to use NativeCall, please do! But think of the generations to come that will use your module. Hopefully, they will like C less then our generation does. Segmentation faults are not -Ofun.

Parenthood

This blogpost is written in memory of Bram Moolenaar — a true hero of text.

Greg Donald reminded me of an old itch of mine. Any object does refer to its base-class but the other way is omitted. So the objective is to teach type objects that are MetamodelX::ClassHOW to know their children.

First I need a place to store the list of sub-classes. Some introspectiveness is always welcome, so a role is it.

role Seepy {
    has @!children;

    method add_child(Mu $obj, Mu $child) {
        @!children.push: $child
    }
    method children(Mu $obj) {
        @!children.List
    }
}

Now I need to teach each child-class to announce its arrival to the parent. This can be done by sub-classing Metamodel::ClassHOW and overloading method add_parent.

class MetamodelX::SeepyHOW is Metamodel::ClassHOW {
    also does Seepy;

    method add_parent(Mu $obj, Mu $parent, :$hides) {
        $parent.^add_child($obj) if $parent.HOW ~~ Seepy;

        callsame
    }
}

I can export my new class-declarator in a composable fashion as explained previously.

our package EXPORTHOW {
}

sub EXPORT($declarator-name = 'registered') {
    use MONKEY-SEE-NO-EVAL;
    OUR::EXPORTHOW::DECLARE := EVAL q:s:to /EOH/;
    package DECLARE {
        constant $declarator-name = MetamodelX::SeepyHOW;
    }
    EOH

    Map.new
}

Let’s do some testing, source can be found here.

use v6.d;
use lib '.';
use seepy 'seepy';

seepy Root { }

seepy Base is Root {

}

seepy Child is Base {

}

seepy Base2 is Root { }
seepy Child2 is Base2 {

}

my $c1 = Child.new;
dd $c1.^parents[0].^children;

my $c2 = Child2.new;
dd $c2.^parents[0].^children;

sub walk-tree(Mu:U $node, &c:(Int $indent, Mu:U $node), $indent? = 0) {
    c($indent, $node);
    for $node.^children -> $child {
        walk-tree($child, &c, $indent + 1);
    }
}

walk-tree(Root, -> Int $indent, Mu:U $node {
    say $node.^name.indent($indent * 2);
});

# OUTPUT:
# (Child,)
# (Child2,)
# Root
#   Base
#     Child
#   Base2
#     Child2

By having inheritance seep through to a root-object, it has become trivial to plot a type tree. Now I can patch Rakudo to do the same to get a full dump of all types defined in Raku.

I take issue with enshittification

With great amusement I watch the discussion of worries about — let’s call them “changes” — of reddit on reddit. The OP wishes Usenet to be back. And yes, we traded robust and decentralised services for “platforms” that require less clicks to reach them. Don’t get me wrong. I don’t wish the good old days back when we had a need for mirrors, because interruptions of the intertubes where quite common. However, I do miss the ability to access documentation without the need of dozens and dozens of machines to all work at the same time. Is docs.raku.org DDoS-safe? I hope so, because running that site on localhost is not exactly trivial.

I like to query the issues for Rakudo and the Problem Solving repo quite often. The latter basically consists of issues. Being on github a simple git clone with the right URL should suffice, right? Well, as it turns out Github is a vendor and issues are the lock-in. You can move issues from one repo to another but there is no export-button. There is an API to fetch them — one by one. If they would not have an API, we would have to use a scraper, increasing the load on that companies servers. Go figure.

While trying to secure Rakudos issues I got the following result:

Message: API rate limit exceeded for user ID 426613., Locations: []

That’s about ⅔ in. I do cache individual issues locally, so in an hour I should have them all. In case you want to do the same here is how far I got within a day. It needs more work and there are external dependencies. Sadly, I have to shell-out to cmark, because none of the modules on raku.land can handle Markdown reliably. That slows things down. Mostly, because hyper and Proc::Async don’t mix too well. The script needs a little more work and plenty of testing, before I can publish it in earnest. I hope the coming week will be sufficient.

Enshittification is a thing and we should prepare for that. There are alternatives to github and issue trackers. Eventually, the “free” services we use will have to pay their investors. With money actually costing money again, that might happen sooner then later. I doubt that shrinking workforces and zero interest-rates mix. There seems to be a Problem Solving-issue in order.

Saying them apart

While reading the source of RakuDoc::To::Text I spotted lizmat doing dispatch by hand. She defines quite a few subs that all take a RakuAST::Doc-type and return a Str. However, they are multi-candidates of the same proto. This allows for better readability, especially when debugging. Can we have multi-candidates with different names? Yes, but we need to cheat.

multi sub trait_mod:<is>(Routine \r, :$renamed) {
    my $orig = r.name;
    INIT {
        use nqp;
        r.set_name(nqp::decont(„$renamed (AKA $orig)“));
    }

    role RenamedMulti[$name] {
        method name { $name }
    }
    r does RenamedMulti[$renamed]
}

proto sub foo(|) is renamed<foo-proto> { * }
multi sub foo(Int $) is renamed<inter> { say &?ROUTINE.name }
multi sub foo(Str $) is renamed<stringer> { say &?ROUTINE.name }
multi sub foo(Failure) is renamed<failer> { fail('oi‽') }

foo(42);
foo('answer');
foo(Failure);

# OUTPUT:
# inter
# stringer
# oi‽
#   in sub failer (AKA failer) at tmp/2021-03-08.raku line 2416
#   in block <unit> at tmp/2021-03-08.raku line 2420

As you can see, I have to (indirectly) use a nqp-opcodes, to change the name of the subroutine, thanks to nqp-trickery in Backtrace.pm6. Why method AT-POS doesn’t just call .name on the Callable, I don’t know and my enquiry on IRC was fruitless. I believe it should but that might break things in CORE.

I will come back to this problem once we got v6.e. RakuAST might make the Backtrace-problem go away without me complaining via an issue. The latter will be my next topic and seems to be a more pressing … issue.

Most trusting

I was looking for a neat way to specify units when working with numbers. When doing dimensional analysis, many physicists like to put units into square brackets to create an additional namespace. We can do the same.

use v6.d;

class Unit { ... }

class SiInt is Int {
    trusts GLOBAL;
    trusts Unit;

    has Unit $!unit;
    method !unit { return-rw $!unit }

    method new(CORE::Int $new) { nextsame }
    method Str { self.Int::Str ~ $!unit.suffix }

    method ACCEPTS(Unit:U $u) { $!unit === $u }
}

class Unit {
    our $.suffix = '';
    our $.long-name = "unit-less";

    method ACCEPTS(SiInt $si) { $si!SiInt::unit === self }
}

class m is Unit { our $.suffix = 'm'; our $.long-name = 'Meter'; }

multi sub postcircumfix:<[ ]>(SiInt $obj, Unit:U $unit) {
    $obj!SiInt::unit === Unit ?? ($obj!SiInt::unit = $unit)
        !! fail(‘Sorry, units can only be set, not changed.’);

    $obj
}
multi sub postcircumfix:<[ ]>(Int $value, Unit:U $unit) { SiInt.new($value)[$unit] }

constant Int = SiInt; # intentional shadowing of CORE::Int

my $i = 42[m];
put [$i, $i.^name]; # 42m SiInt
my Int() $a = 1;
put [$a, $a.^name]; # 1 SiInt

class s is Unit { our $.suffix = 's'; our $.long-name = 'Seconds'; }
multi sub infix:<+>(SiInt $l, SiInt $r) {
    $l!SiInt::unit === Unit ?? callsame()[$r!SiInt::unit]
    !! $r!SiInt::unit === Unit ?? callsame()[$l!SiInt::unit]
        !! $l!SiInt::unit === $r!SiInt::unit ?? nextsame()
           !! fail(„Unit mismatch between $l and $r“)
}

my $s = 60[s];

say $i + $a; # 43m
say $s + $i; # Unit mismatch between 60s and 42m

The idea is to have a numerical type that is by default unit-less. A unit can be added (but not changed) with square bracket postcircumfix. Since I add type-objects for each unit, I don’t have to mess around with strings and can multi-dispatch if needed. Since I want direct access to the unit, I tell the class to trust the package the operators are defined in. (This could be a module, of course.) What happens to be an ENODOC.

I have to use a forward declaration to get ACCEPTS to get hold of $!unit. Subsequently, multi-dispatch works just fine.

multi sub fancy(Int $value where m) { #`[fancy calculation goes here] }
multi sub fancy(Int) { fail ‘Can only be fancy with Unit "m".’ }
fancy($i);

Since SiInt is just an Int all built-ins will work, so long the unit is restored after using them. Being able to trust operators allows them to access the entire class, without having to cheat with use nqp;.

Because Raku treats types as values, I can calculate a compound unit.

class Meters-per-Second is Unit { our $.suffix = 'm/s'; our $.long-name = 'Meters per Second'; }
multi sub infix:</>(m, s) { Meters-per-Second }

sub speed($d where m, $t where s) { ($d / $t).Int.[m/s] }

my Int $fast = speed(500[m], 1[s]);
say $fast; # 500m/s

I’m quite pleased with being able to extend the type-system so easily without having to invent a complete new DSL. This aids composability greatly.