Leaving out considered dangerous

A knowledge seeker asked us why a loop spec allows $i>10 but not $i<10. The reason is that the postcircumfix:«< >» changes the grammar in such a way that it expects a list quote after the $i. As a result you get the following.

loop (my $i=0;$i<10;$i++) {};
# OUTPUT«===SORRY!=== Error while compiling ␤Whitespace required before < operator␤at :1␤------> loop (my $i=0;$i<10;$i++) {};⏏␤    expecting any of:␤        postfix␤»

I tried to illustrate the problem by making the $i>10 case fail as well by defining a new operator.

sub postcircumfix:«> <»($a){}; loop (my $i=0;$i>10;$i++) {};
# OUTPUT«===SORRY!=== Error while compiling ␤Unable to parse expression in postcircumfix:sym«> <»; couldn't find final $stopper ␤at :1␤------> rcumfix:«> <»($a){}; loop (my $i=0;$i>10⏏;$i++) {};␤    expecting any of:␤        s…»

I concluded with the wisdom that that Perl 6 is a dynamic dynamic language. While filing a related bug report I made the new years resolution to put a white space around each and every operator. You may want to do the same.

Perl 6 is Smalltalk

Masak kindly pointed the general public to a blog post that talks about how awesome Smalltalk is.
The example presented there reads:

a < b
  ifTrue: [^'a is less than b']
  ifFalse: [^'a is greater than or equal to b']

The basic idea is that ifTrue and ifFalse are methods on the class Bool. Perl 6 don’t got that and I thought it would be tricky to add because augment enum doesn’t work. After some tinkering I found that augment doesn’t really care what you hand it as long as it is a class. As it happens Rakudo doesn’t check if the class is really a class, it simply looks for a type object with the provided name. The following just works.

use MONKEY-TYPING;
augment class Bool {
    method ifTrue(&c){ self ?? c(self) !! Nil; self }
    method ifFalse(&c){ self ?? Nil !! c(self); self }
}

(1 < 2)
    .ifTrue({say ‚It's True!‘})
    .ifFalse({ say ‚It's False!‘});

If we call only one of the new methods on Bool, we could even use the colon form.

(True)
    .ifTrue: { say "It's $^a!" };

As you likely spotted I went a little further as Smalltalk by having the added methods call the blocks with the Bool in question. Since Block got a single optional positional parameter the compiler wont complain if we just hand over a block. If a pointy block or a Routine is provided it would need a Signature with a single positional or a slurpy.

Please note that augment on an enum that we call a class is not in the spec yet. A bug report was filed and judgement is pending. If that fails there is always the option to sneak the methods into the type object behind Bool at runtime via the MOP.

And so I found that Perl 6 is quite big but still nice to talk about.

UPDATE: There where complains that IfTrue contained an if statement. That’s was silly and fixed.

Awesome and Custom

While toying with roles to find a better example for our class tutorial I believe to have stumbled onto a nice idiom. And so I wrote:

role Unitish[$unit = fail('Please provide a SI unit quantifier as a Parameter to the role Unitish')]

What leads to the following error message when the role argument is missing.

Could not instantiate role 'Unitish':
Please provide a SI unit quantifier as a Parameter to the role Unitish
  in any  at gen/moar/Metamodel.nqp line 2441
  in any protect at gen/moar/stage2/NQPCORE.setting line 802
  in any specialize at gen/moar/Metamodel.nqp line 2428
  in any specialize at gen/moar/Metamodel.nqp line 2644
  in any compose at gen/moar/Metamodel.nqp line 3010
  in any generate_mixin at gen/moar/Metamodel.nqp line 1319
  in any  at gen/moar/Metamodel.nqp line 1235
  in any mixin at gen/moar/Metamodel.nqp line 1270
  in sub postfix:<s> at si.p6 line 12
  in block <unit> at si.p6 line 14

So instead of letting the compiler say: “Dude, the module author wants you to provide a Str!”, I can actually tell the user what the string should look like. The way I’m using it results in a runtime error. The does-operator can be executed at compile time, providing awesome error messages the way you want.

These keys are LTA

While toying around with enums as boolean options to a routine, i found the default error message less then awesome.

Constraint type check failed for parameter '@options'

It would be hard to be even less specific. Let’s create a few exceptions to tell what is going on when things go wrong.

class X::Paramenter::Exclusive is Exception {
has $.type;
    method message {
        "Parameters of {$.type.perl} are mutual exclusive"
    }
}

Now we can check if options of Find::Type are exclusive and complain accordingly.

&& ( exclusive-argument(@options, Find::Type) or fail X::Paramenter::Exclusive.new(type => Find::Type) )

class X::Parameter::UnrecognisedOption is Exception {
    has $.type;
    has $.unrecognised;
    method message {
        "Option { $.unrecognised } not any of { $.type.map({ (.^name ~ '::') xx * Z~ .enums.keys.flat }).flat.join(', ') }"
    }
}

Since enums are containers for types and those got names we can use set operators to check and single out none matching options (basically anything the +@options slurps up we don’t know).

or fail X::Parameter::UnrecognisedOption.new(type => (Find::Type, Find::Options),
    unrecognised => .item ∖ (|Find::Type::.values, |Find::Options::.values) )

Stitching the error message together is a bit more involved because we can get a list of all enum keys in a given enum but those don’t know their qualified name. We have to prefix with the enum name and :: by hand.

class X::Parameter::UnrecognisedOption is Exception {
    has $.type;
    has $.unrecognised;
    method message {
        "Option { $.unrecognised } not any of { $.type.map: { (.^name ~ '::') xx * Z~ .enums.keys.flat } }"
    }
}

This results in a much more awesome error message:

Option 42 not any of Type::File, Type::Dir, Type::Symlink, Options::Recursive, Options::Keep-going

This looks all quite regular. We have a slurpy that is kind of parameterised with one or many enums and those enums may have a flag telling if they act like radio buttons. Sounds like this idiom would fit nicely into a module.

Keys are optional

On my quest to a concurrent File::Find I found the need to have arguments (in this case as Bool) that are of a group of sorts and are mutual exclusive. Enums are very groupy, introduce easy to use names into the scope (directly or via is export) and should be easy to make mutual exclusive. The easy part was a bit naive because typed slurpy arguments are not supported (yet). If there is no easy way, there must be a hard way that is possible.

First let’s define two enums that serve as options to find.

package Find {
  enum Type (<File Dir Symlink>);
  enum Options (<Recursive Keep-going>);
}

Now we can have a where-clause that first checks if all members of a slurpy array are either of type Find::Type or Find::Options. Then we can check how many elements of Find::Options there are. Since there can be only one we complain about exclusiveness if there are to many.

+@options where {
  @options.all (elem) (Find::Type::.values (|) Find::Options::.values)
  && (+(@options.grep: * ~~ Find::Type) <= 1 or die "can only check for one type at a time")
}

In the body of the routine we can use junctions and smart matching to check if options are present.

my Bool $recursive = any(@options) ~~ Find::Recursive;
my %tests = Find::File => {so .f}, Find::Dir => {so .d}, Find::Symlink => {so .l};
@tests.append(%tests{@options.grep: * ~~ Find::Type});

The routine is then called with a list of flags at the end of it’s parameter list.

find(%*ENV<HOME>, include => {.extension eq 'txt'}, exclude => ['cfg', /.xml $/] , Find::File, Find::Recursive, Find::Keep-going);

The same would be possible to do with named arguments but I can’t see a way to do the exclusiveness in a where-clause. I like to have as much argument processing in the signature because it makes it easy to write documentation. Separate all arguments with a newline and then translate type constraints and where-clauses into plain English. Also, having enums as flags feels quite 6-ish and that’s what this blog (post) is about.

It’s lazy all the way down

On my quest to help with the completion of the docs for Perl 6 I found the task to find undocumented methods quite cumbersome. There are currently 241 files that should contain method definitions and that number will likely grow with 6.d.

Luckily I wield the powers of Perl 6 what allows me to parse text swiftly and to introspect the build-in types.

71     my \methods := gather for types -> ($type-name, $path) {
72         take ($type-name, $path, ::($type-name).^methods(:local).grep({
73             my $name = .name;
74             # Some buildins like NQPRoutine don't support the introspection we need.
75             # We solve this the British way, complain and carry on.
76             CATCH { default { say "problematic method $name in $type-name" unless $name eq '<anon>'; False } }
77             (.package ~~ ::($type-name))
78         })».name)
79     }

We can turn a Str into a list of method objects by calling ::("YourTypeName").^methods(). The :local adverb will filter out all inherited methods but not methods that are provided by roles and mixins. To filter roles out we can check if the .package property of a method object is the same then the type name we got the methods from.

For some strange reason I started to define lazy lists and didn’t stop ’till the end.

52     my \ignore = LazyLookup.new(:path($ignore));

That creates a kind of lazy Hash with a type name as a key and a list of string of method names as values. I can go lazy here because I know the file those strings come from will never contain the same key twice. This works by overloading AT-KEY with a method that will check in a private Hash if a key is present. If not it will read a file line-by-line and store any found key/value pairs in the private Hash and stop when it found the key AT-KEY was asked for. In a bad case it has to read the entire file to the end. If we check only one type name (the script can and should be used to do that) we can go lucky and it wont pass the middle of the file.

54     my \type-pod-files := $source-path.ends-with('.pod6')

That lazy list produces IO::Path objects for every file under a directory that ends in ‘.pod6’ and descends into sub-directories in a recursive fashion. Or it contains no lazy list at all but a single IO::Path to the one file the script was asked to check. Perl 6 will turn a single value into a list with that value if ever possible to make it easy to use listy functions without hassle.

64     my \types := gather for type-pod-files».IO {

This lazy list is generated by turning IO::Path into a string and yank the type name out of it. Type names can be fairly long in Perl 6, e.g. X::Method::Private::Permission.

71     my \methods := gather for types -> ($type-name, $path) {

Here we gather a list of lists of method names found via introspection.

81     my \matched-methods := gather for methods -> ($type-name, $path, @expected-methods) {

This list is the business end of the script. It uses Set operators to get the method names that are in the list created by introspection (methods that are in the type object) but neither in the pod6-file nor in the ignore-methods file. If the ignore-methods is complete and correct, that will be a list of methods that we missed to write documentation for.

88     for matched-methods -> ($type-name, $path, Set $missing-methods) {
89         put "Type: {$type-name}, File: ⟨{$path}⟩";
90         put $missing-methods;
91         put "";
92     };

And then at the end we have a friendly for loop that takes those missing methods and puts them on the screen.

Looking at my work I realised that the whole script wouldn’t do squad all without that for loop. Well, it would allocate some RAM and setup a handful of objects, just to tell the GC to gobble them all up. Also there is a nice regularity with those lazy lists. They take the content of the previous list, use destructuring to stick some values into input variables that we can nicely name. Then it declares and fills a few output variables, again with proper speaking names, and returns a list of those. Ready to be destructured in the next lazy list. I can use the same output names as input names in the following list what makes good copypasta.

While testing the script and fixing a few bugs I found that any mistake I make that triggers and any pod6-file did terminate the program faster then I could start it. I got the error message I need to fix the problem as early as possible. The same is true for bugs that trigger only on some files. Walking a directory tree is not impressively fast yet, as Rakudo creates a lot of objects based on c-strings that drop out of libc, just to turn them right back into c-strings when you asked for the content of another directory of open a file. No big deal for 241 files, really, but I did have the pleasure to $work with an archive of 2.5 million files before. I couldn’t possibly drink all that coffee.

I like to ride the bicycle and as such could be dead tomorrow. I better keep programming in a lazy fashion so I get done as much as humanly possible.

EDIT: The docs where wrong about how gather/take behaves at the time of the writing of this blog post. They are now corrected what should lead to less confusion.

Sneaky methods

As one would expect methods can be declared and defined inside a class definition. Not so expected and even less documented are free floating methods declared with my method. Now why would you want:

my method foo(SomeClass:D:){self}

The obvious answer is the Meta Object Protocols add_method-method, as can be found in Rakudo:

src/core/Bool.pm
32:    Bool.^add_method('pred',  my method pred() { Bool::False });
33:    Bool.^add_method('succ',  my method succ() { Bool::True });
35:    Bool.^add_method('enums', my method enums() { self.^enum_values });

There is another, more sneaky use for such a method. You may want to have a look at what is going on in a chain of method calls. We could rip the expression apart and insert a one shot variable, do our debugging output, and continue in the chain. Good names are important and wasting them on one shot variables is unnecessary cognitive load.

<a b c>.&(my method ::(List:D){dd self; self}).say;
# OUTPUT«("a", "b", "c")␤(a b c)␤»

We can’t have no name without an explicit invocant, because Perl 6 wont let us, so we use the empty scope :: to make the parser happy. With a proper invocant, we would not need that. Also, the anonymous method is not a member of List. We need to use postfix .& to call it. If we need that method more then once we could pull it out and give it a name.

my multi method debug(List:D:){dd self; self};
<a b c>.&debug.say;
# OUTPUT«("a", "b", "c")␤(a b c)␤»

Or we assign it as a default argument if we want to allow callbacks.

sub f(@l, :&debug = my method (List:D:){self}) { @l.&debug.say };
f <a b c>, debug => my method ::(List:D){dd self; self};
# OUTPUT«("a", "b", "c")␤(a b c)␤»

In Perl 6 pretty much everything is a class, including methods. If it’s a class it can be an object and we can sneak those in wherever we like.

It’s blocks all the way down

While playing with Perl 6 on glot.io I learned that they really like docker with the simple program:

dir('/')>>.Str.say;

And I wondered if there is a nice idiom for recursing into directories. IO::Path.dir will return a Seq of IO::Path objects. That’s why the >>. hyperoperator works. It wont recurse of cause as there is no sub to recurse with. After some open eye meditation I found what I was looking for quite some time.

A block in Perl 6 is a Callable with one positional argument. That argument is bound to the topic $_. That’s why we can do:

for 1,2,3 { .say }

Recursing into a directory would be easy if we turn the Str '/' into a IO::Path object and check if we got a directory and call the block with that element. That block would need a name, what we could do with my &block = { Nil }, or we use the compile time variable &?BLOCK.

for '.' {
    .Str.say when !.IO.d;
    .IO.dir()>>.&?BLOCK when .IO.d
}

The form .&?BLOCK will treat the call like a method call, what means the object left of . will be the first parameter of the call, where the invocant belongs.

I believe this is a fairly nice example how to use &?BLOCK to get rid of nested loops and one shot variables. It will be added shortly to the docs.

With the kind help of Zoffix the example was golfed down further.

{ .d && .dir».&?BLOCK || .put }(".".IO)

Perl 6 can recurse with the best of them. :)

Disecting subsets

While fighting with htmlily.p6 I found myself in need of a tool that shows me the structure of a pod6 file. Since all nodes are subclasses of Pod::Block and it’s tree structure is walked through simply by recursively descending into .contents, it was easily done. Just recurse and print .^name with some indentation. Putting a stack on walks back is needed because Pod::Block doesn’t provide a .parent method. A bit ugly but not really a problem.

What did confuse me was the fact that .^name on subsets that provide Pod::FormattingCode with a better name didn’t carry a proper type name. And in fact, a subset does inject a new name into the scope but does not change the type of an object that passed a type check against it.

subset FooStr of Str;
my FooStr $s;
sub f(FooStr $s){ say $s.^name };
f("abc");
# OUTPUT«Str␤»

That is fine if you want to use a subset as a type check. Since it passed the check, it must be of that type or a type that behaves in the same way.

I wanted to do introspection, what means turning a type object into a name. Even worse I wanted to turn a type object into a name of a type that is narrower. Since there is no reference from a type to all its sub types, Perl 6 could not help me there. At least not directly.

Luckily I put the subsets into a inline package. When you stick Foo::Bar:: in front of a name, Perl 6 will kindly generate a package for you and put the new types inside. Since a package is a Stash with a funky name, we can iterate over it’s key-value-Pairs. The keys are names and the values are type objects.

Pod::FormattingCode::.pairs.first({ $node ~~ $^a.value }).key

In my case there is a 1:1 relationship between a subset and it’s base type properties, so I can be sure that the first match is correct.

A nice example how using common data structures in the language implementation and exposing them to the programmer, can help Perl 6 to help you.

I don’t need conditionals either

While reading John A De Goes’ blog post about how to rid functions of variables tied to conditions, I wondered how I would do that in Perl 6.

The objective of his first example is to compare two strings case insensitive. Easy!

say "abc".fc eq "Abc".fc;

And then my mind started to wander off. I took a mental note and finished reading about ridding of ifs. Returning to my mental note I concluded: “There must be a nicer way to write that!”. It’s Perl 6 after all.

How about the function combinator?

my &ieq = &infix:<eq> ∘ &fc;

That didn’t work because ∘ cares about the number of arguments of its first operand but doesn’t about its second. What I actually need would be a loop over the two strings calling .fc on each of them.

my @l = <abc Abc>>>.fc;

So I tried to fix the function combinator resulting in a gist. It’s now special cased for the 2nd operand to have just one argument. I felt a bit uneasy about that solution. Maybe because it would require a PR to Rakudo.

And then I realised that I might just bind a pointy to a operator name. I didn’t doc that so I checked and we indeed missed that spot. Short test via camelia.

my &infix:<foo> := -> {}

That worked. So let’s have a case insensitive string compare infix operator.

my &infix:<ieq> = -> |l { [eq] l>>.fc };
say "abc" ieq "Abc";
# OUTPUT«True␤»

Nice, short and no conditionals that could confuse John.