Speeding up Travis

After some wiggling I managed to convince travis to use ubuntu packages to trim off about 4 minutes of a test. Sadly the .debs don’t come with build in zef, what would be another 40 seconds.

As follows a working .travis.yml.

sudo: required
before_install:
    - wget https://github.com/nxadm/rakudo-pkg/releases/download/2017.03_02/perl6-rakudo-moarvm-ubuntu16.04_20170300-02_amd64.deb
    - sudo dpkg --install perl6-rakudo-moarvm-ubuntu16.04_20170300-02_amd64.deb
    - sudo /opt/rakudo/bin/install_zef_as_root.sh
    - export PATH=/opt/rakudo/bin:$PATH
    - sudo chown -R travis.travis /home/travis/.zef/
install:
    - zef --debug install .
script:
- zef list --installed

Using a meta package in conjuction with .debs makes it quite easy to test if a module will work not just with bleeding Rakudo but with versions users might actually have.

Fork All The Things!

As requested by timotimo META6::bin is now able to fork a module on github by looking up its source in the ecosystem and telling git to clone it to the local FS.

meta6 --fork-module=Somebody::Else::Module

As a little bonus it will create a t/meta.t if possible. To be able to do so, META6::bin had to learn how to add dependencies to a META6.json-file.

meta6 --add-dep=Important::Module

I will add pullrequest creation as soon as I figured out how to convice the github api to do my bidding.

UPDATE: Pull requesting is in but not well tested (I don’t have any non-synthetic PRs to send right now). A META6.json is required to get the repo-name automatically. The youngest commit message sports the default PR title.

meta6 --pull-request

Make Children, not War

While teaching META6::bin how to read a config file, I was thinking about handling URIs by creating individual types per schema. That would require a factory what is not to my liking. I don’t use a dymanic language just to implement all that nice redundancy sugested in Design Patterns.

But then I reaslised that I use a dynamic language what means I can have a factory without implementing it. As long as I keep all subclasses in the same compunit then the parent class, that parent can use package introspection to collect all children and implement the factory method in its .new method.

class URL is export {
    my $.subclasses;

    our $.schema;
    has Str $.host;
    has $.raw;

    multi method new(Str $url) {
        $.subclasses //= UNIT::.values.grep({$_ ~~ URL && .schema});

        for $.subclasses.flat {
            return $_.new(:raw($url)) if $url.starts-with(.schema ~ '://');
        }
    }

    method host {
        $!host //= $.raw.substr($.raw.index('://') + 3).substr(0, $.raw.index('/') + 1);
    }
}

class HTTP is URL is export {
    our $.schema = 'http';
}

class HTTPS is URL is export {
    our $.schema = 'https';
}

sub url(|c){
    URL.new(|c)
}

say url('https://foo.com/bar.p6').host;

UNIT::.values returns a list of type objects that we can check against in the constructor. The class variable $.schema is used to pick the right child to return.

One could take that a step further and add a trait to register a child and a thunk with the base class to move the decision making what object to return to the children. That way a child could be moved to a different compunit as well.

Module All The Things!

Melezhik wrote a wrapper around META6::bin what made think that he may not be alone. Further, customisation would required a fork what is less then ideal because one has to care a lot about upstream changes and may even have to cherry pick commits when going down that road.

That’s annoying and unneeded if bin/meta6 is turned into lib/META6/bin.pm6. As it turns out that is surprisingly easy and already done.

I learned a few things I would like to share.

Firstly I wanted to provide the option to call a MAIN just like any normal sub. Simply exporting them doesn’t work as they would be added as MAIN candidates to the script sporting the use META6::bin.

Lets have some code.

use v6.c;

unit module was-a-main;

our proto sub MAIN(|) is export(:MAIN) { * }

multi sub MAIN(Int $a) {
    say "Int candidate";
}

multi sub MAIN(Bool :$a) {
    say "bool candidate";
}

Fist we create a namespace we can refere to, both by putting the file into a directory and with the explicit unit module statement. Then we export a proto as an our sub with a tag called MAIN. By providing the tag in use was-a-module :MAIN we would pull all MAIN candidate into a script and they would be used as normal. Without the tag, the our scope will allow to provide a FQN to call them as normal subs.

use v6.c;

use was-a-main;

was-a-main::MAIN(:a);

In META6::bin I wanted to allow subs to be wrappable. To do so it is required to scope them with our in the module and provide the FQN when wrapping.

use META6::bin :HELPER;

&META6::bin::try-to-fetch-url.wrap({
    say "checking URL: ⟨$_⟩";
    callsame;
});

META6::bin::MAIN(:check);

The sub try-to-fetch-url is used to check if URLs are accessible to catch typos in a META6.info. By wrapping it one can make it verbose. Pretty much all subs are treated the same way, allowing to wrap to your heart’s content.

As melezhik has shown, there is more then one way to have a module created for you. Now it’s both easy and possible.

For Your Convenience

I just created a new module, including a proper skeleton and META6.json, pushing everything github, in just 12 seconds.

(zef install META6::bin)
meta6 --new-module=Foo::Bar

The script will ask for your github PW, the login-name and realname are taken from ~/.gitconfig. The generated README sports a link to travis that allows setup of automated testing with 5 or 6 clicks.

For now there is no way to provide your own templates – that’s for tomorrow.

Fast things should be fast and slow things should be possible!

cale2 asked a hard question

Today cale2 refused to asked an easy question. Let’s start with his hard question.

 <cale2> I need a tutorial on * vs $_ vs $

What is a fairly good question to ask. Those three things are syntactic sugar that can help a great deal to keep lines from wrapping, allowing for less bugs because of context switching.

Let’s start with our good old friend the topic called $_. In Perl 5 it just appeared out of nowhere in every sub. In Perl 6 it appears out of a default for blocks.

my &block = { 'oi‽' };
&block.signature.say;
# OUTPUT«(;; $_? is raw)␤»

The default signature for a block is one positional parameter called $_ So every block has one. There are further statements that set the topic without introducing a new block like with and given (given does introduce a block but it’s special and I wont provide details here).

say $_ with 42;
# OUTPUT«42␤»

Since it’s the default Perl 6 will expect it at quite a few places. Most prominent is when and the objectless method call.

$_ = 42; say 'oi‽' when 42;
.say;
# OUTPUT«oi‽␤42␤»

A lone $ is actually two (and a bit) things. In a Signature it’s a marker for a positional parameter that we can never use (maybe because we don’t care) because it didn’t got a name and therefor doesn’t introduce a symbol in the Routine it applies to. It is useful for protos and free floating Signatures. It’s second use actually introduces a container, again without a symbol. It’s also a state variable whereby it’s initialiser (if there is one) is not handled as state and will be called more then once. We can use the anonymous state variable to count things.

my @a = <a b c d>;
for @a { say @a[$++] };
# OUTPUT«a␤b␤c␤d␤»

We can abuse an anonymous $ to skip values in list assignment. Quite handy when we got a short list returned from a routine.

my ($,$,$c) = <a b c d>;
say $c;
# OUTPUT«c␤»

I like the fact that I don’t have to come up with names for one-shot-variables. The less cognitive load the better.

The Whatever * is the hard part of the question. At times it’s a syntactic sugar marker that we use to tell Perl 6 that we don’t care what will be picked. At other times it means Inf in the sense of All Of Them.

my @a = <a b c d>;
put @a.pick(*);
# OUTPUT«c d b a␤»

If a lone * is used in an argument list it’s turned into the singleton Whatever. Since we can asked for values in signatures we can provide interfaces for multies where the user can state a lack of care.

multi sub foo(Int $i){ $i * 42 };
multi sub foo(Whatever){ <42 oi‽ ♥>.pick };
say foo(*);
# OUTPUT«♥␤» (your results may vary)

If we use * in a statement that involves operators or calls, it forms a block without a scope and acts like a placeholder variable, without losing the information that this block was spawned by a *. The type of the resulting Callable is WhateverCode and a routine could act on it differently then when provided with Sub or Method. Quite in contrast to a real placeholder variable Whatever can be used in a where-clause.

sub foo($a where * < 10){}

This is not a complete answer to his question and that’s why I would like to suggest to cole2 to read the *beep*ing docs.

Once a Week a happy zef

Since zef installs modules to a temporary directory and may or may not set the $*CWD to what we would like, we need to open files in tests carefully. Given that files of interest to a test file are in the same or a child directory of the test, we can operate relative to $*PROGRAM.

my $gitconfig-path = $*PROGRAM.parent.child('data').child('gitconfig');

That would translate to t/data/gitconfig seen from the modules base directory.

Once a Week

Rakudo is still changing quickly for the betterment of mankind (I’m quite sure of that). Once in a while commits will break code that is or was working. Checking for regressions is boring because most of the time it wont happen. As the army of bots that populate #perl6 shows, we like to offload the housekeeping to software and for testing we use travis.

Since a while travis provides build in cronjobs for rebuilding repos automatically. It’s a little hidden.

I missed the cron settings at first because one needs to run a build before it even shows up in the settings tab.

With cron jobs setup to test my modules once a week I will get some angry e-mails every time Rakudo breaks my code.

After clicking the same buttons over and over again I got bored and found the need to automate that step away as well. A meta module that doesn’t do anything but having dependencies to my real modules would save me the clicking and would produce exactly one angry e-mail, what will become more important as the Perl 6 years pass by. The .travis.yml looks like this:

language: perl6
sudo: false
perl6:
- latest
install:
- rakudobrew build-zef
- zef --debug install .

And the META6.json is:

{
"name" : "gfldex-meta-zef-test",
"source-url" : "git://github.com/gfldex/gfldex-meta-zef-test",
"perl" : "6.c",
"build-depends" : [ ],
"provides" : {
},
"depends" : [ "Typesafe::XHTML::Writer", "Rakudo::Slippy::Semilist", "Operator::defined-alternation", "Concurrent::Channelify", "Concurrent::File::Find", "XHTML::Writer", "Typesafe::HTML" ],
"description" : "Meta package to test installation with zef",
"test-depends" : [ ],
"version" : "0.0.1",
"authors" : [
"Wenzel P. P. Peppmeyer"
]
}

And lo and behold I found a bug. Installing XHTML::Writer on a clean system didn’t work because zef uses Build.pm differently then panda. I had to change the way Rakudos lib path is set because zef keeps dependency modules in temporary directories until the last test passed.

my @include = flat "$where/lib/Typesafe", $*REPO.repo-chain.map(*.path-spec);
my $proc = run 'perl6', '-I' «~« @include, "$where/bin/generate-function-definition.p6", "$where/3rd-party/xhtml1-strict.xsd", out => $out-file, :bin;

Please note that Build.pm will be replaced with something reasonable as soon as we figured out what reasonable means for portable module building.

There is a little bug in zef that makes the meta module fail to test. Said bug is being hunted right now.

All I need to do to get a new module tested weekly by travis is to add it to the meta module and push the commit. Laziness is a programmers virtue indeed.

Leaving out considered dangerous

A knowledge seeker asked us why a loop spec allows $i>10 but not $i<10. The reason is that the postcircumfix:«< >» changes the grammar in such a way that it expects a list quote after the $i. As a result you get the following.

loop (my $i=0;$i<10;$i++) {};
# OUTPUT«===SORRY!=== Error while compiling ␤Whitespace required before < operator␤at :1␤------> loop (my $i=0;$i<10;$i++) {};⏏␤    expecting any of:␤        postfix␤»

I tried to illustrate the problem by making the $i>10 case fail as well by defining a new operator.

sub postcircumfix:«> <»($a){}; loop (my $i=0;$i>10;$i++) {};
# OUTPUT«===SORRY!=== Error while compiling ␤Unable to parse expression in postcircumfix:sym«> <»; couldn't find final $stopper ␤at :1␤------> rcumfix:«> <»($a){}; loop (my $i=0;$i>10⏏;$i++) {};␤    expecting any of:␤        s…»

I concluded with the wisdom that that Perl 6 is a dynamic dynamic language. While filing a related bug report I made the new years resolution to put a white space around each and every operator. You may want to do the same.

Perl 6 is Smalltalk

Masak kindly pointed the general public to a blog post that talks about how awesome Smalltalk is.
The example presented there reads:

a < b
  ifTrue: [^'a is less than b']
  ifFalse: [^'a is greater than or equal to b']

The basic idea is that ifTrue and ifFalse are methods on the class Bool. Perl 6 don’t got that and I thought it would be tricky to add because augment enum doesn’t work. After some tinkering I found that augment doesn’t really care what you hand it as long as it is a class. As it happens Rakudo doesn’t check if the class is really a class, it simply looks for a type object with the provided name. The following just works.

use MONKEY-TYPING;
augment class Bool {
    method ifTrue(&c){ self ?? c(self) !! Nil; self }
    method ifFalse(&c){ self ?? Nil !! c(self); self }
}

(1 < 2)
    .ifTrue({say ‚It's True!‘})
    .ifFalse({ say ‚It's False!‘});

If we call only one of the new methods on Bool, we could even use the colon form.

(True)
    .ifTrue: { say "It's $^a!" };

As you likely spotted I went a little further as Smalltalk by having the added methods call the blocks with the Bool in question. Since Block got a single optional positional parameter the compiler wont complain if we just hand over a block. If a pointy block or a Routine is provided it would need a Signature with a single positional or a slurpy.

Please note that augment on an enum that we call a class is not in the spec yet. A bug report was filed and judgement is pending. If that fails there is always the option to sneak the methods into the type object behind Bool at runtime via the MOP.

And so I found that Perl 6 is quite big but still nice to talk about.

UPDATE: There where complains that IfTrue contained an if statement. That’s was silly and fixed.