Home > Raku > Watching new arrivals

Watching new arrivals

Any boring recurring task must be as easy as possible or it will be neglected. I’m quite sure this is why we invented computers. Backups are kinda borring. In fact you want to avoid any form of excitement when it comes to backups. So they must be as easy as possible. I have a script that is triggered by a udev rule when a new device is added. This is working fine when a single disk is plugged in. (This works very well.) I got a usb hub with a few usb sticks that form a btrfs raid5 for making quick backups of my $home whenever I switch the usb hub on. This does not work fine in some cases. Getting a bash script to check if a drive in a set is missing ain’t fun. Mostly because only proper languages come with Set. We do have a proper language.

On linux it is fairly easy to find out if a drive was plugged in. All we need to do is to watch for new files to pop up in /dev/disk/by-id/. We can also learn if new partitions where found. The directory looks like the following.

$ ls -1 /dev/disk/by-id/
ata-CT120BX500SSD1_1902E16BC135
ata-CT120BX500SSD1_1902E16BC2AA
ata-TOSHIBA_HDWQ140_X83VK0GDFAYG
ata-TOSHIBA_HDWQ140_X83VK0GDFAYG-part1
ata-TOSHIBA_HDWQ140_X83VK0GDFAYG-part2
ata-TOSHIBA_HDWQ140_X83VK0GDFAYG-part3
ata-TOSHIBA_HDWQ140_Y8J9K0TZFAYG
ata-TOSHIBA_HDWQ140_Y8J9K0TZFAYG-part1
ata-TOSHIBA_HDWQ140_Y8J9K0TZFAYG-part2
ata-TOSHIBA_HDWQ140_Y8J9K0TZFAYG-part3
usb-SanDisk_Ultra_USB_3.0_4C530001160708110455-0:0
usb-SanDisk_Ultra_USB_3.0_4C530001190708111070-0:0
usb-SanDisk_Ultra_USB_3.0_4C530001220708110370-0:0
usb-SanDisk_Ultra_USB_3.0_4C530001280708111064-0:0
wwn-0x50000398dc60029a
wwn-0x50000398dc60029a-part1
wwn-0x50000398dc60029a-part2
wwn-0x50000398dc60029a-part3
wwn-0x50000398ebb01681
wwn-0x50000398ebb01681-part1
wwn-0x50000398ebb01681-part2
wwn-0x50000398ebb01681-part3

If we look for anything that doesn’t end in '-part' \d+ we got a drive. We can also tell where it’s plugged in by checking the prefix.

sub scan-drive-ids(--> Set) {
    my Set $ret;
    for '/dev/disk/by-id/'.IO.dir.grep(!*.IO.basename.match(/'part' \d+ $/)) {
        $ret ∪= .basename.Str;
        CATCH { default { warn .message } }
    }

    $ret
}

my %last-seen := scan-drive-ids;

Sets don’t got an append method. We can substitude that with ∪=. Now we got a lovely Set of drives in %last-seen that are already there. We now need to wait for new files to pop up and apply set theory to them.

react {
    whenever IO::Notification.watch-path('/dev/disk/by-id/') {
        my %just-seen := scan-drive-ids;
        my %new-drives := %just-seen ∖ %last-seen;
        my %old-drives := %last-seen ∩ %just-seen;
        my %removed-drives := %last-seen ∖ %just-seen;
        %last-seen := %just-seen;

        # say ‚old drives: ‘, %old-drives.keys.sort;
        say ‚new drives: ‘, %new-drives.keys.sort || '∅';
        say ‚removed drives: ‘, %removed-drives.keys.sort || '∅';
    }
}

By binding the Sets to an Associative container we get for and other buildins to behave. If we want to take action if certain disks are added we need to define Sets that contain the right file names.

my %usb-backup-set = Set(<usb-SanDisk_Ultra_USB_3.0_4C530001160708110455-0:0 usb-SanDisk_Ultra_USB_3.0_4C530001190708111070-0:0 usb-SanDisk_Ultra_USB_3.0_4C530001220708110370-0:0 usb-SanDisk_Ultra_USB_3.0_4C530001280708111064-0:0>);

my %root-backup-disk = Set(<ata-TOSHIBA_DT01ACA200_8443D04GS>);

my $delayed-check := Channel.new;
my Promise $timeout-promise;

react {
    whenever IO::Notification.watch-path('/dev/disk/by-id/') {
        my %just-seen := scan-drive-ids;
        my %new-drives := %just-seen ∖ %last-seen;
        my %old-drives := %last-seen ∩ %just-seen;
        my %removed-drives := %last-seen ∖ %just-seen;
        %last-seen := %just-seen;

        # say ‚old drives: ‘, %old-drives.keys.sort;
        say ‚new drives: ‘, %new-drives.keys.sort || '∅';
        say ‚removed drives: ‘, %removed-drives.keys.sort || '∅';

        if %usb-backup-set ∩ %new-drives {
            $timeout-promise = Promise.in(5).then: {
                $delayed-check.send: True;
                $timeout-promise = Nil;
            } without $timeout-promise;
        }

        if %root-backup-disk ∩ %new-drives {
            sleep 2;
            backup-root-and-home-to-disk(%root-backup-disk);
        }

        say '';
    }
    whenever $delayed-check {
        my %just-seen := scan-drive-ids;
        if %usb-backup-set ⊆ %just-seen {
            backup-home-to-usb(%usb-backup-set);
        } elsif %usb-backup-set ∩ %just-seen {
            warn 'drive missing in usb set: ' ~ (%usb-backup-set ∖ (%usb-backup-set ∩ %just-seen)).keys;
            reset-usb-hub;
        }
    }
}

I use the $delayed-check whenever-block to handle the case when one of the usb sticks refuses to come online. The vendorid and deviceid of the usb hub are hardcoded. Please note that state and start don’t mix well.

sub reset-usb-hub(--> True) {
    state $reset-attempt = 0;
    if $reset-attempt++ {
        say ‚already reset, doing nothing‘;
        $reset-attempt = 0;
    } else {
        say ‚Resetting usb hub.‘;
        my $usb_modeswitch = run <usb_modeswitch -v 0x2109 -p 0x0813 --reset-usb>;
        fail ‚resetting usb hub failed‘ unless $usb_modeswitch;
    }
}

The entire script can be found here. I believe the example of watch-path could use a modified version of this script. If you read it you can tell where Sets are used simply by spotting set operators. Making Raku a operator oriented language was a good idea. Thank you Larry.

While turning my backup script from Bash to Raku I had some more findings about shell scripting in a proper language. I shall report about them here in the next few weeks.

Categories: Raku
  1. Samuel Chase
    June 1, 2020 at 06:46

    Wow, set theoretic operators work beautifully here.👌

  2. June 1, 2020 at 14:56

    How do you actually incorporate your script into your system: cron, systemd, manual on boot, or?

    • June 1, 2020 at 23:09

      For now I start it from hand. Starting and restarting it from systemd is an option for later. I need better error handling before I can trust it to run as a demon.

  1. June 1, 2020 at 13:39

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: