Huginn on Janusworx

Huginn: Vive La Pratique Délibérée Avec Les Flux RSS

feedback@janusworx.com (Mario Jason Braganza) — Wed, 15 Mar 2023 09:32:05 +0530

I’ve been learning French, on my own at a snail’s pace over the past year or so.
All because I want to read Émilie du Châtelet, Dumas, Verne, and Montaigne in their native tongue.

Since I cannot do immersion nor give regular practice the time it wants, it’s … slow going But I do, do it. Regularly and deliberately. I’ve built up a vocabulary of about 500-700 words by now and I know if I keep this up, I’ll be able to do read well in time.

One way I practice is to take something in French and read it slowly, looking up words and phrases and slowly building up the missing pieces.
And to do that, I need something in French. I stumbled across the France Pittoresque site and fell in love with it. French and History! Just my kind of site.

Being the lazy bum, that I am, I went looking for an RSS feed, so that I could read their daily article in Reeder.
And hit my first stumbling block.
The good folk at France Pittoresque, don’t provide a feed. Or if they do, I couldn’t find it. Even after looking and looking. And then looking some more.¹

You know where I’m going with this. This is a job for Huginn!

This is how the site looks today

click to embiggen.

And this little item is what I want to get, on a daily basis.

click to embiggen. Actually click any of them to embiggen!

So I whipped up this scenario.

It scrapes that little section I mentioned above and gets me the title, the url and the description of the article

And generates an RSS feed that I can then use in my feed reader!
The feed is normally at:
http://your-huginn-instance/users/[your-user-id-int]/web_requests/[agent-id-int]/some-'secret'-string.xml

It looks more complicated than it is. It’ll get generated the moment you create your agent. One advantage of using a vm is that my feed is accessible from any device.

You can find the scenario code here, if you want to play with it.

Huginn has proven invaluable when it comes to doing all the little things, that I’d rather not do :)
Merci beaucoup, Huginn!

Feedback on this post? Mail me at feedback@janusworx.com

P.S. Subscribe to my mailing list!
Forward these posts and letters to your friends and get them to subscribe!
P.P.S. Feed my insatiable reading habit.

You’re talking to a lazy bum here. So quite possible I overlooked something. ↩︎

Huginn: On Becoming a Cal Newport Podcast Packrat

feedback@janusworx.com (Mario Jason Braganza) — Tue, 14 Mar 2023 17:55:01 +0530

First things first, you should be listening to Cal Newport and reading his books too!
His recent episode on books and reading was lovely!¹

If I remember right, he started doing the podcast, a couple of months after the pandemic broke out and the world shut down. And it felt very much at the time, like one of those friendly voices across the waves in some post apocalyptic movie.
It felt warm, personal and gave me something to do in those dark days.
Ergo, I have a strong connection to it and I began saving every episode, at around episode 10 or so.

Oh, and speaking of slow productivity, it’s amazing to see what Cal has fashioned the podcast into. A slow steady drumbeat of episodic growth has led to what he call the Cal Newport Empire, with media galore.

So, like I was saying, I’ve been downloading them ever since the beginning, because I’ve some sort of unnatural attachment to them. They’re the thing I want to listen to if I’m marooned on a desert island.²

And now that I have Huginn, the computer downloads it for me, in the folder I want, with the naming convention, I want.

Here’s the scenario that does it.

It scrapes the website, once every twelve hours to check for a new episode, and grabs the url and title.
It uses the url to download the episode to a folder.
And at the same time, sends me a notification. The title tells me what’s new :)

I probably could use Huginn to rename and copy the file to where I want, but it runs from a Docker container and does not have access to the parent filesystem.
So this little Python script³ does it for me.

from pathlib import Path
import shutil


downloads_folder = Path('/path/to/downloaded/episode/directory/')
destination_folder = Path('/path/to/destination/directory/')

for audio_file in downloads_folder.iterdir():
 if audio_file.suffix == '.mp3':
 original_name = audio_file.stem
 original_name = original_name.split('-')[2:]
 episode_number, raw_file_name = f'CNPE{original_name[0]}', (' '.join(original_name[1:])).title()

 new_file_name = episode_number+ ' - ' +raw_file_name +audio_file.suffix

 shutil.move(downloads_folder/audio_file, destination_folder/new_file_name)

It looks for any mp3 files in the specified folder, renames them the way I want them named and then moves them to my packrat archive. Et voilà!

The scenario and scripts are here, if you want them.

What else, do you expect a bookworm to tell ya? 😂 ↩︎
with a stock of mp3 players and rechargable batteries ↩︎
scheduled via crontab ↩︎

Huginn: the Rube Goldbergness of Updating Hugo

feedback@janusworx.com (Mario Jason Braganza) — Mon, 13 Mar 2023 19:35:06 +0530

Click the pic to embiggen.
Rube Goldberg illustration via James Vaughan

Ended up too sick to even tinker, since I last wrote.
Well enough now, to scratch out some prose to get my brains in gear.
Let’s start with Huginn adventure #1.
Buckle up!

Background
#

I found Huginn, and am using it as my man friday.¹

The “Problem”
#

I run Hugo on ~~two~~ three machines.
I’m tired of updating to the latest release manually.
The computer can do it for me!

Why this way? Why? Why?!
#

It’s fun!
Yes, there are other (most likely simpler) ways to do it, but Huginn is my new shiny hammer … and well you know!
Did I tell you it’s fun? :)

How now brown cow?
#

Some more background:

I could just apt install, but I’m too posh for that 😂 I want the latest version of Hugo. And the extended version at that.
I depend on two more helpers.
a. Syncthing syncs the folder that the Hugo deb package downloads to, across all my machines
b. Pushover sends me push notifications. Huginn integrates with it.

Having said that, here’s a pic of my four step scenario, followed by the output of each step.

Scrape the Hugo github releases page to get the title.²

Get the latest release title³ and fashion a url out of it.

Download Hugo to a folder.

Send myself a push notification.⁴

And tada!

So we done? No, you silly goose!

We’re done downloading the installer. Said installer now needs installing.
So my very imaginatively titled install-hugo.sh script⁵, which runs once a day via a crontab entry …

takes the file,
checks to see if it’s downloaded in the last day
and if so, installs it.

Mind you, it needs root privileges. And then it also …
4. deletes any stray debs more than a few days old.⁶

#! /usr/bin/env bash

cd /path/to/hugo/deb/file

# Install deb from the last 24 hours
find *.deb -maxdepth 1 -mmin -1440 | xargs -d '\n' dpkg -i

# Remove debs more than 2 days old
find *.deb -maxdepth 1 -mtime +2 | xargs -d '\n' rm -rf 

And now we’re done :)

Holy shiny hammer, Batgirl! It worked!

The scenario, should you install Huginn and want to import and play with, as well as the shell script are here, on Github.

Note: I had originally used the RSS agent in step one, instead of the Website agent I use now, and that was much easier. But it would detect the new feed entry and then just sit there hatching eggs, not doing anything. I wish I had a snapshot (of the missed events, not hatched eggs) but I digress. The website agent, for all its hacky xpath quackery works.

it could have been girl friday too, but I call mine Bertuccio, after Edmond Dantes’ man friday ↩︎
I cannot believe, I go scrape a website and learn all that xpath voodoo, just to get a measly text string. ↩︎
And here’s where Huginn helps. It remembers the last title. And only proceeds if something has changed. ↩︎
the push notification is just like that. something whimsical for me. It’s independent of the download. I should probably have called it Step 3a. or 2b. Never mind now. ↩︎
which is also synced and available across all three machines ↩︎
I let them be for a couple of days, in case a machine is not online that day. ↩︎

Huginn

feedback@janusworx.com (Mario Jason Braganza) — Thu, 02 Mar 2023 05:00:00 +0530

Let’s get the, “What the heck is Huginn?” out of the way.
If you’ve heard of Zapier or IFTTT, that’s what this is. Only better. You host it. You control it. You program it. And everything’s in your control.
If you’ve used Yahoo Pipes, then I tip my hat to you my dear friend and tell the rest of you folk, that I believe all of these new fangled pieces of zaps and agents are the spiritual successor to that pioneering web app. IBM has a similar open source alternative, with all the new fangled hotness called Node-RED

Just like piping and transforming text with Unix utilities locally, Huginn lets us do that with web data. RSS feeds, webhooks, JSON data and the like. Take ’em, ingest ’em, chew on ’em, transform ’em and do what the heck you want with ’em!

Why Huginn?
#

So why did I settle on Huginn? And what do I use it for?
Well, I liked the name. And I wanted something Yahoo-pipesque for myself :)
I used it originally on my desktop, to push my blog posts to twitter and my mastodon account. And then I lost it in a system crash¹ and I didn’t get to putting it up again.

Until a recent bout of sickness, and the urge to tinker led me to put it up on my vm.
I realised that I don’t want to use my programming hammer to tackle every computing hammer that I face. There’s too much to learn and too little life.
So if Huginn lets me do interesting things with json and liquid and duct tape, why ever not?

I still have to do the autopost to my social accounts², but I’ve already set it up to scratch several itches I had.

It scrapes a couple of art websites, that don’t have feeds and then creates RSS feeds so that I can read them peacefully in my RSS Reeder.
I check my Calibre release feed, update my local install of Calibre to the latest hotness, and then get notified via a push notification, using a Rube Goldberg pastiche of Bash, cURL and Pushover.
I love archiving episodes of my favourite podcast (the Cal Newport podcast³), so Huginn and Python and Syncthing and cron all pull together to get it in the right place, with the naming convention I want.⁴

I hope to write a bit more about each of these in the coming days, but needless to say, I’m having more fun playing with Huginn, than I imagined I ever would!

it did work like the little engine that could, without a hitch until then ↩︎
I’ve forgotten how I did it! ↩︎
I think it’s called Deep Questions, but to me it’ll always be the Cal Newport Podcast ↩︎
If you lose all your audio files in a fire Cal, you know who to call! ↩︎

What I Learned Today (8th of February, 2023)

feedback@janusworx.com (Mario Jason Braganza) — Wed, 08 Feb 2023 20:20:06 +0530

Don’t know if it’s the gym or the regular work beat, but I seem to be waking up later and later everyday. I don’t know what to do about this. I feel a little anxious about not having enough hours in the day.

Python
#

Why use comprehensions? Because it reads better.
Like I advice the young ones in DPGLUG on writing.
Write short, choppy sentences.
They read better.
With my problem sets …
- I should stop trying to generalise in advance and solve the specific problem first.
- Read stuff carefully. Don’t do more than is necessary
- Solutions take time to understand too. Budget the time

Linux Mint
#

Nemo, the file browser in Linux Mint always opens the home directory, when I click on it and there’s no apparent way to change it to a preferred directory. I want it to always open Downloads

~~A little bit of web spelunking led to an acceptable work-around.~~
1. Change the command form nemo %U to nemo Downloads
  And that does it!

Update
#

2023-02-09: Does not work. Will look for some other approach. Right now, with the workaround it opens a Downloads window alongside any other folder I click on 😂

Hugo
#

Added a plain, hamburger menu to this blog, using Aditya’s helpful instructions over on the PaperMod Discord.
With all the extra nav items, I’ve added to the header the menu would overflow on my iPhone (and probably on other folks’ phones too). So if this page scooches narrow enough, it will change the links on top to a plain menu.

Huginn / RSS
#

I often have sites that have no RSS feeds, but they’re still interesting enough to me that I want to keep tabs on them for anything new.
While I wrote a tiny program to scrape and generate a custom RSS feed to get my Poetry fix, I needed something more robust, if I wanted to do this with lots of sites.
I realised that my Huggin instance could do that for me. Huginn is documented pretty well, that even a slowpoke like me, could figure out a fix for himself, inside an hour
I also learnt something fun. How do I choose a specific, nested, div on the page when all of them are similarly named? By using the power of XPath! I found it to be a more robust way of picking something from a web page, compared to its alternatives.
It’s also pretty easy to at least get a basic XPath selection. Just inspect your element in your browser’s web inspector and a right/context click should normally give you the option to copy a link as an XPath. Getting to an href link in the div I wanted, led to something like /html/body/div[1]/div[1]/div/div/div/main/div/div[1]/div/header/h2/a/@href and BAM! I got the link!
This solves a really big problem for me. So once again, I’m pretty chuffed today as well :)

Urdu
#

Got done with the “siin” group of letters today.
It is getting more and more confusing to read, but I’m sure I’ll slowly get over it with practice.

Physical Fitness
#

More Cardio today
I’ve gotten comfortable doing an hour of active cardio at around 160, without worrying that my heart would burst. Now to focus on doing the same hour at a slower heart rate, over time.
The pain’s almost gone. Let’s see what the coming days hold.

Huginn on Janusworx

Huginn: Vive La Pratique Délibérée Avec Les Flux RSS

Huginn: On Becoming a Cal Newport Podcast Packrat

Huginn: the Rube Goldbergness of Updating Hugo

Background #

The “Problem” #

Why this way? Why? Why?! #

How now brown cow? #

Huginn

Why Huginn? #

What I Learned Today (8th of February, 2023)

Python #

Linux Mint #

Update #

Hugo #

Huginn / RSS #

Urdu #

Physical Fitness #