Tracking Newsfeed changes
Ever since Google Reader had shut down in 2013, and my subsequent migration to Feedly, I have had wanted to take more control over my RSS feeds.
But writing my own was way overkill. Until it apparently wasn’t anymore.
The plan was to do it in Kotlin:
- lightweight Ktor Client to fetch Feeds
- parsing the various formats using KotlinX Serialization
- writing the Contents to an Exposed Database (initially using by SQLite, now H2)
- serving the content via GReader-compatible API using Ktor Server
and naturally all of the above using Kotlin Coroutines. The project had actually started with Quarkus back then, but that was more trouble than its worth.
Feed Formats
In order to be able to properly parse all the broken Feeds out there, I had started writing my own parsers and started to do a deep dive into Serialization, using my own OPML export as initial data sample and extending it whenever an unparseable feed was encountered.
opmltool
This way I ended up writing a small tool that would read the OPML feed, sort and format it in a deterministic way and then write it back out.
With this tool at hand, I started to track changes to the OPML using git: Every so often I would export my list of feeds, run the tool and commit it into a git repository using the commit message to retroactively describe what had changed and why.
This side product of me toying around with formats, gave me new interesting insights into my own media consumption behaviour.
Cool, where can I try that tool myself?
Short answer: Nowhere (yet)
I know this post is an awfully long way to describe a piece of software that isn’t available. But right now, I don’t feel like releasing it because:
-
It isn’t even nearly ready to be released
Me still using the Reeder’s iCloud sync as a daily driver instead of with this as GReader backend should be the best indicator for that
-
It is my personal playground to learn things
meaning stuff breaks all the time, currently nothing really works because I had to try out Amper as a build system
But even when you cannot use the opmltool to format your OPML subscriptions right now, the method of keeping track of the changes over time as a concept is nevertheless interesting enough to be described.
This sounds strangely familiar
Yes, I have heard about Iconfactory announcing Project Tapestry and Silvio Rizzi teasing his new app.
The idea for this project is old enough that it would have been written in Java instead of Kotlin back then. And when I started the actual coding in 2021, Mastodon, the Fediverse and Feed Syndication were still relatively obscure practices.
This has a slightly different focus of being a versatile toolbox 1 that allows you to filter, aggregate and modify your feeds to your liking, where the other projects appear more of a combined inbox for streams of various sources.
-
Think of it as the modern day equivalent of Yahoo Pipes (I bet that is a name you haven’t heard of for a long time, if you even know what that was) ↩