I Just Wanted to Print a Sandwich

Silverhadch — Tue, 01 Sep 2026 00:00:01 +0000

Or: how I ended up developing KDE Linux.

Now

Then

Most of the people who work on KDE Linux arrived the same way. They were KDE developers first, and then either helped start the project or wandered over to it once it existed. I went the other direction entirely. I was a user. Then a user who filed bugs. Then a user who fixed them. Somewhere in there I became one of the more active contributors, and only after all of that did I become a KDE developer.

So my start in KDE was not really KDE at all. It was KDE Linux.

I was searching for a distro, a very particular one. I wanted the latest KDE, rolling enough, atomic, and flicker-free with systemd-boot. After a lot of searching I stumbled over KDE Linux and started using it.

Oh boy. Young pre-alpha KDE Linux was definitely something.

If I gave you a current KDE Linux image and one of the old ones, you would believe them to be completely separate projects. Stuff broke so comically often back then that I eventually hit the stage of grief where you stop filing bug reports and start fixing things yourself.

So I made myself an Invent account and just started fixing the issues I ran into. From systemd-boot spitting out error messages because the UKI had the wrong permissions, to roughly thirty commits to get network printers working properly, because I wanted to print a picture of a sandwich.

Don't ask why. That will remain a secret you'll have to ask me about in person if we ever meet at Akademy.

That printer saga taught me the first real lesson: unlike my personal projects, here you will get a lot of pushback if something is wrong, and you will go back and forth until it's good enough. If you want to collaborate in a team with actual people, and not just with your friend where the two of you scream at each other over regressions, you need to learn patience.

Patience is the key to everything. You'll see it come up here again. And again.

Pre-Alpha: mostly plumbing

After a while of fixing the distro I was daily driving, it stopped feeling like everything was constantly collapsing. That's when I got invested in seeing this project succeed and started making frequent fixes, to the point where I went from being a random non-KDE contributor to someone the rest of the community started trusting with real work.

A lot of my early work was stabilisation, because as I said, KDE Linux back then was not very stable. And one problem guided us then and still guides us today, hopefully not for much longer: the choice of base distro.

You see, KDE Linux is not a traditional mutable distro. It's an image-based system. Writes to /usr are taboo and there is no pre-installed package manager to install software with.

Now you might have spotted a small problem. Where the heck does /usr come from?

Well, that sentence isn't quite true. There is a package manager that gets used to populate /usr, and then afterwards we rip it out and close the door behind it. KDE Linux currently uses pacman and the Arch Linux repos to build its images. Which means any problem Arch has, KDE Linux inherits. And as far as most Linux users know… Arch is not famously known for being stable.

So KDE Linux had a small problem. The KDE stack we build ourselves in a separate packages pipeline relies on the ABI and versions of certain non-KDE libraries provided by pacman, and therefore by Arch. It is not uncommon for the Arch base to move too fast and brick the KDE stack. We were chasing a target we couldn't possibly catch up with often enough, and we got bricked images out of it.

So it was decided to rely on Arch's infrastructure (that later turned out to be a mistake) and pin to a snapshot. Working together with Harald, we got the packages pipeline to dedicate a repo snapshot for the main image pipeline, so our stack wouldn't die because of our base distro. And since KDE Linux was designed as an image-based system, swapping the base distro shouldn't be disruptive to users should the need ever arise. But let's not get ahead of ourselves.

Now, another thing Arch is known for is the AUR, a public index of build scripts anyone can submit to. That openness is the entire point and it is why the AUR is so useful, but it also means nobody reviews a package on your behalf before it runs on your machine.

For the past year or longer Arch Linux and the AUR have been the victim of repeated DDoS and malware attacks, and KDE Linux got caught in between, because we were using the AUR to a degree. Which meant we had to get the hell out of the crossfire and start removing our AUR dependency bit by bit.

Before KDE Linux I was an AUR packager, and I still package for a couple of distros, so I took over maintainership for some of it and then started helping remove our AUR dependency altogether. And since some of my own AUR packages were now being pulled into KDE Linux, meaning I was now part of a KDE project's supply chain, I started thinking that maybe I should just do it and become a KDE developer.

Throughout the pre-alpha I got some guidance from KDE devs like Nate and Harald, and the KDE community started to grow on me. So I asked Harald to sponsor my KDE developer application.

A short detour about growing up

Before I get to what happened next, I should stop and say something about the person who filed that application.

KDE helped me grow as a developer and as a person.

Because, as some of you might know, I was not that old. I was basically seventeen, almost eighteen, and somewhat immature, and therefore early on I was, let's put it gently, a bit overwhelmed and abrasive in my language towards others. I repeatedly got into conflict with people because of my poor choice of words and insulting behaviour, because I was severely lacking in the social skills department. I've got to admit that.

However, after a while, and after having gotten employ**ed, my social skills improved and I started being cringy instead of abrasive. Depending on who you ask, that might be a downgrade, not gonna lie.

I would say I've grown as a person through interacting with the rest of the KDE community. So it wouldn't be too far-fetched to say that KDE Linux had a positive effect on my life.

The alpha announcement, and the four-day pipeline curse

Eventually KDE Linux reached a certain level of maturity and stability. The pipelines, however, still weren't very happy.

We were approaching the end of the pre-alpha era and preparing a release and an announcement, but the pipelines had a tendency to fall in on themselves every four days. So for a while it wasn't uncommon for the majority of KDE Linux dev work to be plumbing.

Eventually the pipelines started to get along with us too, and the KDE Linux alpha release was announced at Akademy, where I looted a crap ton of stickers that my younger sister then proceeded to take from me because she liked Katie.

How I develop a distro I don't run

Not long after that, the way I worked on the project had to change completely.

Until then, my development process basically amounted to doing sysexts on my daily driver, which was KDE Linux. But getting employ**ed meant I couldn't just stay on an alpha distro, even though KDE Linux was nowhere near as unstable by then as it had been in its early life.

So I moved into the ublue custom images scene and started doing a "KDE Linux but at home" thing, which accidentally turned into a small competition between the two despite me developing both. Ironic. That project eventually died off because the COPR for KDE master builds went unmaintained, so I switched to NixOS, which has been my daily driver ever since.

Which raises a fair question: isn't the alpha stage the part where I made massive changes, features and improvements? How did I do that on NixOS, where building anything outside the Nix way means fighting the fact that nothing lives where a normal build expects it to?

Two things. CI-built images, and the built-in Docker script and QEMU.

(That was three things, but never mind, I'm too tired to move my mouse cursor back and delete it, because I've been writing this in a flow state for two hours.)

Our CI builds images of every MR and pushes them to Harald's server, so you can download an image and boot and test it in QEMU. That is how I develop KDE Linux to this day. There's also the Docker script which builds MR images locally, in case you have a faster machine than the CI, which most people probably do. I also have a second machine that I use to test some images on real hardware.

Alpha: the people who wear cool hats

With the tooling sorted, I could get back to what I actually wanted KDE Linux to be.

Do you know that distro where people wear cool hats? Yes, Fedora.

They're quite known for pushing the Linux desktop into the future and showing off the latest and greatest of Linux. And in case you know what KDE Linux's goal is… it's pretty much that, for KDE. Showing off the latest of KDE. But how can you do that if the latest and greatest of Linux is missing?

So I started joining Fedora Matrix rooms, doing some drive-by contributions and lurking. The change proposals especially caught my eye, along with the fact that both Fedora and KDE Linux want to push Btrfs forward. So I started implementing changes even before Fedora got a chance to do them, like KMSCON.

And one thing Btrfs is known for is snapshots. Small problem, though: Linux distros don't usually create good Btrfs layouts by default. We didn't either, so Harald moved our layout to RootFSv2.

The specific issue is having /home and all its users be one single subvolume. That means, firstly, that only root can take snapshots, because /home is owned by root. And secondly, that every user's data gets rolled back to a given point together, so users can "lose" their data so that another user can recover theirs.

That has some pretty nasty wheel war potential.

So we needed to separate user data from each other. But how? KDE doesn't do useradd and friends. That falls way, way, way, way lower in the shared stack between almost all Linux distros. That's shadow-utils.

Living in the shadows

The work here you won't find in the kde-linux repo, and for a time it looked like I was on hiatus. In reality I had been living in the shadows (pun intended), working behind the scenes to push the Btrfs experience of Linux distros forward, not just for KDE Linux or Fedora, who were also very interested in this approach.

So I worked on shadow-utils for months, from bugfixing the Btrfs backend to adding new features to revamping the whole CI. I've been kind of stuck there, to be honest. Though in a good way. I like the project, and C is my favourite language (shocker, treacherous!).

Eventually the work landed in KDE Linux as RootFSv3 plus snapper for kio-snapshot, which added everything needed for users to be in charge of their own home snapshots, instead of members of the wheel group getting into a wheel war over rolling back all of /home rather than just their own subvolume.

That was a hell of a lot of work, and it showed how patient one has to be. It took a dozen months of mostly waiting.

So always remember: Rome wasn't built in a day. Patience is the key to success.

Modularity, and not reinventing the wheel

You also need to learn modularity, because throwing everything into the main repo is a bad idea for maintainability.

In the early alpha stage I wrote a small helper that improves UX in an interactive zsh session when you try to run a file without the executable bit set. (I got tired of chmod +x-ing things constantly after adding Java support and running .jar files straight from the terminal.)

We also started slowly populating the kde-linux group with a lot of repos over time, since a lot of that work can actually be shared with other distros and eventually landed in the system group. Which also shows: please don't reinvent the wheel. We eventually realised that throwing distro engineering into the kde-linux namespace might not be that great an idea, and that collaboration might be the better thing.

Escaping Arch

Now, remember when I was ranting about Arch up top? It kind of got worse, with our needs slowly not being fulfilled. The attacks on the AUR got worse, stability deteriorated, and the archive infrastructure we depended on turned out to be fragile, which is fair enough, since keeping old snapshots around is not what a rolling release is built for.

So I started the not-so-uncontroversial discussion of finally biting the bullet and fleeing Arch. We and even non-KDE contributors started discussing what we should do.

Because of the shared history with Fedora mentioned above, I was of the opinion that rebasing on Fedora might not be a bad idea, since Fedora and KDE are actually good buddies and our goals align better than with Arch. However, some pointed out that in a way we'd just be exchanging Arch for Fedora, introducing other problems, and still be at the mercy of a base distro.

So a third, more radical approach was suggested by GNOME people, who are also making their own atomic distro, and by the ublue crew (remember the custom images stuff): BuildStream, and taking complete control of the whole stack, so we're the ones in total control rather than our base distro. Which carries the risk of what Uncle Ben warned Peter about. With great power comes great responsibility.

This was around the middle of the alpha. So I started working on a Fedora version of KDE Linux, which later became the foundation for culling our Arch repos. Because up to that point the base-distro agnosticism wasn't entirely true: our stack came from our own self-made Arch repos. So instead I decided to port it to kde-builder-based tarballs, since that only relies on the base distro in one place, defining build dependencies.

We later of course decided to go with the BuildStream approach, and my Fedora work became fedora-plasma-canary, which again comes full circle back to ublue. Lol.

Fun fact: I was actually re-introducing kde-builder-based building of our stack, because the very earliest builds of KDE Linux, before I even took part in it, used kde-builder and didn't build Arch packages. So we basically reverted back to kde-builder, our upstream tooling that we also use for regular KDE development, on and outside of KDE Linux.

Around this time we worked out how decisions actually get made in the project. The running joke is that we are a semi-constitutional monarchy, with Harald as supreme banana emperor and the more active contributors as the magic council of elders. In practice it is a handful of people who talk a lot, usually agree, and sit entirely within KDE's actual governance like everyone else. We had like only one vote so far.

Late alpha: ISO hell

Governance turned out to be the easy problem of that period. The hard one was a file format.

And there is one thing we had to hear an unbearable number of times.

"Where .iso???"

In case you don't know, KDE Linux was distributed as a raw GPT disk. Basically as if your SSD right now were a file. That could be burned onto a USB stick or imported as a hard disk in QEMU. The VM use case was slightly more complicated, because you then need to add another hard disk image to the VM to actually install onto. The old .raw file is a hard drive in file form that you insert into your "PC."

For real installs it didn't matter, because firmware can boot off a hard disk of any kind — which is presumably how the machine you're reading this on got started. So you burn the hard disk file onto a portable hard disk (a USB stick, or a floppy — though it won't fit on a floppy, believe me, I tried, it was 7 GB and 435 MB too large) or a real SSD, if for some reason you have two lying around and are allergic to USB sticks.

Now, GPT hard disks are very flexible. You can do a lot of software gore with them, like putting them on the 9.4 billion terabyte drives that get invented in 2413. And doing cursed things with partitions. Making an image of one is very easy and you don't need to do anything other than have a properly working hard drive for it to work. (Technically you can also image a broken hard drive, but the intention of the KDE Linux live media is to be able to install, and booting is kind of required for that.)

However, ISO 9660 (the filesystem) and El Torito (the requirements for making an ISO 9660 filesystem bootable) are incredibly awful. They are a nightmare to deal with and unbelievably outdated.

Unlike firmware, which will treat a file as a hard disk, please and thank you, these want you to throw everything into an outdated filesystem and insert certain metadata and files in certain places. And then things like systemd-gpt-auto-generator, which rely on GPT byte sectors and which we use to boot, have to be fixed up in the initrd to go from ISO 9660 byte sectors to GPT byte sectors and then boot them. Working around the thing in the first place.

Absolute nightmare. I spent weeks reading specs and source code and more source code to only slightly understand what the hell the .iso wanted from me to be El Torito compatible. After a while I learned systemd had taken a huge part of that work and integrated it. From this day on, I shall fight and die for you, systemd.

And do you know who thought up these horrible ideas, which were never intended to be used outside of optical discs? ISO 9660 came out of a 1985 industry meeting at a hotel in Lake Tahoe and became a standard in 1988. El Torito arrived a decade later, in 1995, as a joint proposal from Phoenix Technologies and IBM, named, so the story goes, after the restaurant in Irvine where it was designed. Its idea of booting is to make the disc pretend to be a floppy, which tells you roughly everything about the era it comes from. The same IBM that gave us the ThinkPad, which is the only reason I didn't throw mine out of the window over this.

Of course this is a gross oversimplification, and don't expect me to explain in detail what these specs said, because I'm trying my hardest to forget the experience. And once again patience saved the day — just the day, mind you, my sanity fell victim to specs older than I am — since I also had to wait until ISO 9660 and El Torito support landed in a systemd release.

Lasciate ogne speranza, voi ch'intrate.

And you know what for? Basically nothing.

It slightly improved the VM install experience, which could also have been solved on the VM software side. Because nobody burns a Linux ISO onto a CD or a DVD anymore in the big 26, and that was the only thing actually gained by complying with El Torito and ISO 9660.

So if you manage to cram the over-7-gigabyte ISO onto a floppy disk, please let me know.

Late alpha: an April Fools joke that got away from me

And since we were just talking about floppy emulation: do you know DOS?

It was quite popular back then, before Microsoft won the DOS wars. How fairly it won is a question people are still arguing about on the internet, and I am not opening that door. Either way, DOS was on its way out once NT got good enough for home use, and the good old days of talking to hardware directly in real mode with clever hyper-optimisations are long over.

However, you can still experience it yourself in KDE Linux. Because as an April Fools joke, I wanted to add DOS support to KDE Linux.

It seems like nobody realised it was a joke. All the other devs thought I was dead serious.

Too awkward to now reveal that it was a joke, I instead decided to extend the Package Compatibility Helper to install certain software on demand via a so-called integration tree. It hooks into mimetypes for opening things by right-clicking, into binfmt (asking the kernel to hand a file off to PCH when it's executed), and into stub binaries in /usr/bin.

For DOS and Windows, PCH first checks against clear mimetypes, and then, if it's still unclear whether you need Wine or DOSBox, we inspect the binary. Because Microsoft was obsessed with throwing stuff into the DOS header. You can tell something is a Win16 program because the DOS header is a subtle ad telling you to go buy Microsoft Windows, or Win32 because the DOS header reminds you that, in case you try to run the Windows version of Kate under DOS, this program cannot be run in DOS mode.

Quite a fun rabbit hole to go down.

Where we are now

If I had to define my work and everyone else's in the late alpha stage, it's finishing the stabilisation, starting the polish, and fulfilling long-standing project goals.

For being an alpha, and soon a beta, this distro has become unrecognisable compared to its pre-alpha state. It's more polished and stable than most released distros I've tried, and I do mean that. Of course I might just be biased, but I can't wait to see where this journey goes, and whether this project will finally give KDE its deserved world domination.

Closing

So, I've been rambling about the important steps and my experiences developing KDE Linux and watching the project grow alongside myself. But all blogs have to end somewhere, and my hands are starting to hurt.

If I have any tips for new contributors who are interested in helping this ship sail, and it's the thing that has been guiding me through this whole journey: cooperation and patience. Not just between KDE devs, but with the entire Linux stack.

The day you plant the seed is not the day you eat the fruit.

Silverhadch on KDE Blogs