Gesture customization mini-sprint

To briefly recap, Natalie Clarius and I applied for an NLnet grant to improve gesture support in Plasma, and they accepted our project proposal. We thought it would be a good idea to meet in person and workshop this topic from morning to evening for three days in a row. Props to Natalie taking the trip from far away in Germany to my parents' place, where we were kindly hosted and deliciously fed.

Our project plan starts with me adding stroke gesture support to KWin in the first place, while Natalie works on making multi-touch gestures customizable. Divvying up the work along these lines allows us to make progress independently without being blocked on each other's work too often. But of course there is quite a bit of overlap, which is why we applied to NLnet together as a single project.

The common thread is that both kinds of gestures can result in similar actions being triggered, for example:

• Showing Plasma's Window Overview
• Starting an app / running a custom command
• Invoking an action inside a running app

So if we want to avoid duplicating lots of code, we'll want a common way to assign actions to a gesture. We need to know what to store in a config file, how Plasma code will make use of it, and how System Settings can provide a user interface that makes sense to most people. These are the topics we focused on. Time always runs out faster than you'd like, ya gotta make it count.

Three days in a nutshell

Getting to results is an iterative process. You start with some ideas for a good user experience (UX) and make your way to the required config data, or you start with config data and make your way to actual code, or you hit a wall and start from the other end going from code to UX until you hit another wall again. Rinse and repeat until you like it well enough to ship it.

On day 1, we:

• Explored some code together, primarily in:
  • KWin, which recognizes gestures;
  • KGlobalAccelD, which manages global shortcut configurations;
  • the KGlobalAccel framework, which asks KGlobalAccelD to to register a global shortcut,
  • and the Shortcuts page in System Settings, a.k.a. kcm_keys.
• Figured out why Natalie's KWin session wouldn't produce systemd logs.
  • (Answer: a leftover config entry.)
• Collected a comprehensive list of gestures (and gesture variants) to support.

On day 2, we:

• Collected a broad list of actions (and action types) to invoke when a gesture is triggered.
• Sketched out UI concepts for configuring gestures.
• Weren't quite satisfied, came up with a different design which we like better.
• Discussed how we can automatically use one-to-one gesture tracking when an assigned action supports it.
• Drafted a config file format to associate (gesture) triggers with actions.

On day 3, we:

• Drafted a competing config file format which adds the same data to the existing kglobalshortcutsrc file instead.
• Reviewed existing gesture assignments and proposals.
• Created a table with proposed default gesture assignments (to be used once gestures are configurable).
• Collected remaining questions that we didn't get to.

What I just wrote is a lie, of course. I needed to break up the long bullet point list into smaller sections. In reality we jumped back and forth across all of these topics in order to reach some sort of conclusion at the end. Fortunately, we make for a pretty good team and managed to answer a good amount of questions together. We even managed to make time for ice cream and owl spottings along the way.

Since you asked for it, here's a picture of Natalie and I drawing multi-touch gestures in the air.

Next up in gestures

So there are some good ideas, we need to make them real. Since the sprint, I've been trying my hand on more detailed mockups for our rough design sketches. This always raises a few more issues, which we want to tackle before asking for opinions from KWin maintainers and Plasma's design community. There isn't much to share with the community yet, but we'll involve other contributors before too long.

Likewise, my first KWin MR for stroke gesture infrastructure is not quite there yet, but it's getting closer. The first milestone will be to make it possible for someone to provide stroke gesture actions. The second milestone will be for Plasma/KWin to provide stroke gesture actions by itself and offer a nice user interface for it.

Baby steps. Keep chiseling away at it and trust that you'll create something decent eventually. This is not even among the largest efforts in KDE, and yet there are numerous pieces to fit and tasks to tackle. Sometimes I'm frankly in awe of communities like KDE that manage to maintain a massive codebase together, with very little overhead, through sheer dedication and skill. Those donations don't go to waste.

At this point I would also like to apologize to anyone who was looking for reviews or other support from me elsewhere in Plasma (notably, PowerDevil) which I haven't helped with. I get stressed when having to divide my time and focus between different tasks, so I tend to avoid it, in the knowledge that someone or something will be left wanting. I greatly admire people who wear lots of different hats simultaneously, and it would surely be so nice to have the aptitude for that, but it kills me so I have to pick one battle at a time.

Right now, that's gestures. Soon, a little bit of travel. Then gestures again. Once that's done, we'll see what needs work most urgently or importantly.

Take care & till next time!

Comments