The Future of KTextEditor

I've been working on something pretty big for katepart for a while now, but it's just struck home how widely applicable this new idea will be for apps utilising the KTextEditor api.

The whole thing started when I was looking for a way to enable custom programmatic syntax highlighting. I first implemented a smart cursor which retained its relative location while the text around it was edited; then extended this into a pair of smart cursors to define a "range" of text. These objects give forth plenty of useful, convenient signals, such as changed(), moved(), etc. This looked like a promising way of implementing custom HL.

Yesterday, it struck me how these smart cursors could replace most of the clumsy mechanisms used today to interface with text editors. As it stands, most 3rd party apps like KDevelop, Quanta etc. all maintain an externally parsed version of the document's buffer. This is not only inefficient, but the hooks to enable on-the-fly parsing (rather than delayed / background parsing) are not there, so there is always some lag time with the parsed versions.

Here's how I envisage this new interface would work:

  • parsing could be implemented with the help of smart cursors and ranges. On-the-fly parsing is made easy by being able to see which parts of the document have changed, and only reparsing that area (and affected external areas too). Thread safety could be achieved easier without a big lock using this method too; no more lag from locking the whole buffer in order to do background parsing.
  • custom syntax highlighting becomes easy when this parsing system is adopted, the information is already there and shared by the editor part. The developer would eg. associate function call ranges with the function call highlighting.
  • KActions could be associated with smart ranges, enabling and disabling themselves automatically as the caret enters and leaves them, and emitting signals with all the information needed to do their jobs so the 3rd party code is kept simple. I think this has huge potential to make 3rd party text editor app development dead easy, which should lead to much heavier use and more great features for eg. kdevelop, quanta, kile, etc.
  • cut or copied ranges could be retained in memory, so persistence of text and its attributes could be achieved when shuffling code around (if the developer so desires)

The KActions idea is the new one, and I think it will be the key to a great successor to the KTextEditor api. For example: the c++ parser surrounds each method declaration with a range. The developer then adds a "go to next method" action to that range, and either writes a small amount of logic or uses some built-in find-next-similar-range logic. The action enables and disables itself depending on the location of the caret. When you right-click the text, the action is automatically added to the context menu. Ranges can be made to change their syntax highlighting in an efficent way (ie kate will know how to do it as efficiently as possible) on mouse and/or caret enter & exit. An extension of this is an eg. "highlight all uses of the variable the caret is over" feature; another is a "change the background colour of the current method slightly" feature - I'm thinking this could result in a more visually pleasing, dynamic and useful user interface.

Much of the KTextEditor interfaces as they currently exist could be re-written in a much simpler and more powerful way using this idea.

Today I also thought of a way for this to scale with large documents and many thousands of smart cursors, so I'm not concerned about performance. I also think that other KTextEditor parts will be able to pick up the api changes without major stress.

So, at the moment I'm porting some of the improvements I made to the kate bidi branch along this line back to head (the bidi work is on hold until Qt 4, I realised the technology in 3 is not ready). There's a lesson for you, if you don't already know it: try not to develop more than one new major feature in a branch, especially when one of those might not be ready for a long time.

I'll be playing with this idea for a while and hopefully come up with something useful soon...


Glad to see you've done this work. This is also very applicable to applications like:

1) Do a spellcheck and dynamically highlight all mispelled words
2) Tell KDevelop to highlight all unused or undefined variables the parser finds laying around
3) Close a brace and the entire block of code/html/whatever briefly highlights a different color then changes back ...

I'm curious, do you have notions of 'style' for this? For instance I'd want the mispelled words done in red squiggly lines (like in most word processors) ... then maybe I want all user-defined objects to be underlined in blue.

Also, you've no doubt ran into ranges and KateArbitraryHighlight within kate. How did those change or did you just replace them completely? *sigh* my current job situation prevents me from hacking around all the time :-/ Regardless, you should post what you have to kwrite-devel

By KDE User at Thu, 02/26/2004 - 03:45

Yes, by applying the same Attribute instance to multiple ranges, it will be akin to a style.

Those classes were written by me, and (reworked) will be part of this plan :)

By Hamish at Thu, 02/26/2004 - 04:41

This is something I've been dreaming of for some time! There are a lot of great ideas mentioned for this already but it's worth noting that as we expand the abilities of the parser in Quanta we've run into definite issues with speed parsing PHP. While it looks like we have this under control we may well end up threading in the future. We've also wanted to do background highlighting. Users have also asked to be able lock regions of text for uses like dynamic templates. One thing I've had a dream of for a long time is to enable multiple markup highlighting in a document by weaving the individual highlighting configurations with the parse. This could simplify the highlighting set up for users and could help to simplify configuration too.

I'm sure Andras and Nicolas will be happy to read this as well as our users. Many thanks for your effort to take on the big challenges and keep after excellence. Developing on KDE is never being far from good news! ;-)

By sequitur at Thu, 02/26/2004 - 06:26

If this pans out, it would be really, really great.

Not for the syntax highlighting that you are aiming for, but for find highlighting.

If anyone has used the Google toolbar for IE, they'll have played with the highlight function, which highlights your search terms throughout the entire document (with different colours for each term in the phrase). It's a superb feature for searching within web pages and it would be even better for Kate, KWord, KDevelop and other applications within which the user uses the search facility a great deal.

It's far, far better than Find/Find Next...

If you could get this working and then enable other applications to use it for 3.3, I'd be very, very happy. It's a killer feature.

By luke chatburn at Thu, 02/26/2004 - 12:35

Hm, I wonder if this would make multiple text selections possible :). Imagine having a XML document and shift-selecting both start and end tag in some line, omitting the element contents in the middle - and then using copy'n'paste.

By eike hein at Thu, 02/26/2004 - 18:09

In KDE 3.2 you could also activate Klipper and select the multiple text sections each separately and then paste them using the paste combo button listing Klipper's clipboard history entries.

By Na at Thu, 02/26/2004 - 19:44

Or for KWord, example:
Select some words in a long text and then apply bold to them. ;)

By Mathias Panzenböck at Fri, 02/27/2004 - 22:20

I'd be all happy to see this in HEAD, as you all mention, any number of applications for this feature can be thought of.

It could possibly be used for the folding tree as well, and for a feature I plan on exploring sometime soon, partial display: Only displaying the current method/class/section of the edited document.

Not to speak of paren checking and such ...

We just can't wait :-)

By KDE User at Sun, 03/07/2004 - 21:27