JUN
11
2005

New Smoke proposal? Bindings broken by design...

I've been following Ashley and Richard's conversations about the new Smoke proposal for KDE4 and I have to say it all reminds me exactly why I have grown to loathe bindings. First, let me say that I have loads of respect for both Ashley and Richard as they are extremely talented developers. Both of them helped me enormously with the now discontinued Qt# bindings. With that said, I think KDE bindings developers and enthusiasts are completely missing the mark for a possible future KDE/Qt binding roadmap.

It seems there is a growing number of KDE'ers with the will to change the KDE binding's situation. If we continue down the path we're now taking for KDE4 we're going to get the current binding's situation perhaps with more visibility in the community. Perhaps the kdebindings module can be cleaned up to the point where it builds reliably well. Perhaps we'll see the adoption of the some of the better bindings (shout out to Ruby, Python and KJSEmbed) in stand alone applications or scripts in Plasma. If these are your only goals, then please, just ignore this post. If however, you want more for KDE bindings then read on...

Let's take the new smoke proposal for instance. Here we already have an enormous library that is about to get even bigger by creating a QObject derived proxy for every non-QObject class in KDE/Qt libraries. Think about that for a second. That means every non-polymorphic class in the current libraries will now have a virtual table. Forget that the hot code paths in KDE/Qt will now have this extra overhead and just think what this will do to the size of this already enormous lib. Further, we'll have to continue overriding each and every existing virtual function and then on top of that we'll need to make it a slot.

What Ashley, Richard and all other bindings developers are doing is actually quite a remarkable achievement, but it is also the source of the problem. I believe the entire concept of bindings as we know it is fundamentally broken. Think about it for a second... what is a binding, really, after all?

Creating a traditional language binding is nothing short of trying to export an entire language's features by writing a shared library. IMHO, the mother of all fundamentally broken designs. Well, actually, there is one that is worse, but you have to read on for that one...

Actually, it is worse than that, because most binding authors see the above as not that challenging enough... no, no, we have to make it harder by writing a program that automatically writes the shared library exporting an entire language's features. LOL.

And what do we get for this back breaking work?

1. A platform that will never match the speed or performance of the original library by design!
2. A platform that will never have the same feature set as the original library by design!
3. The inability to ever write a shared library that can be consumed by the original language by design!
4. The inability to ever write a shared library that can be consumed by other languages bindings "
5. Awkward and arresting incontinuities between the original language's types and the target language "
6. Awkward and arresting incontinuities between the original language's features and the target language "
7. All of this has to be repeated for every new library that we add and every change we make to the current libraries "
8. Third party library makers that are in some way affiliated with KDE/Qt will have the same issues "

In short, a second class development platform by design!

Now, it seems there is a growing number of developers with the will to change this situation. If we continue down the path we're now taking for KDE4 we're going to get the above, but perhaps with more visibility in the community. Perhaps the kdebindings module can be cleaned up to the point where it builds reliably well. Perhaps we'll see the adoption of the some of the better bindings (shout out to Ruby, Python and KJSEmbed) in stand alone applications or scripts in Plasma. If that is the goal, then as I said at the top ;), just ignore this post.

The only real way around the problems I've highlighted above is to, as I've been saying to anyone who will listen, build the binding into the compiler. Actually, it is more like building the binding into the _toolchain_ just like Microsoft has done with the Common Language Runtime. Although, Microsoft is not the first to come up with this idea, they've done the most thorough job of it. One way of solving our problems, of course, is by building a C++ compiler for Mono/PNET based upon the ECMA spec. The advantages here would be an already developed CLR implementation for Windows/Linux/MacOSX. The disadvantages would be the legal uncertainty surrounding this whole path. I'm not in favor of this, myself.

As an aside, I must say that I find it absolutely hilarious the position that the CLR/Mono folks find themselves in. They have this new platform that is supposed to be a binding killer, but they still have to write a binding for their old technologies because they can not be naturally consumed by the new platform. They have to graft their object model that was hacked into the library onto the object model of their new platform. Thinking about how fundamentally broken that is... and I have to say that this is truly the mother of all fundamentally broken designs. Thus, even though they have this new platform, they also are crippled by the problems with traditional bindings. LOL.

The other option would be to adapt LLVM to support a higher level bytecode presentation. I've talked about this in numerous locations and I'm sure some are probably tired of hearing me say it again and again. If you haven't heard about LLVM you can read a little about it here.

To all of you who are tired of hearing me go on and on about this... It is an enormous undertaking, but I can't help but believe that with the renewed interest among KDE developers towards the bindings, and with very talented hackers such as Ashley, Richard, Alexander, Ian, and the rest that we could get much farther and much faster with this approach than with devoting so much time into what I feel is a withering end.

Comments

Currently, LLVM can compile C/C++, they've got the GCC 3.4 parser, it's delicate but you can set it up. Supposedly the PPC code that's generated is very fast, and Apple has been paying some developers to work on it even, and x86 performance is being pushed as well. It also has a backend for Itanium, SPARC and Alpha. There is a lot of interest.

AFAIK, it wouldn't be hard to extend it to run the CLI, it was actually done at MS internally. Even java could be ported.

How do you really want to use it, however?


By S at Sat, 06/11/2005 - 22:29

Yah, I am very familiar with LLVM as I was helping to get Qt to build against it this year. Running the CLI isn't what I'm talking about, however. What I want to do is add-on to LLVM by creating a higher level bytecode representation that the frontends could compile to. Then this would be converted to LLVM bytecode for LLVM to either JIT or spit out native code.


By Adam Treat at Sun, 06/12/2005 - 00:31

Then you might want to look into the following project, www.x-p-s.net. It's a meta-programming system. So you could make DSLs, general languages whatever. I can't think of any reason why you couldn't make a higher level representation byte code with it either. In fact this might be a good test to find any possible deficiencies in XPL core (the base language) -- the base language (XPL) is basically a 1-1 to llvm assembly. It's in the early stages, but it's comming along nicely. The goal is to build up all the various semantics such that we can represent any language we want or at least the ones created thus far. The main hurdles are actually existing tools and of course, working out a good model for respresenting the various semantics.

There is an irc channel irc://irc.oftc.net/xps, same network as the llvm channel.

Mind you, it's still rather raw, but who knows.


By S at Sun, 06/12/2005 - 04:45

The big problem from the standpoint of bindings is that KDE is written in C++, and C++ does not mix well with other languages. Many features that make C++ easier than C for application writers make it harder to interface with other languages. C++ features like overloading and virtual methods are set up so that they "just work" for from the standpoint of the C++ programmer. The features that are nice for C++ programmers are horribly difficult for people who want to use bindings. The GTK+ toolkit is much easier to deal with many in respects because its being written in C requires it to provide, for example, a much more explicit interface to events and signals.

I do not see an easy and cheap solution to the problem. Writing GCC frontends for other languages might permit the languages to interface with C++, but there is still the problems that users of languages expect the languages' standard libraries be present. For example, C# users would expect that at least the CLI base class libs would be available.


By Marcus at Sat, 06/11/2005 - 23:21

Pages