AUG
31
2005

RuDI or tribute to the helping hands in Nove Hrady

Things should be made as simple as possible, but no simpler. -- Albert Einstein

When doing my diploma in theoretical astrophysics in Tübingen I always tried to remember these words from maybe the most brilliant scientist of the last century. Staying to this principle really helped a lot when trying to understand many aspects of even the most complex and difficult.

I think that Einstein's principle fits very well for today's software development process including such complex efforts as a complete and feature rich desktop environment.

Too bad that there are many examples of both proprietary and opensource software which is either too simple or unnecessarily complex. The former often leads to inefficiency in the use of resources and missing completness with regards to features. The later results in hard to maintain software which cannot be reasonably integrated with other offerings and therefore often leads to little monoculture islands. Monoculture cannot be the goal for free software which is all about freedom and a healthy ecosystem built on top of it.

To solve the task of integrating with other systems, components and applications is the main goal of a modern desktop environment.

It seems that reaching the right level of simplicity is more like a dream than reality. It looks more that simply limiting the choice and the customizability is not reaching the goal in any significant manner.

What is going wrong?

On one hand people try to solve the problem by pushing for a single platform. This is what Microsoft has been telling people for years. But going for a monoculture has well known problems including increased vunerability to deseases like viruses or worms. IMHO the worst is that the users are those who suffer most from such a situation as shown in multiple court cases where it was shown that consumers get harmed while monopolies love monocultures. Finally a single platform leads to a lack of cross breeding and lower fertility.

In the case of the Linux Desktop the later means that we require multiple simultaneous approaches to the desktop challenge which not only consists of the far developed KDE but also of other offerings like XFCE and GNOME.

On the other hand people try to make things too simple. E.g. for most of my purposes XFCE is really nice, fast and beautiful but lacks just essential features and does not scale for some of the large installations I am doing like the trading room of a Bank in Frankfurt, Germany with nearly 400 seats all using KDE multi-head with up to 5 monitors each. The lack of the KIOSK lockdown and management framework simply means that XFCE is out of question even though the stock traders mainly run vertical application side by site with Citrix, Konqueror and Kontact.

What is the core issue about complexity?

There is a difference between undesireable complexity and feature richness or large works. The source for complexity are dependencies between the individual components. The problem starts to develop
when the dependency graph starts to become complex and maybe even circular. The second source is unnecessary dependencies. For software the former can be often addressed by redesigning and refactoring while the later means issues with artifical dependencies on 3rd party ABIs, IPC mechanism, filesystem hierarchy, filenames, program names and programs depending on side effects.

As Hao He rephrased Einstein's famous principle: "Artificial dependencies should be reduced to the minimum but real dependencies should not be altered."

Linking to KDE libraries is considered undesireable complexity?

In the past we always asked developers of 3rd party KDE applications to link against the KDE libraries. Only the KDE libraries offered a possibility to fully integrate into the desktop and its many offerings including a common look and feel and network transparency for file I/O. Linking has the well known problem of being dependent on a stable API and ABI. Unfortunately the ABI of the deployed KDE desktops changed much too often during the last 9 years not only because we managed to release three major versions of KDE with each breaking not only binary compatibility but also the ever changing ABI of our favourite C++ compiler.

Also the API was and will be significantly changed between major releases.

Some 3rd party developers tried to escape with linking statically to the KDE libraries. While this helped to ensure that the applications can run on more KDE installations it also means that bug fixes and integration in to the evolving desktop does not work over time.

[image:1396]

RuDI Defined and Explained

RuDI is an architectural approach whose goal is to achieve loose coupling among interacting software components instead of linking to libraries. A service is a unit of work of the desktop done by a service provider to achieve desired end results for a 3rd party service consumer.

The idea of RuDI departs somewhat from that of OO programming, which strongly suggests that you should bind data and its processing together. The results of a service are usually the change of state for the consumer but can also be a change of state for the provider or for both.

How does RuDI achieve loose coupling among interacting software agents? It does so by employing two architectural constraints:

An extensible XML schema allows new versions of services to be introduced without breaking existing services. Existing IPC mechanism like DCOP define an interface which is not as easy extensible as an XML schema. If interfaces do not work, systems do not work. Interfacing is also expensive and error-prone for distributed applications. This means that if we use an XML schema to describe the service we introduce loose coupling instead of a close API/ABI dependency. E.g. the ABI dependency of DCOP makes it less useful from JAVA application when trying to exchange more complex data than plain numbers or strings than really necessary.

Second we send messages over a protocol instead of calling explicit individual member functions. The messages must be descriptive, rather than instructive, because the service provider is responsible for solving the problem. This is like going to a restaurant: you tell your waiter what you would like to order and your preferences but you don't tell their cook how to cook your dish step by step.

Service providers will be unable to understand your request if your messages are not written in a format, structure, and vocabulary that is understood by all parties. Limiting the vocabulary and structure of messages is a necessity for any efficient communication. The more restricted a message is, the easier it is to understand the message, although it comes at the expense of reduced extensibility. XML is a natural choice for expressing these messages.

Extensibility is vitally important. It is not difficult to understand why. The world is an ever-changing place and so is any environment in which a software system lives. Those changes demand corresponding changes in the software system, service consumers, providers, and the messages they exchange. If messages are not extensible, consumers and providers will be locked into one particular version of a service. Despite the importance of extensibility, it has been traditionally overlooked. At best, it was regarded simply as a good practice rather than something fundamental. Restriction and extensibility are deeply entwined. You need both, and increasing one comes at the expense of reducing the other. The trick is to have a right balance.

RuDI must have a mechanism that enables a consumer to discover a service provider under the context of a service sought by the consumer. The mechanism can be really flexible, and it does not have to be a centralised registry. Using a mime-type approach seems reasonable as we already have expirience with this.

[image:1397]

Now lets see how this fits into the Linux desktop. As an example you may look into how RuDI allows for the nice integration of a Qt application in a KDE Desktop. The Qt Application links to the Qt library and the Qt RuDI library to archieve desktop integration. The Qt RuDI library talks to the RuDI Service Provider via an XML wire protocol. DBUS seems to be a natural choice here. The RuDI Service Provider (RSP) than calls the desktop specific service. RuDI services can be directly linked into the RSP so that the execution happens in the RSP process or they can be implemented out of process. In any case an optimization based upon the kdeinit concept with fork() and exec() is a nice optimization for perceived speed, CPU and memory.

In case the Qt application runs on a system which does not provide the RSP it is the job of the QRuDI library to fall back to a Qt only functionality in a transparent manner for the programmer. Of course if some services are not available they don't have to be emulated by the QRuDI library.

The nice thing about going for a protocol instead of an programming interface is that it is much easier to do cross platform and language independent interoperable applications. In my opinion systems like JAVA and in the future maybe MONO are not programming languages anymore they are living in their own universe and actually define their own platform. In order to integrate non native applications into either the KDE or the GNOME desktop RuDI will really help.

Goals of RuDI

First make it possible to create desktop applications in any language of choice which can integrate fully in any Linux desktop which has implemented RuDI. Basically this allows any contributor to the Linux desktop ecosystem to choose their development tools of choice independent of the desktop the application is going to be deployed. In case RuDI or a requested service is not available it is falling back to the old behavior.

Second RuDI wants to enable KDE and GNOME to seperate the desktop workplace from the applications running on top of them. For example this allows for better integration of MONO apps in a KDE version which is shipped by Redhat and might have no direct support for Mono built into neither the KDE nor GNOME desktop. Seperating the applications from the desktop workplace simply means removing unnecessary dependencies and frees many resources due to a decreased complexity. Basically this is the only way to allow 3rd party opensource and proprietary developers to run on their own release schedule while freeing the developers of the desktop from the binary backwards compatibility jail.

This is my first attempt to blog and I am afraid the entry is already rather large. I therefore decided to write in another entry about the results from the RuDI BoF. This BoF is scheduled to start at 3 pm this afternoon in the Marbella room. We will then have time till 5 pm when the KDE working group will start. Everyone attending the aKademy 2005 is welcome to join the discussions.

Please check the works from Hao He http://www.xml.com/pub/au/213 (he is also a physicist :-) ) from the W3C Web service architecture working group about Service oriented architecture (SOA). Much ideas, concepts and wording was taken from him. In addition having a look at WSDL and BPEL.

Last but not least do you now how RuDI is related to Nove Hrady?

References:
http://www.w3.org/TR/ws-arch/ - Web Services Architecture
http://www.w3.org/2001/tag/webarch/ - Architecture of the World Wide Web
http://www.w3.org/TR/wsdl - Web Services Description Language (WSDL)
http://en.wikipedia.org/wiki/WSDL - Wikipedia Web services description language
http://java.sun.com/developer/technicalArticles/WebServices/soa2/ - Service-Oriented Architecture and Web Services: Concepts, Technologies, and Tools
http://weblogs.cs.cornell.edu/AllThingsDistributed/archives/000343.html - Web Services are not Distributed Objects: Common Misconceptions about Service Oriented Architectures
http://msdn.microsoft.com/architecture/default.aspx - Microsoft portal for discussing architecture

Comments

I definitely think integration of loosely coupled components, web-services style, is the best way forward. It works on the web, and it should work well also in the heterogenous world the free software community is. I have only one question? - why do you call it RuDI? I find no explanation of this name, and I don't find any references to it. I think using existing terminologi would work better unless you mean something different, in which case you should explain it.

I often thought that building REST apis for desktop applications would be a good way to integrate applications across ABI boundaries.


By claes at Sun, 09/04/2005 - 17:19

The short version to this answer is because I don't know better.

If you are interested in the long version of the answer write my a pm.

I don't want to use the well established name [http://www.xfront.com/REST-Web-Services.html|REST] because this term is already well understood and RuDI is slighly different. Using the same name for two different things would imho cause confusion and not meet the expectations of the people.

In addition neither RuDI not REST are very [http://appeal.kde.org/|appealing].

Proposals for a better name are very welcome and are definelty worth being considered in this early stage of the project.


By Martin Konold at Thu, 09/08/2005 - 04:03

REST does seems like a good way to do this. Does RuDI include an MVC-style traffic cop? If not, who keeps track of the state changes?


By whalesuit at Thu, 09/29/2005 - 14:24

im wondering about a few things;

i) what is the impact of this approach on the performance

ii) so when plugins are connected in this fashion they cannot bring down the app when they crash?

iii) dont we need this then for konqueror? (the video and flash plugins crash it all the time)


By cies at Sun, 09/04/2005 - 22:45

ad i)
I don't expect a noticable performance penaltity. Actually in many cases (e.g. a File_Open dialog) RuDI is is faster than the traditional approach.

How can it be faster and use less memory than the traditional approch which means linking to a library?

Well basically this boils down to the fact that a single service can be used by many clients. In the case of the File_Open dialog this means that the service is already available and does not need to be created when used (better latency, no trashing...) and that different clients can use the same service simultaneously which also saves memory.

This is similar to the effect that when starting a new konqueror from the panel actually not a new binary is launched but the existing konqueror (possibly hidden) instance is told opens a new window. For the enduser this optimization is invisible except the speedup.

ad ii)
Yes, because RuDi is out of process a failing service cannot crash the application. Actually because the RuDI lib has native fallbacks and error handling it means that in case the service fails the native toolkit specific fallback method is used instead. So e.g. a QApplication is running on KDE it tries to use the KDE File_Open service but in case of an error it falls back to QFile.

But please be aware of the fact the I am currently not aiming at plugins (e.g. gui embedding etc.). RuDI is initially limited to the simpler desktop services which are listed on the [http://www.kdedevelopers.org/node/1396?size=_original|RuDI mindmap].

ad iii)
No, I currently don't look at this kind of embedding though someone might want to elaborate on that.


By Martin Konold at Thu, 09/08/2005 - 03:48

I've been creating and working on a couple of embedded applications that use messaging for a 3 years and have found that loose coupling to be nice and quite workable. Is RuDI going to use asynchronous or synchronous messaging?

Also, in our system we use binary messages and we chose this because of the extra CPU power to marshall/unmarshall XML to/from Binary didn't seem worth the cost.

I would contend that a system using binary messaging can be extensible enough. One way we are considering doing this is having a version field in each message and that "new" data always follows "old" data in the message. That way you have backwards compatibility, extensibility and efficiency.

Cheers,

Wink


By winksaville at Wed, 09/07/2005 - 04:31

RuDI will offer both asynchronous and synchronous services. The implementation is simple because the serverices run all out-of-process.

IMHO binary messages are not well suited for our purposes because we want to be language independent (speak data types) and we use hashing/caching to keep the overhead low. Basically this boils down to avoid the repeated WSDL overhead. I currently don't consider the messaging itself to be critical as the services are typically "fat" compared to the messaging or performance is non critical (e.g. when talking to the screensaver to shut up while displaying a presentation in fullscreen...)

Please be aware of the fact that RuDI is not meant for low level IPC but for higher level desktop services which shall be compatible across the desktops (GNOME/KDE) and allow applications to integrate in either desktop seemlessly.

Please check the [http://www.kdedevelopers.org/node/1396?size=_original|RuDI mindmap] for a list of initial services.


By Martin Konold at Thu, 09/08/2005 - 03:25

That makes sense as far as it goes, but contrast that with our system which is composed of 75+ objects all communicate asynchronously via messaging. The objects range from ISR's handling real time data to application level objects handling user commands which cause messages to be sent between processors communicating via a special bus and/or a LAN.

That is what I see as the biggest advantage of binary messages.

Cheers,

Wink


By winksaville at Thu, 09/08/2005 - 04:22

The "Generic desktop adapter" uses a slightly different approach:

http://www.scheinwelt.at/~norbertf/dadapt/

It doesn't require IPC or XML and uses dlopen() to provide access to "desktop services" like the VFS, File-Choosers,...

Although i agree with lots of your points i suppose using XML over sockets might be too much overhead.

Just an example: Providing filedialogs means that applications also need access to VFS ("network transparency") libraries for downloading and uploading files. Streaming file-data with SOAP or any similar socket protocol is a performance nightmare.


By nf2 at Tue, 12/06/2005 - 02:24