Short personal notice: I am currently in Cologne for a business trip lasting two weeks so I am staying over the weekend. If any KDE people around Cologne want to go for a beer until next Thursday, let me know :)

So, back to the subject. This blog entry is about a rather weird behavior of QBuffer I've debugged recently.

Some friends of mine were seeing a weird problem with some of their code using Qt4 that had previously worked in Qt3.
They broke it down to this minimal test case:

QByteArray data( 20, '\0' );

QDataStream writeStream( &data, QIODevice::WriteOnly );
QDataStream readStream( &data, QIODevice::ReadOnly );

qint32 a = 5;
writeStream << a;

qint32 b;
readStream >> b;

Q_ASSERT( b == 5 );

qint32 c = 2;
writeStream << c;

qint32 d;
readStream >> d;

Q_ASSERT( d == 2 ); // this fails, d == 0

Looking at the content of the byte array "data" confirmed that the write operation had been successful, i.e. "data" looks like this (hex encoded)

0000000500000002000000000000000000000000

So why did the second read operation return 0?

We have already determined that the write operations worked as expected, i.e. "data" contains the 4 byte representations for 5 and 2.

After studying the QDataStream code we concluded that it would not cause the observed effect since it basically calls the QIODevice's read method and then casts the result into the given result type.

So clearly the data QDataStream was seeing in the device was not the 0x00000002. However, a QIODevice::peek() refuted that quite annoyingly

qDebug() << readStream.device()->peek( 4 ).toHex();

Results in "00000002"

Damn!

At this point we were mostly out of ideas so we tried to manually set the read index to specific values:

const qint64 pos = readStream.device()->pos();
readStream.device()->seek( pos );

qint32 d;
readStream >> d; // still no luck, d == 0

const qint64 pos = readStream.device()->pos();
readStream.device()->seek( 0 );
readStream.device()->seek( pos );

qint32 d;
readStream >> d; // HAH! that worked!

Clearly something is going on behind the scenes that is undone or fixed when seeking away from the current position and repositioning again.

Lets expand the code a bit more:

QByteArray data( 20, '\0' );

QBuffer writeBuffer( &data );
writeBuffer.open( QIODevice::WriteOnly );

QBuffer readBuffer( &data );
readBuffer.open( QIODevice::ReadOnly );

QDataStream writeStream( &writeBuffer );
QDataStream readStream( &readBuffer );

qint32 a = 5;
writeStream << a;

qint32 b;
readStream >> b;

Q_ASSERT( b == 5 );

qint32 c = 2;
writeStream << c;

qint32 d;
readStream >> d;

Q_ASSERT( d == 2 ); // this fails, d == 0

This is equivalent to the first code snippet, we just explicitly create the QBuffer objects that handle reading/writing to the QByteArray.

Following the trail we finally discovered that QIODevice, the base class of QBuffer, is buffering reads in some sort of internal buffer.

Meaning our "readStream" was seeing a situation that was out-of-date, i.e. seeing the state of the memory buffer at the time of its first read:
"0000000500000000" instead of the correct "0000000500000002".

Why QIODevice::peek() was clearly bypassing that internal buffer is up to speculation. It was probably easier to implement than to return what the device would actually be using at the next read operation.

Conclusion: when using a QBuffer (directly or indirectly) for reading, remember to always also specify QIODevice::Unbuffered for open flags, otherwise it will waste memory on buffering already in-memory data and messing up read/write behavior.

The correct code for reading and writing a shared memory buffer with two QDataStreams therefore looks like this:

QByteArray data( 20, '\0' );

QDataStream writeStream( &data, QIODevice::WriteOnly );
QDataStream readStream( &data, QIODevice::ReadOnly | QIODevice::Unbuffered );

qint32 a = 5;
writeStream << a;

qint32 b;
readStream >> b;

Q_ASSERT( b == 5 );

qint32 c = 2;
writeStream << c;

qint32 d;
readStream >> d;

Q_ASSERT( d == 2 ); // Finally!

Some things are really really tiny details, yet they can be annoying in way. Something that's been occassionally bugging me is that fact that KDE uses the same wallpaper as KDM background, the splashscreen background and desktop background, yet depending on the screen resolution it may not be exactly the same background - during login the picture may stretch or shrink at certain points. The times when decent monitor screens had a 4:3 ratio are a thing of the past, starting with LCD makers making 5:4 "narrow-screens", then changing their minds and making 16:10 or 16:9 wide-screens. The choice of screen resolutions is not that limited either and that means that the wallpaper has to be scaled ... and that was the problem. Plasma has code to select how to do the scaling, KSplashX has code for that and KDM has code for that, and yes, you guessed it, it's always a different code. So unlucky resolutions get different wallpapers from different code. Since I actually spent some time in the past trying to make the login as seamless as possible, this indeed made me twitch whenever I saw it.

Seeing this again while testing openSUSE 11.3 made me finally spend the time to patch the openSUSE package to use the same selection code in all the three components. We really lack polish in so many places :(. But now it looks like the change is almost not there - there's just a progressbar and logo shown during startup and that changes to the desktop. With compositing enabled there would be also the fade-in animation.

Seeing that 4.4's KDM had no support for differently sized wallpapers, I was about to submit a copy of Plasma's code there when I noticed that trunk has some code for it. Of course, different from the rest again. Also, the login sequence is basically just lucky to be so smooth. The splashscreen is supposed to stay visible until Plasma is ready with its wallpapers and panel layout. And there is code in KSMServer to ensure this. And Plasma uses it. Yet it's apparently not used properly - during the first login, when there is more setup to be done during login, it's perfectly possible to see how the panels are set up. Well ... maybe in time for openSUSE 11.3 + 1.

There is something magical about hacking on things without having much clue about them. It almost feels like a treasure hunt, with mysterious traps all along the way and an elusive treasure maybe at the end. Today's treasure is KDE4's Konsole (preferably not) being awfully slow with some fonts. Specifically, as irony would dictate, the rule could be almost said that the better suited font for Konsole the slower the text rendering is, whereas if the font breaks the text layout completely or hurts to look at, the speed is just fine.

Profiling showed only that the CPU time was all spent in the X server in the nvidia driver, which is really not that helpful with something that is closed-source. It only meant that Qt was feeding to X something that the driver didn't like much. Eventually, I tracked it down to one XRenderCompositeText32() call in Qt (which, in a retrospect, wasn't really a very obscure hint - if there's something slow with drawing, suspecting XRender is always a worthwhile guess).

That showed where the treasure was, but now there was the part of getting it. Not that easy if the knowledge about font rendering doesn't go much further than knowing it's drawing text. Especially when it turned out that disabling the XRender path in the code resulted in some fonts being shifted one pixel down (and only some of them, so that it wouldn't be too simple).

But anyway, to make the story short, for those who want a share in the treasure, the patch is in the Qt bugtracker, waiting for somebody with more knowledge to answer the questions for which I don't know the answers. And home:lllunak:branches:KDE:Qt and home:llunak:branches:KDE:Qt:STABLE repositories have the 4.6/4.5 Qt packages with the patch added.

The KDE BugSquash team is holding another testing and bug triaging day tomorrow, Sunday 9th of November, to help maintain and improve the quality of KDE PIM applications.

Especially versatile applications like KMail and KOrganizer can potentially be tested numerous times by its developers without finding any issues because it requires a certain workflow or data set to trigger them.
Therefore help by as many volunteers as possible is hugely improving the situation, because every person will have different goals, preferences on how to do things (e.g. mouse v.s. keyboard), data sources, amount of data, etc.

If you are interested and have some time tomorrow you'd like to donate to KDE, see the BugSquad's information page.

My sister-in-law got married and the wedding photographer made a DVD with 412 pictures. Unfortunately, my parents-in-law could not read the DVD, either with Linux (which is their main OS) nor with Windows XP. So they gave the DVD to me and I had a look.

The DVD is a DVD-R and my Linux version (Hardy Heron) tried to open it: it showed the dialog asking if it should mount the DVD. A failure message opened asking me to have a look at dmesg. Hmm.

So I did and it said:
[ 2981.006899] UDF-fs: No partition found (1)
[ 2981.125072] ISOFS: Unable to identify CD-ROM format.

So I tried to install some kernel patches for udf.ko, but this did not help. The disc was still not recognized. In the end I decided to call upon the trusted program hexdump. Hexdump should be compulsory at kindergarten!

Browsing through the data with hexdump, I quickly recognized JPEG headers, so I wrote a small program (70 lines) to extract all JPEGs from a binary blob. You can download it.

The first megabyte of the DVD is here.

00000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
*
00008000 00 42 45 41 30 31 01 00 00 00 00 00 00 00 00 00 |.BEA01..........|
00008010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
*
00008800 00 4e 53 52 30 33 01 00 00 00 00 00 00 00 00 00 |.NSR03..........|
00008810 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
*
00009000 00 54 45 41 30 31 01 00 00 00 00 00 00 00 00 00 |.TEA01..........|
00009010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
*
0000a000 01 00 03 00 48 00 00 97 21 87 f0 01 14 00 00 00 |....H...!.......|
0000a010 01 00 00 00 00 00 00 00 08 55 44 46 20 56 6f 6c |.........UDF Vol|
0000a020 75 6d 65 00 00 00 00 00 00 00 00 00 00 00 00 00 |ume.............|
0000a030 00 00 00 00 00 00 00 0b 01 00 01 00 02 00 02 00 |................|
0000a040 01 00 00 00 01 00 00 00 08 31 33 33 41 33 31 32 |.........133A312|
0000a050 43 20 55 44 46 20 56 6f 6c 75 6d 65 20 53 65 74 |C UDF Volume Set|
0000a060 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|

I learnt a something new about XML today, a part of the specification that deals one of the many edge cases that exist in every file format. To illustate this, lets take a look at a few examples. Why is this XML document well-formed :

<test>X</test>

this one also well-formed:

<test>& #9;</test>

But this document isn't:

<test>& #7;</test>

Note that I've added an extra space to these examples as the blogging software used by kdedevelopers.org seems to quote the characters required to make this appear directly.

To find out why it's broken, read on...

As one of my colleagues notes, this statement holds true on its own. However, for those like me too blind to see some things, there are two things about Flash you should know:

1. The latest Flash update fixes various critical security issues.
2. The latest Flash update does not work with anything that is not Gecko-based.

Do the maths yourself. Or just watch it happen as soon as your distribution's update hits your machine.

I'm not going to comment on the first one, but, as a person who has spent a good portion of the last two weeks trying to find a solution, I know some things about the second.

First of all, this is apparently perfectly fine with Adobe. According to their system requirements page, they care only about Firefox, Mozilla or SeaMonkey. Tough luck if you use something else (by the way, according to the latest Linux desktop survey, this something else is at least a quarter of Linux users). Imagine a critical security issue that will exist only when using Flash in Konqueror or Opera but not in Firefox (for whatever reason, like Adobe testing Flash only in Firefox) - care to guess what happens in such case? After all, you can give it a little test - just try to find out what happens if you complain to them that Flash doesn't work for you in Konqueror.

You know, I haven't been a huge fan of Moonlight, but there's definitely at least one thing to like about it - it's open source. Doesn't work? Have a look at the code.

Not so with Flash player. The reason why the latest Flash doesn't work originally appeared to be its new support of some Mozilla XEmbed-based extensions to the plugins API (funny thing about that link is, it says that it finally makes it work with Opera, while in fact it's exactly the opposite). After adding XEmbed support to Konqueror, it still didn't work. The page about the XEmbed extension has demo code hardcoding a hard dependency on Gtk2, so maybe adding Glib2 eventloop support will make it work? After all, the Flash system requirements page mentions this (well hidden in a footnote, if you look close enough), but not really, tough luck, even though the DiamondX testing plugin already works. Do you know what a ballistic approach to debugging is? You either hit, or you don't. Here next hit is that this Flash version doesn't handle properly repeated NPSetWindow() calls, which however happens to be the case with Konqueror. So finally, does it work? Well, kind of, if you don't mind it crashing everytime you leave a page. And it crashes in XtRemoveTimeOut() (incidentaly, a function that should not be called by a Gtk2-based plugin). That's been already taken care of as well - it's enough to give Flash the user agent string from Firefox, suddenly, no crash (I have no idea how Maksim managed to find this out - probably involved sacrificing chicken or something).

So, when can you expect the latest Flash to work with KDE? I have no idea. There are some preliminary patches I've made and committed to SVN, those may or may not work for you, depends on how lucky you'll get. Given that people report random crashes with those, I really meant to say lucky. Right now I'm trying to debug another issue where a proper fix that should remove a race condition actually prevents it from working. I also can't get keyboard focus right, even though debug output from DiamondX seems to look correct and I can't see any significant difference to Firefox. Maybe my fault ... maybe not, given what I know about how Flash handles keyboard input (you don't want to know).Opera people apparently have been more lucky with dealing with this, their latest beta should work better, but I could notice some problems there as well.

So, sorry. E.g. OpenSUSE will probably ship KDE updates for this tomorrow, but I suppose you'll be better off using the one true browser for the time being, if you need Flash. Next time I provide KWin videos only on youtube, somebody please yell at me. A lot. I'll deserve it.

PS: Yes, I know a Flash developer contancted kfm-devel somewhen in the past asking about XEmbed support. That still doesn't make debugging Flash any less painful, which is why the workarounds for it will take unknown time. It also doesn't change anything about Adobe not really giving a damn about Konqueror or Opera, if they could suddenly drop backwards compatibility (even though the code is probably still there, if it can still call XtRemoveTimeOut() and crash on it).

Bugs are bad, I think we all agree on this.
One of the most evil kind of bugs are those that appear only during installation or installation related procedures.

I wrote in an earlier blog that I have been helping the Debian-KDE packagers fix a bug.
Unfortunately, due to a side effect (I hate side effects), a newly installed KDE would no longer find the KControl modules!
The bugs does not appear when upgrading the packages since the old, wrong, directory is still there. So it took someone with a fresh installation to even discover the problem :(

Obviously the programs which are most likely to be bitten by this are installers. For example, a user posted to the debian-kde mailinglist that the Debian installer, or more likely its tasksel stage, have a bug when installing the desktop task, i.e. the KDE desktop is not installed correctly.
Since installers, especially new ones, are mostly used when a new distribution release is about to happen, so their developers are really the poor ones.
Lets hope the DDs can figure it out before the Etch release, would be a pitty to have the Debian desktop weaken while it is having heavy competition from derivatives.

What is the usual procedure to test installers, or how could regular users help testing them? Virtualisation? Spare machine?

Most development I do lately is not directly visible in KDE's source repository, but rather development for other projects connected to or interface with KDE.

Since I am a Debian user, a very happy one :), I am following the debian-qt-kde mailinglist, to know about issues our Debian packagers might encounter.
Sometimes this requires just adding some information to a bug report, sometimes it requires doing some research in bug tracking systems and code respositories and sometimes it requires coding.

One of those which involve coding has been this one: removing the requirement for the SuperKaramba package to depend on XMMS by dlopen'ing libxmms instead of having it as a build-time option.

Those involving doing bug research are obviously less fun, nevertheless quite educating. The one I am currently on is this one:
the packages for KDE breaking KDE's compliance with the freedesktop.org menu specification.
Turns out the reason for the breakage is a hotfix/workaround for a conflict with a GNOME package ages ago!

Since it is very likely that the respective GNOME package has also been fixed by now, the correct KDE behavior can be restored in time for the Debian Etch release :)

So, off to check if I can fix the xdg-utils bug I discovered a few days ago.

P.S.: if you know somebody with knowledge about KURL internals, maybe we can even fix this problem