MAY
3
2006

ThreadWeaver News: Collections and Sequences

Submitted by Mirko Boehm

For those who are not familiar with it: ThreadWeaver is like a
multi-threaded make tool for application developers. It provides means to chop
operations into jobs and declare the way they depend on each other. When
started, the jobs will be executed by a pool of threads, which will
automagically try to find the most efficient order of execution. With the
lately released version 0.5, it now has job collections, job sequences, and
qtestlib based unit tests.

When ThreadWeaver was introduced at last year's KDE conference, a number of
concepts have been suggested for addition to it. One of the concepts where
collections of jobs, another one sequences. Both are now finished and have
been moved from the experimental to the main code base.

As an interesting implementation detail, JobCollections are Jobs
themselves. They can be created and queued, just like any other Job. This
allowed for the removal of all other queueing methods, making the
WeaverInterface class even more simple.

JobSequences in turn are implemented as JobCollections, but will execute all
jobs they contain in the order they where added. This raises an interesting
question - what if I add a number of sequences to a sequence? Intuitively, it
would be expected that the sub-sequences will be executed in the order they in
turn where added. This is exactly what happens. More general, ThreadWeaver's
job collection classes allow recursive queueing of subjobs. A number of unit
tests have been added to showcase this.

The next features to come will be pooled resources (aka Job categories). Think
of local IO intensive Jobs - maybe you want a maximum of 2 of those running at
the same time. There will be special job classes for this purpose.

Currently I am thinking about the best way to add the idea of progress to the
Job class. Of course we can use a special job class, but inheritance does not
seem to be the best way to go. What if what the user wants is a sequence that
has a progress property? On the other hand, I hesitate to simply add a
progress property to the Job base class. Let us see if the future brings a
nice idea of composition ...

ThreadWeaver is in KDE SVN (kdenonbeta/threadweaver).
Release 0.5 has been tagged in tags/others/threadweaver/0.5.

Comments

Have you considered syndicating your blog on PlanetKDE.org? It would bring more visibility to ThreadWeaver.

By vladc at Wed, 05/03/2006 - 21:56

It's only QT4, right?
I have the old QT3 version in my application, and i was looking for updates, but i feel it's no more developed. Am i right?

By rockman at Fri, 05/05/2006 - 09:22

You are absolutely right, there is no Qt 3 version. Threading had a number of unfixable issues with Qt 3, which is why the Qt 3 version has been completely abandoned.

By Mirko Boehm at Fri, 05/05/2006 - 12:41

I wrote a simple threadweaver for amaroK ages ago, that is basically just a job/queue interface. All jobs in my one have the concept of progress, and I encourage all developers who derive jobs to update the progress as far as they are able. This way all background tasks in amaroK show progress info and the user is always kept informed about what is going on.

I also added an easy convenience method so that new jobs automatically get a progressbar added to the statusbar to keep progress info visible to the user.

I encourage you to do the same! Your class is obviously going to be very useful. Put progress information in the class and make it clear in the docs that progress information is important for a graphical enviroment, so they should, as far as they can, emit progress info in sensible steps.

I also suggest a progress QString signal so apps can tell the user what is going on, and also a method for saying "no known finish time, or no progress information can be provided", as those circumstances are common and users should be kept informed.

But good work, and thanks for the inspiration.

By mxcl at Fri, 05/05/2006 - 14:03