ThreadWeaver 0.6: Resource Restrictions and support for MacOs
The next step release of ThreadWeaver has been tagged. Say hi to a couple of new features: Queue policies can be used to adapt the queueing behaviour of jobs. Resource restrictions (a kind of queue policy) can be used to limit the number of jobs of a certain group started at the same time. Job queueing priorities are there to control the order of execution. Jobs can now return whether they are successful. The unit tests have been widely extended. The API polished. The interfaces pimpled. MacOs is now a supported platform. That should be enough for a step release, right?
Queue policies are an abstraction that allows the implementation of almost all queueing requirements. For example, dependencies as used previously are now implemented as a queue policy. Resource restrictions are also a queue policy (see below). The benefit of the abstraction is that Job classes do not need to be modified - queue policies can be assigned to jobs. The code of the Job class has no trace of dependencies or resource restrictions. Much more flexible than before.
Sometimes, it is more efficient to run just one or two jobs of the same kind at the same time. For example, running many jobs that access the hard drive at once may degrade overall performance, because hard drives read faster when reading whole files in one go. That is what resource restrictions are there for - the restriction object sets a numerical limit (say, 2), and of all jobs that have this restriction assigned, only hat many may run at the same time. Notice that, to achieve that, the jobs do not have to be of the same class, many different jobs can have the same resource restriction (or queue policy in general) assigned. In the example above, it is trivial to set up a HardDriveAccessRestriction and assign it to every possible job that is disk I/O intensive.
Job queueing priorities simply order the queue: Threads will start all higher priority jobs before lower priority jobs are executed. Note that this does not affect the CPU priority of the threads, but only the order of queueing.
Other than those larger changes, a number of minor improvements have been added: Jobs can overload a function to return whether their execution was successful. Job sequences will stop executing their elements if a job failed. By now 17 unit tests test all kinds of queueing and execution behaviour. The API has been reviewed for const correctness and a consistent interface. To be able to maintain binary compatibility, all class interfaces have been pimpled. Also, a lot of dependencies on implementation details where eliminated. This should provide for sufficient source compatibility for easy code maintenance (please note that binary compatibility between step releases is not guaranteed yet).
As a result of the added features, it should generally not be necessary to use more than one Weaver job queue. Instead, it is supposedly be more efficient to use one queue with a large number of threads and set up resource restrictions et cetera accordingly. Using ThreadWeaver should be simple and intuitive. Let me know if it is. By now, it is supposed to work on Unix variants and MacOs (since this release - the Apple crowd needs to use those dual cores for something :-).
Upcoming features are Windows support, a standardized benchmark suite for performance measurement, progress and thread activity visualization (in the end, we are making GUI applications, right?), and CMake support for building. As ThreadWeaver is going towards a 1.0 release, the wish list is open for feature requests. Your feedback is much appreciated.
ThreadWeaver is licensed under the LGPL and can be found in KDE SVN (kdenonbeta/threadweaver). Release 0.6 has been tagged in tags/others/threadweaver/0.6.