Randa Review: the buildsystem

Tuesday, 28 June 2011  |  alexander neundorf

I'm back now (since two weeks already) from the KDE Platform sprint in Randa and I have to say it was great.

Randa is located very nice in the Swiss mountains, next to Matterhorn, and very well suited not only for KDE sprints, but also for cycling (I had my bike with me there) :-)

I'm everytime impressed by the exceptional passion and extraordinary talent of the people in our community. Probably this is because our community is "self-filtered": you don't get in for the money, but because of personal motivation. ... and so only people who are highly motivated and "obsessed" by software find their way to us. David, Stephen, Milian, Sune, Aaron, Volker, Kevin (to name just a few) -- what an amazing list of people.

So, what was in it from the buildsystem POV ?

A lot. What I realized is that we are now in a different situation than early 2006 when I introduced CMake into KDE. Back then there were not many free software projects (outside KDE) which used CMake. So it was ok to put our own cmake modules, macros and scripts into kdelibs, and so they were available to "everybody" in our community. Now we have 2011 and now there are a lot of projects outside KDE which use CMake, among them a set of libraries from KDE developers, but which don't depend on kdelibs. For all these projects our CMake extensions are not available, except if they make copies of them in their own projects.

Making our CMake stuff available outside KDE So, one major topic regarding the buildsystem was can we make our extensions available also for non-KDE projects ?

Yes, we can !

The following list of actions all work towards this goal:

  • get automoc into cmake, so every cmake user can use it. Gregory ported it in Randa away from Qt to STL and the crossplatform library cmsys from cmake. Now we have to figure out with the cmake developers in which way this can be integrated nicely into cmake.
  • a small set of modules should qualify for directly going into cmake, e.g. CheckStructHasPointer
  • we have more than 100 Find-modules alone in kdelibs. Getting them into cmake is straightforward. You adopt one of the modules, and post to the cmake mailing list "Here is a new module, I want to be the maintainer." Then you'll get cmake git access and you'll have to maintain it then in cmake, i.e. fix bugs cmake users report, make sure source compatibility is kept, etc. So while straightfoward, it requires work and dedication. Once a Find-module made it into cmake, we cannot rely on it immediately. We'll have to wait until the next version of cmake is released, and if possible until this version has made it at least in the newest versions of the major distributions. So all in all, this doesn't sound too promising for us.
  • Instead, we'll create a separate package, and move the Find-modules into that package. Then everybody who needs a Find-module can depend on this package. It will be in kdesupport, and it will be named "extra-cmake-modules". And it will have relatively short release cycles, so new stuff can become available quickly. Raphael and me will get this going. This will be straightforward, since all the modules in kdelibs have absolutely no dependencies on KDE (of course except the obviously KDE-related ones)

Modularization of KDE Another big topic at the sprint was the repositors split with git. With these many small git repositories it becomes much harder for the developers to build KDE, compared to how it was with our big KDE modules. To help with that, we have kdesrc-build, buildtool, and we found out that Gnome also has something similar called jhbuild.

I had a look at this issue and noticed that CMake basically has built-in support for such things since 2.8.0 with its ExternalProjects support. I worked on this and we now have "Superbuild" CMakeLists.txt for kdesupport and kdegraphics (Superbuild because it consists only of subprojects, so it is the superproject of these projects). Why do we need yet another solution ?

  • it is indeed actually not an additional solution, since it is simply CMake, not a extra new tool
  • it has the additional feature that it can also generate source-packages in flexible
    • granularity, e.g. it can generate a kdegraphics tarball as we had with the big kdegraphics module
  • it should be possible to write an all-in-one CMakeLists.txt, which covers all of KDE SC. Since it also handles the dependencies, it is possible to enable only the project you are interested in, and it tells you which other projects it depends on, so you might enable them to. This way e.g. a source tarball containing everything required to build e.g. kruler can be created.

Superbuild can be found already at http://projects.kde.org/projects/kde/superbuild

Testing We also figured out that we should really try to set up continuous builds for the KDE frameworks, so we can better ensure quality. More on that later.

How to install properly With the more fine grained git repositories as we have them already e.g. in kdesupport and kdegraphics, a lot more packages which contain libraries have to be installed now. Some libraries we have simply install their headers and library files, some also install pkg-config files, some install Config.cmake files for cmake. Even kdelibs doesn't do it in the best possible way, among others because this was new for me back then, and now it has to stay to keep source compatiblity. E.g. kdepimlibs and kdevplatform do it quite good. A cmake-based library should install a FooConfig.cmake file along with a FooConfigVersion.cmake file and export its targets using install(TARGETS ... EXPORT ...). This file can contain all necessary information about the installed package, including version numbers, enabled or disabled features, and, especially important for our Windows crew, the different libraries for debug- and release-versions.

So, what do we do to make more packages install a nice FooConfig.cmake file ?

  • in kdeexamples/ there is a new subfolder buildsystem, which contains now one example, "HowToInstallALibrary", which shows how to "perfectly" install a library from a cmake-based project
  • It will hopefully not be necessary anymore to additionally install a pkg-config file. cmake will support a new command line mode, in which it can be used like pkg-config in purely Makefile-based builds or autotools builds (cmake --find-package -DNAME=LibXml2 -DMODE=COMPILE -DCOMPILER_ID=GNU -DLANGUAGE=C) With the help from David and Ryan (the dconf guy) I was even able to write a working cmake.m4, which autotools-based projects can use to find a library which has been built itself with cmake

All in all, we got a lot done, but the work just started. A first success is that basically our macro_optional_find_package() was accepted into cmake (in a somewhat different way, it will be possible to disable each non-required find_package() via a CMAKE_DISABLE_FIND_PACKAGE_PackageName switch), but it has the same effect. So, one thing which can then also be used outside of KDE.

Next in list, the pkg-config-compatible mode. Let's see.