JUN
18
2013

Compiling *all* of KDE SC from svn/git

On my desktop machine, I have always tried to compile "everything". At a time it was easy: about 18 SVN modules listed in kdesvn-buildrc, trunk for everything, done.
Nowadays there are many more git modules - I currently compile 234 modules, with the help of kdesrc-buildrc (I don't have to list all 234 modules, it expands them for me). Still it's a bit difficult to ensure I really have everything...

Of course I don't use every KDE application, but that's not the point. I want users and distros to be able to actually build the stuff... so it should build for developers, for a start. And in a unified fashion (no special hacks to make things build, it just raises the barrier of entry).

Anyhow, what I want to complain about, is that it's actually impossible to build everything, at this point.
Too many dependencies on very-recent libs, or on libs that aren't packaged by distros (unless I'm missing something).

Here's my list of unsolved issues right now, on an OpenSUSE 12.3 system:

  • kolena needs tesseract library, not shipped by opensuse
  • libnm-qt (and therefore networkmanagement) needs NetworkManager 0.9.8.0, opensuse 12.3 only has 0.9.6.4
  • libkface (and therefore digikam) needs OpenCV >= 2.4.4 (Opensuse has 2.4.1)
  • amarok needs gmock >= 1.4, not shipped by opensuse
  • kdenlive needs MLT >= 0.9.0, opensuse has 0.8.8
  • apper needs packagekit-qt2 >= 0.8.8, opensuse has 0.7.4
  • kamoso needs FindQtGStreamer.cmake or QtGStreamerConfig.cmake, none of which is provided by libQtGStreamer-0_10-0 (no devel package)
  • kolor-manager needs FindOyranos.cmake or OyranosConfig.cmake, none of which is provided by liboyranos-devel
  • libqapt needs aptpkg, not shipped by opensuse
  • trojita uses qmake (kdesrc-build doesn't seem to support that)
  • qoauth tries to install into /usr, which isn't even where my Qt is
  • calligra defines its own iterators, which breaks -DQT_STRICT_ITERATORS
  • bodega-webapp-client - nothing to build
  • unittests in nepomuk-core hang, every time.
  • The kdesvn developer still didn't apply my patch from April (fixing compilation with strict iterators), I'm worried that it's not maintained anymore.
  • playground never builds, and yet there's KDE apps that depend on playground stuff! (hello bluedevil). Either people (and not just me) should ensure that playground builds, or dependencies of KDE SC software should be elsewhere).

I am not happy about things that require too recent libs. I'm using a rather recent distribution, it's 3 months old, and yet there's a lot of stuff I can't compile. NetworkManager in particular is really annoying - no way to build it, with 0.9.6.4. There could at least be a branch that builds with the NetworkManager that people are currently using!

But, to end on a positive note, that's 234-19=215 modules that build just fine :-)

Comments

I'm in full agreement here - this is a problem that the KDE CI system (build.kde.org) runs into regularly. We need to stop depending on the very latest and greatest things here, it's just making things harder for new people to start contributing.

You can see a list at http://build.kde.org/view/External_Deps/ of the things we are building (some because they depend on Qt, others because the system versions do not exist, or are too old) which are not part of KDE. I'd prefer if that list was a touch smaller.


By bcooksley at Thu, 06/20/2013 - 08:12

Yes, there is nothing to build in bodega-web-client, or in the soon to appear bodega-web-manager. They are node.js apps which bootstrap and run from a checkout without any system installation or compiled code.

Playground needs a good scrubbing, indeed. It could probably use a forced reset where sysadmin gives us a time window (say 1 month) and then mothballs everything that isn't claimed or moved.

My 'secret' for building several of the repos you mention is that I used OpenSuse factory ... which is not a satisfying solution to the problem at all of course :) On the other hand, it is difficult to tell app developers they can't develop against a new version of a library until it is in mainstream distros. For the core repositories this makes lots of sense, but for random applications ..

The digikam situation is odd though; libkface should be an optional dep imho.


By Aaron J. Seigo at Thu, 06/20/2013 - 09:05

It sounds like we're missing a solution in projects.kde.org so that modules with nothing to compile are marked as such, and therefore not compiled by kdesrc-build.


By David Faure at Thu, 06/20/2013 - 12:51

Of course we have the solution, I just didn't know it :)
mpyne pointed me to kde-build-metadata (git module) with the file "build-script-ignore" where non-buildable modules can be listed. I'll add bodega-webapp-client there.


By David Faure at Wed, 06/26/2013 - 23:47

You could not ask to people to use latest kde but ask kde developers not to use latest releases.
Sure some retro-compatibility is good and in a perfect world everything should still compile with gcc-2.95 but that has a cost, many project are not so rich on resources.
This and newer version have bug fixed and useful features sometimes.
Specifically about OpenCV-2.4.4 has been released 2013-02-12 and 2.4.5 is already out. Bug opensuse to upgrade it ;-).
Alternatively a rolling distro could be useful to build everything from sources, gentoo + kde overlay for example is often (always?) in position to build whole kde from git easily.


By vivo75 at Thu, 06/20/2013 - 10:54

There's a difference between users and developers, in this context.

Users can use any version of KDE SC they want, they are just required to use the latest KDE SC if they want to report bugs, since, well, what's the point in reporting bugs in old software which has evolved since then? Very likely that the bugs don't exist or don't apply anymore. But still, they have a choice.

On the other hand, I'm talking about developers here: if one doesn't have opencv-2.4, one can't even compile the software in the first place. This is a completely blocking issue, no choice there.

Using another distro is possibly an option for a developer - but this raises the bar of entry, if many distros are just excluded upfront -- and what about automated builds? To reduce the amount of maintainance, build.kde.org should use a stable distro as a base. Like OpenSUSE 12.3, and unlike gentoo.


By David Faure at Fri, 06/21/2013 - 09:04

ok, but then you could not consider only one distro (OpenSUSE) but need to put in all the major players (Fedora, Debian, kubuntu), and provide a db of useable versions. This could well take out the fun from development.
This also mean the developer *MUST NOT* upgrade the library if it's distro of choice is already using it. And many developers are using other distro than the build bot.
> if one doesn't have opencv-2.4, one can't even compile the software in the first place
Well that's not absolutely true, it makes things little more complicated.
A developer could *always* (open source world) compile the library on it's own, well, if we were speaking of libpng or libjpeg which are used by 80% of desktop packages out there ...

So in short IMHO you should not put blockers on developers but educate them to use a newer library only if they *REALLY* need to, and to try to avoid the thing (or also provide compatibility for older versions) if the lib. in question is also used by a lot of packages.


By vivo75 at Mon, 06/24/2013 - 12:28

And most other distributions don't even have the equivalent of software.opensuse.org where you can easily get the newer versions of software you need. Aaron's solution of running openSUSE Factory shouldn't be required... I mean, I'm sure openSUSE would love to make Factory more stable and I'm sure other distro's would love to offer similar options so developers can happily run them without risk, but it simply should not be needed!


By jospoortvliet at Thu, 06/27/2013 - 08:53

Although I have mixed feelings about this in general, I do need to make some corrections:

QtGStreamerConfig.cmake is in gstreamer-0_10-plugins-qt-devel, which provides libQtGStreamer-0_10-devel.

And why is it surprising that an rpm-based system doesn't have apt libraries?

But overall, I can understand issues with not requiring the latest and greatest, although there are certainly cases where it is important.

But several of your issues are with openSUSE not packing something at all yet. But this is a chicken-and-egg problem. Distros aren't going to package a new library if they don't have any software that needs it. By this logic, KDE can never depend on any new libraries until some other important project does so first. But then these complaints just get moved to that other project.

Several of the other problems are packaging issues in openSUSE. I don't see how KDE developers are supposed to be able to anticipate this.

So that really leaves these packages being problems with dependency versions: NetworkManager, OpenCV, MLT, packagekit-qt2. NetworkManager, apparently, has an important requirement there. libkface is now implementing face recognition, a huge change, so I would not be surprised if the change is related to that. MLT 0.9.0 has two major new APIs, so I would suspect that the kdenlive depency is based on those. The only one I am not sure about is PackageKit-Qt2.

But really, out of all the packages, all but 4 dependencies are up-to-date. That seems like a good number. And at least 3 of these appear to be due to major improvements.

So I am not sure what the alternative here is. One one hand, developers can't implement important new features or improvements in trunk because people who aren't developers of those packages might, for some reason, decide to build all of KDE from scratch. The alternative is that people just don't build those couple of packages, or in the absolute worst case enable another repo or two in their package management system to get the dependencies. I don't see why the latter two options are all that bad.


By toddrme2178 at Mon, 06/24/2013 - 13:18

Yeah, I feel your pain. Sadly I gave up hope to see that *everything* builds nowadays. Some of the problems you mentioned can be fixed by adding extra stuff to be built from source, some with hunting down new packages from non-official repositories. But in general building KDE from source is a much worse experience than it was years before.
E.g. in master kphotoalbum didn't build for a long time because it was incompatible with the master kipi-plugins and Jesper told this is normal, they target the latest released version. :)


By amantia at Wed, 06/26/2013 - 21:15

Pages