NOV
9
2011

Cool new stuff in CMake 2.8.6 (2): pkg-config compatible mode added for use e.g. with autotools

Submitted by Alexander Neundorf

After introducing the automoc feature in my last blog, here comes the next part of this series. More will follow.

The new --find-package mode of CMake

Typically, in projects which are built using autotools or handwritten Makefiles, the tool pkg-config is used to find whether and where some library, used by the software, is installed on the current system, and prints the respective command line options for the compiler to stdout.

Since CMake 2.8.6, also CMake can be used additionally to or instead of pkg-config in such projects to find installed libraries.

With version 2.8.6 CMake features the new command line flag --find-package. When called in this mode, CMake produces results compatible to pkg-config, and can thus be used in a similar way.

E.g. to get the compiler command line arguments for compiling an object file, it can be called like this:

   $ cmake --find-package -DNAME=LibXml2 -DLANGUAGE=C -DCOMPILER_ID=GNU -DMODE=COMPILE
   -I/usr/include/libxml2
   $

To get the flags needed for linking, do

   $ cmake --find-package -DNAME=LibXml2 -DLANGUAGE=C -DCOMPILER_ID=GNU -DMODE=LINK
   -rdynamic -lxml2
   $

As result, the flags are printed to stdout, as you can see.

The required parameters are

  • NAME - this is the name of the package, the same way as it is used in
    find_package() calls when using CMake
  • LANGUAGE : which language is used, possible options are C, CXX, Fortran and ASM
  • MODE : either COMPILE, LINK or EXIST
  • COMPILER_ID: this identifies which compiler you are using, so CMake can print
    the options as necessary for this toolchain. When using the GNU compiler, use "GNU"
    here. Many other compilers are compatible to the command line syntax of gcc. Other
    options are "Intel", "Clang", "MSVC" etc.

So, you can insert calls like the above in your hand-written Makefiles.
For using CMake in autotools-based projects, you can use cmake.m4, which is now also installed by CMake.
This is used similar to the pkg-config m4-macro, just that it uses CMake internally instead of pkg-config. So your configure.in could look something like this:

   ...
   PKG_CHECK_MODULES(XFT, xft >= 2.1.0, have_xft=true, have_xft=false)
    if test $have_xft = "true"; then
        AC_MSG_RESULT(Result: CFLAGS: $XFT_CFLAGS LIBS: $XFT_LIBS)
    fi

   CMAKE_FIND_PACKAGE(LibXml2, C, GNU)
   AC_MSG_RESULT(Result: CFLAGS: $LibXml2_CFLAGS LIBS: $LibXml2_LIBS)
   ...

This will define the variables LibXml2_CFLAGS and LibXml2_LIBS, which can then be used in the Makefile.in/Makefiles.

What does that mean for developers of CMake-based libraries ?

You don't have to install pkg-config pc-files anymore, just install a Config.cmake file for CMake, and both CMake-based and also autotools-based or any other projects can make use of your library without problems.
Documentation how this is done can be found here:

What does that mean for developers working on e.g. autotools-based projects, and using a project built with CMake ?

Take a look at the cmake_find_package() m4-macro installed since CMake 2.8.6 in share/aclocal/cmake.m4, it contains documentation, and will help you using that library.
Thanks go to Matthias Kretz of Phonon fame, now working on HPC stuff, who wrote the cmake.m4 from scratch (which was necessary since it had to be BSD-licensed in order to be included in CMake).

Internals

Internally, CMake basically executes a find_package() with the given name, turns the results into the command line options for the compiler and prints them to stdout.
This means it works basically for all packages for which a FindFoo.cmake file exists or which install a FooConfig.cmake file.
There is one issue though: FindFoo.cmake files, which execute try_compile() or try_run() commands internally, are not supported, since this would required setting up and testing the compiler toolchain completely.
It works best for libraries which install a FooConfig.cmake file, since in these cases nothing has to be detected, all the information is already there.

All this stuff is still very new, and has not yet seen wide real world testing.
So, if you use it and find issues, or have suggestions how to improve it, please let me know.

Alex

Comments

What you are proposing here is to replace a sane [1] declarative configuration file with a complicated Turing-complete script, and getting a hard dependency on CMake for autotools-based projects.
I've never been very fond of KDE relying on FindPackage magic [2] instead of pkg-config, but what you are proposing here is definitely the extreme opposite of my idea of "cool".

[1] well, mostly sane, but nothing that cannot be fixed in specs
[2] actually magic practiced by a bad magician, who when faced with several identically looking rabbits just picks one at random

By eckhart wörner at Thu, 11/10/2011 - 14:46

IMO pkg-config is not perfect.
It doesn't work on systems where you decide at install time where you install the package, i.e. Windows and, depending on package type, also OSX. It hardcodes compiler flags. If you decide to use a library with a compiler which is not command-line compatible with gcc, you're out of luck.

Using pkg-config together with cmake brings more issues. You have to take care that both are set up correctly to everything works, although our hard dependency is cmake, which is perfectly capable to work without pkg-config, so it is an additional and kind-of unnecessary dependency.
CMake implements quite some logic to make sure e.g. that your linking command will give you what you expect, i.e. it doesn't generate -Ldir1/ -Ldir2/ -lfoo -lblub, instead it uses the full paths.
It generated command lines like above before, but this turned out to never be fully reliable, since you could get combinations of link directories which would result in not giving you exactly the libraries you want. (e.g. if libblub and libfoo are both in dir1/ and dir2/, and you want blub from dir1 and foo from dir2/). These were real-world problems users reported.
So it changed to using full paths for linking.
How does pkg-config play into this ?
It gives you command line options, i.e. -Ldir1 -Ldir2 -lfoo -lbar. Now, cmake has to parse these command line options, and extract the directories from that. These it can then use as input to find_library(), so it gets a full path.
IMO having to parse command line syntax from gcc is not really a good solution.

Adding cmake as a requirement for autotools-based projects ?
Why not ? If they depend on pkg-config, what's the problem with depending in cmake ? It exists and works everywhere. It should be nowadays more or less as widespread as make itself.

Regarding comment [2], I don't consider this constructive. I can assure you, it is not random.

Alex

By Alexander Neundorf at Thu, 11/10/2011 - 17:06

"IMO pkg-config is not perfect."

I never claimed that.

"It doesn't work on systems where you decide at install time where you install the package, i.e. Windows and, depending on package type, also OSX."

So your point is that Windows users should be able to drop binaries from a zip file to a random folder and expect it to work, something that Linux users cannot expect?
Also, just by looking at Google, there are projects that use pkg-config on Windows.

"It hardcodes compiler flags. If you decide to use a library with a compiler which is not command-line compatible with gcc, you're out of luck."

That's exactly the same thing that happens when your compiler is not compatible with your FindFoo.cmake.

"Using pkg-config together with cmake brings more issues. You have to take care that both are set up correctly to everything works, although our hard dependency is cmake, which is perfectly capable to work without pkg-config, so it is an additional and kind-of unnecessary dependency."

That statement is only true as long as you have a closed system with cmake as its only player, a situation that is quite unlikely. Even with KDE software (I guess that is what "our" refers to), the stack we depend on normally doesn't use cmake, and software that depends on our frameworks doesn't have to use cmake either.

"CMake implements quite some logic to make sure e.g. that your linking command will give you what you expect, i.e. it doesn't generate -Ldir1/ -Ldir2/ -lfoo -lblub, instead it uses the full paths.
It generated command lines like above before, but this turned out to never be fully reliable, since you could get combinations of link directories which would result in not giving you exactly the libraries you want. (e.g. if libblub and libfoo are both in dir1/ and dir2/, and you want blub from dir1 and foo from dir2/). These were real-world problems users reported.
So it changed to using full paths for linking.
How does pkg-config play into this ?
It gives you command line options, i.e. -Ldir1 -Ldir2 -lfoo -lbar. Now, cmake has to parse these command line options, and extract the directories from that. These it can then use as input to find_library(), so it gets a full path.
IMO having to parse command line syntax from gcc is not really a good solution."

I agree with the problem.
However, my conclusion is completely different: 1. add fields to the foo.pkgconfig file that specify the full library paths (you can do that in a backward-compatible way) 2. ask for inclusion to the pkg-config standard 3. everybody profits

"Regarding comment [2], I don't consider this constructive. I can assure you, it is not random."

Sure, it was not constructive, but it pretty much sums up my fundamental problems with FindFoo.cmake files: they are Turing-complete, imperative, and not free of side effects.

By eckhart wörner at Mon, 11/14/2011 - 15:50