# Fail the build if a dependency is not found **URL:** \ **Category:** General\ **Created:** [July 14, 2024, 6:21am UTC](https://swi-prolog.discourse.group/t/fail-the-build-if-a-dependency-is-not-found/7593 "2024-07-14T06:21:03Z")\ **Posts on this page:** 18\ **Page:** 1
**Author:** ![maren](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/maren/32/6605_2.png) [@maren](https://swi-prolog.discourse.group/u/maren)\ **Post date:** [July 14, 2024, 6:21am UTC](https://swi-prolog.discourse.group/t/fail-the-build-if-a-dependency-is-not-found/7593/1 "2024-07-14T06:21:03Z")
Much of SWI-Prolog builds conditionally. For example, regex support is built only when the pcre library is detected by cmake. This gets us a maximally useful SWI-Prolog for a particular system. A downside is that it is not immediately obvious what will actually be built without knowing what packages your build environment provides, and how these influence the SWI-Prolog build. I would like to explicitly enable, say, the pcre package, and have cmake fail if it lacks required dependencies to do this, rather than just disable the pcre package if the pcre library is not found. This does not seem to be possible. I’m pretty unfamiliar with cmake. Is there some way to do what I want? More broadly, is there some way to build a standard SWI-Prolog distribution containing all the things you’d expect for a desktop install, which will fail the build if anything is missing rather than silently turning off features? [edit] One way is to enable `INSTALL_TESTS`, call `check_installation`, and see if any warnings come up. But this can only be done after the build has already run. ---
**Author:** ![brebs](https://avatars.discourse-cdn.com/v4/letter/b/e9c0ed/32.png) [@brebs](https://swi-prolog.discourse.group/u/brebs)\ **Post date:** [July 14, 2024, 8:10am UTC](https://swi-prolog.discourse.group/t/fail-the-build-if-a-dependency-is-not-found/7593/2 "2024-07-14T08:10:22Z")
> [@maren](#): > > built only when the pcre library is detected by cmake Just for clarification - there’s usually 3 levels to consider: 1. The app packaging system - e.g. [Alpine ensures](https://git.alpinelinux.org/aports/tree/testing/swi-prolog/APKBUILD) that the pcre library will be available. 2. cmake looks for (optional) pcre and will use it if found, otherwise not use pcre. 3. As an _override_ - the build is [told to enable pcre](https://git.alpinelinux.org/aports/tree/community/rspamd/APKBUILD), and fail otherwise. Bundling pcre within swi-prolog (mentioned just for completeness of alternatives), would be a [bad choice](https://fedoraproject.org/wiki/Bundled_Libraries). ---
**Author:** ![maren](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/maren/32/6605_2.png) [@maren](https://swi-prolog.discourse.group/u/maren)\ **Post date:** [July 14, 2024, 8:18am UTC](https://swi-prolog.discourse.group/t/fail-the-build-if-a-dependency-is-not-found/7593/3 "2024-07-14T08:18:06Z")
My concern is actually packaging, as I’m trying to make some improvements to the current package definition in Nix. I can study the build system and make sure that the required packages are present in the dependencies. But this is fragile. A next version of SWI-Prolog might introduce a new package that needs a new dependency, or existing dependency requirements might change, and when the packaged SWI-Prolog version is updated to the latest upstream source, there would be no immediate way of knowing that standard features were automatically disabled. Much better to just say what I want and have the build system (or something else) validate that that is what I’ll get. I agree that direct bundling of pcre is bad. ---
**Author:** ![brebs](https://avatars.discourse-cdn.com/v4/letter/b/e9c0ed/32.png) [@brebs](https://swi-prolog.discourse.group/u/brebs)\ **Post date:** [July 14, 2024, 8:37am UTC](https://swi-prolog.discourse.group/t/fail-the-build-if-a-dependency-is-not-found/7593/4 "2024-07-14T08:37:51Z")
> [@maren](#): > > say what I want Yes - you’re wanting the `-DENABLE_PCRE2=ON` option. ---
**Author:** ![jan](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/jan/32/4_2.png) [@jan](https://swi-prolog.discourse.group/u/jan)\ **Post date:** [July 14, 2024, 8:46am UTC](https://swi-prolog.discourse.group/t/fail-the-build-if-a-dependency-is-not-found/7593/5 "2024-07-14T08:46:14Z")
cmake itself has find\_package() where you can add “`REQUIRED`” to make it fail if the dependency is not found. So, in theory we could add some flag, e.g. `SWIPL_PACKAGES_REQUIRED` (better name) and use this to add the `REQUIRED` keyword to all find\_package() calls where we need to drop some feature if the package is not found. Traditionally, SWI-Prolog provides `check_installation/0` which analyses the system for available features and whether or not all foreign libraries can be loaded. So, you can run ``` ./src/swipl -g check_installation -t halt ``` That might be enough? Possibly we need some flags/options to avoid warnings that do not satisfy this usage. For example, is lacking `tcmalloc` an issue or not? It improves memory management on Linux systems for certain multi-threaded workloads (notally web servers) because the default Linux malloc does a poor job here. check\_installation/0 also warns of the executable is not (first) on $PATH or if the user’s environment contains old config files. The nice thing about check\_installation is that it is lightweight and can be run on the not-yet-installed system as well as on the installed system, where it also verifies that the shared objects expected from the OS can actually be loaded. ---
**Author:** ![maren](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/maren/32/6605_2.png) [@maren](https://swi-prolog.discourse.group/u/maren)\ **Post date:** [July 14, 2024, 9:07am UTC](https://swi-prolog.discourse.group/t/fail-the-build-if-a-dependency-is-not-found/7593/6 "2024-07-14T09:07:47Z")
> [@brebs](#): > > Yes - you’re wanting the `-DENABLE_PCRE2=ON` option. There actually is a `SWIPL_PACKAGES_PCRE` build option, which is on by default. But this doesn’t work as an override. Setting that explicitly to ON while not providing pcre in the build environment will still just build SWI-Prolog without pcre. > [@jan](#): > > cmake itself has find\_package() where you can add “`REQUIRED`” to make it fail if the dependency is not found. So, in theory we could add some flag, e.g. `SWIPL_PACKAGES_REQUIRED` (better name) and use this to add the `REQUIRED` keyword to all find\_package() calls where we need to drop some feature if the package is not found. I like this solution. If you like, I could look into seeing if I can modify the build system for this. One problem I saw (at least in the places I looked) is that packages don’t really set anything to say they are being included, but maybe that’s just me misunderstanding cmake right now. > [@jan](#): > > That might be enough? It is workable, and would make sure no dependency mistakes crept into packaging. I think it makes a lot of sense to run this as a post-build check. It would indeed have to be filtered for differences that are accounted for by the packaging explicitly wanting a different kind of output (excluding X/qt for example). Flags for that could work. So could a grep from the output of `check_installation`. Still, failing early when explicitly requested features cannot be built would be nice. If the SWI-Prolog build cannot directly do failure on missing dependencies, I’m thinking of providing some pre-check script to fail the build early for cases I know will not work (but potentially not catching everything), then running `check_installation` after the build. ---
**Author:** ![jan](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/jan/32/4_2.png) [@jan](https://swi-prolog.discourse.group/u/jan)\ **Post date:** [July 14, 2024, 9:21am UTC](https://swi-prolog.discourse.group/t/fail-the-build-if-a-dependency-is-not-found/7593/7 "2024-07-14T09:21:22Z")
> [@maren](#): > > I like this solution. If you like, I could look into seeing if I can modify the build system for this. One problem I saw (at least in the places I looked) is that packages don’t really set anything to say they are being included, but maybe that’s just me misunderstanding cmake right now. Go ahead. I think you only need to define some cmake variable that you add to the argument list of all find\_package() calls that remove features. Note that that is not always trivial. E.g., for ssl we are happy with OpenSSL and LibreSSL. Lacking GMP we use the bundled LibBF, is this that a requirement? XPCE has a bunch of dependencies, several of which are non-fatal. I agree this is the best option for packagers. > [@maren](#): > > It would indeed have to be filtered for differences that are accounted for by the packaging explicitly wanting a different kind of output (excluding X/qt for example). That would be nice in general. As is we get warnings on explicitly disabled features. That is still nice to evaluate the suitability of a given swipl instance for what you are trying to do, but not to check whether this installation satisfies the intend of the packager. It probably needs two modes or two predicates. Possibly we can leave data from CMake in some file to make check\_installation/0 aware of the intended packages? ---
**Author:** ![brebs](https://avatars.discourse-cdn.com/v4/letter/b/e9c0ed/32.png) [@brebs](https://swi-prolog.discourse.group/u/brebs)\ **Post date:** [July 14, 2024, 9:22am UTC](https://swi-prolog.discourse.group/t/fail-the-build-if-a-dependency-is-not-found/7593/8 "2024-07-14T09:22:30Z")
> [@maren](#): > > Setting that explicitly to ON while not providing pcre in the build environment will still just build [SWI-Prolog](https://www.swi-prolog.org/pldoc/doc_for?object=manual) without pcre That’s definitely wrong, then 😀 ---
**Author:** ![maren](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/maren/32/6605_2.png) [@maren](https://swi-prolog.discourse.group/u/maren)\ **Post date:** [July 14, 2024, 9:27am UTC](https://swi-prolog.discourse.group/t/fail-the-build-if-a-dependency-is-not-found/7593/9 "2024-07-14T09:27:34Z")
> [@jan](#): > > Lacking GMP we use the bundled LibBF, is this that a requirement In terminusdb we use gmp to communicate big numbers across fli boundaries. So there it’s actually pretty important to know for sure things built with GMP, and we didn’t somehow get libBF instead. > [@jan](#): > > Possibly we can leave data from CMake in some file to make check\_installation/0 aware of the intended packages? As an extra installable file which is imported at runtime? ---
**Author:** ![jan](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/jan/32/4_2.png) [@jan](https://swi-prolog.discourse.group/u/jan)\ **Post date:** [July 14, 2024, 2:44pm UTC](https://swi-prolog.discourse.group/t/fail-the-build-if-a-dependency-is-not-found/7593/10 "2024-07-14T14:44:32Z")
> [@brebs](#): > > That’s definitely wrong, then 😀 Possibly. It gets messy otherwise though. I am not aware that one can distinguish explicitly set options from default options in CMake. Hopefully I’m wrong. > [@maren](#): > > In terminusdb we use gmp to communicate big numbers across fli boundaries. So there it’s actually pretty important to know for sure things built with GMP, and we didn’t somehow get libBF instead. I see. It is a little shaky as SWI-Prolog redefines the allocation hooks of GMP (but tries to not affect foreign code by calling the default hooks when used not as part of Prolog’s arithmetic). In Janus (the Python binding) and the WASM version I uses (hexadecimal) strings for bigints and rational numbers. > [@maren](#): > > As an extra installable file which is imported at runtime? That would be the idea yes. We could also create library(check\_installation) from some check\_installation.pl.in using CMake. I think both are fine with me. ---
**Author:** ![brebs](https://avatars.discourse-cdn.com/v4/letter/b/e9c0ed/32.png) [@brebs](https://swi-prolog.discourse.group/u/brebs)\ **Post date:** [July 14, 2024, 3:28pm UTC](https://swi-prolog.discourse.group/t/fail-the-build-if-a-dependency-is-not-found/7593/11 "2024-07-14T15:28:45Z")
> [@jan](#): > > can distinguish explicitly set options from default options in CMake It wouldn’t be much of a build system if it couldn’t 😀 The possibilities are: - No override, PCRE is missing, so build doesn’t use PCRE - No override, PCRE is present, so build uses PCRE because it’s preferred - PCRE is overriden to off, (don’t care whether PCRE is present), so build doesn’t use PCRE - PCRE is overriden to on, so the build tries to use PCRE, and fails (with hopefully an intuitive error message) if it can’t Continuing my example, here is [how rspamd recognizes PCRE2](https://github.com/rspamd/rspamd/blob/27b858776562a743a6036d5f6e1ee022d2006f79/CMakeLists.txt). I don’t know if it’s the best way, though. Another simple possibility for this PCRE2 option, is to simply force it on within swi-prolog. Who would complain, i.e. who has a good reason in 2024 for compiling swi-prolog without pcre2? ---
**Author:** ![maren](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/maren/32/6605_2.png) [@maren](https://swi-prolog.discourse.group/u/maren)\ **Post date:** [July 15, 2024, 12:08am UTC](https://swi-prolog.discourse.group/t/fail-the-build-if-a-dependency-is-not-found/7593/12 "2024-07-15T00:08:32Z")
> [@brebs](#): > > Another simple possibility for this PCRE2 option, is to simply force it on within [swi-prolog](https://www.swi-prolog.org/pldoc/doc_for?object=manual). Who would complain, i.e. who has a good reason in 2024 for compiling swi-prolog without pcre2? It’s probably not very common, but there are still constrained systems where you may want to build the smallest swipl you can, ditching any unnecessary dependencies. Being able to turn off things in the build that pull in dependencies you don’t need for a functioning (albeit less-featureful) swipl is great. This is especially the case in very specific deployment scenarios, where you use swi-prolog for a single application only, rather than using it as a more general purpose prolog environment. For example, a prolog http service running on a server, or a swi-prolog embedded in a larger application. If given the option, why would you go for a version that has support for all these features that your application doesn’t even depend on, but which nevertheless pull in extra dependencies? Not everyone will care about doing such specific tuning for their deployments, but I appreciate that the option is there. ---
**Author:** ![brebs](https://avatars.discourse-cdn.com/v4/letter/b/e9c0ed/32.png) [@brebs](https://swi-prolog.discourse.group/u/brebs)\ **Post date:** [July 15, 2024, 12:18am UTC](https://swi-prolog.discourse.group/t/fail-the-build-if-a-dependency-is-not-found/7593/13 "2024-07-15T00:18:21Z")
> [@maren](#): > > that needs a new dependency, or existing dependency requirements might change _All_ packages of other people’s software have this as a potential problem - I don’t think there’s a convenient cure, and updating packages as/when necessary is just the tedious housekeeping that comes with evolving software. It can be mitigated by sensible defaults in the app’s build system - e.g. if PCRE is expected to be intended by around 98% of the users, then the building script could intentionally fail unless e.g. `-DENABLE_PCRE2=OFF` is specified. ---
**Author:** ![peter.ludemann](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/peter.ludemann/32/48_2.png) [@peter.ludemann](https://swi-prolog.discourse.group/u/peter.ludemann)\ **Post date:** [July 15, 2024, 12:42am UTC](https://swi-prolog.discourse.group/t/fail-the-build-if-a-dependency-is-not-found/7593/14 "2024-07-15T00:42:30Z")
> [@maren](#): > > want to build the smallest swipl you can, ditching any unnecessary dependencies If you have a container that doesn’t have PCRE, you can build `swipl` without that dependency. Cmake looks for the dependency in the “global” environment. For example, `findPCRE.cmake` has the line `find_path(PCRE_INCLUDE_DIR NAMES pcre2.h)`, which on my system resolves to `/usr/include/pcre2.h` and which `dpkg -S` tells me is part of package `libpcre2-dev:amd64`; if you don’t want to build with PCRE, then you would need to do the equivalent of `apt remove libpcre2-dev`. [The actual situation is a bit more complicated; there’s also a check for whether `packages/pcre` is present, which is another way of controlling what gets built; `packages/pcre` can be left out by simply not including that submodule.] ---
**Author:** ![maren](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/maren/32/6605_2.png) [@maren](https://swi-prolog.discourse.group/u/maren)\ **Post date:** [July 15, 2024, 8:19am UTC](https://swi-prolog.discourse.group/t/fail-the-build-if-a-dependency-is-not-found/7593/15 "2024-07-15T08:19:49Z")
> [@brebs](#): > > _All_ packages of other people’s software have this as a potential problem - I don’t think there’s a convenient cure, and updating packages as/when necessary is just the tedious housekeeping that comes with evolving software. Sure, there’s probably no way to just make the build system automatically maintain a package, I’m not expecting that much. Package maintenance requires that someone occasionally updates what a package looks like, beyond merely refreshing the imported source, to get with the times. I just want builds to fail when an enabled feature cannot be built, so builds break loudly rather than disable features silently. > [@peter.ludemann](#): > > If you have a container that doesn’t have PCRE, you can build `swipl` without that dependency. The question here is not ‘how do I build SWI-Prolog without pcre’. I know how to do that. The question is, how do we build SWI-Prolog with pcre (or any other feature, or a standard set of features), and have the build fail if it’s not there? Also features should (and are) just disableable by setting a flag, rather than modifying the build environment. ---
**Author:** ![brebs](https://avatars.discourse-cdn.com/v4/letter/b/e9c0ed/32.png) [@brebs](https://swi-prolog.discourse.group/u/brebs)\ **Post date:** [July 15, 2024, 9:18am UTC](https://swi-prolog.discourse.group/t/fail-the-build-if-a-dependency-is-not-found/7593/16 "2024-07-15T09:18:06Z")
There should be many examples within various apps, of using cmake to optionally include PCRE2. Looking within cmake itself, there is: - [CMake/Utilities/cmlibarchive/CMakeLists.txt at ac4f6d513db96ccd6296d85f0a09d54b4c1d5f6c · Kitware/CMake · GitHub](https://github.com/Kitware/CMake/blob/ac4f6d513db96ccd6296d85f0a09d54b4c1d5f6c/Utilities/cmlibarchive/CMakeLists.txt#L243) - [CMake/Utilities/cmlibarchive/build/cmake/FindPCREPOSIX.cmake at ac4f6d513db96ccd6296d85f0a09d54b4c1d5f6c · Kitware/CMake · GitHub](https://github.com/Kitware/CMake/blob/ac4f6d513db96ccd6296d85f0a09d54b4c1d5f6c/Utilities/cmlibarchive/build/cmake/FindPCREPOSIX.cmake) ---
**Author:** ![jan](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/jan/32/4_2.png) [@jan](https://swi-prolog.discourse.group/u/jan)\ **Post date:** [July 15, 2024, 9:28am UTC](https://swi-prolog.discourse.group/t/fail-the-build-if-a-dependency-is-not-found/7593/17 "2024-07-15T09:28:26Z")
With a bit of googling, I stumbled on > This is pretty much what we are looking for. It is rather ugly though. Possibly the dirty stuff can be centralized in a function on `cmake/PackageSelection.cmake`? I think this route is good, so I’m happy to apply this if it can be implemented more or less elegantly. I still think it would be good to have an option that says “all features that rely on dependencies should be present, except those that are explicitly disabled”. I’m afraid I have other priorities for the next couple of months … ---
**Author:** ![maren](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/maren/32/6605_2.png) [@maren](https://swi-prolog.discourse.group/u/maren)\ **Post date:** [July 15, 2024, 10:06am UTC](https://swi-prolog.discourse.group/t/fail-the-build-if-a-dependency-is-not-found/7593/18 "2024-07-15T10:06:28Z")
We might also be able to use tri-state options instead of two-state (boolean) options: [[CMake] How to handle options with more than two possible values?](https://cmake.org/pipermail/cmake/2016-October/064342.html) So the basic idea is to just have a string option instead, and have an `AUTO` (or maybe `UNSET`?) variant in addition to `ON` and `OFF`.