Ideally I’d like to see continuous integration that checks all versions and configurations on each commit. That however is not very realistic. There are a large number of platforms and configurations that need to be tested and building and testing from a clean OS state takes about 20 min CPU time. GitHub CI offers 2,000 min for free and above that the price seems to rise quickly. Doing nothing gets less and less acceptable though. Issues on some particular platform or configuration are not uncommon with releases
As a first step, I’d like to have Docker files for popular Linux distros, ideally with both the oldest supported version and the latest version. Having that is a great reference for required dependencies and possibly different flags build steps, etc. that are sometimes needed.
Now I’m not an experienced Docker developer and I think this can all be a lot smarter and with less code duplication. I have two requests for help
First of all, someone with CI and Docker experience should establish the right reusable structure for these docker files. Note that we also would like to test multiple configurations, such as building using clang, PGO builds, single threaded build, etc. Preferably with as little as possible duplication of work
Ideally we would be able to do stuff like
Test on a specific branch of the git repo
Test a specific configuration on a specific platform
Test a specific configuration on all platforms
Test all configurations on a specific platform
Test everything
Once the structure is fine, people can add configurations and distro versions.
Small correction - according to this page the 2000-minute limit only applies to CI for private repositories. Public repositories have no limit on CI time, although there are still other limits, e. g. how many jobs you can run concurrently.
Thanks. I misread that. That does change the picture. I understand from @ericzinda that docker caching based CI is possible on GitHub.
There is surely still value in a comprehensive set of Docker recipes to build different configurations on different platforms. It both provides a tested way to build some configuration on some platform and if there is anything wrong with some configuration it easy to locally launch a docker image and debug.
For those wanting to learn more but not spend hours reading the following took about 20 minutes.
I started with the quick demo video then read the guides.
Side note:
After learning a bit about Nix script files it seemed obvious that if Nix script could be generated from Prolog then the Prolog would be declarative and IMHO much easier to comprehend and build Nix scripts from Prolog queries.
Also it seems that Nix script could benefit from a type system the same way the JavaScript is typed via TypeScript. For those that know Hindley-Milner type system then this should be obvious.
From what I see, Nix seems just one more packaging system. Any Linux distro is one, we have Macports and Homebrew on MacOS, conan, pip, R packages, etc. All in all, if you combine software you’ve got to make sure all dependencies match. That is a tedious process where every owner of a package balances between backward compatibility and moving forward while trying to follow the decisions of their dependencies. Other people collect and test this stuff and combine it to releases or, more often these days, rolling releases. Every user and packager has preferences for either an aggressive update path or a slow one. The aggressive update plan leads regression due to lack of testing. The slow path leads to bugs because issues fixed upstream take years to be accepted.
There is (I fear) no single solution to this problem. Docker containers are great in establishing a minimal software stack to support a piece of software. Snap, Flatpack, the way mobile devices handle apps are another way to solve it, packaging (as Docker) most dependencies with the application. These schemes lead to duplicated resources and clever techniques to reduce the impact of that. But of course, often applications rely on each others input and output, and thus compatibility is still not guaranteed.
Bottom line is that the entire software eco system evolves on separate islands stacked on top of each other and connected to each other, ultimately running on also evolving hardware. Its pretty much a miracle that works so well as it does
SWI-Prolog is just one of these islands and, like the others, needs to stay in sync and make it as easy as possible for others to stay in sync. This topic is about making all this easier.
There is a Dockerfile snippet for each target that establishes all (build) dependencies for the target.
a YAML file for each target that describes the configurations we want to test by choosing the build tool (Ninja or make) and define environment settings and flags for the configuration, build and test steps.
A Prolog server program that creates a final Dockerfile from this and builds a base image with all dependencies, a checked out Prolog source tree and all configurations built. This file is used to create a Docker base image for this target.
The Prolog server can now run both incremental and clean builds from any branch on any registered git remote. It does so by firing a the base container with a command that checks out the proper source version using git. For incremental builds it now goes to the build directory and just runs ninja or make, followed by ctest. For the clean ones it removes the build directory, recreates it and runs a clean configure, build and test.
The build server connects to a Redis server to receive build requests while it pushes build results to a Redis stream.
The front-facing (Prolog) web server connects to Redis to submit requests and show the results.
The design materialized after discussion with @ericzinda based on his experience using GitHub CI.
There is still some work to do such as turning the servers into proper systemd services, creating more base images and configurations (debug, single threaded, no GMP, etc.). Windows should fit in that using GitHub - SWI-Prolog/docker-swipl-build-mingw: Docker to cross-compile SWI-Prolog for Windows. Hopefully we can eventually get MacOS into this story as well. In principle other OSes can join by making them talk to the same Redis server.
Other developers can be granted access to trigger builds from their forks. Currently only done for @dmchurch to support the refactoring work.
Thanks for the offer. You’re doing plenty already I’ve written enough systemd scripts do be able to do this quicker than describing what needs to be done Its all up and running now with a front facing web server with minimal rights and the actual integration service with enough rights to run docker. We can now start adding configurations and platforms
As SWi Prolog is a major fully open language in the AI area and research world, maybe some should ask Nat Friedman directly to have a contact as a direct support from GitHub ? …
Support is always good Any clue what kind of support GitHub could offer besides hosting the git sources and the already present PR support, Wiki, issues, etc.?
Knowing that SWI Prolog is free, fully open and widely used in research … maybe even be sponsor simplifying your financial GitHub questions and apart from that giving you a more direct tech support when you have some questions on using GitHub.
I did a lot with systemd in a previous 
I’ve written enough systemd scripts do be able to do this quicker than describing what needs to be done 
