Sorry for the somewhat ghoulish and far-fetched question. But it’s actually fairly pragmatic, as we’re about to embark on a new project, and I’m arguing we should use SWI Prolog – because of its extensive libraries, active community and so on.
A reasonable question that gets asked in this context is then: what – if any – are the chances that the system will no longer benefit from the active maintenance and community that underlie it today? I suppose this is a question about the organization of the project, and its governance. Is there a set of people who are ready to take over should Dr. Wielemaker retire?
Any information that will help me make the case is welcome.
I had a discussion along those lines with Jan a while ago and, if i recall correctly, he indicated that there are a number of organizations who have developed internally the skills to work with the source code.
However, I also felt that its time to have commercial organizations working with SWI-Prolog to support swi-prolog by helping increasing the number of people in the community understanding the source code and even have the knowledge of maintaining and improving it.
To this end, and as a first step, I thought that swi-prolog should be documented – quite like those Linux kernel books that annotate the code and have chapters explaining the OS design approach.
There is, for example, a company called SWIMM, who create a git based tool to document open source code, as well as help sync between documentation and source based on gits source code tracking capabilities.
There is also a tool called CodeStory, which i have been using for some time now to document own code, which however, isn’t git based in the version i have – there is a git version in the working.
What if commercial organization could crowd raise fund for Jan so he can dedicate time towards documenting the source, writing technical chapters and perhaps even educating individuals in the community about internal workings through online (and recorded) webinars.
Do you think this could work – assuming of course that Jan would be interested in something like that.
This issue is quite high on the agenda. No, I’m not planning to retire anytime soon. Other possible events in life are harder to plan I’ll make a few remarks:
First of all, SWI-Prolog is open source with a permissive license. That means anyone in this world can take over, either the whole thing or some parts, at any time. Yes, while I’m active and doing a reasonable job this is not that likely to happen. Should that no longer be the case things may change. We have seen that with gcc that became too much for Richard Stallman. There are most likely other examples. Sure, that is likely to change the paste and directions of new development.
There is a (small) number of people who managed to dive deep inside the code. Yes, it is a lot. Some parts are pretty poorly written (notably the older parts). Most of the code is organized in ways that makes navigation fairly easy. All in all, someone with C and Prolog skills will face a few tough months to become familiar with the system. It has been done and (thus) it can be done. Not all these people got a lot of help from me.
Bottom line, open source systems do not die immediately. Even if the lead developer stops there are enough people that will fix bugs, portability problems, etc. The main issue is whether or not there will be coordinated new development.
Having a permissive license, any other Prolog implementation can copy as much as they want to provide enough SWI-Prolog compatibility to seduce the SWI-Prolog users in migrating to their system. This seems to be happening already with a few Prolog systems being newly developed.
Bottom line: if you want to base business on SWI-Prolog you need to think about this, as with any technology stack you decide to use. It is highly unlikely you have to switch in a rush as the worst case scenario is stalling new development that will slowly make the system irrelevant. If you merely use basic Prolog that won’t affect you much and you can switch to a new system in weeks. If you use a lot of SWI-Prolog’s goodies and need more than keeping the system alive you’ll have to invest. How much is hard to predict and largely depends on your needs as well as how the community continues.
The aim with SWI-Prolog Solutions b.v. is to help resolving this problem. How is still an open question. The past months have shown it is easy enough to acquire commercial projects to improve the system (open source) as well as to help companies setting up Prolog based technology. One possible next step is to make this big enough so people can be hired to do the stuff that is needed for the next phase. Anyone with resources that can make this next phase happen are kindly invited to contact me. Think of people who can organize this, create a pool co-developers, etc.
I tried to read some of the early descriptions how the virtual machine works – but didn’t get very far – the text is dense and usually impenetrable for me.
Far too much. Years, while the code is a moving target and thus it would require continuous maintenance. For Linux this supports thousands of kernel developers. For SWI-Prolog it would at best support and handful while a qualified programmer can fairly well understand the inner working of the system in a couple of months.
I’d be much happier with a good text book that explains how SWI-Prolog can be used to achieve your goals. There are so many useful things that are not covered by Prolog text books and are now only densely described in the reference manual and/or scientific publications. Think tabling, threads, engines, continuations, transactions, etc.
In my view, growing the community is the safest insurance for all users.
So why don’t you do this systematically – how about you put up a list of topics you could expand on, each with a funding target, and the community can sponsor each.
I personally, I find the documentation of the more specialized topics, often frustrating – that it is so dense – there is an expression in Hebrew – it shows one measure and hides two measures …
Yes, absolutely. At the end, money is the universal incentive. I wish I was in a position to help with this somehow. As it is, I see no hope in trying to convince anyone around me to start using SWI-Prolog, even if it is technically feasible and probably a good choice The stigma is too strong.
Indeed, a way to move forward is to show that there is a business in selling SWI-Prolog services. This shows that there is money in this, and it shows that others have made the choice to spend money on it. This directly speaks to the people who make both technical and business decisions.
What kind of book do you have in mind? A book with a major publisher or a collaborative effort that is publicly available? What are the trade-offs: how authoritative it feels, who has control over the content, how widely available it is, what are the incentives for the contributor(s)… The list goes on.
It is impossible to take care of all audiences in a documentation. This is why more popular technologies end up having books, online tutorials, blogs, even courses provided by either “established” educational institutions or new breeds of online education.
(One interesting detail here is that there are also business-sponsored and business-developed courses made available through the online outlet of established universities. This is a marketing effort and certainly written off as such in the books of the business; still, students, that is, future workers, have incentives to take the course, and become consumers of the marketing material.)
An issue that I have with any material on any technology is that as a rule of thumb, you can figure out what you can do, but you must invest to figure out what are the hard limits and what you cannot or should not do with this technology. I must repeat: this is not at all specific to SWI-Prolog or its documentation.
I must say that some text in the manual is so dense that I can not figure out what I can do – and, perhaps, more importantly, how to use facilities to achieve goals they were intended for
(continuation comes to mind – which i still don’t fully understand; and the pio library i tried to get my head around yesterday – finding the docu impenetrable, until i found a simple example on stack overflow, which seem to skip everything presented in the docu)
I think there is also the matter that Prolog has evolved since the classic textbooks have been written, and a new generation of text books are needed to gather again the common wisdom that was produced.
To give a personal example, – before I had my long email discussion with Markus – my code was littered with asserts and cuts.
I think my early style of coding was influenced by my lack of experience and the early text books that introduce prolog by showing how the prolog database is programmed and queried, rather use the style of, say, assoc as the paradigmatic example.
Nothing in particular. I guess it mostly depends on the opportunities that arise. Notably someone with the capabilities willing to write this. If then there are practical limitations such as money, access to knowledge, etc. we can try to resolve these. I’m all in favor of community actions, but so far you are one of the few who contributed examples to the new example management. Thanks! Also Prolog - Wikibooks, open books for an open world is not really promising.
Writing such a book is a serious amount of work. Technology books that target a small community are not easy to produce. Too difficult to write and not enough revenues.
It is just a Prolog VM. It isn’t that hard to understand if you know about Prolog VMs. Better, the VM is a tiny part of the SWI-Prolog system. If you don’t like it, it is is a minor (say 1 year) task to design a new VM and reuse 99% of the rest of the system. That might cause loosing some of the detailed features, but is certainly doable.
Thank you all for contributing to this discussion.
I should add a point that may be obvious (I’m not sure): grant-funded development work in Europe and the US increasingly seems to require the open-sourcing of the results of the work. Therefore it is necessary that the system be developed using FOSS tools. Swapping out SWI Prolog for, say, Sisctus, may not be an option.
What makes SWI Prolog so attractive is also the extensive and modern library: to take random examples, yall, or the redis package. Perhaps these are all portable to another system, but I wonder if it’s really that easy?
In any case, there’s no easy answer to this question, understandably. @jan is there a mechanism by which you might be able to hire a post-doc to learn (and perhaps document!) some of the internals? And/or maybe there’s someone in the existing community who is close enough to the core to acquire core maintenance duties? I’m not contesting the BDFL model here – just thinking about managing risk.
I think the opportunity is here – I am sure people in the community would be glad to proof read documentation of the prolog VM.
I personally surely would.
Also, I would suggest using SWIMM or similar tool to document, to have a tool that can track obsoleteness in the documentation vis-a-vis the code as well.
Btw, just to throw it out there – could the prolog VM be documented with Prolog?
I would also add SWI-Prolog has some commercial users whose products heavily depend on SWI-Prlog, see this article:
It is not just this or that side project, so they have considered the risks and are using SWI-Prolog. They have funded the development of some features in the past.
You also should consider your ‘threat model’, what are the actual risks? I would say that with SWIProlog the biggest risk is simply ‘If we hit a bug will it be solved?’ If you don’t use the new features (like tabling) the probability of you hitting a bug is rather low, but nonetheless if you look at the commit log there are other people that have solved bugs even though Jan was the one that committed into master.
What I wrote above is supposing that your team/company is already sold on using prolog, if that is not the case we’re in a totally different game.
EDIT: if the choice is restricted to open source prologs, I honestly don’t think any other alternatives come close to the usability, maturity, features, and support that SWI-Prolog has.
For yall the answer is yes as it is written by Paulo Moura who is very keen on portability. The SWI-Prolog version has some additional declarations to make the development tools cooperate better with it.
The redis package is more complicated. Several Prolog systems have emulated the SWI-Prolog foreign language interface. That makes porting parts fairly easy. Other parts depend on more unique features of SWI-Prolog though. Think of dicts (easy enough to implement for any Prolog system but it must be done in the core system), blobs that are used to encapsulate foreign resources in a safe way and provide garbage collection for them (also not that hard for any system than already has atoms with garbage collection) and threads (hard if you do not have them). In general the more modern libraries are harder to port as they more and more rely on SWI-Prolog features.
That is on the agenda to be resolved. How is still an open question. If anyone reads this who has the necessary skills or wants to learn them and is interested to play a role in this, please contact me. The whole thing depends on people, money and a plan Roughly I see three ways out. One is a big enough stakeholder to make this happen. Another is enough work for SWI-Prolog Solutions that allow hiring someone to take this role and the last is to bring a larger number of stakeholders together to establish paid support that is backed up either by hiring someone or a diverse network of community members capable of covering most areas. Other suggestions?
The question raised by JDS/free-variation, is something that I have raised
in the past both in the old mailing list and at a private discussion with Jan
on ICLP 2015.
I think it is a serious matter and is likely to be a hindrance in people/companies
using SWI-Prolog.
Although SWI is open source and can be cloned, users requiring longevity
cannot depend on the chance that someone competent enough might pick up the project.
My opinion, now and then, has been that a group of people that have required skills and interest
in SWI should form a support committee that will engage in maintaining the project.
To my understanding there are currently at least 2-3 people that understand the deep innards,
2-3 people that understand critical extensions and 2-3 people that have a reasonable
understanding of the overall direction of the project.
Forming a “back-up” committee that will pledge an effort to a continuation project,
that will at a minimum ensure it runs on supported OSes and fix bugs,
can only have a positive impact to people/companies deciding to commit to using SWI.
I’ll make a few remarks:
The stigma is too strong.
Roughly I see three ways out. One is a big enough stakeholder to make this happen. Another is enough work for SWI-Prolog Solutions that allow hiring someone to take this role and the last is to bring a larger number of stakeholders together to establish paid support that is backed up either by hiring someone or a diverse network of community members capable of covering most areas. Other suggestions?