What happens if Dr. Wielemaker is hit by a van?

Discussion
#1

Sorry for the somewhat ghoulish and far-fetched question. But it’s actually fairly pragmatic, as we’re about to embark on a new project, and I’m arguing we should use SWI Prolog – because of its extensive libraries, active community and so on.

A reasonable question that gets asked in this context is then: what – if any – are the chances that the system will no longer benefit from the active maintenance and community that underlie it today? I suppose this is a question about the organization of the project, and its governance. Is there a set of people who are ready to take over should Dr. Wielemaker retire?

Any information that will help me make the case is welcome.

jds

1 Like
#2

Hi jds,

Thank you for raising this question.

I had a discussion along those lines with Jan a while ago and, if i recall correctly, he indicated that there are a number of organizations who have developed internally the skills to work with the source code.

However, I also felt that its time to have commercial organizations working with SWI-Prolog to support swi-prolog by helping increasing the number of people in the community understanding the source code and even have the knowledge of maintaining and improving it.

To this end, and as a first step, I thought that swi-prolog should be documented – quite like those Linux kernel books that annotate the code and have chapters explaining the OS design approach.

There is, for example, a company called SWIMM, who create a git based tool to document open source code, as well as help sync between documentation and source based on gits source code tracking capabilities.

There is also a tool called CodeStory, which i have been using for some time now to document own code, which however, isn’t git based in the version i have – there is a git version in the working.

What if commercial organization could crowd raise fund for Jan so he can dedicate time towards documenting the source, writing technical chapters and perhaps even educating individuals in the community about internal workings through online (and recorded) webinars.

Do you think this could work – assuming of course that Jan would be interested in something like that.

Dan

#3

This issue is quite high on the agenda. No, I’m not planning to retire anytime soon. Other possible events in life are harder to plan :frowning: I’ll make a few remarks:

  • First of all, SWI-Prolog is open source with a permissive license. That means anyone in this world can take over, either the whole thing or some parts, at any time. Yes, while I’m active and doing a reasonable job this is not that likely to happen. Should that no longer be the case things may change. We have seen that with gcc that became too much for Richard Stallman. There are most likely other examples. Sure, that is likely to change the paste and directions of new development.

  • There is a (small) number of people who managed to dive deep inside the code. Yes, it is a lot. Some parts are pretty poorly written (notably the older parts). Most of the code is organized in ways that makes navigation fairly easy. All in all, someone with C and Prolog skills will face a few tough months to become familiar with the system. It has been done and (thus) it can be done. Not all these people got a lot of help from me.

  • Bottom line, open source systems do not die immediately. Even if the lead developer stops there are enough people that will fix bugs, portability problems, etc. The main issue is whether or not there will be coordinated new development.

  • Having a permissive license, any other Prolog implementation can copy as much as they want to provide enough SWI-Prolog compatibility to seduce the SWI-Prolog users in migrating to their system. This seems to be happening already with a few Prolog systems being newly developed.

Bottom line: if you want to base business on SWI-Prolog you need to think about this, as with any technology stack you decide to use. It is highly unlikely you have to switch in a rush as the worst case scenario is stalling new development that will slowly make the system irrelevant. If you merely use basic Prolog that won’t affect you much and you can switch to a new system in weeks. If you use a lot of SWI-Prolog’s goodies and need more than keeping the system alive you’ll have to invest. How much is hard to predict and largely depends on your needs as well as how the community continues.

The aim with SWI-Prolog Solutions b.v. is to help resolving this problem. How is still an open question. The past months have shown it is easy enough to acquire commercial projects to improve the system (open source) as well as to help companies setting up Prolog based technology. One possible next step is to make this big enough so people can be hired to do the stuff that is needed for the next phase. Anyone with resources that can make this next phase happen are kindly invited to contact me. Think of people who can organize this, create a pool co-developers, etc.

9 Likes
#4

What happens if Dr. Wielemaker is hit by a van?

Dr. Wielemaker is tougher than a van :stuck_out_tongue:

3 Likes
#5

We have trams in Amsterdam. They are pretty tough!

#6

How much effort would it take to create something like this for swi-prolog:

https://www.amazon.com/Linux-Core-Kernel-Commentary-Knowledge/dp/1576104699

Edit:

I tried to read some of the early descriptions how the virtual machine works – but didn’t get very far – the text is dense and usually impenetrable for me.

#7

Far too much. Years, while the code is a moving target and thus it would require continuous maintenance. For Linux this supports thousands of kernel developers. For SWI-Prolog it would at best support and handful while a qualified programmer can fairly well understand the inner working of the system in a couple of months.

I’d be much happier with a good text book that explains how SWI-Prolog can be used to achieve your goals. There are so many useful things that are not covered by Prolog text books and are now only densely described in the reference manual and/or scientific publications. Think tabling, threads, engines, continuations, transactions, etc.

In my view, growing the community is the safest insurance for all users.

7 Likes
#8

Jan,

So why don’t you do this systematically – how about you put up a list of topics you could expand on, each with a funding target, and the community can sponsor each.

I personally, I find the documentation of the more specialized topics, often frustrating – that it is so dense – there is an expression in Hebrew – it shows one measure and hides two measures …

Dan

1 Like
#9

Yes, absolutely. At the end, money is the universal incentive. I wish I was in a position to help with this somehow. As it is, I see no hope in trying to convince anyone around me to start using SWI-Prolog, even if it is technically feasible and probably a good choice :frowning: The stigma is too strong.

Indeed, a way to move forward is to show that there is a business in selling SWI-Prolog services. This shows that there is money in this, and it shows that others have made the choice to spend money on it. This directly speaks to the people who make both technical and business decisions.

1 Like
#10

What kind of book do you have in mind? A book with a major publisher or a collaborative effort that is publicly available? What are the trade-offs: how authoritative it feels, who has control over the content, how widely available it is, what are the incentives for the contributor(s)… The list goes on.

#11

It is impossible to take care of all audiences in a documentation. This is why more popular technologies end up having books, online tutorials, blogs, even courses provided by either “established” educational institutions or new breeds of online education.

(One interesting detail here is that there are also business-sponsored and business-developed courses made available through the online outlet of established universities. This is a marketing effort and certainly written off as such in the books of the business; still, students, that is, future workers, have incentives to take the course, and become consumers of the marketing material.)

An issue that I have with any material on any technology is that as a rule of thumb, you can figure out what you can do, but you must invest to figure out what are the hard limits and what you cannot or should not do with this technology. I must repeat: this is not at all specific to SWI-Prolog or its documentation.

1 Like
#12

I don’t think its an either/or …

Although, I have a preference to swish and the like …

#13

I must say that some text in the manual is so dense that I can not figure out what I can do – and, perhaps, more importantly, how to use facilities to achieve goals they were intended for

(continuation comes to mind – which i still don’t fully understand; and the pio library i tried to get my head around yesterday – finding the docu impenetrable, until i found a simple example on stack overflow, which seem to skip everything presented in the docu)

I think there is also the matter that Prolog has evolved since the classic textbooks have been written, and a new generation of text books are needed to gather again the common wisdom that was produced.

To give a personal example, – before I had my long email discussion with Markus – my code was littered with asserts and cuts.

I think my early style of coding was influenced by my lack of experience and the early text books that introduce prolog by showing how the prolog database is programmed and queried, rather use the style of, say, assoc as the paradigmatic example.

Dan

#14

Nothing in particular. I guess it mostly depends on the opportunities that arise. Notably someone with the capabilities willing to write this. If then there are practical limitations such as money, access to knowledge, etc. we can try to resolve these. I’m all in favor of community actions, but so far you are one of the few who contributed examples to the new example management. Thanks! Also Prolog - Wikibooks, open books for an open world is not really promising.

Writing such a book is a serious amount of work. Technology books that target a small community are not easy to produce. Too difficult to write and not enough revenues.

2 Likes
#15

The only important book that should be written, is a book that describes the SWI-Prolog VM. Other Prolog books, blogs and wikis exist like sand by the sea. Without a SWI-Prolog VM book, if a van hits Jan Wielemaker SWI-Prolog will be gone forever.

The SWI-Prolog VM book must be very detailed, not simply refering to some Prolog folklore, which is also about to die out. Really unravel everything about the SWI-Prolog VM. But I don’t know how such a book can nevertheless be made short, and not consuming many years to write.

I recently found, which is rather short:

A Prolog Emulator by Evan Tick, 1987
https://ntrs.nasa.gov/api/citations/19930074153/downloads/19930074153.pdf

Does SWI-Prolog also have a determinism bit, mark a frame without choicepoint, somewhere?

#16

It is just a Prolog VM. It isn’t that hard to understand if you know about Prolog VMs. Better, the VM is a tiny part of the SWI-Prolog system. If you don’t like it, it is is a minor (say 1 year) task to design a new VM and reuse 99% of the rest of the system. That might cause loosing some of the detailed features, but is certainly doable.

#17

There is a misunderstanding, by SWI-Prolog VM I don’t mean the instruction set. The Prolog VM is the virtual machine as a whole. Or what the ISO core standard calls the Prolog processor.

Term I/O, fast load, dynamic database, copy term, garbage collection, attributed variables, tabling are also part of the Prolog VM, CHR is part of the Prolog VM, destructive setarg is part of the Prolog VM. This is all determined by the Prolog VM architecture. The Prolog VM is not only the instruction set for classical Prolog, and therefore only:

pl-wam.c
pl-vmi.c
pl-supervisor.c
pl-comp.c
pl-export

Its also:

pl-read.c
pl-write.c
pl-wic.c
pl-assert.c
pl-prims.c
pl-copyterm.c
pl-gc.c
pl-attvar.c
pl-cont.c
pl-tabling.c
Etc..

Everything that depends on the layout of Prolog terms and Prolog clauses or provides these things, and is not implemented in Prolog itself or only partially. The usual reference manual or user manual doesn’t give an internal view that would help in any way.

In Java the VM and its compiler sits in a JAR file rt.jar and tools.jar, which has size:

rt.jar 67’674’235 Byte (67.7 MB auf dem Volume)
tools.jar 18’314’930 Byte (18.3 MB auf dem Volume)

Its a little bit more than only 1%, its ca. 25% of a JDK install.

#18

Thank you all for contributing to this discussion.

I should add a point that may be obvious (I’m not sure): grant-funded development work in Europe and the US increasingly seems to require the open-sourcing of the results of the work. Therefore it is necessary that the system be developed using FOSS tools. Swapping out SWI Prolog for, say, Sisctus, may not be an option.

What makes SWI Prolog so attractive is also the extensive and modern library: to take random examples, yall, or the redis package. Perhaps these are all portable to another system, but I wonder if it’s really that easy?

In any case, there’s no easy answer to this question, understandably. @jan is there a mechanism by which you might be able to hire a post-doc to learn (and perhaps document!) some of the internals? And/or maybe there’s someone in the existing community who is close enough to the core to acquire core maintenance duties? I’m not contesting the BDFL model here – just thinking about managing risk.

#19

I think the opportunity is here – I am sure people in the community would be glad to proof read documentation of the prolog VM.

I personally surely would.

Also, I would suggest using SWIMM or similar tool to document, to have a tool that can track obsoleteness in the documentation vis-a-vis the code as well.

Btw, just to throw it out there – could the prolog VM be documented with Prolog?

#20

I would also add SWI-Prolog has some commercial users whose products heavily depend on SWI-Prlog, see this article:

It is not just this or that side project, so they have considered the risks and are using SWI-Prolog. They have funded the development of some features in the past.

You also should consider your ‘threat model’, what are the actual risks? I would say that with SWIProlog the biggest risk is simply ‘If we hit a bug will it be solved?’ If you don’t use the new features (like tabling) the probability of you hitting a bug is rather low, but nonetheless if you look at the commit log there are other people that have solved bugs even though Jan was the one that committed into master.

What I wrote above is supposing that your team/company is already sold on using prolog, if that is not the case we’re in a totally different game.

EDIT: if the choice is restricted to open source prologs, I honestly don’t think any other alternatives come close to the usability, maturity, features, and support that SWI-Prolog has.

1 Like