Here’s a blurb from my Erlang’19 paper that says things about scalability:
By adopting a computational model capable of scaling out not only to multiple cores on one machine but also to multiple nodes running on multiple machines, by introducing an option allowing clients to limit the number of network roundtrips it takes to run a query to completion, by embracing the WebSocket transport protocol with its low overhead per message, by offering also a stateless HTTP API, and by leaving ample room for old as well as new forms of caching on both clients, nodes and intermediaries, we have made our best to ensure our high hopes for the scalability of the Prolog Web are not unfounded.
For the Prolog Web to be able to scale really well, nodes must also be able to spawn very many actors, creating and destroying actors must be fast, and the communication among them efficient. Since actors created by the Erlang virtual machine are famous for having exactly those properties, this is certainly yet another reason for us to look closely at Erlang.
Feel free to ask me about what I mean by any of that, and hopefully I’ll be able to answer (although an answer might be in the form of a pointer to a section of my manuscript or the Erlang’19 paper).
Although I haven’t really worked much with the SW technology, I have the same impression. Starting from distributed Prolog might be a much better idea.
Yes, to continue working on the SWI-Prolog implementation would likely be the best way forward. What’s missing more than anything else is the ability to run the PoC online, and for that the sandboxing must be dealt with. I’ve made several attempts at that, but failed each time.
The tutorial included in the PoC also points to a couple of bugs, but nothing as severe as the lack of security.
Note that there are sound technical reasons why dicts and strings are controversial. In particular, strings as implemented in SWI-Prolog have been extensively debated when SWI-Prolog 7 was introduced. Dicts use a functional notation with eager evaluation and an operator that the ISO standard and almost all Prolog systems use for representing lists. I’m not trying to (re)open the debate on these features here (specially in this thread) but Web Prolog work that assumes strings and dicts as materialized in SWI-Prolog risk excluding other Prolog systems from participating at the start gate. Moreover, anyone thinking that this is just other systems being slow in keeping pace, well, that’s naive and ultimately wrong. Both strings and structures in named arguments (aka as dicts) were found in other Prolog systems long before SWI-Prolog 7. So far, no consensus on these features syntax and semantics have emerged from the broad Prolog community, much less any standardization efforts around it. Not trying to rain down on the Web Prolog initiative. Just alerting that there’s work to be done here.
Couldn’t containers provide a solution to the security problem? Spin up duplicate nodes in containers, which can give us load balancing, redundancy and the ability to send a long-running query to another node to try. Definitely scalable! We can kill and spin-up containers at will, ala. Kubernetes Why not use ideas from the big data/serverless communities here? It leaves the realm of programming language and enters that of a framework, but it’s one option.
The only way to get this kind of project truly moving is to follow what Linus said once about open source (paraphrasing): for an open source project to work someone has to be willing to take charge of the code and work on it (without expecting the ‘community’ to do the work); no open source project will succeed unless there is a ‘coder’ that takes ownership and releases early and often.
We have seen this with Linux, SWI-Prolog, gcc, vim, emacs, python, perl, elixir, ruby, netscape (early days of web, before it was a company), keepass, etc. It is only after this ownership and continuous development is in place that a community will truly develop around it.
I think Web Prolog is really needed and a very good idea, but it will never get off the ground until one person takes the code and develops it with a long term commitment, releasing early and often.
Keeping the discussion at the theoretical level is nice, but in the end Web Prolog will never exist. The PoC is a very good start, but it has been frozen in terms of code for long periods of time. This gives the general perception (wether true or not) that the project is simply nice talk, and will not get off the ground until the code base is continuously developed and released often.
I don’t see ‘code ownership’ happening right now for Web Prolog (look at the commit log in github). I do hope some individual appears and takes charge of it.
Yes, that’s probably what has to happen. I’m afraid that one person can’t be me, though. For one thing, I’ve realised that my programming skills are probably not sufficient. I suspect it might require someone like Jan or Paulo or Richard O’Keefe taking charge (although there might be others, of course, I just don’t know… please step forward if you feel like taking on the challenge!).
(BTW, I sent a copy of a draft of my Erlang’19 paper to Richard O’Keefe, and one of his comments (he had many) was:
I should tell you that I am liking what I see very much and would enjoy using this.
As some of you know, Richard isn’t exactly easy to please, so I felt very encouraged. )
Alas, I’m getting old (I’m 60), and the kind of focus required for a serious programming effort tends to wear me out. I’ve suffered several burn-outs during the last ten years, and I’m down to working just 50%, probably for the rest of my career.
Now, on the positive side, my department here in Gothenburg has been very understanding, and I’m allowed to spend most of this time on my own research. So, I plan (and I pledge) to spend at least 15-20 hours a week on the Web Prolog project, if we can get something going. Apart from implementation work, there’s more specification work to tackle as well. The current proposal is just a start, and I’d probably need help with the rest as well. I need to publish papers, of course, and I would also like to finish my book (I’d love to find suitable co-authors for both papers and book).
Very true. But then, this is the case for many things that are generally considered vital such as a module system, threads, C interface, attributed variables, global variables, mutable terms, fix ancient arithmetic and a few more. The only difference is SWI-7 came with syntactical changes to accommodate them better. I think we now know the practical implications of these changes are not that large. If we had a working standardization process that should all have been resolved long ago. You gave it a good try (thanks), but right now the process is stuck and given the distribution of power and interest as well as the ISO procedure this is unlikely to change.
I have decided to stop wasting my time on that and move on. That is not necessarily forever. If an initiative comes up that is prepared to make changes, committed to resolve (most of) the above issues and is backed up with people willing to invest time, I’m on board.
That said, it might be wise for Web Prolog, if it aims to be cross-implementation, to avoid syntax that is fundamentally non-ISO. Of course, the alternative view is that you design Web Prolog to be as good as it can be and hope Prolog systems will accommodate the features it needs.
In theory I could drive such an effort. In practice I would need funding to do it, since currently i am working on an own venture without outside funding.
How are such efforts typically funded – does it require a full time commitment?
I think it would be wise to advertise Web Prolog and its call for participation on more venues. Notably, most Prolog compilers have their own mailing lists. That said, SWI-Prolog is particularly well suited for a Web Prolog implementation given its focus on web technologies. No surprise to see it taking point.
There may (or already there is; sorry if I missed it) a third view of Web Prolog as a set of minimal APIs that allow networked nodes running Prolog (or other logic programming systems) to cooperate, In this view, each system strengths (including proprietary extensions) could be used at full capacity and standardization issues would be minimized as the efforts would focus in the communication between nodes. If all nodes would be capable of advertising and being queried about the Web Prolog profiles they implement, any node would know how to talk to any other node. But maybe all these is already being planned? I have not been able to fully follow the discussion.
One thing that wouldn’t work if only minimal APIs are standardised is what I refer to as code injection, i.e. the ability to inject Web Prolog source code into a pengine (or other kind of actor) to be created. Code injection is involved already in the interaction between SWISH and the backend when a pengine (of the “old” kind) is being created. In my opinion, the injected code has to adhere to a standard, which might be big or small, depending on the profile.
Preserving each system’s strength’s is important, but I believe this is easier if more comprehensive profiles are specified. Also, I propose only less than a dozen of Erlang-ish predicates for taking care of spawning and messaging. I don’t see that they should be very difficult to standardise.
Advertising Web Prolog more broadly is a good idea. Perhaps this can be done by writing something that can be published on the ALP site? Perhaps ALP - as an organisation - would like to see Web Prolog happen? As a way to celebrate the 50th anniversary of the first LP language that has had some success?
Could you give an example for the requirement and use of code injection – i am not familiar with this need – the word injection causes me to think about viruses and other nasty stuff off and on the web
Ok, if you’ve read the paper, you may have noticed the many places where options such as src_text, src_predicates, src_list and src_uri are passed when spawning a pengine or another actor. Such options allow you to inject source code into the private workspace of a local or remote actor and to have them run it. In the web APIs, there are query parameters which serve the same purpose.
Yeah, “code injection” may be a bad choice of term…
Why is this capability crucial in general on the web. What is the need to enable dynamically distributing fact and rule knowledge to augment and change behavior elsewhere.
I can see why its useful, say, in a swish environment – where you want to have an end-user enter code, and have it execute on the server.
Edit:
Its a very powerful feature … but what is driving its need in context of web prolog …
It borrowed it from lisp – a programmable programming language – what’s the name of the lisp book again … where the author said that in the introduction ?
Here’s the title page of the manuscript I’m currently working on (but is not yet ready to release). As you can see, the title contains “programmable”, “Web”, as well as “agents”. I’ll say something more about agents later…
)
– what’s the name of the lisp book again … where the author said that in the introduction ?