Wiki Discussion: SWI-Prolog in the browser using WASM

While I don’t have the time to try this at the moment, it would be interesting to see a Cytoscape.js example done using SWI-Prolog WASM. :slightly_smiling_face:

Think of Cytoscape.js as interactive GraphViz where you can move the nodes around in the browser.

The last Cytoscape.js example created using SWI-Prolog I posted is here.

This is a great example, as it shows practically how to make a small prolog app in an html page. Thanks for sharing it and for adding support for it in the code.

It would be great if we could embed a saved state within an html page (perhaps wiith a base64 encoded data url?) or load a saved state with a method like Prolog.load_saved_state(Url).

Data urls in firefox can be 32MB, so it is quite sufficient for a saved state.

Prolog.load_saved_state(Url) would allow us to develop the app in regular SWI-Prolog and then simply store the saved state in a URL accessible by the web server.

Storing the saved state in a data url would allow us to embed the prolog app and the dependencies within the html page.

This will tremendously simplify dependencies, etc. Of course there would be some limitation on what the code inside the saved state can do, but any browser application would have those limitations.

Mostly thank @PaulBrownMagic :slight_smile:

I’m afraid it ain’t that simple :frowning: States do depend on conditional compilation and the core of the system and these differ between the normal executable and the WASM version. If, as with this example, you want to manipulate the DOM there is no link to the normal desktop version. What could work is to allow loading QLF files from a URL. At the moment I am looking into loading .pl files from a URL from Prolog. If that would work fine you can server a directory with .pl files and dynamically load from there. That is probably doable, though it comes with some problems :frowning:

This project is getting a little big :slight_smile: It is about getting time to seek for (financial) support. If anyone knows about organizations willing to invest in this, please drop me a line.

Wow, that’s really nice, and fast! Thanks for using CBG chords as an example and tagging me, it’s great to see. With an easy WASM prolog I might have to revisit some old client-side ideas!

2 Likes

I liked the example :slight_smile: Can I ask a little favor? The file has no copyright. The various Linux maintainers will complain … Ideally you create a pull request adding a copyright to the file src/wasm/cbg.html. Anything works, as long as it is open source compliant (public domain, BSD-2, BSD-3, MIT, Apache, GPL, etc.) As SWI-Prolog is BSD-2, my preference goes to BSD-2 or public domain. If a PR is too complicated, just send me your decision. Thanks!

1 Like

About the fetch(url).then(...).then(...) form, which uses Promises … it’s probably better for new code to use the newer async syntax instead; it makes error handling easier and it’s a bit easier to read.

Here’s what I did to convert from Promises to async in my example javascript client: Use async functions instead of promises. · kamahen/swipl-server-js-client@6ebfa03 · GitHub

For example, this code:

function fetchFromServer(request, callback) {
    // callback should take a single arg, the response from the server,
    fetch(...)
        .then(response => response.json())
        .then(callback)
        .catch(err => {
            alert('***fetch ' + JSON.stringify(request) + ': ' + err) });
}

became:

async function fetchFromServer(path, request, callback) {
    // callback should take a single arg, the response from the server.
    try {
        const response = await fetch(...);
        callback(await response.json());
    } catch(err) {
        alert('***fetch ' + JSON.stringify(request) + ': ' + err);
    }
}

and calls to fetchFromServer() were prepended by await.

This is great

I feel your pain; I can’t think of an organization at the moment, but I will do the best I can to support this.

I have already started doing so in some places. As I think of it, the js_yield(+Promise, -Result) is in fact await, no? It does (if I understand this correctly) the same: yield from the current (Prolog) execution and resume if the thing we are waiting for is resolved. I was already quite happy after replacing the original (string) request and reply with arbitrary objects to conclude that the only thing you need as input for js_yield/2 is a Promise :slight_smile: If so, we should name it await/2 :slight_smile:

The main issue is that SWI-Prolog cannot yield from everywhere :frowning: Notably it cannot yield from Prolog → C → Prolog calls. Possibly some cases can be fixed by hand (the general case requires a yield mechanism in C). The other nasty part is the debugger. It is probably hard to yield from the debugger as each of the ports comes with its own state that needs to be saved and restored.

Please let everybody who sees potential in this for his/her work look around for opportunities. One big investor is easiest, but several smaller ones work as well. I think I proved that the yield based design is viable. What needs to be done for a nice polished system is roughly this (might have missed something?)

  • Allow yield to work from many more places, including the debugger.
  • Write a proper debugger (we have all the stuff from the GUI traces that is also used by @oskardrums debug adapter, so it is proven to be reusable).
  • Provide asynchronous implementations for many more of the currently blocking primitives.
  • Get engines to the single threaded version. With engines we can have multiple threads of control. Engines do not need threads, but the current engine implementation sits on top of the thread implementation. These would need to be untangled.
  • Error handling, testing.
  • Documentation, demos, …
  • (Modular) packaging and simple deployment of applications.
3 Likes

Got one more step: we can now load files from URLs. This is now available on #wasm_demo, where you can run

?- ['scasp/prolog/scasp'].

to get s(CASP) loaded. It also runs :slight_smile: The sCASP source is made available from the server under /wasm/scasp/. The way it works is

  • If we try to load a file that doesn’t exist in the virtual file system, ensure it has a .pl extension and try to download it as a relative URL.
  • When loading from a URL, all relative paths are loaded relative to the URL of the file being loaded.
  • You can also load from an absolute URL, provided the server allows cross-origin access.

The loading performance is a little disappointing. I also noted that Prolog files from the virtual file system are loaded rather slow. On my machine it takes a little over 2 seconds to load s(CASP) compared to 0.29 using the native SWI-Prolog :frowning:

Haven’t tested whether it also loads .qlf files, but I fear it will try to interpret them as UTF-8 files somewhere along the process …

3 Likes

It was a bit of a challenge :slight_smile: The WASM version can now load .qlf files from a URL. That is nice. Even nicer is that I managed to extend qcompile/2 to include indirectly loaded files except for files from the Prolog system. So, we can do

swipl
?- qcompile('prolog/scasp.pl', [include(user)]).
true.

And end up with a single file scasp.qlf that you can load in any compatible SWI-Prolog instance. If you go to #wasm_demo you an run this and get going:

?- time(['scasp.qlf']).
% 1,671,738 inferences, 0.638 CPU in 0.638 seconds (100% CPU, 2620280 Lips)
true.

The current scasp source conditionally depends on a file that is not part of the WASM version. I created the .qlf file using node /path/to/swipl.js to get system with the same libraries.

Note that these work too. Both times are reported on second invocation, thus using the browser cache.

?- time(['scasp/prolog/scasp']).
% 2,278,937 inferences, 1.465 CPU in 1.465 seconds (100% CPU, 1555588 Lips)
true.

Or, loading from GitHub raw content:

?- time(['https://raw.githubusercontent.com/SWI-Prolog/sCASP/master/prolog/scasp.pl']).
% 2,278,481 inferences, 1.425 CPU in 1.425 seconds (100% CPU, 1598934 Lips)
true.
6 Likes

This is really great :grinning: it starts to make deployment quite feasible.

I don’t have emscripten setup, is there a place where I could download the swipl.js file? I would like to try out generating some qlf files and then loading them in the browser.

As is, there is no swipl.js. If there is no conditional compilation in your code you can just use any recent 8.5.x version to create the .qlf. If there is conditional compilation, you can probably tweak the condition such that you can set some flags to get the result that you need for the browser version.

Streamlining the distribution of the WASM version is still something to consider. Maybe @rla has experience? (How) is the npm version updated? Could/should we have the stuff on a CDN?

Ahh, even better!

I was referring to the wasm node.js wiki section that says:

I guess this is not possible anymore?

It is, but only in the build environment. I see that the npm package does provide swipl.js, but that doesn’t help much as it doesn’t ship with the .data file that provides the Prolog part for the browser. swipl.js is built as a necessary step in the WASM build process to perform the Prolog parts of the build using node.

@jan, @swi sorry for a slow reaction to SWI-Prolog releases. The NPM package https://www.npmjs.com/package/swipl-wasm is still very unstable but it bundles all necessary files including swipl.js, swipl.data and swipl.wasm. They are in dist/swipl/*. The package README contains how to use these files.

I just built and published new version based on 8.5.17. Right now building has to be done manually but I hope we find a solution for that.

Regarding 8.5.17, it does not run on node anymore. The issue is in src/wasm/prolog.js which contains:

HTMLCollection.prototype.toList = function()
{ const ar = [];

  for(let i=0; i<this.length; i++)
    ar.push(this.item(i));

  return ar;
}

This triggers TypeError in nodejs since HTMLCollection does not exist there.

1 Like

Great, this should make it user-friendly to generate qlf files to be deployed on the web without worrying about conditional compilation.

This seems to be a utility/helper method, could it not just be left out of the nodejs version by not compiling it in if HTMLcollection does not exist?

You are doing well :slight_smile: Thanks! Pushed a fix for the HTMLCollection. I still have trouble understanding the browser vs node issue. As I understand it, we have

  • swipl-web.js + swipl-web.wasm + swipl-web.data. These work in the browser. Notably swipl-web.data provides the content for the Emscripten file system that appears as /swipl in the browser.
  • swipl.js and swipl.wasm. These work with node. I think swipl-web.wasm and swipl.wasm are pretty much the same, but I’m not sure. However, there is no swipl.data and instead the files are on the host physical file system.

If this picture is correct, the node version should be shipped with a directory structure that provides the Prolog home. The build system creates swipl.js and swipl.wasm as these components are needed to run the Prolog steps for the build using node. In this setup the Prolog resources are in the home directory of the build tree.

Given the (large) Prolog resources that need to be distributed completely differently, it seems the browser and node versions are pretty much disjoint and the best we can do is to distribute them as different npm packages. Right?

P.s. node src/swipl.js works fine despite the HTMLCollection issue. Why is that?

the new qcompile/2 is quite nice, but it doesn’t seem to handle use_module/2 or reexport/2 properly.

m1.pl:

:- use_module(m2).

m1 :-
   m2.

m2.pl:

:- module(m2, [
   m2/0
]).

:- reexport(m3,[m3/0]).

m2 :-
   format("Hello from m2"),
   m3.

m3.pl:

:- module(m3, [
   m3/0
]).


m3 :-
   format(", and from m3 via re_export").

Test 1 without qcompile/2:

1 ?- m1.
Hello from m2, and from m3 via re_export
true.

Test 2 with qcompile:

1 ?- qcompile('m1.pl',[include(user)]).
true.
$ swipl -l m1.qlf -g m1
ERROR: -g m1: catch/3: Unknown procedure: m2:m2/0

Handling reexport/1,2 was the problem. The code for dealing with .qlf files created from multiple Prolog files is a bit old … Fixed with 7e685450f6117d2d20c2e45d123f635faaa13484.

1 Like

I will take a look. It used to be that .wasm and .data files were common for both nodejs and browser variants but seems like they are not anymore. It used to be:

  • Browser: swipl-web.js + swipl-web.wasm + swipl-web.data
  • Nodejs: swipl.js + swipl-web.wasm + swipl-web.data
2 Likes