SWI-Prolog in the browser using WASM

[jan]

This topic discusses running SWI-Prolog in your browser using WASM (Web Assembler).

Updates

Below we list important updates that are also handled in this wiki page when applicable.

• [2023-05-03] Various updates to the build process. See also Building SWI-Prolog using Emscripten for WebAssembly (WASM)
• [2022-09-19] Released in 8.5.17. Interfaces should from now on be fairly stable. Documentation is at SWI-Prolog -- Manual
• [2022-09-19] Renamed js_yield/2 to await/2 and js_can_yield/0 to is_async/0.
• [2022-09-19] Support loading entire applications from a URL, e.g.

  ?- ['https://raw.githubusercontent.com/JanWielemaker/chat80/master/prolog/chat80.pl'].
  

• [2022-09-15] Added landing page SWI-Prolog WASM demos with access to other demos.
• [2022-09-15] Added library(dom) to provide (partial) compatibility with Tau-Prolog. This was used to port @PaulBrownMagic’s demo for Tau-Prolog, now running at CBG Chords.
• [2022-09-15] Added compiling from <script type="text/prolog"> elements.
• [2022-09-14] Improve handling of asynchronous code. Notable Prolog.forEach() allows enumerating Prolog answers asynchronously and js_yield/2 allows executing arbitrary Promises.
• [2022-09-08] Use JavaScript BigInt and Emscripten -s BIGINT to achieve full transparency between JavaScript and Prolog for big integers.
• [2022-09-06] Pass test suite using ctest
• [2022-09-06] Include most extensions that are meaningful and can be supported on WASM.
• [2022-08-25] Included GMP (Unbounded integers and rational numbers)
• [2022-08-25] Build now also works on MacOS (12.4 on M1)
• [2022-08-19] Start including the foreign extensions
• [2022-08-18] Added high level calling of Prolog predicates by JavaScript and of JavaScript from Prolog. These changes allow for e.g., DOM
• [2022-08-09] Fixed several build (dependency) issues. See updated build instructions.
• [2022-08-08] Fixed yield. Add auto yielding such that the browser remains responsive and queries can be aborted. Allow creating multiple files, Added persistent (using the browser localStorage) command line history and files.
• [2022-08-05] Extended JavaScript binding, run the browser shell using yield.
• [2022-08-03] “Native friend” mechanism for building is no longer needed.
• [2022-08-03] Added -s ALLOW_MEMORY_GROWTH=1 to allow (notably) for bigger stacks)

---

SWI-Prolog in your browser

There are two options for running SWI-Prolog in your browser:

• Using SWISH. This is a server based solution where the actual Prolog code is executed in a sandboxed environment on a (shared) server. SWISH is a comprehensive and mature solution for running SWI-Prolog in your browser that can support many deployment options.
• Actually running it in your browser. This is accomplished by compiling the SWI-Prolog C sources using Emscripten to WASM code. The initial port has been created by @rla. @dmchurch has made some improvements. @josderoo used it to run the EYE reasoner Inspired by the Ciao playground I (@jan) took another look at the WASM port.

A first shot at a SWI-Prolog shell in your browser

A very first version of running SWI-Prolog interactively in the browser was made by @rla. Since then Emscripten evolved and so did the configuration for SWI-Prolog. I have combined the basic test code and @rla’s initial shell to make something that at least performs the very basic tasks. It is online at

• https://dev.swi-prolog.org/wasm

How it works

Normally, SWI-Prolog is an interactive application that reads from and writes to a console. That doesn’t work in your browser because the JavaScript functions called are supposed to complete (quickly). @rla found a nice trick around that: we initialize Prolog and for running a query we put the query into a string that is used for input and we call break/0. This starts a toplevel that reads and runs the query. The price is that the query cannot receive any input (i.e., cannot read). Another problem we are faced with is that long running Prolog queries make the browser unresponsive, i.e., we need to return to the browser’s event loop to create a responsive web page. This is solved using foreign yield, a mechanism to make the VM main function (PL_next_solution()) return such that it can be resumed. Yielding can be dong explicitly using js_yield/2. This is currently used for reading the next toplevel query and, if a query succeeds with a choicepoint, for reading whether the user wants the next result or stop. Finally, the VM auto yields every N inferences, controlled by the Prolog flag heartbeat. The JavaScript wrapper resumes the VM, roughly escaping to the browser event loop every 20ms.

Note that an alternative solution is run Prolog in a web worker. This however isolates Prolog from the page, requiring postMessage() to be used to communicate between the page and Prolog.

The SWI-Prolog web appliance uses the Emscripten virtual file system. SWI-Prolog runs in /prolog and the normal Prolog home, containing the boot file and libraries is in /swipl. You can use library(shell) utilities ls/0, pwd/0, cd/1 to look around.

The editor is a simple HTML textarea. The (Re)consult button copies the content of the textarea into a the file ‘/file.pl’ and calls consult/1 on that file. It has a dropdown to select a file and buttons to add a new file or delete a file. Files are saved to the browser’s localStorage on page unload and reloaded on page load. The query input field provides history using the arrow up/down. The history is also saved to the browser’s localStorage.

Calling between JavaScript and Prolog

High level calling from JavaScript to Prolog is based on automatic conversion between JavaScript data and Prolog data. This follows Consider adding an option to use a different JSON Format · Issue #4 · SWI-Prolog/packages-mqi · GitHub. The table below summarizes the conversion.

JavaScript	→	Prolog	→	JavaScript	Notes
string		atom		string
		string		{$t:"s", v:Text}	(1)
number		integer or float		number or bigint	(2)
bigint		integer		number or bigint	(2)
{$t:"v", v: Id}		variable		{$t:"v", v: Id}	(3)
{$t:"t", Func:[A1,..]}		compound Func(A1, ...)		{$t:"t", Func:[A1,..]}
Array		List		Array
{$t:"partial", v:Array, t:Tail}		Partial List		{$t:"partial", v:Array, t:Tail}
Plain Object		Dict		Object
Other Object		Blob <js_<Class>>(Id)		Object	(4)

Notes

1. Using Prolog.toJSON(term, {string:"string"}) a Prolog string is converted to a plain JavaScript String. This is currently used by Prolog calling JavaScript.
2. JavaScript number is a float. When converting to Prolog and the number is integer we create a Prolog integer, otherwise a Prolog float. When converting from Prolog, a float always becomes a number. Prolog integers below 2^{53} are represented as a JavaScript number. Higher integers are converted to a JavaScript BigInt. Exchanging big numbers for now use a decimal string as intermediate representation.
3. When generated, the Id is an integer. For conversion to a Prolog variable the Id is optional and when two variables share the same Id, they are unified.
4. Plain objects are instances of Object. Such objects are converted to Prolog dicts.
   Other objects that are passed to Prolog are kept in a JavaScript object (Prolog.objects) and represented in Prolog using a blob. (Atom) garbage collecting the blob removes the object from Prolog.objects.

JavaScript calling Prolog

The Module contains a property prolog that represents Prolog. We summarize the most important ones.

Prolog.call(String)

Evaluate a string as a Prolog goal. This is typically used for simple goals such as setting a Prolog flag.

Prolog.query(String, Inputs)

Returns an iteratable object. String is the goal which may contain variables. Inputs is an object that may bind some of the variables in the goal. Solutions are created using Query.next() or Query.once(). The first satisfies the JavaScript iterable protocol. The second returns a dict holding the values for the output variables.

Examples

The example below calls sum_list/2 as once/1, passing a JavaScript Array as a Prolog list and returning an object with the key Sum that represents the result.

function sum_list(list) {
   return Prolog.query("sum_list(List, Sum)", {List:list}).once().Sum;
}

The next example enumerates all answers for p/1

for(const r of Prolog.query("p(X)")) {
  console.log(r.X);
}

Prolog calling JavaScript

Calling JavaScript from Prolog is implemented by :=/2, referring to classical assignment. The predicate can evaluate several JavaScript expressions. We start with some examples:

?- Res := myfunc([1,2,3]).
?- Max := 'Math'.max(10, 20).
?- Out := document.getElementById("output"),
?- Par := document.createElement("p"),
   Par.textContent := "Hello World!",
   _ := document.body.appendChild(Par).

During the conversion, values are converted according to the rules in the table above, converting Prolog strings to a plain JavaScript string except for

• An atom is considered a getter. Dereferencing starts with window.
• A compound is considered a function call. Arguments a are evaluated according to the same rules.
• A term #Term (# is a prefix operator) is taken as a literal value. Thus, createElement(p) would use the global variable p as argument. Using "p" or #p gives the desired result.

Status, issues and future work

Lacking features and issues of the SWI-Prolog WASM port

• Threads Unclear whether the thread support of WASM can support Prolog threads.
• Engines As is, engines require threads. This however is not fundamentally the case while engines could be worthwhile in the JavaScript context to implement reliable coroutines.
  update: current git (Mar 5, 2023) allows building the core with support for engines without including threads. It is merely a proof of concept.
• Synchronous I/O
  • Now deals with reading toplevel queries and prompt for more answers using yield
  • auto yielding keeps the page responsive
  • Yielding is still problematic
    • It cannot happen if there are intermediate Prolog → WASM (C) → Prolog callbacks
    • Auto yielding currently happens at the exit port. Recursive loops that call no other predicates only exit when all is done. Possibly we can also yield at the enter port (after head unification).
• Sync the API with Ciao where possible?

The future of this SWI-Prolog shell

The shell at https://dev.swi-prolog.org/wasm is just a toy. It is intended as a simple platform to help debugging and resolving the above issues. A likely future scenario is to integrate the WASM based solution into SWISH.

---

Building the WASM version

The instructions below have been tested on Ubuntu 22.04 and MacOS 12.4 on an M1. Please edit this page to report on other platforms.

Install Node.js

You need at least v18.16 of Node.js, which is available from GitHub - nodejs/node: Node.js JavaScript runtime. To build this, clone the repository, git checkout v18.16.0 (or whichever version you wish) and follow the instructions at GitHub - nodejs/node: Node.js JavaScript runtime (you may wish to use the command ./configure --prefix=$HOME/.local instead of just ./configure)

Install emscripten and the zlib library

See Download and install — Emscripten 3.1.39-git (dev) documentation. These instructions install emscripten in ~/wasm/emsdk on a Linux machine (requires about 1.1 Gb disk)

cd
mkdir wasm
cd wasm
git clone https://github.com/emscripten-core/emsdk.git
cd emsdk
./emsdk install latest
./emsdk activate latest
source ./emsdk_env.sh

build the zlib dependency

Note: zlib uses the semantic version in the name of the file, as such it is hardcoded in these instructions. If zlib is updated then the zlib name needs to be changed in these instructions accordingly. (As of 2023-05-03 version: 1.2.13)

cd ~/wasm
wget https://zlib.net/zlib-1.2.13.tar.gz
tar -xf "zlib-1.2.13.tar.gz"
cd zlib-1.2.13
emconfigure ./configure --static --prefix=$HOME/wasm
emmake make
emmake make install

The emmake make command generates quite a few warning messages similar to "warning: a function definition without a prototype is deprecated in all versions of C and is not supported in C2x [-Wdeprecated-non-prototype]`

Build the pcre2 library (optional)

Provides library(pcre) to Prolog.

git clone https://github.com/PCRE2Project/pcre2
cd pcre2
git checkout pcre2-10.42
emcmake cmake -DCMAKE_INSTALL_PREFIX=$HOME/wasm -DPCRE2GREP_SUPPORT_JIT=OFF -G Ninja .
ninja
ninja install

Build the GMP library (optional)

As is, using GMP provides faster rational numbers at the cost of a larger binary and (effectively) LGPL license. When omitted, LibBF based unbound integers, random numbers and rational numbers are used, providing exactly the same features.

cd ~/wasm
wget https://gmplib.org/download/gmp/gmp-6.2.1.tar.lz
tar xf gmp-6.2.1.tar.lz
cd gmp-6.2.1
emconfigure ./configure --host=none --disable-assembly --prefix=${HOME}/wasm
make -j
make install

This process is broken on MacOS. See Host CC cannot compile on Mac OS Monterey (12.2.1) · Issue #16358 · emscripten-core/emscripten · GitHub. To work around this, create a file conf holding the content below, make it executable (chmod +x conf) and configure using emconfigure ./conf

#!/bin/sh

export HOST_CC=/usr/bin/clang
./configure --host=none --disable-assembly --prefix=${HOME}/wasm

Download and install SWI-Prolog from the git sources

Download the sources

git clone https://github.com/SWI-Prolog/swipl-devel.git
cd swipl-devel
git submodule update --init

Build the WASM version using Emscripten

cd swipl-devel
mkdir build.wasm
cd build.wasm

Now save the following text to a new file configure and make it executable using chmod +x configure

export WASM_HOME=$HOME/wasm
export GMP_ROOT=$HOME/wasm
source $WASM_HOME/emsdk/emsdk_env.sh
TOOLCHAIN=$EMSDK/upstream/emscripten/cmake/Modules/Platform/Emscripten.cmake
[ -f $TOOLCHAIN ] || echo "Could not find emscripten toolchain"

cmake -DCMAKE_TOOLCHAIN_FILE=$TOOLCHAIN \
      -DCMAKE_BUILD_TYPE=Release \
      -DCMAKE_FIND_ROOT_PATH=$HOME/wasm \
      -DINSTALL_DOCUMENTATION=OFF \
      -DGMP_ROOT=$GMP_ROOT \
      -DNODE_JS_EXECUTABLE=$(type -p node) \
      -G Ninja ..

If you did not build GMP or PCRE, you should also specify -DUSE_GMP=OFF and/or -DSWIPL_PACKAGES_PCRE:BOOL=OFF (this has not been tested).

Now configure and build the version in the build.wasm directory using

./configure
ninja

To update to the latest version one should run the commands below. Removing src/wasm-preload is required to make sure changes to the Prolog libraries are correctly incorporated in src/swipl-web.data which populates /swipl in the WASM version.

git pull
git submodule update --init
cd build.wasm
rm -rf src/wasm-preload && rm -f src/swipl-web.* && ninja

Running using Node.js

If all went right, you can now run the wasm version using node as

$ node src/swipl.js
Welcome to SWI-Prolog (32 bits, version 8.5.15-26-gc1ca50a94)
SWI-Prolog comes with ABSOLUTELY NO WARRANTY. This is free software.
Please run ?- license. for legal details.

    CMake built from "/home/janw/src/swipl-devel/build.wasm"

For online help and background, visit https://www.swi-prolog.org
For built-in help, use ?- help(Topic). or ?- apropos(Word).

1 ?-

Foreign extensions (packages that rely on C/C++ code)

Although Emscripten does support dynamic linking, it is discouraged because it increases the code size and slows down execution. Therefore the Emscripten port uses a new cmake option: -DSTATIC_EXTENSIONS. This adds the foreign libraries (use_foreign_library/1,2) to libswipl and generates src/static_packages.h which is used to make use_foreign_module/1 call the initialization function of the package. This way no changes to the packages themselves is required.

Currently ported foreign extensions are below. This set will be extended. You can verify supported features using check_installation/0.

Library	Prolog code	C code
library(crypt)	Prolog	C
library(double_metaphone)	Prolog	C
library(porter_stem)	Prolog	C
library(readutil)	Prolog	C
library(semweb/rdf_ntriples)	Prolog	C
library(semweb/turtle)	Prolog	C
library(sgml)	Prolog	C
library(sha)	Prolog	C
library(snowball)	Prolog	C
library(uri)	Prolog	C
library(zlib)	Prolog	C
library(pcre)	Prolog	C

Running in your browser

The generated file for your browser is src/swipl-bundle.js. The Prolog shell discussed above can be found in swipl-devel/src/wasm/shell.html. It can be started using a Prolog based web server in the same directory named server.pl. The server must be started in the build directory as it uses src/swipl-bundle.js to find the WASM components.

cd build.wasm
swipl ../src/wasm/server.pl [--port=8000]

How to support such work

As noted in this post, give these GitHub repositories a star SWI-Prolog, Ciao and SWISH

Star history

Discourse Linkify words

Discourse has a plug-in (theme component) that searches post for specific text and automatically converts the text to a link (linkify).

These words with a # prefix were added for this topic

• #changelog
• #devel_commits
• #wasm_demo
• #wasm_wiki

#changelog - A web page to view SWI-Prolog change log(s). There is an option to select a range of versions and select either the development or stable release.

#devel_commits -To understand the changes to the code, review the GitHub commits for SWI-Prolog development repository. To make it easy to identify the link when writing a post noting the GitHub SWI-Prolog development repository commits just add #devel_commits,
Note: That while not always most of the commits related to WASM start with the name WASM.

#wasm_demo - For those that follow along with the post actively, knowing were to find the demo site is easy. For those dropping in the middle via a Google search some years in the future it can become a needle in a haystack search for the site. To make it easy to identify the link when writing a post noting the demo site just add #wasm_demo,

#wasm_wiki - While the discussion thread gets all activity the post that has the crucial facts is not always identified in the post that discuss them. To make it easy to identify the link when writing a post noting this wiki page just add #wasm_wiki