Whats also interesting on the webassembly status page that you
posted. It features Deno, which is also from Ryan Dahl just like nodeJS.
If I am not mistaken Deno features a certain kind of web server,
not using multithreading but multiple workers and each worker
uses coroutines. Further it seems people now compile JavaScript
into WASM. Maybe drop not only 32-bit Windows but also
drop all of your web server implementation and just use an existing
web server. Advantage you get maybe HTTP 2 for free and also other
architectures like coroutine/worker implementation. One way forward
to do such as thing is to raise a WIP(*) with a web server object abstraction,
i.e. defining an API. Maybe the same API could be used for your old
web server(s) and for future web server backends.
P.S:: Trealla Prolog uses wasmtime and not Deno for their web server,
but if I remember well they initially also did something with wasmer, BTW
here is a moon server demo, somehow I recently unconsciously
plagiarized this example by master @guregu:
Disclaimer: I am not saying your web server is bad, I did some testing
it performs quite well. You can also view it as tapping into another
ecosystems, beyond the traditional C programs aproach. But ecosystems
are always a kind of bet, will they survive?
I was more considering to replace the SWI-Prolog client with e.g., libcurl.
There are zillions of web servers around, each with their own merits. The aim of the SWI-Prolog web server is to make it easy to implement web services using a Prolog predicate. Doing that by using another web server framework surely avoids a bit of work and makes it easier to follow developments of the HTTP standard. It also makes you dependent on that framework. Furthermore, accepting the connection and decoding the HTTP request is only a modest part of the web server framework.
WASM is surely not a good choice now as the WASM version of Prolog is slow and limited. You could use e.g. Python Flask using the Janus interface. That probably works fairly well. For serious work I’d first look into embedding SWI-Prolog in e.g., nginx. That would give excellent performance and support for many HTTP features.
But, as you noticed, performance is not too bad. It is probably possible to easily gain a little more by moving reading and parsing the HTTP request header to C. I don’t know how hard it would be to add HTTP 2. If I recall well, @jamesnvc did it for the client? I also do not know how useful it would be, given the typical deployment of the Prolog HTTP server.
The main weak point is that it uses a worker thread to handle a request. The scales relatively poorly for requests that transfer a lot of data while using little CPU, such as serving large files. Serving large files surely is not the target application for the SWI-Prolog web server (although it would not be terribly hard to hand all such requests to a single thread that uses poll() to handle a large number of file downloads).
The crucial and important point would be to first define a:
So that different frameworks become plugable, of course with
a little glue Prolog code for each supported backend. The glue
Prolog code would be the deliverable for each supported backend.
But for the end-user its always the same API. Irrespective what server
framework is used. Currently there is not much convergence, for
example Scryer Prolog invented its own API:
I have also recently started designing and implementing an API, which
already runs on the platforms Python (not using Flask, but http_server
module), JavaScript (using node) and Java (using jdk.webserver or Tomcat
Embedded both work) as proof of concept for a highly polyvalent API.
Finding a common ground here would indeed be useful. SWI-Prolog’s handlers take a broken-down HTTP request and write a document according to the CGI standard to current_output. That is quite simple and flexible. It can even deal with chunked encoding, completing and sending a chunk on flush_output/0. This allows for generating really long documents in limited space.
The implementation is quite involved though. It has grown over the years
Well, there is Prolog code that reads the headers written to the page and combines this with other info to create the final header. So, we can validate and/or rewrite headers. But yes, this is not the most elegant part of the interface The streaming properties are desirable though. I’ve had a user who wrote hundreds of gigabytes ntriples data.
I like the idea of a common API for Prolog HTTP things. I’d be happy to change Trealla’s WASM HTTP API to match it. Thanks for the mention BTW
For some background: Trealla’s server-side support for WebAssembly HTTP stuff targets the old version of Spin’s HTTP component (defined here and here and wit-bindgen generates code like this for using it). Spin is server software that can run any wasm component that speaks CGI or exports certain functions that match its API. It uses wasmtime internally.
Recently, Wasm Components have stabilized and there now exists an official wasi-http spec which aims to standardize an API for HTTP things, which I believe Spin supports now too and is also built into wasmtime (wasmtime serve). I’ll update Trealla to support this eventually. I think the idea is that software like Spin or edge computing platforms like Fastly should be able to run any wasm software that uses this spec. It’s kind of like a newer, fancier version of CGI.
Of course, this is stuff that mostly library/runtime authors need to care about. A nice high level HTTP API for Prolog can abstract all this away, and I think we could find a good API for various Prologs to converge on.
(A bit unrelated but I think a Prolog library that could use arbitrary Wasm Components FFI-style would also be super cool! Trealla’s support is baked in for specific components with generated C code via wit-bindgen. I’d like to experiment with this some time. Might be possible with WASI v0.2 which was just released.)
AFAIK, the stream encoding is set to ASCII while writing the header (possibly ISO Latin 1, see source). Except for the not-so-nice error message, I see little wrong with this. We could recode non-ascii values, as suggested using RFC 2047. I doubt it makes much sense. If a field is supposed to be able to handle non-ASCII, it is typically also specified how to encode it. Often, such fields use base64 encoded UTF-8.
The error is an internal server error, which happens to be correct
The streaming properties are desirable though. I’ve had a user who wrote hundreds of gigabytes ntriples data.