Problem with saved state and http_server at Win10

Hi all,

first, I’m new here but not new to SWI Prolog. I use ist now about 3 years. The context of my question is to build a exe for deployment (SWI Prolog is not available on target machines)

Look at these minimal web server:

:- use_module(library(http/thread_httpd)).
:- use_module(library(http/http_dispatch)).
:- initialization(server(7007)).
:- http_handler(/, say_hi, []).

server(Port) :-
      writef("Starting Server at %d", [Port]),
        http_server(http_dispatch, [port(Port)]).

say_hi(_Request) :-
        format('Content-type: text/plain~n~n'),
        format('Hello World!~n').

If I do a saved state with

swipl --stand_alone=true --autoload=true --foreign=save --verbose=true -o test -c test.pl

then the resulting exe dies after start with error code 0xc0000005 (the errorcode is visible if starting using electronjs, but the exe dies also immediately if starting from powershell directly)

I have no idea whats going on and how I could get my exe of the webserver for my deployment.

Thanks for every help :slight_smile:

Regards

Hans

2 Likes

Got it working with a small change. Attached is your modified program. You need to use initialization/2 to avoid running at program load and you need something to keep the server running. Created the state using

swipl.exe --stand_alone=true --foreign=save -o hs -c hs.pl

Then copied the following files to the same dir:

hs.exe
libgcc_s_seh-1.dll
libgmp-10.dll
libswipl.dll
libwinpthread-1.dll
zlib1.dll

The state contains this:

% unzip -l hs.exe
Archive:  hs.exe
  Length      Date    Time    Name
---------  ---------- -----   ----
       92  2020-07-04 15:44   $prolog/options.txt
   708499  2020-07-04 15:44   $prolog/state.qlf
    86317  2020-07-02 13:25   shlib('x64-win64',readutil)
   114023  2020-07-02 13:25   shlib('x64-win64',memfile)
   306899  2020-07-02 13:26   shlib('x64-win64',sgml2pl)
   106578  2020-07-02 13:25   shlib('x64-win64',uri)
   112566  2020-07-02 13:25   shlib('x64-win64',time)
   100488  2020-07-02 13:25   shlib('x64-win64',files)
   119084  2020-07-02 13:25   shlib('x64-win64',http_stream)
   130752  2020-07-02 13:25   shlib('x64-win64',socket)
---------                     -------
  1785298                     10 files

This used SWI-Prolog 8.3.3, but 8.2.1 should be fine too.

hs.pl (388 Bytes)

Hi Jan,

thank you very much for your answer! If have tried what you suggested (I’ve seen no attatchment but I assume you mean to use

:- initialization(server(7007, main).

If done the file copies as suggested. Now I get the error "

ERROR: c:/program files/swipl/library/socket.pl:182: Initialization goal raised exception:
ERROR: Das angegebene Modul wurde nicht gefunden.

ERROR: In:
ERROR: [17] throw(error(shared_object(open,‘Das angegebene Modul wurde nicht gefunden.\r\n’),context(…,_3366)))
ERROR: [15]
ERROR: [14] with_mutex(‘$foreign’,load_foreign_library(foreign(socket),socket,install_socket))
ERROR: [12] ‘$run_init_goal’(shlib:load_foreign_library(…,install_socket)) at c:/program files/swipl/boot/init.pl:714
ERROR: [11] catch(system:‘$run_init_goal’(…),_3502,system:‘$initialization_error’(_3524,…,…)) at c:/program files/swipl/boot/init.pl:482
ERROR: [10] catch_with_backtrace(system:‘$run_init_goal’(…),_3560,system:‘$initialization_error’(_3582,…,…)) at c:/program files/swipl/boot/init.pl:532

and much more.

But its a progress :slight_smile: BTW I’ve tried this with 8.2.1

Sorry, forgot. Attached to my original post.

Did you check the content of the exe (actually a zip file) and did you copy all dlls and the exe to the same dir?

I didn’t really check on Windows but on Linux using Wine, but that typically gives only more trouble. except that it kindly tells you what dlls it cannot find. Note that the file reported by Prolog may not be the real issue: loading a dll also fails if the dll is there, but one of the indirect dependencies is missing. Instead of including the dlls in the state, you can also copy these to the same dir.

Hi Jan,
unfortunately, it doesn’t work. :pensive:
The zipfile content is identical to your file list, I’ve copied every dll from swipl\bin into the directory of the created .exe. Also I have used administrator shell, too. To be sure that not a path access is a problem, I also put the exe into the swipl directory (which is in PATH).
In any case the result is always the same. See attachted file for the error messages from powershell error.pl (16,0 KB)
I would like to help here, but I think this would require very deep knowlegde of SWI Prolog. I also have no Windows C++ development environment.

My interpretation: The socket.pl requires a module which can not be load. But foreign=save is given. So either the qsave_Program doesnt see a (indirect) dependency or the load of one dll fails by any other reason.

Do you have some interpretation out of the error messages?

cheers

Hans

If you copy socket.dll (etc.) to the deployment dir you should not use --foreign=save. This option adds the dlls to the saved state. For running, the system copies the dlls to the current temp directory and tries to load them. Could it be that your (or more generally modern) versions of Windows disable executing files from the temp directory?

You could check by setting TMP or TEMP to a local writeable directory where you surely can run files.

Another option is to simply zip your installed SWI-Prolog installation, add your Prolog file to it and a script to start bin/swipl.exe <myprog.pl> SWI-Prolog doesn’t need to be installed to use it. Installing doesn’t do much more than adding it to the desktop and/or start menu and associating .pl files with it. Neither is needed to run the system.

Hi Jan,

thank you very much! The information about what happes with the dll during start was very helpful.
I think you’re right, there must be a problem with TMP/TEMP. In win10 there is a system TEMP and a user TEMP. I’ve have not deeper investigated in because the solution is simple:

I have just copied all *dll from swipl/bin to my development directory, and as you pointed out not use --foreign=save

Now it works! Also my “real code” using Pengine and electronjs is now running fine.

So the basic receipt is (for deployment by an packager, in my case electronjs for Win10)

  • copy all swipl/bin *.dll files to development directory
  • create saved state with

swipl --stand_alone=true -o outputfile -c sourcefile.pl

  • ensure that your installer copies the dlls from the step above beside our *exe in your installation directory.

Tested with 8.2.1 64bit

Thank you very much for your help!

Cheers

Hans

2 Likes

That is generally too much of course. The once you would need is the list I gave before

and the ones that would be saved using --foreign=save.

Would be nice if some Windows expert could confirm this and find a way around. So, the real problem is that I have a .dll in the state (which is a zip file) and I want to load it. I guess there are three possible routes:

  • Save the dll to a file (but, then I need a safe place where I can do so and TEMP doesn’t seem to be that, or …)
  • Do not compress the dll in the zip and somehow tell Windows to load the dll from some file offset.
  • Load the dll into memory and somehow tell Windows to use it.

This is a very informative topic that you have started both in your question and the followup post.

If you don’t mind would you consider creating a Wiki post or two on this?

For the two post I am thinking

  1. How to create an executable that can run on a different machine that does not have the full SWI-Prolog installed.
  2. How to create a web server using SWI-Prolog

To make a wiki post just create a new topic and select wiki as the category. I can take care of the other details such as making it editable by all and the admin only abilities such as changing the owner to wiki.

Of if you prefer to just write something simplier or note a specific part that might be of interest to others you can use the category Nice to know

Thanks :grinning:

2 Likes

Hi Jan,

of course I agree, what I’ve written is a work-around.
I’ll take your points and investigate a little bit the Win10 policies.

Cheers

Hans

Honestly, I think at the moment what I do is a work-around, which is not “wiki-proof” :slight_smile: But in general you’re right, if we have a win10 solution it may be a good idea to describe it in the wiki. My context is to deploy an SWI Prolog App with electronjs, and this maybe worth an article.
Related to the webserver: there is a fine tutorial by Anne Ogborn, very easy to understand.

Cheers

Hans

2 Likes

Thanks.

I should have noted that the site has a Wiki for Useful Prolog References. I added the link to Anne’s tutorial and awarded you a badge. I know most people don’t care about badges but it is nice to get them to show that people are being appreciated.

Remember that most users, including you, can edit the wiki post directly, so please add to it if you know of something.

Hey, this is very kindly, thank you :slightly_smiling_face:
Indeed, a wiki is a good place for collecting. If’ve got some progress with my stuff (especially electronjs) then I will write an wiki page for it :wink:

Cheers

Hans

1 Like

I think that this tutorial is slightly out of date … some guidance on which modules to use was pushed to the documentation: Add documentation pointing to http_server for most use cases. by kamahen · Pull Request #135 · SWI-Prolog/packages-http · GitHub
@anniepoo - I don’t know how to update your tutorial … is it in a wiki somewhere?

3 Likes

the tutorial is at https://github.com/Anniepoo/swiplwebtut

The make file is a bat for windows, but asciidoctor can be used on linux fairly obviously

Hi Jan,

is there anything new from your side ? I’ve started to read a bit about Win10 Basics, for example this: Dynamic-Link Library Search Order - Win32 apps | Microsoft Docs to understand more of the background. My opinion is maybe the layout of the UWP would be nice for SWI under Win10

Cheers

Hans

I’m not aware there is a pending issue. Reading back I see --foreign=save doesn’t work, but that is barely an issue. You end up with an executable that needs a bunch of dlls anyway and IMO adding a few more doesn’t make it worse. Moreover, adding the DLLs is faster as it avoids the step to save the DLLs from the state to a temp file system. DLLs in the state would be mostly interesting if we could execute them directly from the saved state, but AFAIK very few operating systems can do that and Windows is (again AFAIK) not one of them. Worse, cleanup is dangerous. On Unix we can save the DLL (shared object there) to /tmp, load it and delete it immediately. That immediately removes its name from the filesystem and the file data will automatically be freed as the process terminates. On Windows that does not work and thus we need to do the delete ourselves at process termination. That is far more likely to fail.

You can use current_foreign_library/2 to get all the DLLs used by your program and loaded by means of use_foreign_library/1. You still need libswipl.dll and the DLLS required by it. Q: Is there a Windows command line utility or Win32 API to get the DLL dependencies? That would allow us to reliably automate this process. In the past I did have a Prolog script that would take the current executable and save the state as well as all required DLLs into a directory. That wasn’t really reliable though as it is hard to find the indirect DLL dependencies :frowning: So, these were hard wired and that always ages …

2 Likes

I have added win_process_modules/1, so we now can make the call below. That allows for generating a directory with all one needs to run some process on Windows a lot easier. It should also help debugging the not-so-uncommon cases where the system misbehaves because an old version of some DLL was copied to some unexpected place.

1 ?- win_process_modules(X), maplist(writeln,X).    
z:/home/janw/src/swipl-devel/build.win64/src/swipl.exe
c:/windows/system32/ntdll.dll
c:/windows/system32/kernel32.dll
c:/windows/system32/kernelbase.dll
c:/windows/system32/msvcrt.dll
z:/home/janw/src/swipl-devel/build.win64/src/libswipl.dll
c:/windows/system32/advapi32.dll
z:/home/janw/src/swipl-devel/build.win64/src/libgcc_s_seh-1.dll
z:/home/janw/src/swipl-devel/build.win64/src/libwinpthread-1.dll
c:/windows/system32/psapi.dll
c:/windows/system32/shell32.dll
c:/windows/system32/shlwapi.dll
c:/windows/system32/user32.dll
c:/windows/system32/gdi32.dll
c:/windows/system32/version.dll
c:/windows/system32/shcore.dll
c:/windows/system32/ole32.dll
c:/windows/system32/rpcrt4.dll
c:/windows/system32/winmm.dll
c:/windows/system32/msacm32.dll
c:/windows/system32/ws2_32.dll
z:/home/janw/src/swipl-devel/build.win64/src/libgmp-10.dll
z:/home/janw/src/swipl-devel/build.win64/src/zlib1.dll
c:/windows/system32/imm32.dll
X = ...
2 Likes

Hi Jan,

this is great :grinning:
BTW I’dont see this all as an issue in the sense of a “bug” or disengagement. I see it as a point which could be improved for Win10 users, gaining more comfort and helps to use it in an company IT environment. Thanks for your time!

Cheers

Hans