I would like to study a code sample that uses the pengine_input predicate that shows how you return input to the server from the client code side. My main question is what method do I use on the client side Javascript code to return the pending input to the server?
Also, if there ends up being a large number of pending input requests, could that affect negatively the server as far as thread usage or memory is concerned? I am worried that scenario could arise if I start doing asynchronous operations in my Node.js server that is the client to my PEngines server, that are part of the processing chain. For example, what happens if I get a large number of the following events:
A client side request for user A from browser occurs
PEngines code executes a pengine_input call (user A)
My Node.JS server executes an asynchronous call to an outbound server or perhaps even to a human (i.e. - a very slow response situation)
While waiting for a response, client request for user B occurs
PEngines code executes a pengine_input call (user B)
…
And so on and so on. So in the case of a response being dependent on a large number of external, slow services, I worry that the interleaved operations might lead to a “pile up” of open threads or SWI-Prolog resources associated with the pending pengine_input calls, and I am wondering if that situation could cause problems. Is this possible?
The most extensive example is probably the SWISH server. See web/js/runner.js. And yes, you can create all sorts of loops that will blow up the server. That is simply a fact for services that can contact other services. What to do against that depends a lot on the setup.
Thanks Jan. Is there a number or set of numbers I should watch closely in the PEngines admin panel that will help me estimate the amount of memory one of my PEngines sessions is holding onto while waiting for a pending pegine_input() response? If there is/are, I’ll set up a test case where I launch a bunch of Node.JS calls into my PEngines server that will all trigger a pengine_input() call. I won’t respond to this call on purpose, just to see how the numbers change.
Pengines has become a bit overloaded term. I think you are talking about this repo. Be a bit careful with this code as it is not maintained. The core Pengines are distributed with SWI-Prolog itself and resides here. I.e., to run a Pengine server you do not need anything but the Prolog release. Then there is SWISH, which started of as an application for the original pengines web app, but which grew into being a stand alone project. The core pengines as distributed with Prolog and SWISH are both considered key features of the system and are actively maintained. Finally there is @torbjorn.lager’s new Web Prolog.
For resource tracking I’m working on webstat, which should eventually become a complete web based dashboard to track what is going on inside a SWI-Prolog process. There is a long way to go … Until then your tools are the OS monitoring tools, statistics/0, the xpce based thread monitor and zillions of predicates that can tell you details about the state of the system but that are scattered al over the place and hard to use (that is what webstat should solve).
There is some stuff of general interest such as overall resource usage and profiling (though currently only of the main thread). Most is about tabling as its development started to gain insight about what was going on in a large tabled program.
Any thoughts on identifying orphan PEngine sessions? Here is the scenario I want to handle:
My Node.JS server queries My PEngines server (i.e. - SWI-Prolog using its PEngines features, aka the “main” repo)
The PEngine session executes a pengine_input() request to my Node.JS server
My Node.JS server never responds, due to a crash, an Exception, etc.
Now the PEngine session is an “orphan”
Is there a facility in Prolog/PEngines to detect a sessions that are stale? For example, some way to scan for any open sessions that have not had any activity in X seconds?
If not, would a viable approach be as follow:
From inside my Prolog code, create a wrapper predicate called something like guard_pengine_input()
That predicate asserts the session ID and the current date/time before issuing the pengine_input() call. For example “assert(pengine_input_called(InstanceId, Date)).”
Retract that from the dynamic database when the session that issued the pengine_input() call concludes.
Periodically scan the dynamic data base using the query “pengine_input_called(InstanceId, Date).”. Kill any sessions whose Date value is over X seconds old.
If that is a viable strategy, what is the best mechanism for periodically scanning the dynamic database? Is there a native SWI-Prolog predicate for setting up something like an internal cron job? If not, then should I set up a predicate marked as “safe” for PEngine use that my Node.JS server calls periodically on a Javascript interval, and have this query do that orphan detection work?
Pengines are protected by two time limits: one on wall time while computing and one on wall time while waiting. If either elapses the Pengine stops. See pengines.pl settings as well as SWISH debug.pl plugin.
