A bundled Python interface

[jan]

@EricGT

?- py_shell.
ERROR: Python error <class 'AttributeError'>:
ERROR:   'NoneType' object has no attribute 'write'

I guess you tried this using swipl-win, e.g., the version that provides its own console window? That unfortunately does not work. It works fine for me using swipl.exe in a Windows console or PowerShell. The problem is that Python writes to the console, but under swipl-win there is no console.

I don’t know whether this can be fixed. Does anyone know whether (and how) we can rebind Python’s read/write to the user?

Note that as long as you make Python calls that do not read or write to the console, all should work fine.

[EricGT]

@jan

I guess you tried this using swipl-win, e.g., the version that provides its own console window?

Yes.

On Windows because I start SWI-Prolog so often, e.g. a new top level to test a predicate then close, I keep an icon for SWI-Prolog in the Windows taskbar

and that starts it with

“C:\Program Files\swipl\bin\swipl-win.exe” --win_app

---

Thanks for the info.

Starting SWI-Prolog on Windows to use py_shell this way works as expected.

Using a Windows command prompt

Microsoft Windows [Version 10.0.19045.3208]
(c) Microsoft Corporation. All rights reserved.

C:\Users\Groot>swipl
Welcome to SWI-Prolog (threaded, 64 bits, version 9.1.12-18-g7e6f7de02)
SWI-Prolog comes with ABSOLUTELY NO WARRANTY. This is free software.
Please run ?- license. for legal details.

For online help and background, visit https://www.swi-prolog.org
For built-in help, use ?- help(Topic). or ?- apropos(Word).

1 ?- py_version.
% Janus embeds Python 3.11.4 (tags/v3.11.4:d2340ef, Jun  7 2023, 05:45:37) [MSC v.1934 64 bit (AMD64)]
true.

2 ?- py_shell.
Python 3.11.4 (tags/v3.11.4:d2340ef, Jun  7 2023, 05:45:37) [MSC v.1934 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> from janus import *
>>> once("writeln('Hello world!')")
Hello world!
{'status': True}

---

There are test cases for swipl but they are not included with an install. To install and run the test.

Start Windows command prompt

Microsoft Windows [Version 10.0.19045.3208]
(c) Microsoft Corporation. All rights reserved.

C:\Users\Groot>cd "C:\Users\Groot\Documents\GitHub"

C:\Users\Groot\Documents\GitHub>git clone https://github.com/SWI-Prolog/packages-swipy.git
Cloning into 'packages-swipy'...
remote: Enumerating objects: 446, done.
remote: Counting objects: 100% (446/446), done.
remote: Compressing objects: 100% (253/253), done.
remote: Total 446 (delta 287), reused 341 (delta 182), pack-reused 0Receiving objects:  80% (357/446)
Receiving objects: 100% (446/446), 113.78 KiB | 3.07 MiB/s, done.
Resolving deltas: 100% (287/287), done.

C:\Users\Groot\Documents\GitHub>swipl
Welcome to SWI-Prolog (threaded, 64 bits, version 9.1.12-18-g7e6f7de02)
SWI-Prolog comes with ABSOLUTELY NO WARRANTY. This is free software.
Please run ?- license. for legal details.

For online help and background, visit https://www.swi-prolog.org
For built-in help, use ?- help(Topic). or ?- apropos(Word).

1 ?- pwd.
% c:/users/groot/documents/github/
true.

2 ?- working_directory(_,'C:/Users/Groot/Documents/GitHub/packages-swipy').
true.

3 ?- pwd.
% c:/users/groot/documents/github/packages-swipy/
true.

4 ?- [test_janus].
true.

5 ?- test_janus.
[51/51] xsb_call:reverse ......................... passed (0.003 sec)
% All 51 tests passed in 0.458 seconds (0.359 cpu)
true.

[jan]

@EricGT

git clone https://github.com/SWI-Prolog/swipy.git

which gives an error for me when used

``

Sorry. Must be (notethe packages-)

pip install git+https://github.com/SWI-Prolog/packages-swipy.git#egg=janus

[EricGT]

Thanks.

No need to say sorry. I am more than happy to work through the bumps to see this work. As my git skills are not as great as yours was not sure if I was doing something wrong. Thanks for the update.

It also explains why @anon419083 was seeing the Git account authorization errors; I too was getting authentication request that I have never seen before using Git. Once the correct URL was used it worked as expected.

[jan]

How to deal with Janus on MacOS

It seems that dealing with the new Janus package is pretty much settled for Windows and Linux. On Windows we have the Python binary packages that provide the fairly version independent python3.dll, so we can search %PATH% for this and be happy. On Linux the packages take care of the dependencies and source installation is easy if you want something different.

On MacOS, I find on my machine Python frameworks in LibreOffice (3.8), Xcode (3.8 and 3.9), /Library (3.9), Homebrew (3.9 and 3.10) and Macports (3.10).

I guess installation in Homebrew or Macports should be fine: just add a dependency on Python3 and the package manager should do its job. But, what about the binary we distribute? We can’t really tie it to a particular Python version, but there is no equivalent of python3.dll. LibreOffice including its own version of Python should of course work, but this can hardly be a good solution Suggestions?

[JamesH]

From within Python it should be possible by changing the values of sys.stdin, sys.stdout and sys.stderr to some “file like” objects.

I’ve used this approach in some simple cases and it was pretty straightforward. No additional changes were needed.

It can be done by directly changing the values. There are also packages like contextlib.redirect_stdout.
James

[jan]

@JamesH

From within Python it should be possible by changing the values of sys.stdin, sys.stdout and sys.stderr to some “file like” objects.

That sounds promising. Do you have a pointer how this works? Maybe some example code? SWI-Prolog’s I/O is quite flexible. A stream has a block of functions that is pretty much the same as C’s FILE with some extensions. Should be possible

[peter.ludemann]

@jan

LibreOffice including its own version of Python should of course work, but this can hardly be a good solution Suggestions?

You can run pip (or any Python program) under a particular version of Python, e.g. python3.12 pip install . or python3 pip install .. (There’s also pip3 install . …)
This is sometimes necessary on systems where python still points to python2.

(I tried installing swipy on my Debian (Chromebook) system which has a local version of python3.12, and it failed. I haven’t yet figured out what the errors mean but I suspect it’s somehow using the wrong pip.)

[JamesH]

Just something like this:

import sys
sys.stdout = open(“output.txt”, “w”)
print(“This will be written to the file”)

I looked at the source code for contextlib and this is basically what it does, but using some abstract classes and setattr instead of “=” to make it more general.
So if you can produce Python bindings for whatever streams are used for input/output to the Windows interface and then assign them to sys.stdout/sys.stdin it should work.

There are alot of articles discussing this approach, eg: https://www.devdungeon.com/content/using-stdin-stdout-and-stderr-python which includes an example redirecting it to a string buffer.

[Boris]

It is very exciting to see this happen! It truly provides a natural way of embedding Prolog into an existing Python codebase!

I am using archlinux on my personal machine (yeah I know…). Using Python from Prolog works just fine. Installing Janus for Python was a bit roundabout.

I cannot use pip install, I get:

error: externally-managed-environment

× This environment is externally managed
╰─> To install Python packages system-wide, try 'pacman -S
    python-xyz', where xyz is the package you are trying to
    install.
    
    If you wish to install a non-Arch-packaged Python package,
    create a virtual environment using 'python -m venv path/to/venv'.
    Then use path/to/venv/bin/python and path/to/venv/bin/pip.
    
    If you wish to install a non-Arch packaged Python application,
    it may be easiest to use 'pipx install xyz', which will manage 
    virtual environment for you. Make sure you have python-pipx
    installed via pacman.

note: If you believe this is a mistake, please contact your Python
installation or OS distribution provider. You can override this, at
the risk of breaking your Python installation or OS, by passing
--break-system-packages.
hint: See PEP 668 for the detailed specification.

(At this point I discovered that there exists another Python janus, which confused me for a second)

Using pipx (the second option from the error message above) doesn’t seem to work out of the box. Something happens and something is installed but the Python janus cannot find the SWI-Prolog binary.

Using a venv and pip works; however, for some reason venv installed an outdated pip in the virtual environment (??) and then pip complained and told me to update it:

Processing /home/boris/install/swipy
  Preparing metadata (setup.py) ... done
Installing collected packages: janus
  DEPRECATION: janus is being installed using the legacy 'setup.py
install' method, because it does not have a 'pyproject.toml' and the
'wheel' package is not installed. pip 23.1 will enforce this behaviour
change. A possible replacement is to enable the '--use-pep517' option.
Discussion can be found at https://github.com/pypa/pip/issues/8559
  Running setup.py install for janus ... done
Successfully installed janus-0.1.0

[notice] A new release of pip available: 22.3.1 -> 23.2.1
[notice] To update, run: python -m pip install --upgrade pip

The Janus installation seemed to work despite the warnings and notices but I was not willing to get to the bottom of it. Python versions have been a sad story for a decade now so I am not surprised, just annoyed

If you upgrade the pip inside your virtual environment and then install Janus everything goes smoothly:

$ python -m venv ./swipyvenv
$ cd swipyvenv/
$ bin/pip --version
pip 22.3.1 from /home/boris/code/swipyvenv/lib/python3.11/site-packages/pip (python 3.11)
$ bin/python -m pip install --upgrade pip
Requirement already satisfied: pip in ./lib/python3.11/site-packages (22.3.1)
Collecting pip
  Using cached pip-23.2.1-py3-none-any.whl (2.1 MB)
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 22.3.1
    Uninstalling pip-22.3.1:
      Successfully uninstalled pip-22.3.1
Successfully installed pip-23.2.1
$ bin/pip --version
pip 23.2.1 from /home/boris/code/swipyvenv/lib/python3.11/site-packages/pip (python 3.11)
$ bin/pip install ~/install/swipy
Processing /home/boris/install/swipy
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: janus
  Building wheel for janus (pyproject.toml) ... done
  Created wheel for janus: filename=janus-0.1.0-cp311-cp311-linux_x86_64.whl 
  Stored in directory: /tmp/pip-ephem-wheel-cache-i1f_5jro/wheels/
Successfully built janus
Installing collected packages: janus
Successfully installed janus-0.1.0
$ bin/python
Python 3.11.3 (main, Jun  5 2023, 09:32:32) [GCC 13.1.1 20230429] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from janus import *
>>> once("writeln(hello)")
hello
{'status': True}
>>> 

Or is there a good reason why an older version of pip was installed by default? This is a question for seasoned Python developers I guess.

EDIT: apparently this is all on purpose. Which pip version were you using in these examples @jan?

[jan]

@Boris

(At this point I discovered that there exists another Python janus, which confused me for a second)

Good to know I’ll throw this into the discussions about Janus (which happen by mail and Zoom).

@Boris

DEPRECATION: janus is being installed using the legacy 'setup.py
install' method, because it does not have a 'pyproject.toml' and the
'wheel' package is not installed. 

Is there some seasoned Python developer who can help here?

@Boris

Which pip version were you using in these examples @jan?

I’m using stock Python and pip from Ubuntu 22.04; Python 3.10 and pip 22.0.2. Python 3.11 on MacOS and Windows. Dunno the pip version. Some complain there is a newer version. I ignore that

The brief experience with Python tells me the version issues are much worse than with (SWI-)Prolog

[jan]

That looks easy. I guess the route is to subclass the Python stream class and call out to C to talk to SWI-Prolog’s I/O?

Will look into that later. Unless, of course, there is a volunteer

[Boris]

@jan

@Boris

(At this point I discovered that there exists another Python janus, which confused me for a second)

Good to know I’ll throw this into the discussions about Janus (which happen by mail and Zoom).

It shouldn’t be a real problem but at least archlinux has a python-janus and ubuntu and debian both have a python3-janus packages in their package databases

[JamesH]

That sounds like a good approach. The class to subclass is probably _io.TextIOWrapper.
I don’t have a Windows machine set up for this kind of thing unfortunately, otherwise I’d try it.

[jan]

I’m playing with something simple. It works for output, but not yet for input. In Python, we do

import io
import sys

class PrologIO(io.TextIOWrapper):
    def __init__(self, plstream, *args, **kwargs):
        self.prolog_stream = plstream
        super().__init__:__call__(args, kwargs)
    def write(self, s):
        once("janus:py_write(Stream, String)", {'Stream':self.prolog_stream, 'String':s})
    def readline(self, size=-1):
        once("janus:py_readline(Stream, Size, Line)", {'Stream':self.prolog_stream, 'Size':size})['Line']

def bindIO():
    sys.stdin  = PrologIO("user_input",  io.BytesIO(), line_buffering=True)
    sys.stdout = PrologIO("user_output", io.BytesIO(), line_buffering=True)
    sys.stderr = PrologIO("user_error",  io.BytesIO(), write_through=True)

And in Prolog this. The ?>> prompt such that we see we are in charge for now Python complains with object.readline() returned non-string though I should also do something with the prompt. Seems Python already writes its prompt, which upsets SWI-Prolog’s line editor.

py_write(Stream, String) :-
    format(Stream, '~s', [String]).

py_readline(Stream, _Size, Line) :-
    prompt1('?>> '),
    read_line_to_string(Stream, Read),
    (   Read == end_of_file
    ->  Line = ""
    ;   string_concat(Read, "\n", Line)
    ).

Comments welcome (notably on my Python )

[jan]

@jan

    def readline(self, size=-1):
        once("janus:py_readline(Stream, Size, Line)", {'Stream':self.prolog_stream, 'Size':size})['Line']

Alright, better add return before that Now have to deal with the prompt.

[jan]

@jan

Now have to deal with the prompt.

So, it seems the prompt is issued by input() That raises an “audit event”, which I can indeed register and handle. That tells me what the
prompt is, but doesn’t seem to allow me to avoid it getting printed. Did I miss something here.

To some extend I have something working by backspacing over the written prompt and then give it again to the Prolog prompt. Not all I/O environments will like this though. Any clue about something better?

[JamesH]

I don’t fully understand the problem, but it appears that you can change the various prompts used by the REPL through the values sys.ps1, sys.ps2, and sys.ps3.

For example if I change ps1 and ps2 I can get the following kind of output:

>>> for i in range(3):

... print(i)

...

0

1

2

>>> import sys

>>> sys.ps1 = "### "; sys.ps2 = "--- "

### for i in range(3):

--- print(i)

[JamesH]

My message got cut off for some reason. I was saying maybe setting the prompts (ps1, ps2) to empty strings would give the desired behavior.

sys.ps1 and sys.ps2 are documented at
https://docs.python.org/3.10/library/sys.html
James

[jan]

@anon95304481

Yes python has some line reader and even a history buffer.

Thanks. I want the thing the other way around though. If I have Python embedded into Prolog and Prolog’s I/O redefined to use some window rather than file handles 0,1,2, I want Python’s I/O to use Prolog’s. Well, that now works (be it a bit ugly to my taste) And yes, Windows doesn’t get Unicode copy/paste > 0xffff to work very well. Works fine in Linux and MacOS …

@JamesH

My message got cut off for some reason. I was saying maybe setting the prompts (ps1, ps2) to empty strings would give the desired behavior.

That would improve things a bit, but then we have no prompt left I now implemented this:

• Use the audit event to see what the prompt is. Set a flag that we will be prompted.
• Now, if the write handler sees this flag and the string written is the prompt, it simply ignores it
• Pass the prompt to the readline() handler.

That works flawlessly. The only small issue is that the Prolog command line history now captures both the Prolog and the Python commands if you use py_shell/0. Well, that also has it charm. This write hack smells a little fragile, but it surely is a step ahead. Tested in the Qt based swipl-win console. We automatically install this hack if Prolog I/O is not connected to the file handles 0,1,2. If Prolog uses 0,1,2, we do nothing.