A bundled Python interface

Request For Comments
67

For Windows 10 (Update)

Installed https://www.swi-prolog.org/download/daily/bin/swipl-w64-2023-08-07.exe which is the latest daily build for Windows. (One day newer than used in last post)

Welcome to SWI-Prolog (threaded, 64 bits, version 9.1.12-18-g7e6f7de02)
SWI-Prolog comes with ABSOLUTELY NO WARRANTY. This is free software.
Please run ?- license. for legal details.

For online help and background, visit https://www.swi-prolog.org
For built-in help, use ?- help(Topic). or ?- apropos(Word).

?- py_version.
% Janus embeds Python 3.11.4 (tags/v3.11.4:d2340ef, Jun  7 2023, 05:45:37) [MSC v.1934 64 bit (AMD64)]
true.

?- py_shell.
ERROR: Python error <class 'AttributeError'>:
ERROR:   'NoneType' object has no attribute 'write'
ERROR: In:
ERROR:   [11] janus:py_call(janus:interact(),_46632)
ERROR:    [9] toplevel_call(user:user:py_shell) at c:/program files/swipl/boot/toplevel.pl:1173
ERROR: 
ERROR: Note: some frames are missing due to last-call optimization.
ERROR: Re-run your program in debug mode (:- debug.) to get more detail.
?-

Yesterday received this error but knew there were a few more patches to the swipy code so wanted to try again with newer code. Still same problem.

Additional info

C:\Users\Groot>where python
C:\Users\Groot\AppData\Local\Programs\Python\Python311\python.exe
C:\Users\Groot\AppData\Local\Microsoft\WindowsApps\python.exe

C:\Users\Groot>where python3.dll
C:\Users\Groot\AppData\Local\Programs\Python\Python311\python3.dll

C:\Users\Groot>powershell
Windows PowerShell
Copyright (C) Microsoft Corporation. All rights reserved.

Try the new cross-platform PowerShell https://aka.ms/pscore6

PS C:\Users\Groot> get-command python

CommandType     Name                                               Version    Source
-----------     ----                                               -------    ------
Application     python.exe                                         3.11.41... C:\Users\Groot\AppData\Local\Programs\Python\Python311\python.exe

PS C:\Users\Groot> exit

C:\Users\Groot>python
Python 3.11.4 (tags/v3.11.4:d2340ef, Jun  7 2023, 05:45:37) [MSC v.1934 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>>
68

In the documentation here it notes

git clone https://github.com/SWI-Prolog/swipy.git

which gives an error for me when used

C:\Users\Groot\Documents\GitHub>git clone https://github.com/SWI-Prolog/swipy.git
Cloning into 'swipy'...
info: please complete authentication in your browser...
remote: Repository not found.
fatal: repository 'https://github.com/SWI-Prolog/swipy.git/' not found

However using the link as noted in the repository

image

it works.

C:\Users\Groot\Documents\GitHub>git clone https://github.com/SWI-Prolog/packages-swipy.git
Cloning into 'packages-swipy'...
remote: Enumerating objects: 427, done.
remote: Counting objects: 100% (427/427), done.
remote: Compressing objects: 100% (239/239), done.
Receiving objects:  82% (351/427)used 330 (delta 177), pack-reused 0
Receiving objects: 100% (427/427), 110.28 KiB | 3.94 MiB/s, done.
Resolving deltas: 100% (274/274), done.

Was that a typo? or a use of a earlier repository for the code? or something else?

69

I guess you tried this using swipl-win, e.g., the version that provides its own console window? That unfortunately does not work. It works fine for me using swipl.exe in a Windows console or PowerShell. The problem is that Python writes to the console, but under swipl-win there is no console.

I donā€™t know whether this can be fixed. Does anyone know whether (and how) we can rebind Pythonā€™s read/write to the user?

Note that as long as you make Python calls that do not read or write to the console, all should work fine.

70

Yes.

On Windows because I start SWI-Prolog so often, e.g. a new top level to test a predicate then close, I keep an icon for SWI-Prolog in the Windows taskbar

image

and that starts it with

ā€œC:\Program Files\swipl\bin\swipl-win.exeā€ --win_app

Thanks for the info.

Starting SWI-Prolog on Windows to use py_shell this way works as expected.

Using a Windows command prompt

Microsoft Windows [Version 10.0.19045.3208]
(c) Microsoft Corporation. All rights reserved.

C:\Users\Groot>swipl
Welcome to SWI-Prolog (threaded, 64 bits, version 9.1.12-18-g7e6f7de02)
SWI-Prolog comes with ABSOLUTELY NO WARRANTY. This is free software.
Please run ?- license. for legal details.

For online help and background, visit https://www.swi-prolog.org
For built-in help, use ?- help(Topic). or ?- apropos(Word).

1 ?- py_version.
% Janus embeds Python 3.11.4 (tags/v3.11.4:d2340ef, Jun  7 2023, 05:45:37) [MSC v.1934 64 bit (AMD64)]
true.

2 ?- py_shell.
Python 3.11.4 (tags/v3.11.4:d2340ef, Jun  7 2023, 05:45:37) [MSC v.1934 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> from janus import *
>>> once("writeln('Hello world!')")
Hello world!
{'status': True}

There are test cases for swipl but they are not included with an install. To install and run the test.

Start Windows command prompt

Microsoft Windows [Version 10.0.19045.3208]
(c) Microsoft Corporation. All rights reserved.

C:\Users\Groot>cd "C:\Users\Groot\Documents\GitHub"

C:\Users\Groot\Documents\GitHub>git clone https://github.com/SWI-Prolog/packages-swipy.git
Cloning into 'packages-swipy'...
remote: Enumerating objects: 446, done.
remote: Counting objects: 100% (446/446), done.
remote: Compressing objects: 100% (253/253), done.
remote: Total 446 (delta 287), reused 341 (delta 182), pack-reused 0Receiving objects:  80% (357/446)
Receiving objects: 100% (446/446), 113.78 KiB | 3.07 MiB/s, done.
Resolving deltas: 100% (287/287), done.

C:\Users\Groot\Documents\GitHub>swipl
Welcome to SWI-Prolog (threaded, 64 bits, version 9.1.12-18-g7e6f7de02)
SWI-Prolog comes with ABSOLUTELY NO WARRANTY. This is free software.
Please run ?- license. for legal details.

For online help and background, visit https://www.swi-prolog.org
For built-in help, use ?- help(Topic). or ?- apropos(Word).

1 ?- pwd.
% c:/users/groot/documents/github/
true.

2 ?- working_directory(_,'C:/Users/Groot/Documents/GitHub/packages-swipy').
true.

3 ?- pwd.
% c:/users/groot/documents/github/packages-swipy/
true.

4 ?- [test_janus].
true.

5 ?- test_janus.
[51/51] xsb_call:reverse ......................... passed (0.003 sec)
% All 51 tests passed in 0.458 seconds (0.359 cpu)
true.
71

Sorry. Must be (notethe packages-)

pip install git+https://github.com/SWI-Prolog/packages-swipy.git#egg=janus
72

Thanks.

No need to say sorry. I am more than happy to work through the bumps to see this work. As my git skills are not as great as yours was not sure if I was doing something wrong. Thanks for the update.

It also explains why @anon419083 was seeing the Git account authorization errors; I too was getting authentication request that I have never seen before using Git. Once the correct URL was used it worked as expected.

1 Like
73

How to deal with Janus on MacOS

It seems that dealing with the new Janus package is pretty much settled for Windows and Linux. On Windows we have the Python binary packages that provide the fairly version independent python3.dll, so we can search %PATH% for this and be happy. On Linux the packages take care of the dependencies and source installation is easy if you want something different.

On MacOS, I find on my machine Python frameworks in LibreOffice (3.8), Xcode (3.8 and 3.9), /Library (3.9), Homebrew (3.9 and 3.10) and Macports (3.10).

I guess installation in Homebrew or Macports should be fine: just add a dependency on Python3 and the package manager should do its job. But, what about the binary we distribute? We canā€™t really tie it to a particular Python version, but there is no equivalent of python3.dll. LibreOffice including its own version of Python should of course work, but this can hardly be a good solution :frowning: Suggestions?

74

From within Python it should be possible by changing the values of sys.stdin, sys.stdout and sys.stderr to some ā€œfile likeā€ objects.

Iā€™ve used this approach in some simple cases and it was pretty straightforward. No additional changes were needed.

It can be done by directly changing the values. There are also packages like contextlib.redirect_stdout.
James

75

That sounds promising. Do you have a pointer how this works? Maybe some example code? SWI-Prologā€™s I/O is quite flexible. A stream has a block of functions that is pretty much the same as Cā€™s FILE with some extensions. Should be possible :slight_smile:

76

You can run pip (or any Python program) under a particular version of Python, e.g. python3.12 pip install . or python3 pip install .. (Thereā€™s also pip3 install . ā€¦)
This is sometimes necessary on systems where python still points to python2.

(I tried installing swipy on my Debian (Chromebook) system which has a local version of python3.12, and it failed. I havenā€™t yet figured out what the errors mean but I suspect itā€™s somehow using the wrong pip.)

77

Just something like this:

import sys
sys.stdout = open(ā€œoutput.txtā€, ā€œwā€)
print(ā€œThis will be written to the fileā€)

I looked at the source code for contextlib and this is basically what it does, but using some abstract classes and setattr instead of ā€œ=ā€ to make it more general.
So if you can produce Python bindings for whatever streams are used for input/output to the Windows interface and then assign them to sys.stdout/sys.stdin it should work.

There are alot of articles discussing this approach, eg: https://www.devdungeon.com/content/using-stdin-stdout-and-stderr-python which includes an example redirecting it to a string buffer.

78

It is very exciting to see this happen! It truly provides a natural way of embedding Prolog into an existing Python codebase! :clap:t2:

I am using archlinux on my personal machine (yeah I knowā€¦). Using Python from Prolog works just fine. Installing Janus for Python was a bit roundabout.

I cannot use pip install, I get:

error: externally-managed-environment

Ɨ This environment is externally managed
ā•°ā”€> To install Python packages system-wide, try 'pacman -S
    python-xyz', where xyz is the package you are trying to
    install.
    
    If you wish to install a non-Arch-packaged Python package,
    create a virtual environment using 'python -m venv path/to/venv'.
    Then use path/to/venv/bin/python and path/to/venv/bin/pip.
    
    If you wish to install a non-Arch packaged Python application,
    it may be easiest to use 'pipx install xyz', which will manage 
    virtual environment for you. Make sure you have python-pipx
    installed via pacman.

note: If you believe this is a mistake, please contact your Python
installation or OS distribution provider. You can override this, at
the risk of breaking your Python installation or OS, by passing
--break-system-packages.
hint: See PEP 668 for the detailed specification.

(At this point I discovered that there exists another Python janus, which confused me for a second)

Using pipx (the second option from the error message above) doesnā€™t seem to work out of the box. Something happens and something is installed but the Python janus cannot find the SWI-Prolog binary.

Using a venv and pip works; however, for some reason venv installed an outdated pip in the virtual environment (??) and then pip complained and told me to update it:

Processing /home/boris/install/swipy
  Preparing metadata (setup.py) ... done
Installing collected packages: janus
  DEPRECATION: janus is being installed using the legacy 'setup.py
install' method, because it does not have a 'pyproject.toml' and the
'wheel' package is not installed. pip 23.1 will enforce this behaviour
change. A possible replacement is to enable the '--use-pep517' option.
Discussion can be found at https://github.com/pypa/pip/issues/8559
  Running setup.py install for janus ... done
Successfully installed janus-0.1.0

[notice] A new release of pip available: 22.3.1 -> 23.2.1
[notice] To update, run: python -m pip install --upgrade pip

The Janus installation seemed to work despite the warnings and notices but I was not willing to get to the bottom of it. Python versions have been a sad story for a decade now so I am not surprised, just annoyed :smiley:

If you upgrade the pip inside your virtual environment and then install Janus everything goes smoothly:

$ python -m venv ./swipyvenv
$ cd swipyvenv/
$ bin/pip --version
pip 22.3.1 from /home/boris/code/swipyvenv/lib/python3.11/site-packages/pip (python 3.11)
$ bin/python -m pip install --upgrade pip
Requirement already satisfied: pip in ./lib/python3.11/site-packages (22.3.1)
Collecting pip
  Using cached pip-23.2.1-py3-none-any.whl (2.1 MB)
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 22.3.1
    Uninstalling pip-22.3.1:
      Successfully uninstalled pip-22.3.1
Successfully installed pip-23.2.1
$ bin/pip --version
pip 23.2.1 from /home/boris/code/swipyvenv/lib/python3.11/site-packages/pip (python 3.11)
$ bin/pip install ~/install/swipy
Processing /home/boris/install/swipy
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: janus
  Building wheel for janus (pyproject.toml) ... done
  Created wheel for janus: filename=janus-0.1.0-cp311-cp311-linux_x86_64.whl 
  Stored in directory: /tmp/pip-ephem-wheel-cache-i1f_5jro/wheels/
Successfully built janus
Installing collected packages: janus
Successfully installed janus-0.1.0
$ bin/python
Python 3.11.3 (main, Jun  5 2023, 09:32:32) [GCC 13.1.1 20230429] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from janus import *
>>> once("writeln(hello)")
hello
{'status': True}
>>>

Or is there a good reason why an older version of pip was installed by default? This is a question for seasoned Python developers I guess.

EDIT: apparently this is all on purpose. Which pip version were you using in these examples @jan?

79

Good to know :frowning: Iā€™ll throw this into the discussions about Janus (which happen by mail and Zoom).

Is there some seasoned Python developer who can help here?

Iā€™m using stock Python and pip from Ubuntu 22.04; Python 3.10 and pip 22.0.2. Python 3.11 on MacOS and Windows. Dunno the pip version. Some complain there is a newer version. I ignore that :slight_smile:

The brief experience with Python tells me the version issues are much worse than with (SWI-)Prolog :slight_smile:

80

That looks easy. I guess the route is to subclass the Python stream class and call out to C to talk to SWI-Prologā€™s I/O?

Will look into that later. Unless, of course, there is a volunteer :slight_smile:

81

It shouldnā€™t be a real problem but at least archlinux has a python-janus and ubuntu and debian both have a python3-janus packages in their package databases :frowning:

82

That sounds like a good approach. The class to subclass is probably _io.TextIOWrapper.
I donā€™t have a Windows machine set up for this kind of thing unfortunately, otherwise Iā€™d try it.

83

Iā€™m playing with something simple. It works for output, but not yet for input. In Python, we do

import io
import sys

class PrologIO(io.TextIOWrapper):
    def __init__(self, plstream, *args, **kwargs):
        self.prolog_stream = plstream
        super().__init__:__call__(args, kwargs)
    def write(self, s):
        once("janus:py_write(Stream, String)", {'Stream':self.prolog_stream, 'String':s})
    def readline(self, size=-1):
        once("janus:py_readline(Stream, Size, Line)", {'Stream':self.prolog_stream, 'Size':size})['Line']

def bindIO():
    sys.stdin  = PrologIO("user_input",  io.BytesIO(), line_buffering=True)
    sys.stdout = PrologIO("user_output", io.BytesIO(), line_buffering=True)
    sys.stderr = PrologIO("user_error",  io.BytesIO(), write_through=True)

And in Prolog this. The ?>> prompt such that we see we are in charge for now :slight_smile: Python complains with object.readline() returned non-string though :frowning: I should also do something with the prompt. Seems Python already writes its prompt, which upsets SWI-Prologā€™s line editor.

py_write(Stream, String) :-
    format(Stream, '~s', [String]).

py_readline(Stream, _Size, Line) :-
    prompt1('?>> '),
    read_line_to_string(Stream, Read),
    (   Read == end_of_file
    ->  Line = ""
    ;   string_concat(Read, "\n", Line)
    ).

Comments welcome (notably on my Python :slight_smile: )

84

Alright, better add return before that :slight_smile: Now have to deal with the prompt.

1 Like
85

So, it seems the prompt is issued by input() That raises an ā€œaudit eventā€, which I can indeed register and handle. That tells me what the
prompt is, but doesnā€™t seem to allow me to avoid it getting printed. Did I miss something here.

To some extend I have something working by backspacing over the written prompt and then give it again to the Prolog prompt. Not all I/O environments will like this though. Any clue about something better?

87

I donā€™t fully understand the problem, but it appears that you can change the various prompts used by the REPL through the values sys.ps1, sys.ps2, and sys.ps3.

For example if I change ps1 and ps2 I can get the following kind of output:

>>> for i in range(3):

... print(i)

...

0

1

2

>>> import sys

>>> sys.ps1 = "### "; sys.ps2 = "--- "

### for i in range(3):

--- print(i)