Janus and Python venv

Most people in the python world use environments for their apps, something like

$ python -m venv venv
$ . venv/bin/activate  # notice the dot which sources the script
(venv)
$ pip install mido
[...]

I did a quick try and environments don’t seem to work in Janus, is there a way to tell it to use the current active environment?

Thanks. My quick try just works fine. What is goind wrong?

$ python -c "import sys; print('in venv') if sys.prefix != sys.base_prefix  else print('not in venv')"
not in venv

$ . venv/bin/activate      
(venv)
$ python -c "import sys; print('in venv') if sys.prefix != sys.base_prefix  else print('not in venv')"    
in venv

(venv)
$ swipl
2 ?- py_shell .
Python 3.11.5 (main, Sep  2 2023, 14:16:33) [GCC 13.2.1 20230801] on linux
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> import sys; print('in venv') if sys.prefix != sys.base_prefix  else print('not in venv')
not in venv
>>> 

[Moved topic]

I see. I was building the Janus interface as Python package in the venv. That works fine. Here the problem is that the embedded Python does not listen to to the virtualenv. Does anyone know how to handle this? The virtualenv seems to make two changes to the environment:

  • set VIRTUAL_ENV
  • modify PATH to add $VIRTUAL_ENV/bin

I found quite a couple of suggestions on how to handle this, but mostly contradicting, not working, etc. :frowning:

Some links and remarks

Seems we must

  • set sys.prefix to $VIRTUAL_ENV
  • Avoid preset site-packages. Docs says this is done using -I, but this only part of the story: using Python inside a venv also removes the system wide local packages and -I does not; it only removes the user one. It seems venv removes all directories with site-packages and dist-packages
  • Add the site-packages from $VIRTUAL_ENV. Only, this is (for me) in $VIRTUAL_ENV/lib/python3.10/site-packages. Form where do I get the 3.10? Is this always the major.minor version?

Pushed d08f3497bf3e1bff90d077fd5c89c0e06e8f4cd7

. venv/bin/activate
swipl
?- py_version.
% Janus embeds Python 3.10.12 (main, Jun 11 2023, 05:26:28) [GCC 11.4.0]
% Janus: using venv from '/home/janw/venv'
true.

Updates sys.prefix and sys.path as described above. Works using Python 3.10 on Linux. No clue whether there is something OS/Python version specific. Not unlikely, so please report problems.

I installed a python package in the venv, and I was able to import the package from janus py_shell, so it looks like it is working well. Thanks!

Hi,

How to make janus work with conda virtual env? I set export VIRTUAL_ENV=~/miniconda3 and janus still doesn’t recognize it. I also have the conda env activated, so when I run which python3 outside of swipl shell I get ~/miniconda3/bin/python3.

A bit more information is required to say anything sensible. I think you are talking about Janus as SWI-Prolog library (as opposed to Janus as Python package for embedding Prolog into Python), right?

It seems we are also talking about some non-Windows OS. Which?

The SWI-Prolog build creates janus.so/?? as a SWI-Prolog plugin that is loaded by janus.pl That shared object binds to libpython. This should be the libpython that belongs to the Python you want to run. Now, it all depends on the OS, how Prolog is build/installed and the Python versions.

Sorry for not providing details.

Yeah, I was referring to the SWI-Prolog library and my setup is calling python code from prolog. I’m on Ubuntu 22.04.

The issue was janus library didn’t use the conda environment that was active and was using the python binary under /usr/bin. Anyways, I was able to resolve the issue using the suggestions in this thread. Essentially, building SWI-Prolog from the source and setting -DPython_ROOT_DIR when running cmake.

As a side-effect, building SWI-Prolog from the source resulted in losing REPL functionalities such as going through previously run history via Up/Down keys and tab completion.

You probably missed some dependencies. See Prerequisites for Debian based systems (Ubuntu, Mint, ...) for the complete list. In this case you probably missed libedit-dev, but you may be missing more functionality. Install all of them, then in the build dir run

cmake .
ninja
ninja install

The cmake . will also tell you about missing dependencies. Building SWI-Prolog with missing dependencies just drops some functionality. Only libz is a hard requirement.

I ran to the same error on a new Ubuntu system and came back to this thread. I see the solution that worked for me earlier required rebuilding SWI-Prolog with -DPython_ROOT_DIR. I wonder if there is a solution that doesn’t require building SWI-Prolog from the source.

In addition, I noticed the following when trying to make Janus work for a virtual env without rebuilding.

The python version in my virtual env is 3.10.20 and the global python version is 3.12.3. I’m using SWI-Prolog version 10.1.5. If there is a mismatch between the Janus embedded python version (here the global python 3.12.3) and the virtual env version, Janus won’t recognize packages installed in the virtual env even if the environment is activated prior to running swipl

source venv/bin/activate
swipl
?- py_version.
% Interactive session; added `.` to Python `sys.path`
Warning: Janus: venv dirrectory '/home/xabush/code/.venv' does not contain "/home/xabush/code/.venv/lib/python3.12/site-packages"
% Janus 1.5.2 embeds Python 3.12.3 (main, Mar 23 2026, 19:04:32) [GCC 13.3.0]
% Janus: using venv from '/home/xabush/code/.venv'
true.
?- py_module_exists(pysam).
false.

However, if I set the virtual env python version to match that of Janus, it works and loads the modules installed in the virtual env.

source venv/bin/activate
swipl
?- py_version.
% Interactive session; added `.` to Python `sys.path`
% Janus 1.5.2 embeds Python 3.12.3 (main, Mar 23 2026, 19:04:32) [GCC 13.3.0]
% Janus: using venv from '/home/xabush/code/.venv'
true.

?- py_module_exists(pysam).
true.

It looks like Janus expects the virtual environment python version to match the one it was compiled with. I’m wondering if there is a way of overriding this functionality as one might use a different python version for a virtual env and still want to interface with that python version.

Difficult. If you are working in a venv and you build SWI-Prolog into this venv, all should work fine. venv also works fine of the Python version matches the (global) Python against which Janus was compiled.

Otherwise, SWI-Prolog is not aware of venv. In theory, we could do the following

  • If we create a venv, we generate a new Prolog janus module in that VM. That still requires the SWI-Prolog source around as the janus package is embedded in the full source build infrastructure. Alternatively we need to define an alternative CMake configuration to merely build the C part of Janus outside the Prolog tree,
  • If we git janus.so inside the venv, janus.pl could detect the venv and load janus.so from the venv rather than the originally built one.

Than we may still be confronted with different Janus versions in different Prolog versions you may have floating around :frowning:

For short, hard :frowning: In part because the Python C API is bound to a specific minor version of Python. If that was more general stable (the Windows version provides a python3.dll that binds to a range of Python 3.x versions) it would have been possible to play some tricks with the dynamic loader to combine the right parts.

If anyone is inspired or has an idea, please share and make a PR.

edit there is one alternative. Inside the venv, ensure the desired swipl is in PATH. Now build the Python side of Janus using pip. Then, in Python, load Janus and run janus.prolog(). That gives you a Prolog REPL loop.