# Janus: calling python exec fails **URL:** \ **Category:** General\ **Created:** [November 7, 2023, 5:08am UTC](https://swi-prolog.discourse.group/t/janus-calling-python-exec-fails/6953 "2023-11-07T05:08:14Z")\ **Posts on this page:** 13\ **Page:** 1
**Author:** ![swi](https://avatars.discourse-cdn.com/v4/letter/s/51bf81/32.png) [@swi](https://swi-prolog.discourse.group/u/swi)\ **Post date:** [November 7, 2023, 5:08am UTC](https://swi-prolog.discourse.group/t/janus-calling-python-exec-fails/6953/1 "2023-11-07T05:08:14Z")
I am wondering why the following code fails, is there anything I am doing wrong? ```prolog :- use_module(library(strings)). main(R) :- S={|string|| import nltk from nltk.tokenize import sent_tokenize, word_tokenize nltk.download('punkt') # Download the punkt tokenizer models if you haven't already. def truecase_text(text): # Tokenize the text into sentences and then words. sentences = sent_tokenize(text) truecased_sentences = [] for sentence in sentences: words = word_tokenize(sentence) truecased_words = [] for word in words: # Use NLTK's built-in `title()` function to truecase each word. truecased_word = word.title() truecased_words.append(truecased_word) # Reconstruct the truecased sentence from the truecased words. truecased_sentence = ' '.join(truecased_words) truecased_sentences.append(truecased_sentence) # Reconstruct the truecased text from the truecased sentences. truecased_text = ' '.join(truecased_sentences) return truecased_text # Example usage: text = "this is an example. it demonstrates truecasing." truecased_text = truecase_text(text) print(truecased_text) |}, py_call(exec(S),R). ``` when I run it I get a `frame does not exist` error: ```prolog 2 ?- main(R). ERROR: Python 'SystemError': ERROR: frame does not exist ERROR: In: ERROR: [11] janus:py_call(exec("import nltk\nfrom nltk.tokenize import sent_tokenize, word_tokenize\n\nnltk.download('punkt') # Download the punkt tokenizer models if you haven't already.\n\ndef truecase_text(text):\n # Tokenize the text into sentences and then words.\n sentences = sent_tokenize(text)\n truecased_sentences = []\n\n for sentence in sentences:\n words = word_tokenize(sentence)\n truecased_words = []\n\n for word in words:\n # Use NLTK's built-in `title()` function to truecase each word.\n truecased_word = word.title()\n truecased_words.append(truecased_word)\n\n # Reconstruct the truecased sentence from the truecased words.\n truecased_sentence = ' '.join(truecased_words)\n truecased_sentences.append(truecased_sentence)\n\n # Reconstruct the truecased text from the truecased sentences.\n truecased_text = ' '.join(truecased_sentences)\n\n return truecased_text\n\n# Example usage:\ntext = \"this is an example. it demonstrates truecasing.\"\ntruecased_text = truecase_text(text)\nprint(truecased_text)\n"),_412) ERROR: [9] toplevel_call(user:user: ...) at /swi/home/boot/toplevel.pl:1235 ERROR: ERROR: Note: some frames are missing due to last-call optimization. ERROR: Re-run your program in debug mode (:- debug.) to get more detail. 3 ?- ``` ---
**Author:** ![jan](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/jan/32/4_2.png) [@jan](https://swi-prolog.discourse.group/u/jan)\ **Post date:** [November 7, 2023, 8:23am UTC](https://swi-prolog.discourse.group/t/janus-calling-python-exec-fails/6953/2 "2023-11-07T08:23:15Z")
> [@swi](#): > > ```prolog > ERROR: Python 'SystemError': > ERROR: frame does not exist > > ``` Good question. Probably there are more informed answers in a Python forum šŸ™‚ This applies for anything you want to exec(): ```prolog 101 ?- py_call(exec("import sys\n"), V). ERROR: Python 'SystemError': ERROR: frame does not exist ``` That could be the reason why e.g. `Py_Run()` and similar interfaces exist? This does work: ```prolog 104 ?- py_run("import sys\n", py{}, py{}, R, []). Correct to: "janus:py_run(\"import sys\\n\",py{},py{},R,[])"? yes R = @(none). ``` Possibly we should make py\_run/5 public? Iā€™m afraid I donā€™t really understand this interface though. Alternatively we could decide that this stuff **must** be in a Python module. That was surely the original intend of the Janus developers. Does that make sense? ---
**Author:** ![swi](https://avatars.discourse-cdn.com/v4/letter/s/51bf81/32.png) [@swi](https://swi-prolog.discourse.group/u/swi)\ **Post date:** [November 7, 2023, 10:28am UTC](https://swi-prolog.discourse.group/t/janus-calling-python-exec-fails/6953/3 "2023-11-07T10:28:09Z")
> [@jan](#): > > Good question. Probably there are more informed answers in a Python forum šŸ™‚ This applies for anything you want to exec(): Seems to be a prolog problem, not a python one, it works fine in the py\_shell: ```prolog 25 ?- py_shell. Python 3.11.5 (main, Sep 2 2023, 14:16:33) [GCC 13.2.1 20230801] on linux Type "help", "copyright", "credits" or "license" for more information. (InteractiveConsole) >>> exec("print('hello')") hello >>> ``` > [@jan](#): > > Possibly we should make py\_run/5 public? yes, I think this is very useful, to define python function within prolog by passing a string. EDIT: hmm, how do you call a function defined with py\_run/5? ```prolog 37 ?- py_run("def myfun(): print('myfun')",py{},py{},R,[]). Correct to: "janus:py_run(\"def myfun(): print('myfun')\",py{},py{},R,[])"? yes R = @(none). 38 ?- py_call(myfun(),R). ERROR: python_builtin `myfun' does not exist ERROR: In: ERROR: [10] janus:py_call(myfun(),_13326) ERROR: [9] toplevel_call(user:user: ...) at /swi/home/boot/toplevel.pl:1235 39 ?- ``` ---
**Author:** ![jan](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/jan/32/4_2.png) [@jan](https://swi-prolog.discourse.group/u/jan)\ **Post date:** [November 7, 2023, 10:57am UTC](https://swi-prolog.discourse.group/t/janus-calling-python-exec-fails/6953/4 "2023-11-07T10:57:32Z")
> [@swi](#): > > Seems to be a prolog problem, not a python one, it works fine in the py\_shell: There seems to be something special when making calls to Python directly from C. It probably has to do with ā€œPython framesā€, but do not ask ā€¦ Iā€™m a Prolog expert slowly understanding the Python environment. > [@swi](#): > > ```prolog > py_run("def myfun(): print('myfun')",py{},py{},R,[]). > > ``` The question is how you get this defined in some module and if you canā€™t/donā€™t specify the module, where it ends up. That too is more a Python question. An unqualified function call by py\_call/2 resolves the function as a built-in. ---
**Author:** ![swi](https://avatars.discourse-cdn.com/v4/letter/s/51bf81/32.png) [@swi](https://swi-prolog.discourse.group/u/swi)\ **Post date:** [November 7, 2023, 11:40am UTC](https://swi-prolog.discourse.group/t/janus-calling-python-exec-fails/6953/5 "2023-11-07T11:40:32Z")
> [@jan](#): > > The question is how you get this defined in some module and if you canā€™t/donā€™t specify the module, where it ends up. That too is more a Python question. It seems to be stored in module ā€˜Noneā€™, so I tried this: ```prolog 42 ?- py_run("def myfun(): \n print('myfun')\nprint('module: ',myfun. __module__ )",py{},py{},R,[]). Correct to: "janus:py_run(\"def myfun(): \\n print('myfun')\\nprint('module: ',myfun. __module__ )\",py{},py{},R,[])"? yes module: None R = @(none). 43 ?- py_call('None':myfun(),R). ERROR: Python 'TypeError': ERROR: 'NoneType' object is not callable ERROR: In: ERROR: [10] janus:py_call('None':myfun(),_12586) ERROR: [9] toplevel_call(user:user: ...) at /swi/home/boot/toplevel.pl:1235 ``` You canā€™t add a function to a module (python) without creating a file. ---
**Author:** ![jan](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/jan/32/4_2.png) [@jan](https://swi-prolog.discourse.group/u/jan)\ **Post date:** [November 7, 2023, 12:38pm UTC](https://swi-prolog.discourse.group/t/janus-calling-python-exec-fails/6953/6 "2023-11-07T12:38:04Z")
> [@swi](#): > > ```prolog > module: None > > ``` This is from Prolog not ā€˜Noneā€™, but the Python object _None_ represented as `@none` in Prolog (at least, that is what I suspect). I think it trying to tell us that `myfun()` has no module. Next question is you get access to a function object that is not part of a module? Normally, functions are simply attributes of their module. ---
**Author:** ![swi](https://avatars.discourse-cdn.com/v4/letter/s/51bf81/32.png) [@swi](https://swi-prolog.discourse.group/u/swi)\ **Post date:** [November 7, 2023, 5:37pm UTC](https://swi-prolog.discourse.group/t/janus-calling-python-exec-fails/6953/7 "2023-11-07T17:37:12Z")
> [@jan](#): > > Next question is you get access to a function object that is not part of a module? Normally, functions are simply attributes of their module. I can call it with py\_run/5, but inside the same string: ```prolog 29 ?- py_run("def myfun(): \n print('hello I am myfun')\nprint('module: ',myfun. __module__ )\nmyfun()",py{},py{},R,[]). Correct to: "janus:py_run(\"def myfun(): \\n print('hello I am myfun')\\nprint('module: ',myfun. __module__ )\\nmyfun()\",py{},py{},R,[])"? yes module: None hello I am myfun R = @(none). ``` Notice the python code inside py\_run: ```python def myfun(): print('hello I am myfun') print('module: ',myfun. __module__ ) myfun() ``` The call to `myfun()` inside the py\_run string works fine, because it prints: ```prolog module: None hello I am myfun R = @(none). ``` however, it does not work with py\_call/1: ```prolog 30 ?- py_call(@(none):myfun()). ERROR: python_builtin `@' does not exist ERROR: In: ERROR: [10] janus:py_call(@(none):myfun()) ERROR: [9] toplevel_call(user:user: ...) at /swi/home/boot/toplevel.pl:1235 ``` EDIT: the use case is one prolog source file (script, for example) that includes python source code in a string. The simple solution is to write the string to a file and import that, but this does not work in a read-only filesystem. ---
**Author:** ![jan](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/jan/32/4_2.png) [@jan](https://swi-prolog.discourse.group/u/jan)\ **Post date:** [November 7, 2023, 6:20pm UTC](https://swi-prolog.discourse.group/t/janus-calling-python-exec-fails/6953/8 "2023-11-07T18:20:07Z")
> [@swi](#): > > `30 ?- py_call(@(none):myfun()).` That will never work as, even if @none maps to the Python _None_, this has no attributes. I think you should read this as ā€œmyfunc has no moduleā€. > [@swi](#): > > EDIT: the use case is one prolog source file (script, for example) that includes python source code in a string. The simple solution is to write the string to a file and import that, but this does not work in a read-only filesystem. I understand. You have to figure out how to create a Python module from a string. If you know how to do that it might already be possible using the current interface. If not, we can think about making it possible. ---
**Author:** ![swi](https://avatars.discourse-cdn.com/v4/letter/s/51bf81/32.png) [@swi](https://swi-prolog.discourse.group/u/swi)\ **Post date:** [November 7, 2023, 8:40pm UTC](https://swi-prolog.discourse.group/t/janus-calling-python-exec-fails/6953/9 "2023-11-07T20:40:11Z")
> [@jan](#): > > You have to figure out how to create a Python module from a string. Figured out how to do it: ```prolog :- use_module(library(strings)). pyfunc :- S={|string|| def pyhello(num): print("hello ",num+1) print("module",pyhello. __module__ ) |}, py_call(globals(),Globals,[py_object(true)]), py_call(exec(S,Globals),R). ``` ```prolog 1 ?- pyfunc. module __main__ true. 2 ?- py_call(' __main__':pyhello(5)). hello 6 true. ``` What was missing was the `globals()` object for the exec call, which is why it complained about missing a frame and also the reason why it worked in `py_shell` (because the globals was already there). See [python - How to load a module from code in a string? - Stack Overflow](https://stackoverflow.com/questions/5362771/how-to-load-a-module-from-code-in-a-string) to make a module with a different name. EDIT: the documentation for py\_call/3 specifies: > Call Python and return the result of the called function. Call has the shapeā€˜[Target][:Action]\*`, where Target is either a Python module name or a Python object reference. Each Action is either an atom to get the denoted attribute from current Target or it is a compound term where the first argument is the function or method name and the arguments provide the parameters to the Python function. could you give an example on how to pass several `Actions` to `py_call`? ---
**Author:** ![jan](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/jan/32/4_2.png) [@jan](https://swi-prolog.discourse.group/u/jan)\ **Post date:** [November 8, 2023, 10:21am UTC](https://swi-prolog.discourse.group/t/janus-calling-python-exec-fails/6953/10 "2023-11-08T10:21:35Z")
> [@swi](#): > > ```prolog > py_call(globals(),Globals,[py_object(true)]), > py_call(exec(S,Globals),R). > > ``` Note that you can do ``` py_call(exec(S, eval(globals()), R). ``` Iā€™m still unsure whether we want the eval() there. > [@swi](#): > > could you give an example on how to pass several `Actions` to `py_call`? This implements _chaining_. There is an example in the library to add a dir to `sys:path` as ``` py_call(sys:path:append(Dir)). ``` Typically, you want as few as possible intermediate explicit Python objects references in Prolog. To create such as reference we must create an atom. This will be garbage collected later, which implies work and later reclaim of the Python object than needed. It is also relatively complex as the atom garbage collector cannot grab the Python GIL needed to decrement the reference count (would deadlock too easily). So, the Python objects referenced is added to a queue and this queue is emptied whenever Prolog gets the GIL for some other reason. That makes chaining and eval() attractive: more compact, faster and fewer objects known to Prolog. But, even the explicit route is pretty fast and there is typically no reason to not let code clarity prevail. Thanks for the link. That allows us to define `py_load(Module, Source)`. Would that be good? Better name? py\_module? py\_create\_module? Longer names are not very attractive as it pushes the Python code further to the right with normal layout. ---
**Author:** ![jan](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/jan/32/4_2.png) [@jan](https://swi-prolog.discourse.group/u/jan)\ **Post date:** [November 8, 2023, 1:07pm UTC](https://swi-prolog.discourse.group/t/janus-calling-python-exec-fails/6953/11 "2023-11-08T13:07:41Z")
> [@jan](#): > > `py_load(Module, Source)`. Added to the current git. Note that this is just a POC. Suggestions for a better name and better handling when reloading the Prolog file are welcome. ---
**Author:** ![swi](https://avatars.discourse-cdn.com/v4/letter/s/51bf81/32.png) [@swi](https://swi-prolog.discourse.group/u/swi)\ **Post date:** [November 8, 2023, 6:00pm UTC](https://swi-prolog.discourse.group/t/janus-calling-python-exec-fails/6953/12 "2023-11-08T18:00:49Z")
> [@jan](#): > > `py_load(Module, Source)`. This is great! Most especially replacing the module if the string hash changes. This is very useful for reloading. I think a better name is `py_module(...)`, since the primary effect is to create/modify a module. ---
**Author:** ![jan](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/jan/32/4_2.png) [@jan](https://swi-prolog.discourse.group/u/jan)\ **Post date:** [November 8, 2023, 6:26pm UTC](https://swi-prolog.discourse.group/t/janus-calling-python-exec-fails/6953/13 "2023-11-08T18:26:19Z")
> [@swi](#): > > I think a better name is `py_module(...)` Agree. Pushed a change.