# Swiplserver: problem with create\_dict on python 3.10 **URL:** \ **Category:** Help!\ **Tags:** bug\ **Created:** [May 25, 2022, 5:45am UTC](https://swi-prolog.discourse.group/t/swiplserver-problem-with-create-dict-on-python-3-10/5427 "2022-05-25T05:45:25Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![Losbarthos](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/losbarthos/32/4037_2.png) [@Losbarthos](https://swi-prolog.discourse.group/u/Losbarthos)\ **Post date:** [May 25, 2022, 5:45am UTC](https://swi-prolog.discourse.group/t/swiplserver-problem-with-create-dict-on-python-3-10/5427/1 "2022-05-25T05:45:25Z")
I am using: - Ubuntu 22.04 LTS - python `3.10` - the last version of `swiplserver==1.0.2` (pip install swiplserver) - SWI-Prolog (threaded, 64 bits, version 8.4.2) My problem now is, it is not possible anymore to create [dictionaries](https://www.swi-prolog.org/pldoc/man?section=bidicts). An example code so far: ```prolog from swiplserver import PrologMQI, PrologThread with PrologMQI() as mqi: with mqi.create_thread() as prolog_thread: result = prolog_thread.query("dict_create(Dict, Tag, [1-""a"", 2-""b""]).") print(result) ``` the expected result from swipl: ```prolog ?- dict_create(D,proof, [1-"a",2-"b"]). D = proof{1:"a", 2:"b"}. ``` If I debug the code in python, the problem is, the variable `data` in the function `_receive` of file `prologmqi.py`. `data` keeps the value null and so it leads into some endless loop. ---
**Author:** ![ericzinda](https://avatars.discourse-cdn.com/v4/letter/e/b3f665/32.png) [@ericzinda](https://swi-prolog.discourse.group/u/ericzinda)\ **Post date:** [May 27, 2022, 2:35am UTC](https://swi-prolog.discourse.group/t/swiplserver-problem-with-create-dict-on-python-3-10/5427/2 "2022-05-27T02:35:29Z")
I verified that this doesn’t work on the latest build, but the latest build has had some fixes so that it doesn’t hang anymore, but it does return an exception (which is what a recent fix fixed). I’m curious though, did this exact code used to work using swiplserver? Edit: The problem boils down to the fact that this code throws (everything after `dict_create` is the code that MQI runs to serialize the result: ```prolog ?- dict_create(D,proof, [1-a,2-b]), term_to_json:term_to_json(D, Json), with_output_to(string(Json_String), (current_output(Stream), json:json_write(Stream, Json))). ERROR: Type error: `text' expected, found `1' (an integer) ERROR: In: ERROR: [11] with_output_to(string(_176784),(current_output(_176794),json: ...)) ERROR: [10] ''(user:user: ...) ERROR: [9] toplevel_call(user:user: ...) at /Applications/SWI-Prolog.app/Contents/swipl/boot/toplevel.pl:1162 ``` I’ll dig in tomorrow and see if I can figure out the fix. Edit: SWI Prolog dicts support keys that are integers, JSON does not. json\_write/2 (and json\_write\_dict/2) throw if you try to serialize a dict with an integer key to JSON and that’s what’s going on here. Edit: I’m trying to see how pengines solves this issue (so I can copy that approach) or if it exists there too, but I’m not sure how to get a pengine set up to be serializing raw JSON as a response. I did figure out that using the pengines reply\_json/1 predicate hits the same issue, so I suspect the same issue exists there: ```prolog http_json:reply_json(proof{1:a,2:b}). Content-type: application/json; charset=UTF-8 { ERROR: Type error: `text' expected, found `1' (an integer) ERROR: In: ... ``` ---
**Author:** ![ericzinda](https://avatars.discourse-cdn.com/v4/letter/e/b3f665/32.png) [@ericzinda](https://swi-prolog.discourse.group/u/ericzinda)\ **Post date:** [May 27, 2022, 11:33pm UTC](https://swi-prolog.discourse.group/t/swiplserver-problem-with-create-dict-on-python-3-10/5427/3 "2022-05-27T23:33:42Z")
OK, the problem is that the format used “on the wire” by MQI is JSON since that is the most interoperable, supported, etc. I.e. it is friendly to mostly languages. The dict you are creating is a valid dict in SWI Prolog (and in Python for that matter) but it is not a valid JSON key in either language (or in the JSON specification). If you convert a dict with integer keys to JSON, Python converts the keys to strings: ```prolog >>> json.dumps({1:"a", 2:"b"}) '{"1": "a", "2": "b"}' ``` But SWI Prolog throws an exception, as you’ve seen. If you change the keys to be an atom, it works fine (although it does strip the tag, which is the behavior of term\_to\_json/2): ```prolog ?- dict_create(D,proof, [key1-a,key2-b]), term_to_json:term_to_json(D, Json), with_output_to(string(Json_String), (current_output(Stream), json:json_write(Stream, Json))). D = Json, Json = proof{key1:a, key2:b}, Json_String = "{\"key1\":\"a\", \"key2\":\"b\"}", Stream = (0x6000029ebf00). ``` Options for MQI are: 1. Leave it as is: Workarounds are many: use an atom key, wrap what you are doing in a predicate that massages the data into another form, etc. There are other things that can’t be serialized to JSON, like the query `open_null_stream(X)` that have to be wrapped as well so it isn’t unprecedented. 2. Update json\_write and json\_write\_dicts to do what Python does. Seems like this could affect a lot of code… 3. Create a local version of json\_write that does what Python does but only applies to MQI. I don’t like forking this kind of low level routine for lots of reasons… 4. Provide a predicate that does that conversion that people can use. This predicate could also be used to convert any tags so that they get returned and to do other conversions over time if there are other JSON serialization cases like this that appear. Something like `json_formatter(JSON_in, JSON-out, Options)`. 5. Use the predicate from #4 by default to always (or optionally) do the conversion. This is the approach taken for attributes in MQI currently. @jan any thoughts here? My inclination is to go with 4 or 5. #4 has the bonus of not being in the code path unless the user needs it, doesn’t add a bunch of options to MQI, etc. But I am curious if pengines has a different approach to this issue when sending back JSON (or if the predicates built in #4 could be useful for that scenario as well). ---
**Author:** ![jan](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/jan/32/4_2.png) [@jan](https://swi-prolog.discourse.group/u/jan)\ **Post date:** [May 28, 2022, 1:12pm UTC](https://swi-prolog.discourse.group/t/swiplserver-problem-with-create-dict-on-python-3-10/5427/4 "2022-05-28T13:12:40Z")
Hmmm. JSON cannot fully represent a Prolog dict in a natural way. We could of course invent some (verbose) JSON representation of any dict. I doubt this makes much sense for MQI. After all, MQI is there to allow some other language do delegate part of a computation to Prolog. IMO the application should design a JSON data scheme that interfaces data relevant to the application. As any JSON object can be represented naturally in Prolog, there is no problem. And yes, besides that Prolog can represent data in ways that poorly map to JSON. That should barely be an issue for an application. In other words, I think a user of MQI (or any other binding to Prolog or even between two arbitrary languages) should design an interface that consists of a data schema and functions/methods/predicates/… that can be called and implement that. This opposed to making arbitrary calls on Prolog and feed the results thereof back to some other call on Prolog. In part this cannot work differently. For example, you cannot open a stream in Prolog through MQI and use that stream. Prolog streams are represented by _blobs_ that can be serialized (written), but deliberately cannot be _read_. This allows Prolog to perform garbage collection on such objects. As a result, if you need to use some functionality through MQI that involves a Prolog stream you must define a more high level predicate in Prolog that performs all actions that create, use and close the stream and call that through MQI. So, I would not change anything … P.s. As for Pengines, the same applies. SWISH uses the `json-html` format for replies which hold the overall structure of the reply in JSON, but represents Prolog terms as HTML. ---
**Author:** ![EricGT](https://avatars.discourse-cdn.com/v4/letter/e/f1d935/32.png) [@EricGT](https://swi-prolog.discourse.group/u/EricGT)\ **Post date:** [May 28, 2022, 1:50pm UTC](https://swi-prolog.discourse.group/t/swiplserver-problem-with-create-dict-on-python-3-10/5427/5 "2022-05-28T13:50:52Z")
> [@jan](#): > > JSON data scheme That is what I was thinking. There is already a specification for JSON schema, see: [JSON Schema](https://json-schema.org/) The down side of a JSON schema is that 1. The noted schema is not official AFAIK. 2. Many people don’t use a schema when given the chance. 3. AFAIK there is no SWI-Prolog JSON schema validator. They do exist for other programming languages ([ref](https://json-schema.org/implementations.html#validators)) 4. Each message type would need a new JSON schema. > [@jan](#): > > So, I would not change anything … 👍 * * * I do like MQI I just don’t like changes to SWI-Prolog that are specialized. * * * Related topic: [json\_dict/2 - Helpful for learning how to use JSON with SWI-Prolog dict](https://swi-prolog.discourse.group/t/json-dict-2-helpful-for-learning-how-to-use-json-with-swi-prolog-dict/4450) Expand the notes section for some really useful information. ---
**Author:** ![ericzinda](https://avatars.discourse-cdn.com/v4/letter/e/b3f665/32.png) [@ericzinda](https://swi-prolog.discourse.group/u/ericzinda)\ **Post date:** [May 31, 2022, 6:15pm UTC](https://swi-prolog.discourse.group/t/swiplserver-problem-with-create-dict-on-python-3-10/5427/7 "2022-05-31T18:15:38Z")
> [@jan](#): > > IMO the application should design a JSON data scheme that interfaces data relevant to the application. As any JSON object can be represented naturally in Prolog, there is no problem. And yes, besides that Prolog can represent data in ways that poorly map to JSON. That should barely be an issue for an application. Yes this makes sense. I do think I need to clarify that the JSON Serialization format will not serialize this particular case to text since it is valid JSON in Prolog and could be confusing. I’ll create a pull request that adds a couple of items to the [“Mapping Prolog Terms into JSON”](https://www.swi-prolog.org/pldoc/man?section=prolog-canonical-json): - Pointing out that dicts with keys that are integers can be converted to the Prolog JSON format, but won’t serialize to txt - Pointing out the (maybe obvious) point that there are objects like Streams that won’t convert at all And then update the [message format](https://www.swi-prolog.org/pldoc/man?section=mqi-message-format) and [toplevel differences](https://www.swi-prolog.org/pldoc/man?section=mqi-toplevel-differences) section of MQI to point there.