Bug in character escape handling

Hi Jan,

Welcome to SWI-Prolog (threaded, 32 bits, version 9.1.6-27-gfcef8c3fe)

?- write_term('te\\st', [character_escapes(true), quoted(true)]).
'te\\\\st'
true.

?- write_term('te\\st', [character_escapes(false), quoted(true)]).
'te\\\\st' <-- wrong, should be 'te\st'
true.

Cheers!
Holger

Hi Holger,

This surely is a bug. The first should write 'te\\st', no? Is this some edit work? Note that I’ve added Discourse code block markers (```) around the code.

The second is surely wrong though and does reproduce. It relates to commit cbe691f8af61ccaf466147868a316b96c110fedb. That is because atom_to_term/2 calls PL_write_term(), the C api to the writer with quoted, but without char escapes. This seems fixed in the wrong place. In fact, all over the place PL_WRT_QUOTED is used, but without character escapes enabled, probably including a lot of external packages.

Another complication is that the character_escapes Prolog flag is module local, so we could handle mixed code with and without character escapes in the same program. write/1 and friends use the setting from the user module.

This makes the whole thing a bit of a mess :frowning: May I ask why you use character_escapes(false)?

Anyway, this needs some thought. I’m tempted to change PL_write_term() to accept, next to PL_WRT_CHARESCAPES a PL_WRT_NO_CHARESCAPES and, if none of these is present use the flag in the user module. Practically, that would imply that many conversions to quotes text start escaping as quoted write does. As is, only the quote and \ are escaped, which does allow to read the term back unambiguously.

Alternatively we could have three modes for PL_write_term(): unquoted, quoted with normal escaping and quoted with minimal escaping (only the quote and \\). Three modes would introduce fewer incompatibilities, but otherwise seems wrong. Note that this mostly affects the C interface, but for example also term_to_atom/2 which directly using the C interface for converting a term to a string (atom).

Thoughts are welcome. Surely first needs some consideration.

Hm, somehow Discourse messed up my message.
The problem occurred after we had ported an older application from (good old) 5.9.7 to a new SWI-Prolog version.
Rewriting the app which is using portray/1 and other tricks is unfortunately not an option.
Here a screenshot with old and new behaviour:

That confirms the answer to the first query in your original message is wrong. Discourse should be ok with \, but they need to be in single, double or triple backquoted text.

The bug was introduced in 2010 according to the git logs :slight_smile: It will be fixed, but I need a night sleep on the how.

I don’t see an issue with portray/1, although print/1 (and ~p) have changed to produce quoted output.

I didn’t mean that portray/1 itself has a problem, but by using portray/1 in the app and all sorts of other “dirty” tricks, I don’t get how I could “heal” the problem in any other way.

Thanks for your support
Holger

Most of the old “dirty tricks” should still work. SWI-Prolog is good at dirty tricks because if you really want you can work around almost all policies.

I have created a PR following the proposal described before. If anyone sees problems with this, please comment on the PR (or here).

Unfortunately we can probably not fix the stable series as I see no sensible way to fix the bug and remain compatible while I think the impact of this 13 year old bug is smaller than the incompatibility. While I do not expect many problems with these changes it is not unlikely that it will affect some applications :frowning:

Merged this into master. If you see issues, please still report. Better fix things before the next release :slight_smile:

Hi Jan,

works now as expected.

Cheers
Holger