Progressing the SWI-Prolog environment

[jan]

You use Windows, no? Which version? On my VM I have some problems with the console that sometimes not all output is written. Typing or clicking on the console shows the remaining output. I hope this is some VM artifact. The problem does not happen when running the Windows binary under Linux+Wine.

And yes, this does not add much for the Windows user. The console has several enhancements to to scrolling/resizing, ANSI sequence handling (quite a few) and performance. All tools can now render color glyphs, though support for UTF-16 on Windows for characters > 0xffff is very partial.

This mostly removes a large amount of Windows specific code from the sources

[abaljeu]

Yes, I use Windows, and roughly 7/10/11 are similar looking. But even though I develop C# gui’s and know all the Microsoft technologies such as WPF and Windows Forms, my inclination is to build GUIs on Node.js, or webasm and have the browser define everything. I use HTML and CSS. It’s cross-platform and looks good.

Bottom line here, I’m just saying XPCE is irrelevant to me. Prolog can generate html if I want it to. But I use Prolog as my constraint engine, my logic computation system not my user interface.

[jan]

I can produce something similar when resizing the console while entering text, i.e., paste a couple of times, resize the console. If I do not resize, all works fine.

I fear that is hard to fix considering the way libedit and the console window work on Windows.

[jan]

The behaviour is still most likely caused by libedit having a different idea about the screen width than the console or possibly some other issue with the font metrics. I can run

 ?- tty_size(R,C).
 R = 26,
 C = 80.

Now I can verify this works as it should using

 ?- format('~`.t~80|A~n').

Which should print a line of . and an A on the next line. If this is incorrect, please try to resize and see whether it works correctly afterwards.

[stassa.p]

I just found out about Epilog. I guess I was away from computer for a while.

I love the splittable windows and the overall look and feel but I already have a bit of an issue: it seems that either my emacs editor preferences file is not loaded, or the commands in it are not executed, and the same for my Prolog preferences file.

First issue, in appdata/roaming/swi-prolog/xpce/Defaults (i.e. on windows) I have the following option:

emacs_buffer.unicode_encoding: utf8

This was mainly to display quantifier characters correctly in the editor, but now they are shown as “wrongly encoded bytes”:

(Tailrec) ∃.P,Q ∀.x,y,z: P(x,y)← Q(x,z),P(z,y)
(Identity) ∃.P,Q ∀.x,y: P(x,y)← Q(x,y)

Whereas this is what I see in the console for the same output:

(Tailrec) ∃.P,Q ∀.x,y,z: P(x,y)← Q(x,z),P(z,y)
(Identity) ∃.P,Q ∀.x,y: P(x,y)← Q(x,y)

Second issue, my custom theme file is not loaded correctly. I have this in my appdata/roaming/swi-prolog/init.pl:

:- use_module(library(theme/dork)).

And the dork.pl file is indeed in that location, but I have the dark.pl theme in the editor and the console instead.

What has changed and how do I load the right configuration for the editor and console?

Edit: another thing. In the old swipl-win console it was easy to change the console font and size. In the new one I guess the only way is to change the editor preferences?

Edit 2: I’m on version 9.3.33

[jan]

@stassa.p

emacs_buffer.unicode_encoding: utf8

The great mystery of encodings Whether or not it reads the config file is unclear, but this doesn’t help. It determines how the file is saved if it contains non-ASCII characters. When saving, PceEmacs adds a BOM marker to the file. Then it can read it again.

You can get the right result using ?- set_prolog_flag(encoding, utf8)., but there is a problem If you set this in the Epilog console, it is set in the console thread, but not in the main thread, which is the thread in which xpce runs and loads the file … Using the init.pl file won’t help either as this too is loaded in the Epilog thread. What does help for me is running

 swipl-win -Dencoding=utf8

I guess we need a more elegant solution. What probably also helps is, in init.pl

:- in_pce_thread(set_prolog_flag(encoding, utf8)).

But this too is not so elegant

@stassa.p

:- use_module(library(theme/dork)).

It seems the Epilog theme setup ignores this What probably helps is to add in the Defaults file

display.theme: dork

I’ll try to modify the code such that if a theme is already loaded it will not try again.

@stassa.p

Edit: another thing. In the old swipl-win console it was easy to change the console font and size. In the new one I guess the only way is to change the editor preferences?

Just resizing the font can be done with the Ctrl–, Ctrl-+ and Ctrl-=. Permanent changes indeed require editing the Defaults file. It used the tt and boldtt font aliases by default and the resource are terminal.font and terminal.bold_font. There are now three resources to control fonts:

• font.scale scales all fonts
• font.pango_families maps sans, serif and mono to a Pango font family
• font.system_fonts maps alias names to fonts.

Ideally, I guess we should at some point provide a GUI to modify the Defaults file …

Thanks for the comments!

[stassa.p]

@jan

:- in_pce_thread(set_prolog_flag(encoding, utf8)).

But this too is not so elegant

I’m fine with that and it indeed works to get back my quantifiers. It’s also the easiest thing on windows and probably the most portable between OSs no?

@jan

display.theme: dork

I tried this and I can confirm it loads the theme file, but the the theme now displays all wrong. This dork theme is the one I shared a while ago, based on your dark theme. It looks dark in the earlier versions of SWI (up until 9.3.24 which was the last version I had before today, and I guess predates epilog) but in 9.3.33 the colours are all yellow on white almost like the colours are inverted. Anyway I guess the old theme file is no longer read correctly by the system so I’ll have to see about adjusting it based on the dark theme.

@jan

Thanks for the comments!

Thanks for your hard work!

[stassa.p]

Yay, got my dork theme back. It was just a matter of updating the theme file to use the new code for pce_set_defaults/1 and pce_style/2 that had changed since the last version I used.

Unfortunately my theme looks awful and is basically unreadable with non-module code, but that was the case before epilog too, so it has nothing to do with it.

[stassa.p]

Hm. Just realised that the new Open File dialog isn’t navigable with the keyboard. Is that … permanent? I really try to avoid touching the mouse as much as possible (wrists).

Besides, is there an option to have Open File open the OS file explorer, instead of its own file navigator? It would be nice to be able to see a graphical representation of the file system and also decide whether to show/hide hidden files etc.

Edit:

More observations. The key bindings I was used to no longer work on windows. For example, to replace I would press (left) Alt + Shift + % I guess to do a \e%, but this doesn’t do anything in the new editor.

I tried changing this in the editor preferences:

key_binding.style:	emacs

But that doesn’t seem to do it.

Version 9.3.33-14-g6544cc402, 64 bits, on windows 11.

Edit 2: Also, the coloriser is very, very broken now in my editor. I hope it’s OK to upload a small image:

Note how some parentheses and commas are orange, but only some. Orange is the variable colour. Also see the broken q1 in the last clause and those weird underscores before the second literal in clauses one and two and the last clause.

[jamesnvc]

A guess about the highlighting & underscores – is that mixed tabs & spaces in the file? I’ve seen somewhat similar things with the Prolog LSP trying to highlight in a file with tabs…oh, and also windows-style newlines can cause things to get in this weird state too (e.g. this issue addressed with this commit, I think). Jan, is it possible that this issue is causing the built-in editor to get confused? (as I recall, the problem is that the automatic newline translation when reading the file results in the file offsets getting out-of-sync, because when reading CRLF it consumes two characters but since it converts it to a single newline by default, it only counts the offset as increasing by one).

[jan]

@stassa.p

Besides, is there an option to have Open File open the OS file explorer, instead of its own file navigator?

That used to be supported for Windows and MacOS. It was lost in the migration to SDL3, but with some luck it can be restored.

@stassa.p

to replace I would press (left) Alt + Shift + I guess to do a \e, but this doesn’t do anything in the new editor.

The binding is still \\e%, but SDL reports Alt-Shift-% as Alt-Shift-5 (on US keyboard). Binding the latter is easy, but only works on some keyboard layouts. I’ll see whether there is a more general way out. ChatGPT suggests SDL3 has ways to figure out that Shift-5 is actually %. Now we know it sometimes lies … I never noticed because for some reason I always use ESC-%. That works fine.

@stassa.p

Edit 2: Also, the coloriser is very, very broken now in my editor. I hope it’s OK to upload a small image:

@jamesnvc

Jan, is it possible that this issue is causing the built-in editor to get confused?

There was another report where the problem vanished after removing some German non-ascii characters. There was another change though and I did not manage to reproduce. And no, file encoding should not be the issue. The encoding plays a role when loading the file into the editor. Once loaded though, ^M is already removed and the file is no longer touched (the Prolog highlighting code reads from the editor rather than the file). There is a known problem with Unicode combining characters.

This is a serious issue but I need something to reproduce. @stassa.p, can you send the file (or some other file showing the problem) to bugs@swi-prolog.org in such a way that it does not get re-encoded by some smart filter during transmission? I guess best it to zip/tar it. Please include an SHA1 or other checksum (if there is a program for that on Windows), so I can be sure I really have the same file.

[jan]

@stassa.p

More observations. The key bindings I was used to no longer work on windows. For example, to replace I would press (left) Alt + Shift + I guess to do a \e, but this doesn’t do anything in the new editor.

Ok. It turns out SDL3 has no way to deal with this (ChatGPT lied ). You can deal with this for a known keyboard layout though. For example. for a US keyboard, where the % is Shift-5, you can do this:

Create a config file in <config dir>/swi-prolog/xpce/emacs/fundamental_mode.pl. Files in this directory that are name *_mode are loaded when the editor initializes the named mode. fundamental is the root of all modes. In this file, add

:- module(my_fundamental_mode, []).
:- use_module(library(pce)).

:- emacs_extend_mode(fundamental,
		     [ replace = key('\\e\\S-5')
		     ]).

:- emacs_end_mode.

This binds replace to Alt-Shift-5 (in addition to the existing ESC-% binding).

[stassa.p]

Thanks, I’ll send you the file but it looks fine today after I restarted my Prolog session. Also I changed it a bit since I took the screenshot yesterday, so it’s no longer the same file exactly, although AFAICT I didn’t touch the encoding.

@jan

This binds replace to Alt-Shift-5 (in addition to the existing ESC-% binding).

Alright, I can do that, thanks! But I think there’s more problems with the Emacs-like keybindings because after I set:

@stassa.p

key_binding.style: emacs

• all my keybindings stopped working, like even Ctrl-X, Crl-S didn’t work to save the file anymore. I commented the config line back out and now Ctrl-S works (not sure about Ctrl-X, Ctrl-X, I’ll have to try it).

More concerningly, I no longer have any command history in the Epilog console, but this seems to be an older issue. I’ll send you the file and write a more complete comment in a bit.

@jan

That used to be supported for Windows and MacOS. It was lost in the migration to SDL3, but with some luck it can be restored.

Thanks, that’d be great. The new file navigator dialog is not great.

[stassa.p]

@jan

Ok. It turns out SDL3 has no way to deal with this (ChatGPT lied ).

Oh no! ChatGPT was wrong? Aaaaw…

(This is why I don’t get invited to parties)

[jan]

@stassa.p

More concerningly, I no longer have any command history in the Epilog console,

You mean if you use key_binding.style: emacs? I’ll have a look. Could be some issue with the dynamic rebinding. Keyboard handling is quite different in SDL3, so xpce needed some changes as well. That could be incomplete.

@stassa.p

Thanks, that’d be great. The new file navigator dialog is not great.

It seems SDL3 provides a portable interface to the native file dialog of the OS. This time ChatGPT was right I think that is what we need And yes, xpce’s own file dialog is 30 years old …

[stassa.p]

@jan

You mean if you use key_binding.style: emacs? I’ll have a look. Could be some issue with the dynamic rebinding. Keyboard handling is quite different in SDL3, so xpce needed some changes as well. That could be incomplete.

I don’t think it’s the keybindings because !! gives me an empty history too.

@jan

It seems SDL3 provides a portable interface to the native file dialog of the OS. This time ChatGPT was right I think that is what we need And yes, xpce’s own file dialog is 30 years old …

I thought that was some kind of SDL3 default, I didn’t realise it was xpce’s one.

@jan

This time ChatGPT was right

Hmph

[stassa.p]

Some more information on this. I went back to the version I had in February, 9.3.20-23-ge2557e66e, to see if I had history with it. I do, but only recent history.

Let me try to organise this. Here’s what I mean:

• Recent history: commands entered in the last Prolog session before the current one.
• Older history: commands entered in all previous Prolog sessions (up to the max history lines).

I try to invoke history with up-arrow, or !!. To test what I have, I start a new Prolog session by loading a Prolog file, call it f. The behaviour I’m used to is that this Prolog session should give me both the recent and older history, i.e. all the history, but only for the commands I entered in a session loading the same file, f. So the way things worked until now, I could start a new Prolog session loading a project load-file from a different directory each time and I would get the older history of the commands that I had entered in any session started in that directory (so for that project specifically, which was nice because I could always go back and pick up where I left).

Here’s what I have:

• Version 3.33, epilog console: no recent history, no older history.
• Version 3.33, powershell: only recent history, no older history.
• Version 3.24, old xpce-based console: no recent history, no older history.
• Version 3.24, powershell: only recent history, no older history.
• Version 9.3.20-23-ge2557e66e, old xpce-based console: only recent history, no older history.
• Version 9.3.20-23-ge2557e66e, powershell: only recent history, no older history.

In other words, my long-time history seems to be gone, and the only version where things seem to work as I remember them (i.e. there seems to be new history being recorded and recalled in both the IDE console and the OS command-line, powershell) is version 9.3.20-23-ge2557e66e.

Btw, in version 9.3.20-23-ge2557e66e, in the old xpce-based console, I get this command that I don’t think I’ve ever entered:

 _HiStOrY_V2_

Which seems to be some kind of system thing? I don’t get this in any other version.

Sorry if I’m still reporting this in a confusing manner. Please let me know what more information you need or if there’s something you want me to try.

[jan]

@stassa.p

I don’t think it’s the keybindings because !! gives me an empty history too.

@stassa.p

_HiStOrY_V2_

I think that explains some of the situation. If the system has a commandline editor, it by default does not initialize the !! based history. That is mainly because some people do not like that ! can get ambiguous at the toplevel. You can enable !! based history using

:- set_prolog_flag(history, 50).

in your init.pl (adjust 50 to the number of commands you want to be recalled).

Next, the versions before the move the SDL used a little Win32 console I wrong ages ago in native Windows code. The new one uses the Epilog terminal emulator in xpce and then uses BSD libedit for the commandline editor. The _HiStOrY_V2_ comes from the history save/load of this commandline editor. Before that, the old console used a different format for saving the history. I think you should

• Remove (or empty) the dir-history directory in the config dir to get rid of the old history
• Make sure there is no setting for the readline Prolog flag in init.pl.

Note that the libedit commandline editor now also works for swipl in the Windows console. As with the non-Windows versions, you can now also run PceEmacs, the GUI tracer, etc. from the native console version. There a catch though: in that case the tools and your program run in the same thread and the tools are not responsive while your program runs.

One of the good things about this entire move is that the system now functions practically identical on all supported platforms.

Let me know your findings!

P.s. Should we make numbered history access default?

[stassa.p]

Uh oh, I see I posted my previous comment about history in this thread by mistake. I meant to post it in this new topic:

Command line history gone

I reported some issues I had with the new SWI version 3.33 in this thread: To be able to work as fast as before while I get used to the new Epilog console and editor, I’ve re-installed my previous Prolog version, 9.3.24, alongside the new one. After I did that, I noticed that I no longer have any command-line history in any of my Prolog projects. What I mean by that is that up-arrow doesn’t fill in any commands, and ?- !!. says there’s nothing there: 1 ?- !!. ! No such event ! No such even…

Apologies for the confusion. I guess the newer topic can be merged into this one?

@jan

I think that explains some of the situation. If the system has a commandline editor, it by default does not initialize the !! based history. That is mainly because some people do not like that ! can get ambiguous at the toplevel. You can enable !! based history using

:- set_prolog_flag(history, 50).

in your init.pl (adjust 50 to the number of commands you want to be recalled).

I added that in my init.pl file in an effort to resolve my history issues, but I still don’t get any history in the newer versions (9.3.24 and 9.3.33, which I keep referring to as 3.33- sorry).

Btw, I guess all versions pick up the same init.pl file so that could cause trouble, for instance if there’s a readline setting for one version that’s not supported by another. Is there a way to have an init.pl file specific to a version, e.g. in a folder named the same as a version? Sounds hacky though.

@jan

Next, the versions before the move the SDL used a little Win32 console I wrong ages ago in native Windows code. The new one uses the Epilog terminal emulator in xpce and then uses BSD libedit for the commandline editor. The _HiStOrY_V2_ comes from the history save/load of this commandline editor. Before that, the old console used a different format for saving the history.

Thanks, that clarifies things a bit. So the history files written by earlier versions (prior to the switch to SDL?) can’t be read, or updated by, newer versions (after the switch)?

@jan

• Remove (or empty) the dir-history directory in the config dir to get rid of the old history
• Make sure there is no setting for the readline Prolog flag in init.pl.

So that’s where the history is! The hint is in the name I guess

I can also see a dir_history.bak for backup I guess.

I had a look in my dir_history and there’s a bunch of files (40 of them) in there that hold the history for all my projects going back to 2022. They’re just plaintext files so I can see all the stored commands in my text editor. I like that! But if I remove dir-history then I lose all of them. That I don’t like!

The files in dir_history are named with hashes that I’m guessing must record the path to the directory of the Prolog session that created the history file. Is that correct?

Is there a way to re-activate those history files, and make them available to their respective SWI-Prolog sessions again in the newer SWI-Prolog versions? By “respective” I mean any SWI-Prolog session starting from the same directory as the session that originally created the history file (assuming my model about that is correct)?

I would really like to be able to hold on to my command history. I start so many different projects that it’s difficult to remember what I was doing in every one. I usually document the important queries but not always.

Clearly, I have some attachment issues.

[stassa.p]

@jan

• Remove (or empty) the dir-history directory in the config dir to get rid of the old history
• Make sure there is no setting for the readline Prolog flag in init.pl.

I did as you say (after keeping a backup of my dir_history directory so I can possibly resurrect it later) and here’s what I found:

1. Version 9.3.33-14-g6544cc402, Epilog console: A new dir_history directory is created when I start a new Prolog session by clicking <swi_root>/bin/swipl-win.exe, but no history files are created in this directory when I make queries at the top-level. So there’s no history recorded.
2. Version 9.3.33-14-g6544cc402, powershell command-line: As above but this time when I make a few queries at the top-level a new history file is created so I do get history.

The same goes for version 9.3.24 (which happens to be the earlier version I had on my machine before updating to 9.3.33 and that I re-installed side-by-side to 9.3.33 until I get used to the new one). A new dir_history is created and stays empty if I start the session from the IDE, but I get a history file if I start the session from powershell.

I noted that the name of the new history files, created by version 9.3.33 in lower-case and I wonder if that is what messes up with the newer console versions finding the older history files. Or at least maybe that’s part of it?