Is there a simple way to get shell echo in Prolog variables ? The following seems not work.
?- with_output_to(string(X), shell("echo hello")).
hello
X = "".
Thanks in advance.
Can use e.g.:
run_sh_command(StrCmd, OutputLines, ErrorLines, ExitCode) :-
setup_call_cleanup(
process_create(path(sh), ['-ce', StrCmd], [process(PID), stdout(pipe(OutputStream)), stderr(pipe(ErrorStream))]),
( read_stream_lines(OutputStream, OutputLines),
read_stream_lines(ErrorStream, ErrorLines) ),
( process_wait(PID, ExitCode),
close(OutputStream),
close(ErrorStream) )
).
run_command(LstCmd, OutputLines, ErrorLines, ExitCode) :-
LstCmd = [ExeName|Args],
setup_call_cleanup(
process_create(path(ExeName), Args, [process(PID), stdout(pipe(OutputStream)), stderr(pipe(ErrorStream))]),
( read_stream_lines(OutputStream, OutputLines),
read_stream_lines(ErrorStream, ErrorLines) ),
( process_wait(PID, ExitCode),
close(OutputStream),
close(ErrorStream) )
).
read_stream_lines(Stream, Lines) :-
read_line_to_codes(Stream, Line1),
read_stream_lines_(Line1, Stream, Lines).
read_stream_lines_(end_of_file, _, Lines) :-
!,
Lines = [].
read_stream_lines_(Codes, Stream, [Line|Lines]) :-
atom_codes(Line, Codes),
read_line_to_codes(Stream, Line2),
read_stream_lines_(Line2, Stream, Lines).
?- run_sh_command(['echo hello'], OutputLines, ErrorLines, ExitCode).
OutputLines = [hello],
ErrorLines = [],
ExitCode = exit(0).
?- run_command(['echo', 'hello'], OutputLines, ErrorLines, ExitCode).
OutputLines = [hello],
ErrorLines = [],
ExitCode = exit(0).
1 Like
Thank you for quick and instructive solution. Replacing read_lines/2, which was undefined, with read_string/3, it works. The solution is so instructive, I am considering to use it to handle terminal command osascript on macOS to use Finder GUI from Prolog, for exampe, choose folder, etc.
Apologies, I’ve added my (now renamed) read_stream_lines.
1 Like
AFAIK, you don’t need the PID and wait for this simple case. Just reading up to end-of-file should do. Actually, this may deadlock if the process has too much output because the process may block if stdout or stderr pipe is full. If you are not interested in the error output, do not rebind it (it will go to the terminal) and without the PID thing all should work. If you want both stdout and stderr things get a bit more tricky. The simplest portable option is to use a thread to read one of the outputs and the main thread for reading the other. On POSIX systems you can also use wait_for_input/3 to watch both streams.
There is a lot of example code in library(git). library(process) is modeled after the SICStus library with the same name. Various people have proposed a more high level library. Most use cases seem pretty unique though. Unfortunately it is quite easy to do it wrong though and potentially leak resources or deadlock 
True. The routines I posted focus on completeness - sometimes the exit code of the process is important, as well as not processing the output until the exit code has been checked.
As a simple example, an exit code of 1 from grep may be acceptable.
Agree. Just, waiting for the process while it writes the output to a limited buffer no one is reading for will deadlock if the process writes more than the buffer size. AFAIK, a POSIX pipe by default has a 4K buffer.
Once, pipe option for open was enough for my purpose on communication between emacs and prolog. Recently I noticed that pipe option was dropped. So I have to recover the pipe option. Though I am not familiar with process handling in Prolog, my purpose for use is very simple like getting shell echo. I hope it is not so difficult to recover the dropped pipe for open. I am going to read library(process).
I just tried it on a 40MB text file containing 50,003 lines - it works.
?- run_command([cat, 'large.txt'], L2, Err, Exit), length(L2, Len).
...
Err = [],
Exit = exit(0),
Len = 500003.
Also works with an 80MB text file.
1 Like
??
102 ?- open(pipe('echo hello'), read, In).
In = <stream>(0x5643a72f7000).
103 ?- read_string($In, _, S).
S = "hello\n",
In = <stream>(0x5643a72f7000).
It might have been a bad idea to add it, but there are no plans for removing it. library(process) is much more powerful though.
You are right. You wait after having completed the read. That is fine. Sorry for the confusion.
You still do have a problem if the process writes a lot of data to stderr. In that case it will block, so it never completes the output to stdout.
Ah, this hangs:
?- run_command([sh, '-c', 'cat large.txt >&2'], L, Err, Exit), length(L, Len).
Sorry, I read wrongly help(open):
Use of the
pipe(Command) feature is deprecated. The predicate process_create/3 from
library(process) provides a richer and more portable alternative for
interacting with processes including handling all three standard streams.?- getinfo(‘echo Hello’, X).
In fact, I have found comment lines of queries in my codes, which use open(pipe(--), --, --) internally, which still works.
?- getinfo("echo Hello", X).
?- getinfo_codes('echo Hello', X).
?- getinfo("osascript -e 'return POSIX path of (choose folder)'", X).
?- getinfo("date +%Y-%m-%d", X).
?- getinfo_codes("date +%Y-%m-%d", X), smash(X).
Possibly we need some different wording here. I think the general meaning of deprecated for features of software artifacts is “don’t use this as we plan to remove this in due time”. Most of SWI-Prolog’s use of deprecated in documentation merely says “don’t use this because there are better alternatives” , while there is no intend to remove the deprecated functionality. Better means more (de-facto) standard, more portable, cleaner, faster, …
Dealing with functionality that is obsolete remains a bit hard. Keeping it in improves backward compatibility. That is an important, but unfortunately probably the only advantage. Applications get better (as defined above) if they replace usage of deprecated functionality.
This uses a thread, to prevent hanging:
run_sh_command(StrCmd, OutputLines, ErrorLines, ExitCode) :-
run_command([sh, '-ce', StrCmd], OutputLines, ErrorLines, ExitCode).
run_command(LstCmd, OutputLines, ErrorLines, ExitCode) :-
LstCmd = [ExeName|Args],
setup_call_cleanup(
message_queue_create(ErrorQueue),
setup_call_cleanup(
process_create(
path(ExeName), Args, [
process(PID),
stdout(pipe(OutputStream)),
stderr(pipe(ErrorStream))
]
),
( thread_create(
( read_stream_lines(ErrorStream, ErrorLines),
thread_send_message(ErrorQueue, ErrorLines)
),
ThreadId
),
read_stream_lines(OutputStream, OutputLines),
process_wait(PID, ExitCode),
thread_join(ThreadId),
thread_get_message(ErrorQueue, ErrorLines)
),
( close(OutputStream),
close(ErrorStream)
)
),
message_queue_destroy(ErrorQueue)
).
read_stream_lines(Stream, Lines) :-
read_line_to_codes(Stream, Line1),
read_stream_lines_(Line1, Stream, Lines).
read_stream_lines_(end_of_file, _, Lines) :-
!,
Lines = [].
read_stream_lines_(Codes, Stream, [Line|Lines]) :-
atom_codes(Line, Codes),
read_line_to_codes(Stream, Line2),
read_stream_lines_(Line2, Stream, Lines).
This outputs Codes rather than lines:
run_command_to_codes(LstCmd, OutputCodes, ErrorCodes, ExitCode) :-
LstCmd = [ExeName|Args],
setup_call_cleanup(
process_create(
path(ExeName), Args, [
process(PID),
stdout(pipe(OutputStream)),
stderr(pipe(ErrorStream))
]
),
setup_call_cleanup(
message_queue_create(ErrorQueue),
( thread_create(
( read_stream_to_codes(ErrorStream, ErrorCodes),
thread_send_message(ErrorQueue, ErrorCodes)
),
ThreadId,
[at_exit(close_stream(ErrorStream))]
),
read_stream_to_codes(OutputStream, OutputCodes),
process_wait(PID, ExitCode),
thread_join(ThreadId),
thread_get_message(ErrorQueue, ErrorCodes)
),
message_queue_destroy(ErrorQueue)
),
close_stream(OutputStream)
% Hangs if closing here: close_stream(ErrorStream)
).
% Prevents "stream... does not exist (already closed)" error
close_stream(Stream) :-
( is_stream(Stream)
-> close(Stream)
; true
).
read_stream_to_codes(Stream, Codes) :-
fill_buffer(Stream),
read_pending_codes(Stream, Codes, Tail),
( Tail == []
-> true
; read_stream_to_codes(Stream, Tail)
).
?- run_command_to_codes([echo, hello], L, E, Ex).
L = `hello\n`,
E = [],
Ex = exit(0).
Strangely, close(ErrorStream) run on the main thread hangs - raised as Closing stream used in thread hangs · Issue #1174 · SWI-Prolog/swipl-devel · GitHub
Although I should understand the reason why pipe for open is deprecated,
it has at least one merit for my application, that is, which could be applied to call osascript for macOS Finder GUI.
To move along recommendation, first I tried to apply @breb’s run_command_to_codes
to osascript, but it seems not work
% ?- run_command_to_codes([osascript, " -e 'return POSIX path of (choose folder)'"], X, _, _).
%@ X = [].
However preparing a unix shell bin script so that it can be found on the $PATH, it works.
choosefolder:
#! /bin/sh
osascript -e 'return POSIX path of (choose folder)'
% ?- run_command_to_codes([choosefolder], X, _, _), atom_codes(A, X).
%@ X = [47, 85, 115, 101, 114, 115, 47, 99, 97|...],
%@ A = '/Users/cantor/Documents/texnotes/\n'.
I don’t know for now how to avoid preparing the small unix shell script to use directly
run_commnad_to_codes.
This works:
run_command_to_codes([osascript, '-e', 'return POSIX path of (choose folder)'], O, E, Ex),
atom_codes(OA, O), atom_codes(EA, E).
To parse such simple single-line output, can use e.g.:
first_line_codes([], []).
first_line_codes([H|T], Line) :-
% Newline is Unicode 10
( H == 10
-> Line = []
; Line = [H|L],
first_line_codes(T, L)
).
I’d use H == 0'\n for readability and portability 
Note that all this stuff is in library(readutil).
If portability is not an issue I’d use strings for most of this. They are much more compact, faster and easier to work with, unless you want to run a DCG on the result. If that is the case you can still use string_codes/2.