Windows, unit test for clib/process

General
1

The unit test fails at two “subtests”:

  1. First one is something with /dev/null:
test(stream_input, [condition(has_exe(wc)), X == "0"]) :-
    open('/dev/null', read, In, [type(binary)]),
    process_create(path(wc),
                   ['-c'],
                   [stdin(stream(In)), stdout(pipe(Out))]),
    close(In),
    read_process(Out, X0),
    split_string(X0, "", " \r\n", [X]).

I invoked it from MSYS2 that has a “wc” program. The test fails since there’s no such device /dev/null on Windows. I tried to fix it by replacing by “NUL:”, but then I get a bad file descriptor because one cannot set the type to binary. This works, but I am unsure if this makes sense:

test(stream_input, [condition(has_exe(wc)), X == "0"]) :-
    process_create(path(wc),
                   ['-c'],
                   [stdin(null), stdout(pipe(Out))]),
    read_process(Out, X0),
    split_string(X0, "", " \r\n", [X]).

Obviously, that would be called on Windows only.

  1. The other one seems a bug to me:
% Seems to be able to kill itself on MacOS if this is ran
% concurrently.  Too quick reuse of PIDs?
test(kill_gone, [ error(existence_error(process, PID)),
                  condition(\+((current_prolog_flag(windows, true),
			        current_prolog_flag(apple,true))))
		]) :-
    process_create(path(sleep), [2], [process(PID)]),
    process_kill(PID),
    process_wait(PID, X),
    assertion(X == killed(15)),
    process_kill(PID).

Doesn’t this mean something like “if not on Windows and Apple at the same time?”, and shouldn’t it read as “not on Win and not on Apple”, e.g. like this

condition((+ current_prolog_flag(windows, true), + current_prolog_flag(apple,true)))

2

Hmm. This is Wine rather than Windows, but most of the time that works the same.

?- open('NUL', read, In, [type(binary)]).
In = <stream>(00000000012a9b10).

?- read($In, I).
I = end_of_file,
In = <stream>(00000000012a9b10).

I could live with that. It surely is cross-platform. Seems to open ‘nul’ on Windows and ‘/dev/null’ otherwise.

:slight_smile: Guess the ‘,’ should be a ‘;’. May also explain why the test still poses problems on Apple.

Do you make a PR?

3

I can get this to work from msys2 bash using cmd /c "wc -c < NUL", whereas wc -c < NUL does not work and so does cmd /c wc -c < NUL either. This probably creates trouble in process_create.

I’ll happily make a PR for the cross-platform solution.

1 Like
4

One more soon from udp_sockets, CRLF woes, again.

5

Note that the current implementation for UDP sockets provide an encoding option. That may be enough if you set that to octet.

6

The problem appears in the last subtest, hello(String), with String = character codes from 0 to 1000. I cropped it to 0…20 to make it readable.

test(string, Got == got(hello(String))) :-
    numlist(0, 20, Codes),
    string_codes(String, Codes),
    trip(hello(String), Got, [as(term),encoding(utf8)]).

Just changing the encoding(utf8) to encoding(octet) in the unit test did not solve the problem.

I think it is somewhere else, here, in clib/socket.c, line 608:

  switch(as)
  { case PL_VARIABLE:  cvt = CVT_ALL;             break;
    case PL_ATOM:      cvt = CVT_ATOM;            break;
    case PL_STRING:
    case PL_CODE_LIST: cvt = CVT_STRING|CVT_LIST; break;
    case PL_TERM:      cvt = CVT_WRITE_CANONICAL; break;
    default:           assert(0);                 return FALSE;
  }

  if ( !PL_get_nchars(Data, &dlen, &data, cvt|CVT_EXCEPTION|rep) )
    return FALSE;

PL_TERM matches, so PL_get_nchars uses “write_canonical” is chosen, which ends up in PL_get_text, see here, src/os/pl-text.c, around line 342:

    for(enc = encodings; *enc != ENC_UNKNOWN; enc++)
    { size_t size;
      IOSTREAM *fd;

      r = text->buf;
      size = sizeof(text->buf);
      fd = Sopenmem(&r, &size, "w");
      fd->newline = SIO_NL_POSIX; // <<<<<<<<<<<< added this
      fd->encoding = *enc;
      if ( PL_write_term(fd, l, 1200, wflags) &&
           Sputcode(EOS, fd) >= 0 &&
           Sflush(fd) >= 0 )
      { text->encoding = *enc;
        text->storage = (r == text->buf ? PL_CHARS_LOCAL : PL_CHARS_MALLOC);

The memory stream uses CRLF again :slight_smile:

698×394 65.4 KB

I think the ink blot on the finger is unrelated. But when I add the line with SIO_NL_POSIX, it works fine. The question is, of course, how to solve this, since one cannot change this behavior globally. One option that could be considered might be to add one more flag to PL_get_text, would you accept this? PL_get_nchars has an argument “flags” that it just hands over to PL_get_text. Maybe there’s room for another bit that suppresses the “CR”.

7

I’ve made a PR (actually, several, bear with me, I don’t speak Git) with a new flag REP_NL_POSIX for your kind consideration.

8

I’ve seen them. Will need some careful consideration. The text exchange functions never had anything to do with the newline representation, only with various different ways to encode characters and convert that to/from Unicode. I hope there is a sensible solution to keep it that way :slight_smile: