You could maybe leave out “on a Mac”? The behavior you show is most probably the same across platforms. A precursory search tell me that the \r is was used as a new line on “Classic Mac OS” shipped until 2001.
And could you use read_string/5 to give it your own “end of line”?
?- open_string('abc\rdef', S), read_string(S, "\r", "", _, T).
S = <stream>(0x5baa5dba2600),
T = "abc".
The current definition of read_line_to_string/2 specifically doesn’t use \r as an end of line.
Historical note: I read an interesting explanation on the origins of the use of two end-of-line characters (\r\n) in some OSs (I thought it was Windows though). It has to do with how teletypes worked and signal was transmitted at the time
The explanation seemed to make sense, but of course I took it for granted don’t know if anyone has ever heard/read something similar
On a teletype, there was a movable printhead (also called a “carriage”) … when the printhead/carriage reached the end of line, the “return” code would send it back to the beginning of the line. The “newline” code would advance the roller to the next line. (So, if you wanted to overtype something, you could either use backspace, or you could use “return” and forward space)
The carriage return took some time, so if you immediately followed it with more text, you’d get “random” overtyping. To handle this, you would send some “null” characters that did nothing (I think that the “null” moved the carriage slightly up and down, as if a space was being typed). “Null” was different from “space” in that it didn’t move the carriage one position to the right.
So, at end of line, you’d send \r to start the carriage back to the beginning of the line, then \n to advance the roller, then some number of \0s.
(There were some additional complexities if you had a human operator rather than punched paper tape, at least on the oldest teletype machines … the operator had to type at a steady speed, which most typists couldn’t do. My father told me that in his office there was one person who had been a teletype operator and when he first sat down at a typewriter, the other typists all got up and gathered around in amazement at the regular sound of his typing)
The old teletypes also explain some other codes such as tab, vertical-tab, form-feed, bell, shift-out, shift-in, etc.
I don’t think the OS matters at all, does it? It seems now that you would like to support multiple mutually exclusive end-of-line markers? I don’t see how you can make \r\n be a single end of line but at the same time both \r and \n be individually also end of line.
You’d need some heuristic that decides what kind of EOL it should expect in this file. I am certain this is what your Notepad is doing.
What happens on your Windows 10 if you give it a file like this:
abc\r\ndef\rghi\nx
??
You should probably also try out the different orders in which the three linebreaks appear in your file.
When reading input from the stream, if newline is None, universal newlines mode is enabled. Lines in the input can end in '\n', '\r', or '\r\n', and these are translated into '\n' before being returned to the caller. If newline is '', universal newlines mode is enabled, but line endings are returned to the caller untranslated. If newline has any of the other legal values, input lines are only terminated by the given string, and the line ending is returned to the caller untranslated.
Internally, text in SWI-Prolog is supposed to be Unicode where lines are separated by single \n. Translation on I/O is done by streams using the encoding and newline properties. As is, it supports posix, which does no translation and dos which simply removes \r and emits newlines as \r\n. \r terminated lines as (I think) existed on old MacOS are not supported. That is probably not perfect, but it seems good enough in practice. Notably blindly removing \r is a little dubious, but it “fixes” common cases where some lines use \r\n and others just \n while some have \r\r\n, etc.
A possibly useless observation: one difference between read_string/5 and most other “read line” offers is that read_string/5 takes a set of characters in the “end of line” argument (SepChars). Others (Python, awk, getline/getdelim in glibc, and so on…) either use a char, or a string. For a “set of characters” as in read_string/5 you need to use a higher level mechanism like a regular expression.
In Dogelog Player in release 1.1.5 I had compression
enabled by default. Currently for release 1.1.6 I have
disabled it, and I am experimenting with a separate predicate
get_compress/2 besides get_code/2. The predicate get_compress/2 is to support some legacy code that assumes
compression. Especially stateful compression, based on a stream
state. The new uncompressed behaviour get_code/2 is
based on the observation that SWI-Prolog can live with it. So
Dogelog Player should also be able to live with it. Now exploring
the corners that break when using uncompressed get_code/2.
If you have a string, I/O has done its job, i.e., strings are already supposed to be in canonical form. Translation holds for text streams created from files and a few other cases, such as process pipes. In other places (e.g., sockets), you have to set the encoding and newline mode yourself as these are typically dictated by some OS independent standard. I guess you can also set the newline mode for a stream that originates from a string. Probably you cannot change the encoding as this is assumed to be fixed.
And, as is, translation is only enabled by default on Windows.
But when I use Notepad on Windows 10 for the generated file it shows
me two lines, which is correct interpretation of '\r\r\n' as one line
separator '\r' and then one line separator '\r\n':
I guess it is a matter if taste. \r\r\n is the same as \r\n on the good old teletype Bill Gates is old enough to know that I told you, the dos newline mode simply removes \r.
don’t know if anyone has ever heard/read something similar
Bill Gates is old enough to know that