# Preserving CR with LF when converting text files to character codes

**URL:** <https://swi-prolog.discourse.group/t/preserving-cr-with-lf-when-converting-text-files-to-character-codes/2591>\
**Category:** Nice to know\
**Created:** [July 3, 2020, 11:23am UTC](https://swi-prolog.discourse.group/t/preserving-cr-with-lf-when-converting-text-files-to-character-codes/2591 "2020-07-03T11:23:57Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![EricGT](https://avatars.discourse-cdn.com/v4/letter/e/f1d935/32.png) [@EricGT](https://swi-prolog.discourse.group/u/EricGT)\
**Post date:** [July 3, 2020, 11:23am UTC](https://swi-prolog.discourse.group/t/preserving-cr-with-lf-when-converting-text-files-to-character-codes/2591/1 "2020-07-03T11:23:57Z")

</div>

When using SWI-Prolog on Windows and reading text files for conversion into characters codes and needing to preserve the CR with LF some special attention is needed.

Normally a file would be opened as a text file and read but that results in the CR with LF being converted to just a LF.

To preserve the CR with LF use set\_stream/2 with `newline(posix)`.

Example code

```prolog
to_codes(File_path) :-
    setup_call_cleanup(
        open(File_path,read,Stream),
        (
            set_stream(Stream, newline(posix)),
            read_stream_to_codes(Stream,Codes),
            format('Codes: ~w~n',[Codes])
        ),
        close(Stream)
    ).

```

Example file

![image](https://global.discourse-cdn.com/free1/uploads/swiprolog/original/2X/b/b4387baa7f88a8aa982e5cadeab965519cff73bf.png)  
which is

`; test` followed by `cr` with `lf`.

Example run

```prolog
?- to_codes('C:/Users/Groot./test.abnf').
Codes: [59,32,116,101,115,116,13,10]
true.

```

* * *

A case in point where preserving CR with LF is needed is when working with ABNF files.

ABNF (Augmented BNF) is the BNF used for [IETF](https://ietf.org/) (Internet Engineering Task Force) [RFPs](https://en.wikipedia.org/wiki/Request_for_proposal) (Request For Proposal) and documented in [RFP 5234](https://tools.ietf.org/pdf/rfc5234.pdf).

There are several good but dated references regarding SWI-Prolog and ABNF ([search](https://www.google.com/search?q=swi+prolog+%22%22abnf%22&sa=X&ved=2ahUKEwjtr_Oc1bHqAhX6l3IEHQJNA1IQ5t4CMAJ6BAgEEAY&biw=1745&bih=881&dpr=1.1))

---

<div class="post-metadata">

**Author:** ![jan](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/jan/32/4_2.png) [@jan](https://swi-prolog.discourse.group/u/jan)\
**Post date:** [July 3, 2020, 11:54am UTC](https://swi-prolog.discourse.group/t/preserving-cr-with-lf-when-converting-text-files-to-character-codes/2591/2 "2020-07-03T11:54:30Z")

</div>

type(binary) and encoding(octet) are, AFAIK, equivalent. ISO only defines `type`. SWI-Prolog also has `encoding`.

If you want to read a text file using some encoding and preserving CR/LF you can use the normal open and then use `set_stream(In, newline(posix))` to disable the CR/LF --\> LF translation.

---

<div class="post-metadata">

**Author:** ![EricGT](https://avatars.discourse-cdn.com/v4/letter/e/f1d935/32.png) [@EricGT](https://swi-prolog.discourse.group/u/EricGT)\
**Post date:** [July 3, 2020, 11:58am UTC](https://swi-prolog.discourse.group/t/preserving-cr-with-lf-when-converting-text-files-to-character-codes/2591/3 "2020-07-03T11:58:39Z")

</div>

Thanks. 😀 I will fix the first post.