# Byte ordering/conversion predicates? **URL:** \ **Category:** Predicate\ **Created:** [August 3, 2021, 9:31pm UTC](https://swi-prolog.discourse.group/t/byte-ordering-conversion-predicates/4243 "2021-08-03T21:31:52Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![dmchurch](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/dmchurch/32/3427_2.png) [@dmchurch](https://swi-prolog.discourse.group/u/dmchurch)\ **Post date:** [August 3, 2021, 9:31pm UTC](https://swi-prolog.discourse.group/t/byte-ordering-conversion-predicates/4243/1 "2021-08-03T21:31:52Z")
Is there any sort of builtin SWI-Prolog functionality for doing bit representation conversion (uint32 ↔ int32, i.e. taking the two’s complement) and or byte-order fixing, like libc’s `ntohl` and friends? More generally, is there anything for doing binary-encoding of numeric data for interop with other systems? ---
**Author:** ![peter.ludemann](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/peter.ludemann/32/48_2.png) [@peter.ludemann](https://swi-prolog.discourse.group/u/peter.ludemann)\ **Post date:** [August 3, 2021, 9:51pm UTC](https://swi-prolog.discourse.group/t/byte-ordering-conversion-predicates/4243/2 "2021-08-03T21:51:58Z")
There are some serialization operations as part of library(protobufs) in [contrib-protobufs/protobufs.c at 137a9d87ccc990d29e7ae337bb4a97ad8af05982 · SWI-Prolog/contrib-protobufs · GitHub](https://github.com/SWI-Prolog/contrib-protobufs/blob/137a9d87ccc990d29e7ae337bb4a97ad8af05982/protobufs.c) (there’s also a byte-ordering fix - it does little-endian ordering, and not the network big-endian ordering … I assume it works with a big-endian machine but have never tested) These are currently not exported by library(protobufs), but I can easily change that (just remove some "%"s) ---
**Author:** ![dgelessus](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/dgelessus/32/3106_2.png) [@dgelessus](https://swi-prolog.discourse.group/u/dgelessus)\ **Post date:** [August 3, 2021, 10:12pm UTC](https://swi-prolog.discourse.group/t/byte-ordering-conversion-predicates/4243/3 "2021-08-03T22:12:22Z")
For signed/unsigned integer conversion, at least int → uint is easily done with bitwise operators (`UInt16 is Int16 /\ 0xffff`). The other direction is a bit more annoying (`UInt16 >= 0x8000 -> Int16 is UInt16 - 0x10000 ; Int16 = UInt16`, or all in one expression: `Int16 is UInt16 \/ (\0xffff * ((UInt16 >> 15) /\ 1))`). A proper library or builtins for this would be more convenient of course. Regarding converting integers from/to packed binary data, there was some related discussion a few months ago: > [@How to send a string + string length on a stream?](https://swi-prolog.discourse.group/t/how-to-send-a-string-string-length-on-a-stream/3779): > > I want to write a predicate that takes a stream and a string and writes the string length first as a 64 bit integer followed by the string as UTF8 encoded. Is this in the right direction? Also: I cannot figure out how to turn a Prolog number into bytes (i.e. turn 2 into binary 0x10). Any tips there? sendString(Stream, String) :- % make sure it is UTF8 encoded set\_prolog\_flag(encoding, utf8), % get a set of UTF8 codes from the string stringCodes(String, Codes), % Count and … In that thread, @peter.ludemann also suggested the internal protobuf predicates mentioned above: > FWIW, I would also be interested in having binary un-/packing operations supported in SWI, and ideally not just as undocumented internal predicates that you have to call with a module prefix 🙂 I haven’t done much binary data parsing/generating with SWI yet, but it’s something I want to properly try out at some point. ---
**Author:** ![dgelessus](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/dgelessus/32/3106_2.png) [@dgelessus](https://swi-prolog.discourse.group/u/dgelessus)\ **Post date:** [August 3, 2021, 11:23pm UTC](https://swi-prolog.discourse.group/t/byte-ordering-conversion-predicates/4243/5 "2021-08-03T23:23:56Z")
I think `Int16 is (Int + 0x8000) /\ 0xffff - 0x8000` will work to truncate any integer to signed 16-bit and wrap around if the input is out of range. Could be useful if you need to simulate signed 16-bit two’s complement arithmetic for some reason. This is shorter than my original solution too, not sure why I didn’t try this earlier. ---
**Author:** ![dgelessus](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/dgelessus/32/3106_2.png) [@dgelessus](https://swi-prolog.discourse.group/u/dgelessus)\ **Post date:** [August 3, 2021, 11:48pm UTC](https://swi-prolog.discourse.group/t/byte-ordering-conversion-predicates/4243/7 "2021-08-03T23:48:04Z")
Tried looking for some “prior art” related to binary un-/packing in Prolog, but there really doesn’t seem to be much. - I found [some](https://stackoverflow.com/q/26683738) StackOverflow [questions](https://stackoverflow.com/q/29752355), but the answers all suggest manual bit twiddling with either (is)/2 or library(clpfd). - Logtalk has a [`cbor`](https://github.com/LogtalkDotOrg/logtalk3/blob/179c9dbaaa9fa6c7dfe3c9604b75807ad4755c6d/library/cbor/cbor.lgt) library in pure Logtalk, and it also does all un-/packing by hand with lists and bit manipulation. - Had a quick look over the usual collection of other Prolog systems (SICStus, Ciao, YAP, ECLiPSe, Scryer) and couldn’t find any relevant builtins or libraries. ---
**Author:** ![grossdan](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/grossdan/32/23_2.png) [@grossdan](https://swi-prolog.discourse.group/u/grossdan)\ **Post date:** [August 4, 2021, 3:43am UTC](https://swi-prolog.discourse.group/t/byte-ordering-conversion-predicates/4243/8 "2021-08-04T03:43:37Z")
> [@dmchurch](#): > > binary-encoding of numeric data for interop with other systems @peter.ludemann, perhaps there is a use case here to factor our those routines from protobufs into a separate library. ---
**Author:** ![peter.ludemann](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/peter.ludemann/32/48_2.png) [@peter.ludemann](https://swi-prolog.discourse.group/u/peter.ludemann)\ **Post date:** [August 4, 2021, 3:55am UTC](https://swi-prolog.discourse.group/t/byte-ordering-conversion-predicates/4243/9 "2021-08-04T03:55:32Z")
> [@grossdan](#): > > perhaps there is a use case here to factor our those routines from protobufs into a separate library. Go for it. I’m afraid I won’t be able to do much for the next month or so. But before you do it, be sure that there’s a genuine need. 😉 ---
**Author:** ![grossdan](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/grossdan/32/23_2.png) [@grossdan](https://swi-prolog.discourse.group/u/grossdan)\ **Post date:** [August 4, 2021, 5:31am UTC](https://swi-prolog.discourse.group/t/byte-ordering-conversion-predicates/4243/10 "2021-08-04T05:31:35Z")
Just as my better half says – i am very good at suggesting ideas to others 🙂 ---
**Author:** ![EricGT](https://avatars.discourse-cdn.com/v4/letter/e/f1d935/32.png) [@EricGT](https://swi-prolog.discourse.group/u/EricGT)\ **Post date:** [August 4, 2021, 7:51am UTC](https://swi-prolog.discourse.group/t/byte-ordering-conversion-predicates/4243/11 "2021-08-04T07:51:43Z")
> [@dmchurch](#): > > More generally, is there anything for doing binary-encoding of numeric data for interop with other systems? AFAIK there is none as comprehensive as you seek. However for [bit field](https://en.wikipedia.org/wiki/Bit_field) see; > To see them listed as code use: ```prolog listing('$syspreds':map_bits/N). ``` Note: This is one of the rare cases where some of the clauses have two cuts in the same clause. * * * I don’t know if Jan W. would accept a PR adding more genialized such predicates in the boot level code but it does give one a general idea of what is needed to create a PR as a separate module. * * * Quite a while ago I did [post](https://swi-prolog.discourse.group/t/example-parse-of-binary-data-using-dcg/1370) some code that worked with streams of byte data and did some of these types of conversions. [binary.pl](https://swi-prolog.discourse.group/uploads/short-url/2qMLZeeMcgB5VXFaVTdbNtzbHrY.pl) (9.9 KB) ---
**Author:** ![jan](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/jan/32/4_2.png) [@jan](https://swi-prolog.discourse.group/u/jan)\ **Post date:** [August 4, 2021, 8:25am UTC](https://swi-prolog.discourse.group/t/byte-ordering-conversion-predicates/4243/12 "2021-08-04T08:25:50Z")
> [@EricGT](#): > > I don’t know of Jan W. would accept a PR adding more genialized such predicates in the boot level code but it does give one a general idea of what is needed to create a PR as a separate module. I’m not against a library with basic serialization helper predicates. In most cases I think one should write the low-level serialization operations for implementing some protocol in an external language. In some cases there are well maintained good external libraries on top of which you can build (in others that doesn’t make life much easier, so you just as well do it from scratch). If you write the low-level operations for a specific protocol in e.g., C, there is typically not much need for a separate Prolog library. Note that there exist quite a few ways to serialize basic data types such as integers, floats and strings. Most of these require just a few lines of C and, especially when it comes to handling strings, a low level implementation performs a lot better as we can directly map from a C `char*` to a Prolog atom or string rather than collecting the characters one by one in a list and use string\_codes/2, etc. ---
**Author:** ![peter.ludemann](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/peter.ludemann/32/48_2.png) [@peter.ludemann](https://swi-prolog.discourse.group/u/peter.ludemann)\ **Post date:** [September 5, 2021, 5:36pm UTC](https://swi-prolog.discourse.group/t/byte-ordering-conversion-predicates/4243/13 "2021-09-05T17:36:45Z")
There’s a related discussion for Python: [Mailman 3 C API for converting Python integers to/from bytes sequences - Python-ideas - python.org](https://mail.python.org/archives/list/python-ideas@python.org/thread/V2EKXMKSQV25BMRPMDH47IM2OYCLY2TF/#V2EKXMKSQV25BMRPMDH47IM2OYCLY2TF)