Hello everybody, Some times ago, I’ve stumbled on the QOI image format . The interesting thing about this image format is that you can read the specification in about 5 min and write a decoder in about the same time ^^ It is also a format that is optimized to be very fast while getting a quite ok …

Sounds great. I’m no expert here but- when you say you profiled the code- did you check IO performance? I have a hunch that this is the bottleneck.

In my experience, code like this are difficult to improve by profiling. The protobufs library has some low-level predicates for things like converting 4 bytes into an integer (e.g., protobufs:int32_codes/2, used in payload//2). I don’t know how much this speeds things up; I inherited that code and …

[image] peter.ludemann: Also, I think you could spread things up by changing qoi_decode/2 to use read_file_to_codes/3 and phrase/2 . I second this. Without any further setup, phrase_from_stream/2 will buffer at most 4096 bytes. That means a lot of tiny reads are going to happen over the course…

thanks for all your reply ! [image] burlingtonbertie99: did you check IO performance? So, going on a hunch, I thought that prolog arithmetics would be so slow that I did not thought that IO would ever be a bottleneck. I’ll report back the influence of the IO on parsing speed. [image] pe…

About if-then-else vs auxiliary predicate with first-argument indexing - the trade-offs are similar to those between if-then-else and switch in C. There is one more thing: the call to the auxiliary predicate requires shifting the parameters, so that overhead might make things worse. Sometimes it he…

[image] maren: I second this. Without any further setup, phrase_from_stream/2 will buffer at most 4096 bytes. That means a lot of tiny reads are going to happen over the course of parsing this. Much better to just read in the whole thing at once. Do you have evidence? In my experience the r…

[image] jan: Do you have evidence? Intuition based on io in other languages. Which turns out to be wrong here. I just tried it out with a simple DCG that just reads the entire file 1 code at the time, doing nothing else. If anything, the larger file buffer makes things slightly slower. Apolo…

[image] peter.ludemann: Also, I think you could spread things up by changing qoi_decode/2 to use read_file_to_codes/3 and phrase/2 . Also this turns out to actually be the worst option. It’s extremely memory hungry, which in turn (I think) requires lots of garbage collection to happen during e…

No, phrase_from_file/2 is the fastest. prereading the file into a string, then opening that string as a stream and phrase_from_stream/2 is slightly slower. Slowest of all is read_file_to_codes/3 followed by phrase/2.

Quite Ok Image (QOI) format challenge

General

peter.ludemann August 28, 2024, 5:06pm 10

So, read_file_to_string, open_string, phrase_from_stream is faster than phrase_from_file? That surprised me, but shows how unintuitive it can be to speed up code.

Topic		Replies	Views
Pure QOI (Quite Ok Image) format implementation General	30	311	September 14, 2024
Read_stream_to_codes Predicate	15	1628	March 25, 2020
Using DCGs for reading and writing in shell "pipes" Help!	6	981	December 28, 2019
Phrase_from_file vs phrase Help!	29	2977	September 14, 2020
A tokeniser I've written. Any suggestions on how to improve it? Help!	28	3073	April 3, 2019

Quite Ok Image (QOI) format challenge

Related topics