Not sure I get this. If the public API is called with BUF_MALLOC it should make sure no strings are “stacked”. Note that with the old “ring” mechanism this was not an issue, which is probably why these issues arose. How is another issue. Ideally, if the BUF_MALLOC is present, the last action that modifies the string should do the malloc().
For example, PL_cvt_i_schar() calls _PL_cvt_i_char(), which calls PL_get_text(). PL_get_text() has a call to findBuffer(BUF_STACK), so PL_get_text() assumes it’s inside a mark-string-buffers?
Similarly, PL_get_nchars() calls PL_get_text(), which can call findBuffer(BUF_STACK).
(It also calls PL_save_text() in a different code path; where there’s a call to findBuffer(BUF_STACK) if text->storage isPL_CHARS_LOCAL and flags are not BUF_MALLOC.)
So, my interpretation (which might be wrong) is that there are a number of code paths that can use the string-buffer stack even if the flags don’t specify BUF_STACK.
Yes. Despite the name it is a private function though, so it is up to the caller to deal with this.
All in all, it seems the introduction of the string stacking rather than using a ring has some more consequences than expected 
Now, it is not as I feared as calls to foreign predicates automatically clears stacked strings created during its execution. So, the problem is limited to foreign predicates creating a lot of strings and foreign code using these predicates while not called from Prolog (i.e., the main is foreign).
We probably should add a version of the string mark/release macros that checks the flags to include BUF_MALLOC and be a no-op otherwise. That way we can keep the code fairly clean.
As an interim measure, should I add calls to PL_mark_string_buffers() and PL_release_string_buffers_from_mark() in all the C++ functions that return a string? (Using PlStringBuffers, so it’s easy to do; and looking at the C code, it doesn’t seem to add much overhead.)
I’m typically more in favour of fixing the bug where it is rather than working around it elsewhere. The overhead is mostly getting the thread specific engine twice
It’s not that urgent considering that normal Prolog execution is not affected. If mostly affects getting strings as UTF-8 that are not ASCII and getting wchar_t* strings that are in fact ISO Latin 1. Both, eiher many of them inside a single foreign call or not called from Prolog.
Pushed a fix (1beb59ebea5e802869f0e4a4668d122375f127c2). Not sure this covers all suspect functions. Using a bit more modern coding style the code also got simpler. One keeps learning 
Do these bug fixes take care of BUF_DISCARDABLE? (That is, should the test be for ~BUF_STACK rather than for BUF_MALLOC?)
Which test? Both BUF_STACK and BUF_DISCARDABLE may use the stack, though BUF_DISCARDABLE avoids it in some cases. The user gets a pointer to a stacked buffer or a direct pointer into Prolog memory. Using BUF_STACK, it is safe to make other calls to the API while using BUF_DISCARDABLE it is not.
BUF_MALLOC should ensure all is clean in the end except for the allocated end result.
That’s what wasn’t clear to me. I’ll take another look at the documentation and update it if necessary. (And also review the C++ API implementation.)
1 Like