The `public` predicate property

Help! Predicate
1

According to the manual the public predicate directive (SWI-Prolog -- Manual) “has no semantics.” I’m not sure that’s really the case because whenever I reload a module containing user defined arithmetic functions I get a slew of warnings like:

Warning: /Users/ ... /functionalarithmetic/dot_arithmetic.pl:145:
Warning:    Local definition of user:fill/3 overrides weak import from dot_arithmetic

one for each user defined arithmetic function. Each function predicate, e.g., fill/3 is exported by the module.

In any of my testing, these warnings don’t seem to have any material impact. They seem to be caused when the arithmetic_function directive is expanded causing a :- public(Pred). to be executed. As a test, I’ve removed this directive from the expansion and the warnings disappear and no ill effects have yet to be noticed.

So what does this directive actually do and will removing it from the expansion of arithmetic_function have any ill effects.

2

It creates the predicate and sets a flag. The flag is only used by a couple of tools. It causes the cross-referencer not to flag the predicate as “not called” and it causes the code obfuscating library not to rename the predicate.

It tells the system that is predicate is called from somewhere, typically either through some callback that cannot be understood by the tools or from a foreign C routine.

If you get this message it may mean the public directive is associated with the predicate in the wrong module and probably some similar situations. It tells you dot_arithmetic exports fill/3 that is somehow also defined in user and imported from dot_arithmetic using use_module/1.

3

So the only configuration that I can get working is when the arithmetic function is defined in user space and the predicate is exported, e.g., something like:

:- module(dot_arithmetic, [fill/3]).

:- arithmetic_function(user:fill/2).

fill(Obj, N, Result) :. ...

But that produces the warning message when the module is reloaded (not on original consult). AFAIK there are no use_module calls involved. Could there be some lingering piece of info left from the previous load?

As I said, there doesn’t seem to be an operational impact, just the confusing warning.

4

The library was a hack … The proper way would be to use

:- arithmetic_function(fill/2).

And modify the expansion logic such that if fill/2 is imported into the current context fill/2 must be rewritten. That would make the library act properly on the module system. Feel free to submit a PR that makes this work.

5

A bit of a learning curve, but very pleased with the final result. All warning messages are gone and I was able to remove a meta-predicate declaration for arithmetic_expression_evaluation which was complicating using : as a function, e.g., for a slice operation. A few other things were also simplified in the process.

I’ll submit a PR in the next couple of days with my changes after updating the test suite. One thing I noticed is that the current test for flag/3 is failing, presumably because arithmetic expression in the third argument isn’t being expanded. Should that be fixed as part of this exercise? Any other such cases come to mind?

6

Sounds promising!

That sounds scary. It is a meta predicate. We’ll see …

For sure the test suite needs to remain passing. In extreme cases that may mean change the test. flag/3 must remain capable of processing arithmetic expressions. I don’t think I care too much about user defined functions. It is mostly used for flag(f, N, N+1) to have a rather efficient atomic global counter.

7

Admittedly my understanding of the requirement for the meta-predicate directive is a but fuzzy. In this particular case the call in arithmetic_expression_value is always to a fully module qualified predicate, so there seems to be no reason the expression argument needs to be defined as “meta”. In fact the requirement to derive the “imports from” information, seems to mean there’s no need for the arithmetic_function directive to be a meta-predicate either. Seems to all work, but maybe there’s a use case I’m missing.

If you mean the existing test_arithfunc.pl, I can’t believe it’s been working for some time now. It claims to be using module arithfunc, although it’s probably getting arithmetic via autoloading. And some of the tests are incompatible with plunit. Is there some other test suite that is involved here?

In my experience using standard global variables (thread based) and doing your own arithmetic is much more efficient:

?- set_flag(x,0),time((between(1,1000000,_),flag(x,N,N+1),fail));get_flag(x,N).
% 6,000,001 inferences, 2.218 CPU in 2.689 seconds (82% CPU, 2705633 Lips)
N = 1000000.

?- nb_setval(x,0),time((between(1,1000000,_),nb_getval(x,X),X1 is X+1,nb_setval(x,X1),fail));nb_getval(x,N).
% 6,000,002 inferences, 0.300 CPU in 0.300 seconds (100% CPU, 20014551 Lips)
N = 1000000.

Perhaps it’s for thread safety, but flag seems to be extraordinarily slow for what it does.

For efficiency, it would be nice to have something similar using global vars. Even a hard coded increment by N for operational measurements would be nice. (OM’s in clpBNR eat up about 4% of the the constraint engine execution time.)

8

It was indeed very old and not functional as the test was compiled conditionally on a no longer true condition. Thanks. Updated and pushed.

? For me the difference is 0.593 sec vs. 0.276 sec. That is on a standard compiled system on Ubuntu 20.04. The main difference is that flags are shared between threads though and global variables are not.

9

OK. I’ll add my tests to this version in the PR.

Now that’s interesting - clearly an OS library dependency. Also note the CPU percentage - MacOS version is definitely off in OS land for a significant amount of time. But still a factor of 2 on Ubuntu; hopefully users will know the difference between process globals and per-thread globals.

Per-thread operational measurements are exactly what I need, so it’s a moot point. But I will fix arithmetic to expand the flag predicate if needed; it’s a trivial addition.

Just curious, is the arithmetic expression in flag subject to optimization?

10

Wouldn’t bother. I’ve removed the test that verifies this used to work when there were real user defined functions.

flag/3 is defined in Prolog (?- edit(flag).).

flag(Name, Old, New) :-
    Old == New,
    !,
    get_flag(Name, Old).
flag(Name, Old, New) :-
    with_mutex('$flag', update_flag(Name, Old, New)).

update_flag(Name, Old, New) :-
    get_flag(Name, Old),
    (   atom(New)
    ->  set_flag(Name, New)
    ;   Value is New,
        set_flag(Name, Value)
    ).

Flags are an old legacy thing of SWI-Prolog. They should be replaced by something better. Well, for most usage there are better alternatives.

11

Because of the atomic update, this wasn’t as trivial as I first thought. So not going to happen for now.

So the answer is no (Value is New). For efficiency reasons that really seals the deal for per-thread globals if that’s all you need. I do have to wonder why mutex’s are that much slower on MacOS. From my 2009 MacPro:

?- time((between(1,1000000,_),Value is 1+1,fail)).
% 2,000,001 inferences, 0.179 CPU in 0.179 seconds (100% CPU, 11154806 Lips)
false.

?- time((between(1,1000000,_),with_mutex('$flag',Value is 1+1),fail)).
% 3,000,001 inferences, 2.288 CPU in 2.862 seconds (80% CPU, 1311235 Lips)
false.

Luckily, not a feature I need.

1 Like
12

On Ubuntu, compiled with PGO (Profile Guided Optimization), the difference gets a little bigger as the non-mutex implementation gets faster, but still way below MacOS. Ubuntu 20.04, gcc 9.3.0, AMD 3950X:

101 ?- time((between(1,1000000,_),Value is 1+1,fail)).
% 2,000,003 inferences, 0.100 CPU in 0.100 seconds (100% CPU, 20060335 Lips)
false.

102 ?- time((between(1,1000000,_),with_mutex('$flag',Value is 1+1),fail)).
% 3,000,001 inferences, 0.267 CPU in 0.267 seconds (100% CPU, 11256823 Lips)
false.

Which version do you use on MacOS these days? The app bundle (compiled using gcc 10 with PGO), Macports? Homebrew? The two latter ones both use MacOS Xcode clang and are a lot slower …

13

MacOS 10.13.6 (High Sierra), bundle version of 8.3.18

I’ve remained on High Sierra due to the age of my MacPro (2009, 2.26Ghz Xeon), but it’s very stable and I think general performance is acceptable considering its age. (But my fanless laptop is around 25% faster single threaded.)

Re mutex, the other thing I mentioned is is the CPU utilization. Whereas this little benchmark CPU is 100% on Ubuntu, it’s consistently only around 80% on MacOS. To me this suggests there’s quite a different mutex design strategy on MacOS; it isn’t just a minor optimization issue. This is all a little surprising; mutex is hardly a new concept.

P.S. Interesting how a topic on public predicate directive evolved into a discussion of OS kernel functionality.