A general `-Dflag=value` commandline option

Request For Comments
1

As all new features in the devel tree, although already available in the GIT version, there is room for changes …

I’ve pushed a patch that makes swipl handle -Dflag=value (or -D flag=value) as a generic mechanism to set Prolog flags from the commandline. This provides a generic way to specify modes of operation or configure (user) code. As is, the flags are applied after the initial saved state is loaded.

Many of the existing commandline options also (indirectly) manages a Prolog flag, some of them read-only flags as they can only be modified in the early boot phase. The plan is to make the -Dflag=value a synonym, so swipl --no-threads and swipl -Dthreads=false are synonyms. These are pretty similar as is. swipl -Dtraditional would become synonym to swipl --traditional. This would be new as a runtime switch between traditional and native mode is not possible.

Comments? Connection to other current practices?

2

This looks like a nice improvement that could make some command lines more concise. OTOH, there’s some cost to adding a new notation as it makes the decision of which notation to use at a given case a bit more complex, especially if the functionalities overlap. So I wonder if there’s a concrete distinction/use case for favoring this notation over the current alternatives:

  • -g "create_prolog_flag(foobar,true,[access(read_only),type(boolean)])" for introducing new flags
  • -g "set_prolog_flag(foobar,true)" for existing flags
  • dedicated command line arguments for special flags that must be set early during Prolog initialization (e.g. --(no-)threads)
1 Like
3

Good question. I’m not a big fan of having to rely on -g command. It makes the commands long and quoting gets hard, in particular when part of the command comes from variables in the shell script (you rarely want to type this stuff into an interactive shell) and multiple layers of scripts with their own quote rules are involved. So, -Dflag=value is typically a lot more user friendly. It is also consistent with what several other applications do. Think about (C) compilers, cmake, make (although make uses simply name=value) and more.

Second is when to apply the flag. Using -g ... is applied after loading all code ((user) init scripts, command-line Prolog files, etc). That is needed as it is intended to run some goal using the code you just loaded. That is too late for affecting the loading process.

The dedicated commandline arguments are processed “when needed”. We can do the same for the -D flags that have a well defined meaning (and I think we should). An advantage of the dedicated commandline arguments is that we can easily document them in the usage summary. On the other hand, using flags from the commandline we get more control and we only have to document the flags rather than the commandline options and the flags.

Another issue is that undefined commandline options lead to an error. Undefined flags (currently) just create the flag as an r/w flag where the type is derived from the value. I’m unsure whether or not that is a good idea. Note that, unlike ISO, SWI-Prolog allows adding flags at runtime. I think that is good as it allows libraries to add flags (as our apply_macros library now does). In the ISO case each library has to invent its own way to manage settings, deal with the interaction with threads, etc.

Maybe we should be more helpful. One could be to only allow setting new flags using a notation

swipl -Dflag=type:value

This may be ambiguous, so possibly this?

swipl -Dflag:type=value

But, we don’t always know when the flag is defined. Using -D on a flag that is introduced in a library that is loaded later is fine IMO.

Another trick to inform the user could be to copy cmake’s behavior and warn if a flag is set, but never queried. So, you get

swipl -Dmyflag=42 ...
<normal output>
WARNING: user flag "myflag" was set but never used.

That should be fairly easy to implement. Both the code using current_prolog_flag/2 on this flag would silence this message as well as a call to create_prolog_flag/3 on this flag.

Slightly related to this is that the typing of flags is weak. Currently, the options are boolean, integer, float, atom or term. Notably atom should probably allow to enumerate the set of admissible atoms. Numbers might have a range.

1 Like
4

Right, that makes a lot of sense. These two distinctions (more robust quoting story and time of application) are good reasons to support this new notation IMO.

Thanks!

5

We might want more fine-grained control over when the flags are applied. For example, I might want optimisation when loading the source, but also want to run unit tests (currently the optimisation flag turns off unit tests). This could be difficult to define, because Prolog blurs the distinction between compile time and run time.
We might also want to have sub-flags or combination flags (e.g., a general optimise flag and also optimise/apply, optimise/yall, no-optimise/plunit … the / is to make command line flags easier, otherwise I’d suggest optimise(yall) etc.).

[Sent from my cell phone while travelling, please excuse typos]

6

Do you not mean where here? When refers to time AFAIK (forgive my command over English if this is wrong). The When for -D flags should be defined as “as early as needed”, which in practice implies right after parsing the commandline options for a handful and after loading the state (as now for all of them) for the others. Loading the state itself is not affected by any flags (I think) and all initialization/1 directives are executed after the load completes.

As is, I’m reluctant to turn flag names into terms. That would seriously impact (and complicate) the administration and lookup. Note that many of them are accessed in performance critical parts of the core C code.

We now have flags like apply_macros, so to optimize but not rewrite the apply family, one can use (eventually also -Doptimise)

swipl -Dapply_macros=false -O ...

Similarly, we can enable testing and optimize using

swipl -Dplunit_load=always -O ...

The disadvantage seems lack of uniformity. On the other hand, plunit is not “optimized or not”, but is “loaded or not”. Another example from this family are the debug/3 and assertion/1 statements that are normally removed when optimizing, but can be kept using swipl -Doptimise_debug[=true]. Oops, inverted meaning of your intended sub-flags :frowning:

So, in your view we should renamed apply_macros to optimise_apply (makes sense and as the flag is just introduced, this can still be done). optimise_debug should invert its meaning (not happy about the change, but I guess acceptable for the sake of consistency) and there could be an optimise_tests that can affect the default for plunit_load? I think these are all. Given that, we can invent new flags in the same spirit.

As is, -Dflag (no =value) is a shorthand for -Dflag=true. Should we also have -Dno-flag to be a shorthand for -Dflag=false?

7

I was thinking about whether the flag is applied when loading the code (“compiling”) or when running (e.g., can unit tests run). But this might not make sense. I got a bit lost when reading the plunit code (what does “loading” mean? - it seems to be whether some term expansion is done?).

optimise_apply could take the values true or false, so no need for debug_apply (is that what you meant by optimise_debug?)

-Dflag=no, Dflag=false, -Dnoflag, -Dno-flag would all mean the same (and -Dflag, -Dflag=yes, -Dflag=true would mean the same). I’ve seen and used variants of these; it’s annoying to remember whether it’s no or no-, so it’s best if both are allowed. (Similarly, should it be optimise_apply or optimise-apply? :slight_smile:

I don’t think I’ve ever seen 100% consistency in flags, probably because every system adds flags until there seem to be too many, and by then it’s too late to make them consistent.

8

Typically flags are designed to affect either compilation or running. I think this is not a big issue. But, if you insist on the commandline, -D acts as early as needed and -g set_prolog_flag(...) acts after loading.

No. The optimise_debug flag controls whether debug/3 and assertion/1 calls are replaced by true (and if the compiler uses optimise, be removed completely). The optimise_debug flag uses three values: default, false and true, where the initial default means use the value of the optimise flag. I now implemented this schema for several of the optimise_ sub-flags: initialize them as default. In that case the library decides how to interpret that.

For now, we have optimise_apply, optimize_debug and optimize_clpfd, where the first two default to the optimise flag and the latter defaults to true.

That is how the boolean commandline options are interpreted. Here, we refer to Prolog flags and these normally are atoms that do not require quotes. So, I think (and just added) that -Dno-flag is ok to mean set_prolog_flag(flag,false). -Dnoflag is getting ambiguous though as noflag may well be an existing flag. For the flag values, this is now left to set_prolog_flag/2, which accepts false/off or true/on (only). As you typically omit the = for Boolean flags, this might be enough?

I like this quote :slight_smile: Slowly getting a bit more consistency :slight_smile:

9

This is a nice improvement to the design.