It might be a good idea to look at Python click Welcome to Click — Click Documentation (8.0.x) to make sure that with the new api the user could implement all the different use cases and features that python click supports.
I don’t think we need to provide all those features directly (prolog is much more flexbible), but rather to use the listed features/options of python click as a check-list of use cases to make sure that the api doesn’t prevent implementing any of them.
Something is happening with replies lately (are you @swi in particular posting through email?) Instead of posted as part of the original thread they come as a new thread with a title [Original thread] - reply [N]. It isn’t this post only it has been happening to others too.
This is just a comment (and no longer meta): the library that Jan is trying to replace, library(optparse), is indeed an attempt at porting optparse from Python, and it seems that Click itself is based on optparse. It might be that having less use cases and features is a good thing.
Being a simple programmer, I personally failed to grasp library(optparse) once and ended up finding somewhere in the source (almost by accident) the precursor to argv_options. Features don’t have to be in direct conflict with simplicity but sometimes they are…
Offtopic rant (please stop reading now!), I have been noticing that as Python has been defacto standard for “backend development” for some years now, and with the influx of programmers from all kinds of backgrounds into Python, the Python ecosystem is becoming quite byzantine.
Thanks for your thoughts. Just reading the table of contents of “click” makes me think their aims are different from mine. I mostly wanted something simple, lightweight and not as slow as optparse. The complexity of optparse first caused me to add argv_options/3 as I found myself often in situations where I just needed some basic option parsing and doing it all from the ground up every time is a bit too tedious. argv_options/3 was a bit too simple though. No short options, no help and not even very basic checking for existence of options or argument types. So I wanted something that added these with as little as possible complexity for the user.
One more thing I wanted was the ability for plugins to add additional options. That works in a clean way now.
From the click intro we get stuff we do not have. Below is a more elaborate list of things you may want, but is not provided.
For all this stuff, we could define option attributes that handle these. You make a choice between what we have now (nothing) to a very complex specification language that can deal with most cases. Right now the user needs to do the checking and adding derived (default) options after argv_options/3. Is that bad? Yes and no IMO. The code tends look a bit ugly. Suppose that if x is given y defaults to true. We get ugly stuff like below and something on top that calls a sequence of similar “fixers” and checks.
fix_y(Options0, Options) :-
option(x(_), Options0),
\+ option(y(_), Options0),
!,
Options = [y(true)|Options0].
fix_y(Options0, Options).
We might want something more logic here. Something like
y(true) :- x(_), \+ y(_).
And some s(CASP) like constraint notation, e.g. this to define that you cannot ask for both the html and json options to be true.
false :- html(true), json(true).
Something along these lines could deal with prompts, defaults and much more. This could be implemented in a separate library to keep things lightweight. It requires a comfortable notation syntax, a forward reasoner and some ways to turn at least the common constraints into messages (nearly) automatically. If someone can design and implement such a beast, please contribute it!
This is more along the lines of what I was thinking (I don’t see anything about plugins in the docs though?).
A possibility is a hook to process options; the hook could allow users to handle all those cases you mentioned, like defaults, required, inter-option constraints. All these are better left to a user definable hook since they can change in multiple ways.
I agree with keeping the api simple.
EDIT: does it currently handle multi-line help and wrapping long lines?
I think this happens with posts of type “Announce”, I don’t really see much benefit.
There is a note. The spec being a called (hook) predicate means you can easily distribute the rules over multiple files. This is more flexible than a huge data structure. It is typically also much easier to find syntax errors in small clauses as opposed to large data structures, in particularly if these involve long strings.
I think this is not the right way though as it makes it hard to deal with relations and constraints between options. I think the way would be to have a nice set of declarative rules that post process the options, adding derived options, defaults (if desirable) and find conflicts (constraints). That exploits the power of Prolog rather than trying to mimic the complicated procedural way others need to deal with this 
Yes. It automatically computes the with for the option columns, formats help texts and adjusts to the console width if known (assuming 80 columns when unknown). It also uses colors, respecting the theming.
This was once introduced after endless discussions as a reply to new releases. Many of these responses are adequate though and there is no clear boundary between announcements for early work, RFC and discussion 
We changed the Announce category almost a year ago (ref) to not allow replies. It is also noted in the about for the Announcement category
Note : Use of this category will not allow for replies to the announcement. The intent being that each person should start a separate topic as needed to avoid a cacophony of divergent replies.
The reason for the change was an announcement was made and turned into several off-topic discussions that were a distraction for the OP and others. As such the admins here decided to clean up the category and not allow replies. Now each reply is flagged for moderator intervention and not automatically posted.
If you notice the Announcement category has no replies.
If I am not busy I will let the OP of the reply know why the change was made with a personal message.
If you are releasing new or updated software and would like the topic to receive replies then use the category Releases which does allow replies.
Great, I see it now.
Makes a lot of sense; it could be make generic enough that combining a set of basic (logical) rules can cover all the cases.
Great!
This is good to know, that is the best option. The poster chooses Releases if he wants replies or Announcement if not, nice and clean. Thanks!
A few generic comments on options, based on my experience over the years.
It’s possible to have hundreds of options if you’re using a lot of imported code whose behavior can be tweaked (I saw this at Google, where we had to modify the Linux kernel to allow longer command lines).
It’s convenient for each imported module to be able to specify how it’s tweaked using some kind of “global” data structure. Prolog’s multifile/1 is an obvious way of doing this; opt_parse/5 has an opposite approach that requires specifying all the options in a single place. (And there needs to be a way of detecting parameter name clashes between modules.)
Keywords are used for more than just optional parameters (despite what the author of library(optparse) says); if there are multiple required parameters, it’s helpful to be able to specify them with keywords to avoid confusion (e.g., I can never remember the order of the parameters to the Unix ln -s command).
Python’s way of doing command lines doesn’t scale. Google’s approach (which has its own warts) does scale: abseil / Flags abseil / The Abseil Flags Library
Thanks. Seems we are pretty much in good shape. There is (as yet) no check for duplicate flag definitions. It is also not completely obvious this is desirable. Multiple plugins may define the same flag. There is only a problem if two plugins define the same flag in some incompatible way (notably with different types).
Scaling should be pretty much ok. Clause indexing should be enough 
It is not entirely clear how to support tools like git that use
git [options] command [options] argument ...
The best option is probably to have something that allows to stop parsing options at the first positional argument. Then the first call returns the first set of options and as positional arguments the command and its options. We can then check the command and call the command’s entry point which can call argv_options/3 on the remainder. We can add that to argv_options/4.
edit added an option options_after_arguments(false) to make this possible. I think this type of option processing now works nice enough. It requires some work, but setting up an application with plugins requires some wok anyway.
In using opt_help(usage,"p version 1.0 \nUsage: p [options]"):
#!/usr/bin/env swipl
:- initialization(main, main).
main(Argv) :-
argv_options(Argv,_Pos,_Opts).
opt_type(s,stats,boolean).
opt_type(statistics,stats,boolean).
opt_type(v,verbosity,boolean).
opt_help(stats,"Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nunc at purus a elit mattis tempus. Morbi ligula ante, dapibus vestibulum posuere ac, luctus vel metus. Quisque aliquam justo vitae eleifend pretium. Quisque dignissim metus laoreet convallis dignissim. Curabitur varius enim quis justo hendrerit faucibus. Suspendisse potenti. \n \n Nullam vehicula aliquet consequat. Duis nec molestie enim. Etiam volutpat felis ante, quis sollicitudin orci tristique a. Suspendisse scelerisque purus consequat purus placerat bibendum. Vestibulum elementum aliquet feugiat. Ut in iaculis urna. Sed molestie orci urna, eu convallis purus faucibus cursus. ").
opt_help(verbosity,"Increase verbosity").
opt_help(usage,"p version 1.0 \nUsage: p [options]").
argv_options/3 prints the following:
Usage: swipl /tmp/p.pl p version 1.0
Usage: p [options]
Options:
-h, -?, --help Show this help message and exit
-s, --statistics Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nunc at purus a elit mattis tempus.
Morbi ligula ante, dapibus vestibulum posuere ac, luctus vel metus. Quisque aliquam justo
vitae eleifend pretium. Quisque dignissim metus laoreet convallis dignissim. Curabitur varius
enim quis justo hendrerit faucibus. Suspendisse potenti. Nullam vehicula aliquet consequat.
Duis nec molestie enim. Etiam volutpat felis ante, quis sollicitudin orci tristique a.
Suspendisse scelerisque purus consequat purus placerat bibendum. Vestibulum elementum aliquet
feugiat. Ut in iaculis urna. Sed molestie orci urna, eu convallis purus faucibus cursus.
-v Increase verbosity
In my opinion, I think it would be more useful if opt_help(usage,...) would completely override the string printed before the options.
Thanks for your great work!
EDIT: Also, is there a way to override the -h option and specify an alternative option for help? e.g. a program where you always specify a host may want to use -h <host> and only use --help for help (or maybe some other short option also).
EDIT 2: Perhaps opt_type(Something,help,boolean) can be used for changing the help option? and maybe even opt_help(help,Something) if the user wants to completely override what is printed?
I’m not sure about that. Notably the default line figures out how the program was started, i.e., as swipl script.pl ... or from a saved program created from script.pl. So, you get either
Usage: swipl script.pl [options] ...Usage: myprog [options] ...It would be nice to be able to insert text at various places in the help output. I guess at the top, between the usage line and the options and at the bottom? What about
opt_help(help(header), "....").
opt_help(help(usage), "....").
opt_help(help(footer), "....").
Using a term avoids conflicts with options. We now also allow the content of these to be a list of line elements such that you can use ansi(Style, Format, Args). Pushed that and use it in s(CASP).
edit also implemented your opt_type(_, help, boolean) proposal.
I think this is very nice, as a default. But I think there should be an option to override the program name.
The reason is that you may have your script in a very long path: /nfsmount/myclustermachine/myuserid/bin/scripts/p.pl that you don’t want printed or you have symbolic or hard links by which the program behaves differently (git used to do this, don’t know if still does it). Do you want to print the hard-link/symlink name or the source file it references? Do you want the whole path printed or not?
I think an option like opt_help(help(program),Something) to override the program name would solve this, what do you think?
Nice! This is much better than what I thought originally.
I’m not yet convinced. You know, less is better … unless more is better or course As is, it tries to be smart, getting the way you invoked the program. If that is as an absolute path because you’ve got multiple of these things or it is not in your PATH, you get a long name. That could be a good idea.
The hard coded thing surely has the disadvantage that it may be simply incorrect. That wouldn’t be the first and probably won’t be the last time. The automatic is correct (I hope, there are quite a few ways to invoke Prolog). Most cases it will also look nice and short.
hehe you are a hard one to convince 
Well, you managed with quite a few aspects of the new option parser Thanks!
By the way, I think it may be a good idea to include the new argv_options/3 in 8.4.1 stable. It is much better for anyone handling command line arguments.
Yes. That was also my thinking. Unless someone defines opt_type/3 it should not have any compatibility consequences. It is already in the stable git. There you can see what will be in 8.4.1. I do the cherry picking a bit irregularly. Typically once or twice a week in this stage of development.