This is an interesting topic. I hope it is OK to ask a few questions.
If I understand correctly, the topic is term/goal expansion, code re-writing, meta predicates, and so on. The discussion is both on the level of “why do we need it?” as well as “how do we implement it?” Is that correct?
There are two seemingly conflicting opinions in circulation, and I am summarizing them as I have understood them (so probably I am wrong):
- Code re-writing in general points to a weakness in the language or the implementation.
- Code that writes (code that writes (code …)) is the ultimate weapon of the programmer and it is a highly desirable property of a system to support that.
Are those two statements relevant to the discussion? Are they really in conflict? Is there any resolution in sight?
I will now list a few known examples of the need for inlined predicates, code rewriting and so on. Can you explain what would be you desired final, perfect solution for each case?
- “Anonymous”, one-off predicates, or “lambdas” as in library(yall);
- Meta-predicates in general, as in “algorithm as argument to an algorithm”;
- Meta-predicates as re-written by library(apply_macros);
- DCGs, dicts: both seem to be cases of “syntax sugar”, but writing code without them would be just silly;
- Re-writing for pragmatic reasons.
Is the last one the more general case of DCGs and dicts as well as library(apply_macros)?
Other examples would be using term expansion instead of memberchk/2 (at the bottom). Another related example would be expanding the definition of a DFA to code that implements the DFA (where predicates are states and predicate calls are transitions), instead of interpreting it. The common in all examples is that the expanded code behaves better (determinism, backtracking, efficiency…) while the interface (what the programmer has to write) stays sane.
Usability: the only important question for me personally is: if I use mechanism X (for any X), can the compiler/IDE still help me out? Spot errors? Provide useful warnings?
(The last paragraph above comes from personal experience. In the few code bases I have seen, in every single one, there is a complex code generation/transformation lurking somewhere in the depths, and it provides 90% of the business value. It also has to be regularly “improved”.)