:-multifile and DCG rules being stubborn, predicate fails immediately

General
1

I decided to split my growing DCG rules (ast.pl) across one file per handler, I added the module search path but things are not working and I also have the error:

|    Warning: /Users/sean/Documents/code/prolog/the-f-word-project/src/ast/ast.pl:37:
Warning:    Local definition of ast:inst/4 overrides weak import from defun

I understand what causes this message but right now I am not sure what to do about it and feeling a little confused about just how to break out my big sprawling ast file into smaller more manageable files.

Ideally I want a single file for each syntax form, so one for defun, defvar, emit, blah blah. I currently have the following folder structure from the project root

src/
    ast/
        ast.pl
        inst/
            defun.pl, .plt
            defvar.pl, .plt
    tokeniser
    coder

I have this code in my load.pl which sets up the module paths (IIUIC) so that I can then import my modules and they get found:

:- multifile user:file_search_path/2.
:- prolog_load_context(directory, Dir),
    asserta(user:file_search_path(myapp, Dir)),
    format(atom(Src), '~s/src', [Dir]),
    asserta(user:file_search_path(myapp, Src)),
    format(atom(Ast), '~s/src/ast', [Dir]),
    asserta(user:file_search_path(myapp, Ast)),
    format(atom(Inst), '~s/src/ast/inst', [Dir]),
    asserta(user:file_search_path(myapp, Inst)),
    format(atom(Tok), '~s/src/tokeniser', [Dir]),
    asserta(user:file_search_path(myapp, Tok)),
    format(atom(Coder), '~s/src/coder', [Dir]),
    asserta(user:file_search_path(myapp, Coder)).

If there’s a shorter cleaner way I’d love to know by the way! :slight_smile: In my main ast.pl file I have included the module that contains the extracted code, now under a common predicate name of inst//2 for -instruction- as that’s what they are processing. My main ast file:

:- multifile inst//2.
:- use_module(myapp(defun)).

and in my defun.pl file which lives in src/ast/inst/defun.pl I have this at the top of the file:

:- module(myapp(defun)).
:- multifile inst//2.

inst( ..... ) --> ....code.,....

When I step through the code, at the point of calling it, it immediately fails, it doesn’t even seem to try calling into the imported module. When I execute listing(inst) I can see that the code in both files present:

:- multifile ast:inst/4.

:- multifile defun:inst/4.

defun:inst(defun(Pos),
    :
defun:inst('defun+'(Pos),
    :
defun:inst('defun#'(Pos),
    :
    : etc.

The call site is:

built_in_func(Out) -->
    [Term], {
        Term =.. [Fn, _Pos],
        reserved_word(Fn),
        bifmap(Fn, RealFn),
        debug(ast, "built_in_func: calling as: ~w", [inst(Term,Out)])
    },
    inst(Term, Out).  % <-- fails immediately
%    call(RealFn, Start, Out).

and the code I expected to be called is:

:- module(defun, [inst//2]).
:- multifile inst//2.
inst(defun(Pos), defun(Pos, pub(Name), Args, Body)) -->
    defun_(defun_function_name, Pos, Name, Args, Body).
:
: much more code...

This is the first time I have tried to split my code out like this but it means I can write tests in the same file now they are going to be small little units of brilliantly logically beautiful code… :expressionless:

So, what have I done wrong / not done in order to break out a large file into smaller files and use the module system?

Thank you in advance,
Sean.

2

OK, After posting, you always remember -something- you read somewhere… for me it was remembering the extended module:predicate calling format so for now, I have modified my code to look like this, using the functor name as the module name, which isn’t such a hardship and would actually enforce the rule of one module per reserved word handler which is fine. Heck, maybe this -is- the answer I was looking for ?

built_in_func(Out) -->
    [Term], {
        Term =.. [Fn, _Pos],
        reserved_word(Fn),
        bifmap(Fn, RealFn),
        debug(ast, "built_in_func: calling as: ~w", [inst(Term,Out)])
    },
    RealFn:inst(Term, Out).  % call individual handler! eg defun:inst()

This means at least that I can continue to break the code out into separate file in lieu of a more pertinent answer.

3

OK, well, I’ve stopped doing that pattern as the number of warnings is growing with each one I farm out:

|    Warning: /Users/sean/Documents/code/prolog/the-f-word-project/src/ast/ast.pl:37:
Warning:    Local definition of ast:inst/4 overrides weak import from assign
Warning: /Users/sean/Documents/code/prolog/the-f-word-project/src/ast/ast.pl:38:
Warning:    Local definition of ast:inst/4 overrides weak import from defun
Warning: /Users/sean/Documents/code/prolog/the-f-word-project/src/ast/ast.pl:39:
Warning:    Local definition of ast:inst/4 overrides weak import from defvar
Warning: /Users/sean/Documents/code/prolog/the-f-word-project/src/ast/ast.pl:40:
Warning:    Local definition of ast:inst/4 overrides weak import from emit

So what -is- the best way to break out a large file into smaller ones ?

4

Thanks again to @pmoura and he doesn’t even know it yet!
:slight_smile:

I dived in and found this article:
http://swi-prolog.996271.n3.nabble.com/About-quot-multifile-quot-td6839.html

The nugget of hope was shown and I now have a fully working solution, no warnings or other flies in the ointment and I can now happily break up my code. Here is my solution thanks due to that article:

in may MAIN ast file:

:- multifile ast:inst//2.
:- use_module(myapp(assign)).
:- use_module(myapp(defun)).
:- use_module(myapp(defvar)).
:- use_module(myapp(emit)).

and then in my other files, a few packed in here…

:- module(defun, [ast:inst//2]).
:- multifile ast:inst//2.

ast:inst(defun(Pos), defun(Pos, pub(Name), Args, Body)) -->
    defun_(defun_function_name, Pos, Name, Args, Body).

:- module(emit, [ast:inst//2]).
:- multifile ast:inst//2.

ast:inst(emit(Pos), emit(Pos, Terms)) -->
    all_single_or_sexp(Terms).
:
: etc.
:

As those great scholars Bill and Ted would say, “Excelleeeennnnt!”
As with APL, it does what you expect, you just have to know what to expect!

Thanks Paulo!

5

Paulo got too annoyed once and he left this list :frowning:

6

Noooooooo! Really?

7

Glad you got your across module predicates sorted. Regarding doing this in a cleaner way:

:- multifile user:file_search_path/2.
:- prolog_load_context(directory, Dir),
    asserta(user:file_search_path(myapp, Dir)),
    format(atom(Src), '~s/src', [Dir]),
    asserta(user:file_search_path(myapp, Src)),
    format(atom(Ast), '~s/src/ast', [Dir]),
    asserta(user:file_search_path(myapp, Ast)),
    format(atom(Inst), '~s/src/ast/inst', [Dir]),
    asserta(user:file_search_path(myapp, Inst)),
    format(atom(Tok), '~s/src/tokeniser', [Dir]),
    asserta(user:file_search_path(myapp, Tok)),
    format(atom(Coder), '~s/src/coder', [Dir]),
    asserta(user:file_search_path(myapp, Coder)).

You could do this instead:

:-  prolog_load_context(directory, Dir),
    asserta(user:file_search_path(myapp, Dir)),
    asserta(user:file_search_path(src, myapp(src))),
    asserta(user:file_search_path(ast, src(ast))),
    asserta(user:file_search_path(inst, ast(inst))),
    asserta(user:file_search_path(tokeniser, src(tokeniser))),
    asserta(user:file_search_path(coder, src(coder))).

Using this means you’ll need to be more explicit with your imports:

:- multifile ast:inst//2.
:- use_module(myapp(assign)).  % I can't see where this actually is
:- use_module(inst(defun)).
:- use_module(inst(defvar)).
:- use_module(myapp(emit)).  % I can't see where this actually is

This has the advantage that your code tells you about where the file is, so when you’re looking for it somewhere in myapp, you at least know what directory it’s in and you’re only a query away from getting the whole path to it, rather than searching.

1 Like
8

Thanks Paul, I’ll study that and decide if I go with it ot now…it does look a lot cleaner though at the cost of being clearer about imports as you say.

Thanks.

9

Better. Note that you only have to assert the one that you compute dynamically from prolog_load_context/2. The others can just be facts in the file. You can also dynamically add (say) myapp_root as dynamically computed alias and then do

user:file_search_path(myapp, myapp_root(src)).
user:file_search_path(myapp, myapp_root(src/ast)).
...

Does it make much sense to use a directory structure and then smash these directories to the same alias in Prolog, creating a flat namespace?

I also have my doubt that multifile is a nice solution. If you can avoid it, that is typically a good idea.

2 Likes
10

I don’t like multifile/2 either it feels somehow kludgy but I can’t think of another way, being something of a newcomer to the environment.

Is there some glaringly obvious feature I have missed?

I’ve spent a (comparatively) long time for me learning Prolog thus far but I have not really picked up too much about the SWI environment. I had a heck of a time when trying some of the http server stuff for instance.

I will study your comment in depth regarding expressing the search path as facts, for whatever reason I sometimes find the official docs either confusing or unclear to my beginner and it puts me at odds with what I am trying to do until I finally get it, usually after much gnashing of teeth etc.

Thanks.