Difference between ls/1 and use_module/1 when finding a module

General
1

After updating to the latest daily build for windows I get an error when I load a module using an explicit path, like “lib/module.pl” but not when I use a user-set file search path like “lib(‘module.pl’)”. At the same time, ls/1 can find the file with both paths.

Here’s what the whole setup looks like on disk:

.
│   config.pl
│   load.pl
│
├───lib
│       target_module.pl
│
└───src
        module_path.pl

Here’s what the error looks like. Below, the module_path.pl module is trying to load the lib/target_module.pl module, taking its path from a predicate in the config.pl module. It first calls ls/1 with the same path:

% lib/target_module.pl   
ERROR: c:/module_paths/src/module_path.pl:5:
ERROR:    source_sink `lib/'target_module.pl'' does not exist
Warning: c:/module_paths/src/module_path.pl:5:
Warning:    Goal (directive) failed: module_path:(config:target_path(_14496),ls(_14496),use_module(_14496))
Welcome to SWI-Prolog (threaded, 64 bits, version 9.3.8-27-g80f956be2)
SWI-Prolog comes with ABSOLUTELY NO WARRANTY. This is free software.
Please run ?- license. for legal details.

Note that the first line is output from ls/1 and it can see the target path.

I think use_module/1 raises an error because there is no lib/target_module.pl path under the same directory as module_path.pl - but then why does ls/1 not raise a similar error?

The example is a little convoluted- there may be an easier way to show the same errors.

Here are the file contents:

% load.pl
:-prolog_load_context(directory, Dir)
,asserta(user:file_search_path(project_root, Dir)).

user:file_search_path(src, project_root(src)).
user:file_search_path(lib, project_root(lib)).

:-use_module(src(module_path)).

% config.pl
:-module(config, [target_path/1]).

% Raises error
target_path(lib/'target_module.pl').

% Doesn't raise error.
%target_path(lib('target_module.pl')).

% Gives wearning.
%target_path(lib/target_module).

% src/module_path.pl
:-module(module_path,[]).

:-use_module(project_root(config)).

:-config:target_path(P)
  ,ls(P)
  ,use_module(P).

% lib/target_module.pl
:-module(target_module, [pred/1
                        ]).
pred(1).

What is the best way to upload this example? Should I send an archive to bugs@swi-prolog.org ?

2

If I read this right, this is not a bug, but a deliberate change. Before, a relative path for use_module/1 (etc.) was resolved first relative to the file in which it appears and then relative to the process working directory. The “then” part no longer works.

So, you need to load '../lib/target_module'. This works in the old system as well and makes sure you can load the code regardless of the process working directory.

Alternatively, setup file search paths early in the load process and use these.

1 Like
3

Thanks Jan, to be honest the syntax that raises the error does seem wrong so I thought the (new) error probably wasn’t a bug but the result of a purposeful change.

How about ls/1 though? Shouldn’t it also have the same behaviour? If I call it from the same file as use_module/1, shouldn’t it only look for the path relative to the file?

4

I guess that is debatable. The impact is in absolute_file_name/3. I don’t know by heart what ls/1 uses. If I recall correctly (see docs), absolute_file_name/3 interprets relative files as relative to the file for directives.

1 Like
5

From what I can see, ls/1 ultimately calls name_to_files_/3 (note the underscore) which calls absolute_file_name/3 but only if the input is a compound of arity 1. It calls expand_file_name/2 otherwise.

With the path I’m looking for in my example above, lib/'target_module.pl', which is a compound with arity 2 (//2) this is more or less what happens, from what I can see:

% This is the name_to_files_/3 clause that should be reached from my directive; 
% Exists is set to 'true' in the ls/1 call:
name_to_files_(Spec, Files, Exists) :-
    file_name_to_atom(Spec, S1),
    expand_file_name(S1, Files0),
    (   Exists == true,
        Files0 == [S1],
        \+ access_file(S1, exist)
    ->  warning('"~w" does not exist', [S1]),
        fail
    ;   Files = Files0
    ).

% This is the result, simulated at the top-level:
?- config:target_path(P), shell:file_name_to_atom(P,A), expand_file_name(A,Fs), access_file(A,exist).
P = lib/'target_module.pl',
A = 'lib/target_module.pl',
Fs = ['lib/target_module.pl'].

I can’t see what access_file/2 does (it’s foreign) but I guess it takes the path as relative to the process, unlike use_module/1? Otherwise there would be a warning that the file does not exist.

If I call absolute_file_name/3 on 'lib/target_module.pl' with the option access(exist) which is called if the input is an arity-1 compound and I think is doing something similar to the access_file/2 check, I do get an error.

% name_to_files_/3 clause hit when Spec is an arity-1 compound:
name_to_files_(Spec, Files, _) :-
    compound(Spec),
    compound_name_arity(Spec, _Alias, 1),
    !,
    findall(File,
            (   absolute_file_name(Spec, File,
                                   [ access(exist),
                                     file_type(directory),
                                     file_errors(fail),
                                     solutions(all)
                                   ])
            ;   absolute_file_name(Spec, File,
                                   [ access(exist),
                                     file_errors(fail),
                                     solutions(all)
                                   ])
            ),
            Files).

% In module_path.pl:
:-config:target_path(P)
  ,absolute_file_name(P,Abs,[access(exist)])
  ,writeln(Abs).

?- make.
ERROR: c:/module_paths/src/module_path.pl:33:
ERROR:    source_sink `lib/'target_module.pl'' does not exist
Warning: c:/module_paths/src/module_path.pl:33:
Warning:    Goal (directive) failed: module_path:(config:target_path(_12252),absolute_file_name(_12252,_12332,[access(exist)]),writeln(_12332))
% c:/module_paths/src/module_path compiled into module_path 0.00 sec, 0 clauses
true.

So is this consistent behaviour or not? I have no idea. What is the intent of treating compounds of arity 1 in a special way?

(Sorry for the multiple edits- I hope you get to read the whole thing in one go)

Edit: hang on, I think I can answer my own question: the point of checking for an arity-1 compound is to catch terms Alias(Relative) like it says in the docs for absolute_file_name/3. So really the mistake in my code is to use ‘lib’ as the start of a path rather than an alias. But I still don’t know whether the behaviour of ls/1 makes sense or not.

6

Hard to say. As is, if you use Alias(File) it uses the logic of absolute_file_name/3, which works relative to the source being loaded or, if no source is being loaded, relative to the working directory of the process. All other usage of ls/1 does not use absolute_file_name/3 and (thus) uses the working directory of the process as base.

I guess it is what it is :slight_smile: I wrote ls/1 when I was using the minix os which had, due to lacking memory manager on i386, no sharing fork() and thus a system call would double the memory usage. A set of Prolog shell replacement predicates was a lot faster and did not waste memory :slight_smile: It is still somewhat useful to have OS agnostic quick look around options.

1 Like
7

I think that makes sense. name_to_files_/3 essentially wraps the OS ls functionality and adds the ability to handle the Alias(Relative) syntax.

That said, I’m trying to remember where it was that I first learned I could use that syntax and whether there is a primer for it in the docs. I think I just noticed the use_module(library(something)) directives in various files and documentation examples and cottoned on to it. Which probably means it’s a little bit of an advanced feature not immediately accessible to new entrants. I might be wrong- I’ll have to check the documentation.

8

Here’s another little gotcha that I found today. Below, the first bit of syntax, with the path as a quoted atom, raises an error when I try to pass it to use_module/1, whereas the second bit of syntax, with the path as a //2 term does not raise any errors:

% Raises error when atomic path is passed to use_module/1
experiment_file('lib/grid_master/data/solvers/grid_solver.pl',grid_solver).

% Raises no error:
experiment_file(lib/grid_master/data/solvers/grid_solver,grid_solver).

It might seem strange to pass the path explicitly as a quoted atom, as in the first uncommented line above, but the reason was that I was unsure whether that path would be recognised as a path if I passed it around as a term, rather than an atom.

Also, in general my intuition with SWI-Prolog is that paths are relative to the SWI-Prolog process, rather than relative to any source files. So I load all my source files with a “project load file” that also sets file search paths. I start my project by consulting the project load file. Then I refer to loaded files assuming paths are relative to the project load file. After the recent changes this tactic has caused quite a few errors.

My project load file is quite an informal thing so I guess everyone’s use cases will differ but knowing that every path I may pass around my project is relative to the directory where the project load file is run from gave me a certain sense of security.

Btw, here’s my project load file for Louise:

I guess such a thing is more common when one is using a full-blown IDE to manage a project, like people do with Visual Studio etc.

Anyway I would maybe argue for making everything relative to the initial process, but I suppose that’s a bit like a debate between which one is best, chocolate or strawberry ice cream.

Strawberry. Yech. :stuck_out_tongue: