For short, user is the place to be if you do not use modules. SWI-Prolog also allows using it for defining stuff that affects all modules, such as global operators and predicates. That has great value for dealing with emulating other Prolog systems, but should probably be handled differently.
system sits above user and is where all built-in predicates reside or are imported to and is the direct super-module of the libraries. Typically, don’t touch. Nothing really prevents you from doing so though.
prolog is a module that have become fashionable for holding hooks. As system should not be touched, this is not the ideal place. Quintus derived systems used user for this, but this pollutes the user module which we should (IMO) normally leave empty (unless you decide not to use modules at all). So, by intend the prolog module only contains multifile predicates. It is not reserved in any way though, so if you want to misuse it for anything else your are free to do so.
Note that module inheritance affects:
- operators (if the operator is not defined or cancelled locally, look in the super)
- predicates (if a predicate is undefined, look in the super)
- Macro expansion (term/goal): expand locally, next feed the result through the expansion rules of the super (pipeline).