The choice between Prolog and C is not so much about tricky. Some things cannot be done in pure Prolog, such as opening a file. So this must be in C. Then there are notably term manipulations that can be done way more efficiently in C, in part because C is faster but also for a large part because we can temporary modify the terms we are working on to avoid the need for tables to keep track of the state. This saves space and turns many operations into O(n), where n is the complexity of the term. Then there is simple deterministic stuff that needs to be fast, such as sort/2 and variations.
I don’t think the (SWI-)Prolog VM is conceptually that different from the Java one. There are some clear differences:
- The (SWI-)Prolog VM instruction set is not stable. This means you cannot load VM code on a different version (let alone a different Prolog). There is no real reason why this shouldn’t be possible in theory.
- A Prolog VM has a rather different instruction set. Many instructions are related to unification.
- The WAM is a register VM, but despite some wrong names in the sources, the SWI-Prolog VM is based on a minimal version of the ZIP, which passes arguments over the stacks rather than using registers.
- I don’t know how the JVM deals with stuff such as OS access. The (SWI-)Prolog VM can deal with such things in two ways: add a VM instruction or add a foreign predicate. Most not very time critical stuff uses foreign predicates as that keeps the VM small, is easier to manage and even allows loading such extensions at runtime.