The performance improvements by adding “combined instructions” suggest two hypotheses (they’re not mutually exclusive):
- the VM interpreter overhead is sometimes significant
- the I-cache is not being fully utilized
It’s not clear that generating compiled code (e.g. a JIT) would give much more benefit compared to adding “combined instructions” – the resulting “bloat” of compiled code could hurt the I-cache. Adding combined instructions might get the best of both worlds: remove interpreter overhead where it matters but maintain code compactness (and also avoid the complexities of a JIT).
The results of the Python “Unladen Swallow” project show that compiling out the interpreter loop isn’t always worth the effort, especially when the VM instructions are moderately complex. You might want to look at the disappointing benchmarks and discussion of how difficult it is to build a JIT using LLVM. (There are also some specific aspects of Python that might or might not be relevant – if you want me to pontificate on them, you merely have to ask.)