Internal data representation changes in GIT version

Announce
1

The changes announced with Cooking: internal data representation have been pushed to the swipl-devel.git master branch.

Users should be aware that internally a lot has changed. The commits affect nearly 4,000 lines spread over 115 files. Unfortunately, while all tests pass, some regression is to be expected. Please test and report issues

Consequences

  • The QLF format is incompatible. You may have to delete old *.qlf files from the build directory if you do an incremental build. You can do so by running this in the build directory:

    swipl qlf clean -ra

  • The 64 bit version should be fully compatible

  • On 32 bit version (still distributed for Windows, can be build on small embedded Linux systems such as Raspberry Pi (although these also move to 64 bits) and the WASM version) there are a few more consequences:

    • The C API type PL_atomic_t changed from 32 to 64 bits (used for dict keys)
    • The system can use all system memory for the Prolog stacks rather than 128Mb per stack. Stack usage is practically doubled.
    • Program size (compiled code) is not affected.
    • Internal hash tables map 64 bit to 64 bit. This should have little impact on most programs, but space used for tabling is significantly more. Future versions may enhance on that.

Next steps

Timing for the next steps is still unclear. Assuming these changes hold, several steps come into view. For example:

  • Get rid of relative addressing for Prolog data. That almost surely will improve performance. It will also simplify the garbage collector and allows for Prolog data that does not live on the stack of the running engine. Think about shared ground data or allow engines to access each others data.
  • Simplify the data type tags. That should simplify the system and improve performance.
  • Consider using 32 bit VM instructions on 64 bit hardware. That would seriously reduce the size of compiled code on 64 bit machines.
1 Like