# VM code refactor - thoughts? **URL:** \ **Category:** Request For Comments\ **Tags:** on-topic-only\ **Created:** [April 19, 2021, 11:31pm UTC](https://swi-prolog.discourse.group/t/vm-code-refactor-thoughts/3845 "2021-04-19T23:31:34Z")\ **Posts on this page:** 20\ **Page:** 5
**Author:** ![grossdan](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/grossdan/32/23_2.png) [@grossdan](https://swi-prolog.discourse.group/u/grossdan)\ **Post date:** [May 13, 2021, 2:17am UTC](https://swi-prolog.discourse.group/t/vm-code-refactor-thoughts/3845/150 "2021-05-13T02:17:28Z")
Yet another thing to worry about – semantics of types and correctness definitions … 🙂 Dan ---
**Author:** ![peter.ludemann](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/peter.ludemann/32/48_2.png) [@peter.ludemann](https://swi-prolog.discourse.group/u/peter.ludemann)\ **Post date:** [May 14, 2021, 8:12pm UTC](https://swi-prolog.discourse.group/t/vm-code-refactor-thoughts/3845/151 "2021-05-14T20:12:50Z")
An interesting article on optimizing Python and Javascript, although it’s The Register, so some of it might not be completely accurate: [The quest for faster Python: Pyston returns to open source, Facebook releases Cinder, or should devs just use PyPy? • The Register](https://www.theregister.com/2021/05/06/the_quest_for_faster_python/) JavaScript has a JIT implementation (Google’s V8 engine) which beats Python on simple benchmarks but often isn’t a net win for “production” code because many of the performance-critical parts of Python’s libraries hae been rewritten in C. [Pyston](https://github.com/pyston/pyston) gives a ~30% performance boost by using a JIT – which fits with [my observation of interpreter bytecode loop’s overhead](https://swi-prolog.discourse.group/t/vm-code-refactor-thoughts/3845/140) when the individual VM instructions are moderately complex (which is the case for both Python and Prolog). ---
**Author:** ![EricGT](https://avatars.discourse-cdn.com/v4/letter/e/f1d935/32.png) [@EricGT](https://swi-prolog.discourse.group/u/EricGT)\ **Post date:** [May 15, 2021, 8:44am UTC](https://swi-prolog.discourse.group/t/vm-code-refactor-thoughts/3845/153 "2021-05-15T08:44:32Z")
This is tagged `On-Topic-Only` topic, please clean up the off topic items. ---
**Author:** ![dmchurch](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/dmchurch/32/3427_2.png) [@dmchurch](https://swi-prolog.discourse.group/u/dmchurch)\ **Post date:** [May 15, 2021, 5:08pm UTC](https://swi-prolog.discourse.group/t/vm-code-refactor-thoughts/3845/155 "2021-05-15T17:08:52Z")
I feel like it was more likely about the Rust digression? (shrugs) Either way, it doesn’t make that much difference to me. ---
**Author:** ![peter.ludemann](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/peter.ludemann/32/48_2.png) [@peter.ludemann](https://swi-prolog.discourse.group/u/peter.ludemann)\ **Post date:** [May 15, 2021, 5:44pm UTC](https://swi-prolog.discourse.group/t/vm-code-refactor-thoughts/3845/156 "2021-05-15T17:44:39Z")
An article about Python’s plans for a 2x speed improvement: > **[Guido van Rossum aiming to make CPython 2x faster in 3.11](https://www.theregister.com/2021/05/13/guido_van_rossum_cpython_3_11/)** > > 'I got bored sitting at home while retired ... I chose to go back to my roots' I’m guessing that there are more pay-offs in Python for JIT-like optimizations; a quick look at the instructions in [ceval.c](https://github.com/python/cpython/blob/80b089179fa798c8ceaab2ff699c82499b2fcacd/Python/ceval.c) shows fairly short instructions sequences for LOAD\_FAST etc. There are a few interesting comments in that code - computed gotos vs threaded code (15-20% performance difference) - opcode prediction (common sequences) - “periodic things” (related to async; don’t know if it’s applicable to Prolog) It’ll be interesting to see what things the Python folks use to get their 2x performance improvement; and whether they can actually get 2x. (BTW, if you want to talk with any of them, I probably can make introductions via friend-of-a-friend; and their work will be opensource anyway, so you can see who is doing what.) ---
**Author:** ![dmchurch](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/dmchurch/32/3427_2.png) [@dmchurch](https://swi-prolog.discourse.group/u/dmchurch)\ **Post date:** [May 15, 2021, 8:50pm UTC](https://swi-prolog.discourse.group/t/vm-code-refactor-thoughts/3845/159 "2021-05-15T20:50:26Z")
> [@anon95304481](#): > > I think Jan W. suggested a different exploration. Namely enlarging the instruction set I mean if @jan wants to enlarge the instruction set, more power to him! I won’t claim nearly as much expertise on operation and optimization of the WAM, so I’m sticking to the lower layers for my optimization work 😃 ---
**Author:** ![peter.ludemann](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/peter.ludemann/32/48_2.png) [@peter.ludemann](https://swi-prolog.discourse.group/u/peter.ludemann)\ **Post date:** [May 20, 2021, 3:34pm UTC](https://swi-prolog.discourse.group/t/vm-code-refactor-thoughts/3845/160 "2021-05-20T15:34:21Z")
A couple more articles about what’s being proposed for Python; maybe a few of these ideas would be helpful for Prolog but most of them seem Python-specific. > **[The 2021 Python Language Summit: CPython Performance Improvements at Instagram](https://pyfound.blogspot.com/2021/05/the-2021-python-language-summit-cpython.html)** > > Dino Viehland gave a presentation at the 2021 Python Language Summit about improvements to CPython's performance at Instagram. Cinder is ... > **[Faster Python: Mark Shannon, author of newly endorsed plan, speaks to The...](https://www.theregister.com/2021/05/19/faster_python_mark_shannon_author/)** > > The biggest challenge? 'Backwards compatibility of features that we might not even know we have' Personal opinion: in the end, they’ll probably get some nice overall boost – perhaps 1.2x or 1.3x – but I’m skeptical of 2x unless they make some changes to the language. Similarly, I’m not hopeful for huge gains with Prolog, much as I’d like to see them, without changes to the language; and that’s already been tried, e.g. with Mercury or (in a sense) Erlang. A concrete example: Instagram tried “inline caching for byte code” - which gave a 5% improvement in production. It takes quite a few such changes - and considerable effort - to get a 30% improvement because not every program is helped by such “optimizations”; in some cases the optimizations make things slower. ---
**Author:** ![dmchurch](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/dmchurch/32/3427_2.png) [@dmchurch](https://swi-prolog.discourse.group/u/dmchurch)\ **Post date:** [May 20, 2021, 3:41pm UTC](https://swi-prolog.discourse.group/t/vm-code-refactor-thoughts/3845/161 "2021-05-20T15:41:06Z")
I’ll certainly be taking a look at these! At some point, in my copious spare time 😂 To be honest, though? I absolutely am hoping to make some fairly ridiculous gains in swipl’s efficiency. I think an order of magnitude is probably too much to hope for, but 2x is where I’m setting my “initial goal” sights. This coming largely from my inspection of generated assembly in swipl, at least for the amd64 case. But we’ll see! And we’ll certainly have a bit of a ways to go before we get there 😅 ---
**Author:** ![grossdan](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/grossdan/32/23_2.png) [@grossdan](https://swi-prolog.discourse.group/u/grossdan)\ **Post date:** [May 20, 2021, 4:19pm UTC](https://swi-prolog.discourse.group/t/vm-code-refactor-thoughts/3845/162 "2021-05-20T16:19:51Z")
You know this high speed assembly based library of data structures that was posted about earlier, made me wonder – what if the VM core is rewritten in assembly – what would this achieve? ---
**Author:** ![jan](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/jan/32/4_2.png) [@jan](https://swi-prolog.discourse.group/u/jan)\ **Post date:** [May 20, 2021, 4:21pm UTC](https://swi-prolog.discourse.group/t/vm-code-refactor-thoughts/3845/163 "2021-05-20T16:21:01Z")
Sounds good 🙂 Not sure where you want to look. One of the candidates is for 64-bit to change the data representation. As is, a Prolog word has - 2 GC bits - 3 tag bits - 2 “storage” bits that tells that the value is relative to the base of a specific stack. On 64-bit machines we have at least 8 bits that we can freely use. We could omit the _storage_ bits and have direct pointers. This requires some changes to the stack shifter and garbage collector. It surely make a pointer lookup much more direct. AFAIK at least some CPUs ignore the high bits on a pointer and thus you can use the value directly as a pointer rather than the current steps: shift the word right by 7bits to get rid of the above, get the base pointer for the stack specified by the storage bits from the `LD` struct and compute the final address. That also means 6 bits for the tag at no price, so we can get rid of more indirect ways to separate the types. Just by heart, the types are - var - ref - rational with subtypes - real rational (MPQ) - big integer (MPZ) - 64 bit int - inline it (fits in the word-7 bits) - float - atom - string - compound With 6 bits they can all get their own tag. I don’t think it is is particularly hard to do so. The main challenge is to keep the code working on 32-bit hardware. I don’t care too much about people with an old 32-bit only desktop. It is a pity to loose embedded devices though. Also WASM is 32 bits … ---
**Author:** ![dmchurch](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/dmchurch/32/3427_2.png) [@dmchurch](https://swi-prolog.discourse.group/u/dmchurch)\ **Post date:** [May 20, 2021, 4:31pm UTC](https://swi-prolog.discourse.group/t/vm-code-refactor-thoughts/3845/164 "2021-05-20T16:31:00Z")
> [@grossdan](#): > > what if the VM core is rewritten in assembly – what would this achieve? I’ve certainly had some thoughts along those lines, and I’m not above writing assembly versions of certain routines if that’s the only way to get the performance! That said, I’d prefer to figure out a way to write the C code that convinces the _compiler_ to generate the assembly we’re hoping for—and, if we can make the data flows clear enough and the data structures self-describing enough, it might even be able to manage it. After all, when you resort to writing assembly, you’re effectively forking your codebase, once for each platform you target for optimizations. And I’d _prefer_ not to have to maintain that many implementations 😄 I know _I’m_ a polyglot who enjoys learning new assembly dialects for fun, but I’m not gonna assume anyone else is, and the last thing I’d want is for me to be the only person capable of maintaining the VM code! ---
**Author:** ![dmchurch](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/dmchurch/32/3427_2.png) [@dmchurch](https://swi-prolog.discourse.group/u/dmchurch)\ **Post date:** [May 20, 2021, 4:40pm UTC](https://swi-prolog.discourse.group/t/vm-code-refactor-thoughts/3845/165 "2021-05-20T16:40:28Z")
> [@jan](#): > > One of the candidates is for 64-bit to change the data representation. I’d had that thought as well! Glad to hear you’re open to the idea 😄 And yes, I’m certainly invested in maintaining 32-bit support, since I _did_ after all get into this by poking at the WASM build 🙂 Out of curiosity, what would the downside be to adding one more metadata bit to the (32-bit) representation, bringing it up to an even 8 bits? Would it just mean that the compiler is one bit more likely to turn something into an externalized int, or would it break Prolog-level compatibility? I ask because it may in fact end up being more efficient to do byte-level operations to isolate the tag and the value, rather than bit-level. Hmm, I suppose it would drop the max stack size on 32-bit from 128MB down to 64MB, which isn’t a small consideration. But if that could be avoided? ---
**Author:** ![swi](https://avatars.discourse-cdn.com/v4/letter/s/51bf81/32.png) [@swi](https://swi-prolog.discourse.group/u/swi)\ **Post date:** [May 20, 2021, 6:06pm UTC](https://swi-prolog.discourse.group/t/vm-code-refactor-thoughts/3845/166 "2021-05-20T18:06:25Z")
> [@jan](#): > > is to keep the code working on 32-bit hardware I think this is extremely important, there are many server applications for SWI prolog on older and cheaper 32 bit single board computers ---
**Author:** ![jan](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/jan/32/4_2.png) [@jan](https://swi-prolog.discourse.group/u/jan)\ **Post date:** [May 20, 2021, 6:19pm UTC](https://swi-prolog.discourse.group/t/vm-code-refactor-thoughts/3845/167 "2021-05-20T18:19:45Z")
> [@dmchurch](#): > > Hmm, I suppose it would drop the max stack size on 32-bit from 128MB down to 64MB, which isn’t a small consideration. But if that could be avoided? That is indeed the reason. Is there a difference between `x>>7` and `x>>8`? I’d be surprised. I very much doubt there is a way to avoid limiting the stack size as a result. Do keep in mind that using many threads is not an exception for SWI-Prolog programs. Each of them has three stacks! Anyway, crucial is to make 64 bit faster. Loosing a little on 32-bit is probably not a big issue as long as it is not too bad. ---
**Author:** ![grossdan](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/grossdan/32/23_2.png) [@grossdan](https://swi-prolog.discourse.group/u/grossdan)\ **Post date:** [May 20, 2021, 7:01pm UTC](https://swi-prolog.discourse.group/t/vm-code-refactor-thoughts/3845/168 "2021-05-20T19:01:59Z")
If nothing else, but for a benchmark, having one assembly based version on the most popular processor architecture (x86?) might be worthwhile. ---
**Author:** ![dmchurch](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/dmchurch/32/3427_2.png) [@dmchurch](https://swi-prolog.discourse.group/u/dmchurch)\ **Post date:** [May 20, 2021, 7:17pm UTC](https://swi-prolog.discourse.group/t/vm-code-refactor-thoughts/3845/169 "2021-05-20T19:17:41Z")
It’s certainly possible! One of the potential advantages of using assembly is that we wouldn’t be restricted nearly as much by C calling conventions and the like, and moreover it would allow for _composeability_—if each VM instruction is represented by a predefined set of assembler instructions, then one could simply concatenate the requisite instruction implementations together to form a callable native predicate implementation. That said, it’s not anything I’ve got planned for the near future, there are too many other bits of the refactor that have to come first. Once that’s done, though, we’ll see 🙂 Are you volunteering to help write the assembly? ---
**Author:** ![grossdan](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/grossdan/32/23_2.png) [@grossdan](https://swi-prolog.discourse.group/u/grossdan)\ **Post date:** [May 21, 2021, 2:37am UTC](https://swi-prolog.discourse.group/t/vm-code-refactor-thoughts/3845/170 "2021-05-21T02:37:23Z")
I can’t deny (to myself) the excitement of delving into assembly – and also started to briefly looking at the youtube videos of fasm. > **[FASM](https://en.wikipedia.org/wiki/FASM)** > > FASM (flat assembler) is an assembler for x86 processors. It supports Intel-style assembly language on the IA-32 and x86-64 computer architectures. It claims high speed, size optimizations, operating system (OS) portability, and macro abilities. It is a low-level assembler and intentionally uses very few command-line options. It is free and open-source software. > All versions of FASM can directly output any of the following: flat "raw" binary (usable also as DOS COM executable or SYS driver), obj... If i can (timewise) to do some assembly programming, and can get the help to understand how to do this well in the x86 environment – i.e. to make best use of how the processor within its hardware architecture is intended to be programmed and used – then would be glad to do it. Dan ---
**Author:** ![dmchurch](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/dmchurch/32/3427_2.png) [@dmchurch](https://swi-prolog.discourse.group/u/dmchurch)\ **Post date:** [May 21, 2021, 4:29am UTC](https://swi-prolog.discourse.group/t/vm-code-refactor-thoughts/3845/171 "2021-05-21T04:29:34Z")
Haha, sounds great! I’ll definitely let you know when there’s a good time to start thinking about assembler implementations. In the meantime, if you haven’t seen it before, you might find it interesting reading to skim through the x64 instruction set manual: > **[64-ia-32-architectures-software-developer-instruction-set-reference-manual-325383.pdf](https://www.intel.com/content/dam/www/public/us/en/documents/manuals/64-ia-32-architectures-software-developer-instruction-set-reference-manual-325383.pdf)** > > 0 B and the x64 optimization manual: > **[64-ia-32-architectures-optimization-manual.pdf](https://www.intel.com/content/dam/www/public/us/en/documents/manuals/64-ia-32-architectures-optimization-manual.pdf)** > > 0 B ---
**Author:** ![grossdan](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/grossdan/32/23_2.png) [@grossdan](https://swi-prolog.discourse.group/u/grossdan)\ **Post date:** [May 21, 2021, 4:35am UTC](https://swi-prolog.discourse.group/t/vm-code-refactor-thoughts/3845/172 "2021-05-21T04:35:13Z")
thank you. Yes, noticed the first manual references on the fasm site and glanced through it … i then noticed MMX and similar, and was wondering how this could be put to use as well … i.e. if we remove portability considerations and focus on benchmarking – and having one swi version for a broadly used architecture that is really fast. Dan ---
**Author:** ![peter.ludemann](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/peter.ludemann/32/48_2.png) [@peter.ludemann](https://swi-prolog.discourse.group/u/peter.ludemann)\ **Post date:** [May 21, 2021, 4:51am UTC](https://swi-prolog.discourse.group/t/vm-code-refactor-thoughts/3845/173 "2021-05-21T04:51:50Z")
I once worked on an all-assembler implementation of Prolog (IBM Prolog/370), which was written by a genius programmer. I’m also a good IBM/370 assembler programmer (or was – it’s been some years), and there was one other expert on the project … nevertheless, it was not easy to maintain the code, and the IBM/370 has a cleaner instruction set than x86 and also has 16 registers. ARM (M1) is probably a nicer target than x86. (Or RISC-V or PowerPC or just about anything for that matter) But do you want to be maintaining multiple code bases? I would suggest first finding hotspots and then looking at the assembler that the C compiler generates for them. There are also some additional optimization options that might help, plus link-time optimizations. My 2-cents’ worth. [Previous page](https://swi-prolog.discourse.group/t/vm-code-refactor-thoughts/3845.md?page=4) [Next page](https://swi-prolog.discourse.group/t/vm-code-refactor-thoughts/3845.md?page=6)