# The subtle costs of garbage collection

**URL:** <https://swi-prolog.discourse.group/t/the-subtle-costs-of-garbage-collection/6287>\
**Category:** General\
**Tags:** discussion\
**Created:** [February 21, 2023, 7:36pm UTC](https://swi-prolog.discourse.group/t/the-subtle-costs-of-garbage-collection/6287 "2023-02-21T19:36:45Z")\
**Posts on this page:** 1\
**Showing post:** 38

<div class="post-metadata">

**Author:** ![jan](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/jan/32/4_2.png) [@jan](https://swi-prolog.discourse.group/u/jan)\
**Post date:** [March 2, 2023, 10:06am UTC](https://swi-prolog.discourse.group/t/the-subtle-costs-of-garbage-collection/6287/38 "2023-03-02T10:06:18Z")

</div>

> [@peter.ludemann](#):
>
> The reason for the big speed-up was that thread switching did bad things to the cache; I presume that this would still apply today, perhaps even moreso (I’ve observed ~3x performance differences on one Prolog program, between two machines that differ by 10x in the amount of cache).

Possibly on some workloads. In the early days of multi-threaded SWI-Prolog running things concurrently typically meant a performance improvement with 2 or 4 threads, after which the performance started dropping down, often quickly getting below the performance of 1 thread. At least on Linux I no longer observe that on most workloads.

Using this driver code

> **Driver**
>
> ```prolog
> :- meta_predicate
> conc_table(+,0),
> conc(+,0).
> 
> conc_table(Max, Goal) :-
> format('~t~w~15|~t~w~15+~t~w~15+~n', ["#threads", "total", "per run"]),
> between(0,Max,Step),
> Conc is 1<<Step,
> call_time(conc(Conc, Goal), Time),
> PerSol is Time.wall/Conc,
> format('~t~D~15|~t~3f~15+~t~3f~15+~n', [Conc, Time.wall, PerSol]),
> fail.
> conc_table(_, _).
> 
> conc(N, Goal) :-
> length(Threads, N),
> maplist(thread_create(Goal), Threads),
> maplist(thread_join, Threads).
> 
> ```

we get this result. Machine is 16 cores, 32 threads. So, you see close to perfect speedup up to 16 threads (total time remains almost flat). Some more speedup to 32 (hyper threading) and after that flat performance.

```prolog
?- conc_table(11, rtest_chats(100)).
       #threads total per run
              1 0.674 0.674
              2 0.668 0.334
              4 0.690 0.173
              8 0.723 0.090
             16 0.806 0.050
             32 1.201 0.038
             64 2.400 0.038
            128 4.640 0.036
            256 9.139 0.036
            512 18.337 0.036
          1,024 36.686 0.036
          2,048 73.649 0.036

```

There is still a reason for not using system threads: scheduling gets unfair as each of the threads has equal priority. Well, you can work around that using scheduling settings.

My assessment is roughly this, where details depend on workload and also OS (and possibly hardware).

- Threads use more resources (stack, OS resources)
- Fibers probably scale better with many often blocking tasks
- Threads scale better with many CPU bound tasks
- Fibers are easier to program with
- If you use threads+fibers you get probably best scalability. On the other hand, where old OSes used to have that for implementing POSIX threads, e.g., Linux doesn’t bother.
- With only fibers you can only use one CPU. With threads+fibers you can scale to multiple cores, but you get most of the complications of preemptive multi-threaded programming in return.

---

_[View the full topic](https://swi-prolog.discourse.group/t/the-subtle-costs-of-garbage-collection/6287)._