I was trying to ground my understanding of the performance cost of dictionaries and compared to compound terms. Where I expected it to be slower, but not by substantial amounts.
When actually measuring it while doing a field swap, I found a few things surprising.
- Dictionary was about 5x slower than compound terms (or 10x if you subtract baseline; but I don’t know if that’s a fair thing to do).
- Delta between dictionary and compound terms is greater in SWI than the equivalent of clojure’s immutable data structures (clojure is about 1.5x difference between map and vectors, and generally 8x faster than prolog; but I’m not certain if this is just problems in my methodology)
- No-op is surprisingly slow, taking about 600 ms compared to 10-20ms in other languages (I’m guessing this is just some prolog VM cost for backtracking and such?)
Now, I am in no way certain my methodology is correct. I’d be more than happy to be told I’m doing something silly or expecting something I shouldn’t be, this is a learning exercise for me after all.
Is this all actually expected? Would also love to be enlightened to the details on where the costs are going. Thanks!
Prolog benchmark
Code
baseline_no_op :- true.
swap_field_term(
foo(A, B, C, D, E),
foo(A, B, C, E, D)).
swap_field_dict(Foo1, Foo2) :-
Foo2 = Foo1.put(_{d: Foo1.e, e: Foo1.d}).
run_test :-
format("Baseline no op x 10M~n"),
forall(between(1, 3, _),
time(forall(between(1, 10_000_000, _),
baseline_no_op))),
nl,
nl,
format("Swap field term x 10M~n"),
forall(between(1, 3, _),
time(forall(between(1, 10_000_000, _),
swap_field_term(foo(a, b, c, d, e), _)))),
nl,
nl,
format("Swap field dict x 10M~n"),
forall(between(1, 3, _),
time(forall(between(1, 10_000_000, _),
swap_field_dict(foo{a:a, b:b, c:c, d:d, e:e}, _)))),
nl,
nl.
Output
Baseline no op x 10M
% 20,000,000 inferences, 0.648 CPU in 0.648 seconds (100% CPU, 30872506 Lips)
% 19,999,999 inferences, 0.670 CPU in 0.670 seconds (100% CPU, 29870419 Lips)
% 19,999,999 inferences, 0.657 CPU in 0.657 seconds (100% CPU, 30431102 Lips)
Swap field term x 10M
% 19,999,999 inferences, 1.021 CPU in 1.021 seconds (100% CPU, 19579253 Lips)
% 19,999,999 inferences, 1.015 CPU in 1.015 seconds (100% CPU, 19707627 Lips)
% 19,999,999 inferences, 1.018 CPU in 1.018 seconds (100% CPU, 19646922 Lips)
Swap field dict x 10M
% 120,000,002 inferences, 5.323 CPU in 5.323 seconds (100% CPU, 22545290 Lips)
% 119,999,999 inferences, 5.269 CPU in 5.269 seconds (100% CPU, 22774026 Lips)
% 119,999,999 inferences, 5.403 CPU in 5.403 seconds (100% CPU, 22208489 Lips)
The surprising bits to me:
- Dictionaries is coming at 5x the time
- Baseline takes more than 100ms
Clojure benchmark
Disclaimer: I don’t know prolog nor clojure well enough to know if this is truly apples to apples. I suspect it’s not, but I couldn’t tell you how.
Code
(defn no-op [] true)
(println "baseline: " (no-op))
(dotimes [_ 3]
(time
(dotimes [_ 10000000]
(no-op))))
(defn swap-dict-test [x]
(assoc x
:d (:e x)
:e (:d x)))
(println "dict: " (swap-dict-test {:a :a :b :b :c :c :d :d :e :e}))
(dotimes [_ 3]
(time
(dotimes [_ 10000000]
(swap-dict-test {:a :a :b :b :c :c :d :d :e :e}))))
(defn swap-array-test [vec]
(assoc vec
3 (vec 4)
4 (vec 3)))
(println "array: " (swap-array-test [:a, :b, :c, :d, :e]))
(dotimes [_ 3]
(time
(dotimes [_ 10000000]
(swap-array-test [:a, :b, :c, :d, :e]))))
Output
baseline: true
"Elapsed time: 8.7682 msecs"
"Elapsed time: 6.775901 msecs"
"Elapsed time: 5.1539 msecs"
dict: {:a :a, :b :b, :c :c, :d :e, :e :d}
"Elapsed time: 738.496829 msecs"
"Elapsed time: 684.561426 msecs"
"Elapsed time: 635.70142 msecs"
array: [:a :b :c :e :d]
"Elapsed time: 488.002514 msecs"
"Elapsed time: 486.411614 msecs"
"Elapsed time: 487.424913 msecs"
Note: Dictionary updates only took ~1.5x the time.
Javascript (no-op only)
I only checked no-op here since there are no immutable data structures, so that would definitely be not-apples-to-apples.
console.time(); x = []; for (i = 0; i < 10000000; i++) { x.push(1); x.pop(); }; console.timeEnd();
VM181:1 default: 19.4169921875 ms