Cannot allocate memory (tcmalloc) building .qlf file + re_replace crash

I’ve never had much luck with upgrading Linux major versions (I ended up wiping my disk and installing Ubuntu 18.04 from scratch when my upgrade from 16.04 failed). So, I need to put aside a few hours to get WSL 2, and I haven’t felt the incentive to do that. :wink:

1 Like

One more data point (Ubuntu 16.04 a.k.a. WSL) – when I rebuilt, it didn’t get tcmalloc (because it’s not on my system); also, it didn’t built library(pcre), it seems. (Yes, my unit tests requre the infamous library(pcre).)

As for tcmalloc – as I recall, its main raison d’être is for mostly lock-free allocation in a heavily threaded environment (which is what Google servers tend to be). Better memory usage would be a bonus.

I’ve sent @Jan the strace output by private message.

The critical line seems to be:

read(4, "% -*- mode: Prolog -*-\n\n%% Utili"..., 4096) = 4096
stat("/home/peter/src/pykythe/pykythe/pykythe_utils.pl", {st_mode=S_IFREG|0644, st_size=18949, ...}) = 0
access("/home/peter/src/pykythe/pykythe/pykythe_utils.pl", R_OK) = 0
stat("/home/peter/src/pykythe/pykythe", {st_mode=S_IFDIR|0775, st_size=4096, ...}) = 0
stat("/home/peter/src/pykythe/pykythe/must_once.pl", {st_mode=S_IFREG|0644, st_size=4133, ...}) = 0
access("/home/peter/src/pykythe/pykythe/must_once.pl", R_OK) = 0
stat("/home/peter/src/pykythe/pykythe", {st_mode=S_IFDIR|0775, st_size=4096, ...}) = 0
stat("/home/peter/src/pykythe/pykythe/pykythe_utils.pl", {st_mode=S_IFREG|0644, st_size=18949, ...}) = 0
access("/home/peter/src/pykythe/pykythe/pykythe_utils.pl", R_OK) = 0
stat("/home/peter/src/pykythe/pykythe", {st_mode=S_IFDIR|0775, st_size=4096, ...}) = 0
read(4, "Sha1Hex    % succeed if SHA1 is "..., 4096) = 2134
futex(0x7f9c408a2b58, FUTEX_WAKE_PRIVATE, 1) = 1
futex(0x7f9c408a2b08, FUTEX_WAKE_PRIVATE, 1) = 1
close(4)                                = 0
clock_gettime(0xfffcc436 /* CLOCK_??? */, {tv_sec=0, tv_nsec=164634143}) = 0
read(3, "-, +, -, +, -, +).\n\n:- style_che"..., 4096) = 4096
read(3, "    kyfact_signature_node/6,\n   "..., 4096) = 4096
rt_sigprocmask(SIG_BLOCK, ~[QUIT BUS SEGV CONT STOP PROF RTMIN RT_1], [], 8) = 0
clock_gettime(0xfffcc436 /* CLOCK_??? */, {tv_sec=0, tv_nsec=165555194}) = 0
mmap(NULL, 270336, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f9c409b0000
munmap(0x7f9c409f2000, 204800)          = 0
clock_gettime(0xfffcc436 /* CLOCK_??? */, {tv_sec=0, tv_nsec=165686111}) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
clock_gettime(0xfffcc436 /* CLOCK_??? */, {tv_sec=0, tv_nsec=166182431}) = 0
rt_sigprocmask(SIG_BLOCK, ~[QUIT BUS SEGV CONT STOP PROF RTMIN RT_1], [], 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
clock_gettime(0xfffcc436 /* CLOCK_??? */, {tv_sec=0, tv_nsec=166271291}) = 0
mmap(NULL, 1099511631872, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = -1 ENOMEM (Cannot allocate memory)
write(2, "[FATAL ERROR: at Wed Jun 10 10:3"..., 43) = 43
write(2, "Could not allocate memory: Canno"..., 49) = 49
write(2, "]\n", 2)                      = 2
futex(0x7f9c408a2b5c, FUTEX_WAKE_PRIVATE, 1) = 1
futex(0x7f9c408a2b08, FUTEX_WAKE_PRIVATE, 1) = 1
nanosleep({tv_sec=0, tv_nsec=100000000}, 0x7ffc4e385140) = 0
close(3)                                = 0

It seems to have finished processing one use_module and has read one more 4096-byte block of source code and then crashed on an mmap. (I presume that the use_module is “lazy” because otherwise the ordering of file opens in the strace output doesn’t make sense.)

That is clearly making you run out of memory :slight_smile: Need to think what is next.

1099511631872 == 0x10000001000

And now the problem has mysteriously gone away. (Yes, I did make some small changes to my sources, but when I went back to what I think I had before, the problem was also gone)

Oh well … I did seem to cause it to happen one other time (during a compile step), so perhaps it’ll re-occur some time in the future …

Bug has been reproduced and fixed with https://github.com/SWI-Prolog/swipl-devel/commit/0a0d9800134b361eb2344f29cece8b0a4b571666

1 Like

Yeah. That is clearly the same one :slight_smile: