SSL(0A000126) : unexpected eof while opening many web sites with http_open

General
1

Hello

Opening many web sites (like twitter.con, pirellihangarbicocca.org, …) using

  Url="https://twitter.com/", http_open(Url, In, [cert_verify_hook(cert_accept_any)]), load_html(In, DOM, []),close(In).

I get

ERROR: SSL(0A000126) : unexpected eof while reading
ERROR: In:
ERROR:   [19] sgml:sgml_parse(sgml_parser(94669959974976),[document(...),...|...])
ERROR:   [17] setup_call_catcher_cleanup(sgml:new_sgml_parser(...,...),sgml:parse(...,...,...,<stream>(0x561a11520a00,0x561a1124c000)),_110940,sgml:free_sgml_parser(...)) at /usr/lib/swi-prolog/boot/init.pl:679
ERROR:   [15] sgml:load_structure_from_stream(<stream>(0x561a11520a00,0x561a1124c000),_111012,user:[...|...]) at /usr/lib/swi-prolog/library/sgml.pl:429
ERROR:   [14] setup_call_catcher_cleanup(sgml:open_any(<stream>(0x561a11520a00,0x561a1124c000),read,<stream>(0x561a11520a00,0x561a1124c000),true,...),sgml:load_structure_from_stream(<stream>(0x561a11520a00,0x561a1124c000),_111086,...),_111054,sgml:close_any(true)) at /usr/lib/swi-prolog/boot/init.pl:679
ERROR:   [10] '<meta-call>'('<garbage_collected>') <foreign>
ERROR:    [9] toplevel_call('<garbage_collected>') at /usr/lib/swi-prolog/boot/toplevel.pl:1173
ERROR: 
ERROR: Note: some frames are missing due to last-call optimization.
ERROR: Re-run your program in debug mode (:- debug.) to get more detail.
^  Exception: (17) setup_call_catcher_cleanup(sgml:new_sgml_parser(sgml_parser(94669959974976), [dtd(dtd(94669959093424, html5))]), sgml:parse(sgml_parser(94669959974976), user:[dialect(html5), dialet(html), max_errors(-1), syntax_errors(quiet)], _39290, <stream>(0x561a11520a00,0x561a1124c000)), _155658, sgml:free_sgml_parser(sgml_parser(94669959974976))) ? creep
   Call: (19) sgml:free_sgml_parser(sgml_parser(94669959974976)) ?

I suppose it is related to SSL , but I can get the url with curl, the browser and other programming languages.

My env is

matteo@debian:~$ openssl version
OpenSSL 3.1.4 24 Oct 2023 (Library: OpenSSL 3.1.4 24 Oct 2023)

matteo@debian:~$ swipl --version
SWI-Prolog version 9.0.4 for x86_64-linux

matteo@debian:~$ uname -a
Linux debian 6.5.0-5-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.5.13-1 (2023-11-29) x86_64 GNU/Linux

Reading some blog posts I also tried to set th option “SSL_OP_IGNORE_UNEXPECTED_EOF” in /etc/ssl/openssl.cnf but running the prolog code I get out of memory / cannot allocate memory…

Thanks in advance
Matteo

2

I’m running version 9.1.21 (nightly in Win11, built from source in Ubuntu:WSL), and cannot reproduce your problem.

Indeed, apart the fact that https://twitter.com is unreachable (being redirected to x.com ? that in turn appears to redirect back to twitter.com ?), load_html/3 succeeds.

Maybe you should first try to upgrade your swipl version…

1 Like