# Building a port scanner **URL:** \ **Category:** Help!\ **Created:** [June 30, 2020, 12:40pm UTC](https://swi-prolog.discourse.group/t/building-a-port-scanner/2580 "2020-06-30T12:40:46Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![EricGT](https://avatars.discourse-cdn.com/v4/letter/e/f1d935/32.png) [@EricGT](https://swi-prolog.discourse.group/u/EricGT)\ **Post date:** [June 30, 2020, 12:40pm UTC](https://swi-prolog.discourse.group/t/building-a-port-scanner/2580/1 "2020-06-30T12:40:46Z")
SWI-Prolog (threaded, 64 bits, version 8.3.2) on Windows 10 In creating a port scanner for a web site tcp\_connect/2 is wrapped with setup\_call\_catcher\_cleanup/4. The catcher is catching the error and the clause to the handle the error is getting called but the code is not completing successfully and the error message is being printed. The code uses `:- set_prolog_flag(report_error,false).` to suppress the printing of the error message so I don’t know why the error message is being printed. The code runs correctly for an open port, e.g. 80, and causes a timeout exception for a port that is not handled, e.g. 79, but is not completing successfully to display the desired result: `Port 79: closed` Example run ```prolog ?- port_scan_range(80,80). Port 80: open true. ?- port_scan_range(79,79). ERROR: Unhandled exception: Socket error: Connection timed out ERROR: In: ERROR: [13] socket:tcp_connect((0000000006D179B0),'140.211.166.101':79) ERROR: [12] setup_call_catcher_cleanup(user:tcp_socket((0000000006D179B0)),user:tcp_connect((0000000006D179B0),...),_22258,user:(...,...)) at c:/program files/swipl/boot/init.pl:562 ERROR: [10] port_scan_range(79,79) at c:/users/eric/documents/notes/discourse swi-prolog osu osl/osu osl prolog/tcp_test.pl:33 ERROR: [9] ERROR: ERROR: Note: some frames are missing due to last-call optimization. ERROR: Re-run your program in debug mode (:- debug.) to get more detail. Timeout error caught ?- current_prolog_flag(report_error,Flag). Flag = false. ``` Code ```prolog :- set_prolog_flag(debug_on_error,false). % Do not drop into debugger on error. Errors and exceptions are different with errors being more severe. :- set_prolog_flag(report_error,false). % Do not print error messages to screen. % ----------------------------------------------------------------------------- port_scan_range(Start,End) :- must_be(positive_integer,Start), must_be(positive_integer,End), Start =< End, IP_address = '140.211.166.101', between(Start,End,Port), port_scan(IP_address,Port,Result), format('Port ~w: ~w~n',[Port,Result]). port_scan(IP_address,Port,Result) :- setup_call_catcher_cleanup( tcp_socket(Socket), % Open stream socket based on TCP/IP which uses IP address and port number, i.e. INET socket tcp_connect(Socket, IP_address:Port), Catcher, ( catcher(Catcher,Result), tcp_close_socket(Socket) ) ). catcher(exit,open). catcher( exception( error(socket_error(wsaetimedout, 'Connection timed out'), _Context) ), closed ) :- format('Timeout error caught~n',[]). catcher(Catcher,unknown) :- format('Catcher default case~n',[]), format('~w~n',[Catcher]). ``` ---
**Author:** ![EricGT](https://avatars.discourse-cdn.com/v4/letter/e/f1d935/32.png) [@EricGT](https://swi-prolog.discourse.group/u/EricGT)\ **Post date:** [June 30, 2020, 1:49pm UTC](https://swi-prolog.discourse.group/t/building-a-port-scanner/2580/2 "2020-06-30T13:49:36Z")
By changing setup\_call\_catcher\_cleanup/4 to setup\_call\_cleanup/3 and using catch/3 for the goal it works as desired. Also changed the predicate catcher/2. ```prolog port_scan(IP_address,Port,Result) :- setup_call_cleanup( tcp_socket(Socket), catch( ( % Open stream socket based on TCP/IP which uses IP address and port number, i.e. INET socket tcp_connect(Socket, IP_address:Port), Result = open ), Catcher, catcher(Catcher,Result) ), tcp_close_socket(Socket) ). catcher( error(socket_error(wsaetimedout, 'Connection timed out'),_Context), closed ) :- !. catcher(Catcher,unknown) :- format('Catcher default case~n',[]), format('~w~n',[Catcher]). ``` Example run ```prolog ?- port_scan_range(79,79). Port 79: closed true. ?- port_scan_range(80,80). Port 80: open true. ``` ---
**Author:** ![EricGT](https://avatars.discourse-cdn.com/v4/letter/e/f1d935/32.png) [@EricGT](https://swi-prolog.discourse.group/u/EricGT)\ **Post date:** [June 30, 2020, 2:02pm UTC](https://swi-prolog.discourse.group/t/building-a-port-scanner/2580/3 "2020-06-30T14:02:54Z")
Jan W., when you get a chance, is tcp\_close\_socket/1 needed? The documentation notes two cases where is it is needed but since this is port scanning and not using tcp\_open\_socket/3 I am not sure. ---
**Author:** ![jan](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/jan/32/4_2.png) [@jan](https://swi-prolog.discourse.group/u/jan)\ **Post date:** [June 30, 2020, 2:12pm UTC](https://swi-prolog.discourse.group/t/building-a-port-scanner/2580/4 "2020-06-30T14:12:46Z")
> [@EricGT](#): > > is [tcp\_close\_socket/1](https://www.swi-prolog.org/pldoc/doc_for?object=tcp_close_socket/1) needed? Yes, you do not create streams for the socket, so you must close the socket yourself. The simplest way is probably this: ```prolog catch(setup_call_cleanup( tcp_socket(S), tcp_connect(S, IP_address:Port), tcp_close_socket(S)), error(_,_), fail). ``` If this succeeds, the port is open, else it is closed. You may be more restrictive on the errors under which you consider the port closed, but I’m not sure that is needed. Note that the `report_error` flag is probably not even used anymore (should check) and the `debug_on_error` flag just deals with whether or not the debugger is started when an error is uncaught. In this case you need catch/3 (not that the various \*\_cleanup predicates do **not** catch the error, they just guarantee that resource cleanup is called regardless of success/error/fail/cut/… that terminates the goal that needs the resource. ---
**Author:** ![EricGT](https://avatars.discourse-cdn.com/v4/letter/e/f1d935/32.png) [@EricGT](https://swi-prolog.discourse.group/u/EricGT)\ **Post date:** [June 30, 2020, 2:26pm UTC](https://swi-prolog.discourse.group/t/building-a-port-scanner/2580/5 "2020-06-30T14:26:19Z")
Complete version after suggestions by Jan W. and making port\_scan\_range/3 a proper failure driven loop. ```prolog port_scan_range(Start,End) :- must_be(positive_integer,Start), must_be(positive_integer,End), Start =< End, IP_address = '140.211.166.101', between(Start,End,Port), port_scan(IP_address,Port,Result), format('Port ~w: ~w~n',[Port,Result]), fail. port_scan_range(_,_). port_scan(IP_address,Port,Result) :- catch( setup_call_cleanup( tcp_socket(Socket), ( % Open stream socket based on TCP/IP which uses IP address and port number, i.e. INET socket tcp_connect(Socket, IP_address:Port), Result = open ), tcp_close_socket(Socket) ), error(_,_), Result = closed ). ``` Example run ```prolog ?- port_scan_range(79,80). Port 79: closed Port 80: open true. ``` ---
**Author:** ![peter.ludemann](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/peter.ludemann/32/48_2.png) [@peter.ludemann](https://swi-prolog.discourse.group/u/peter.ludemann)\ **Post date:** [June 30, 2020, 4:42pm UTC](https://swi-prolog.discourse.group/t/building-a-port-scanner/2580/6 "2020-06-30T16:42:57Z")
I would suggest using forall/2 rather than a failure-driven loop – forall/2 is easier to debug. With a failure-driven loop, you can’t tell if the range part fails (which is fine) or the port\_scan part fails (which is a bug). [BTW, there’s no need for the `Start= **Author:** ![EricGT](https://avatars.discourse-cdn.com/v4/letter/e/f1d935/32.png) [@EricGT](https://swi-prolog.discourse.group/u/EricGT)\ **Post date:** [June 30, 2020, 8:32pm UTC](https://swi-prolog.discourse.group/t/building-a-port-scanner/2580/7 "2020-06-30T20:32:50Z") For those that know about [port scanning](https://en.wikipedia.org/wiki/Port_scanner), a typical scan will scan a thousand or more [ports](https://en.wikipedia.org/wiki/List_of_TCP_and_UDP_port_numbers). Also it is better to return no reply if the port is not available which on the client side typically results in a timeout when trying to open the port. Since the timeouts can take a few seconds and multiply that by a thousand, it can take some time. 5 seconds \* 1000 = 5,000 seconds 5,000 seconds \* (1 minute / 60 seconds) = 83.3 minutes 83.3 minutes - 60 minutes/hour = 1 hour 23 minutes Since port scanning can be done in parallel it is worth the effort to use [multithreading](https://www.swi-prolog.org/pldoc/man?section=threads). Update of code using a [pool thread](https://www.swi-prolog.org/pldoc/man?section=threadpool). Based on [test case](https://github.com/SWI-Prolog/swipl-devel/blob/ca64f340ac3977ff68e38a5b231d657ae7e62421/src/Tests/library/test_thread_pool.pl#L80-L91). ```prolog start_scan_pool(Number_of_threads,Options) :- thread_pool_create(scan_pool,Number_of_threads,Options). stop_scan_pool :- thread_pool_destroy(scan_pool). scan_pool(IP_address,Low_port,High_port,Number_of_threads) :- start_scan_pool(Number_of_threads,[]), time( ( findall(Id, ( between(Low_port,High_port,Port), thread_create_in_pool(scan_pool,port_scan(IP_address,Port),Id,[]) ), Ids), join_all(Ids) ) ), stop_scan_pool. % ----------------------------------------------------------------------------- join_all([]). join_all([H|T]) :- thread_join(H, Status), assertion(Status == true), join_all(T). % ----------------------------------------------------------------------------- port_scan(IP_address,Port) :- catch( setup_call_cleanup( tcp_socket(Socket), ( % Open stream socket based on TCP/IP which uses IP address and port number, i.e. INET socket tcp_connect(Socket, IP_address:Port), Result = open ), tcp_close_socket(Socket) ), error(_,_), Result = closed ), format('Port ~w: ~w~n', [Port, Result]). ``` Example run. ```prolog ?- scan_pool('140.211.166.101',1,9,3). Port 2: closed Port 3: closed Port 1: closed Port 5: closed Port 4: closed Port 6: closed Port 7: closed Port 8: closed Port 9: closed % 40,303 inferences, 0.031 CPU in 63.044 seconds (0% CPU, 1289696 Lips) true. ``` A more comprehensive example run. ```prolog ?- scan_pool('140.211.166.101',1,1023,50). ... % 37,861 inferences, 0.047 CPU in 441.204 seconds (0% CPU, 807701 Lips) true. ``` **EDIT** More threads means less time. 1024 threads checking 65536 ports in ~23 minutes. ```prolog ?- scan_pool('140.211.166.101',1,65536,1024). Port 22: open Port 80: open Port 443: open % 2,464,802 inferences, 6.891 CPU in 1369.216 seconds (1% CPU, 357704 Lips) true. ``` 8192 threads checking 65536 ports in ~4.5 minutes. ```prolog ?- scan_pool('140.211.166.101',1,65536,8192). Port 80: open Port 22: open Port 443: open % 2,424,842 inferences, 6.469 CPU in 259.009 seconds (2% CPU, 374855 Lips) true. ``` @anniepoo At [https://www.swi-prolog.org/pldoc/man?section=threadpool](https://www.swi-prolog.org/pldoc/man?section=threadpool) you asked for an example. 🧙 ---
**Author:** ![peter.ludemann](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/peter.ludemann/32/48_2.png) [@peter.ludemann](https://swi-prolog.discourse.group/u/peter.ludemann)\ **Post date:** [June 30, 2020, 8:47pm UTC](https://swi-prolog.discourse.group/t/building-a-port-scanner/2580/8 "2020-06-30T20:47:51Z")
Or just use `concurrent_forall/3` 😉 [Newly added: see [Size limitations on qsave\_program?](https://swi-prolog.discourse.group/t/size-limitations-on-qsave-program/2560/11)] ---
**Author:** ![EricGT](https://avatars.discourse-cdn.com/v4/letter/e/f1d935/32.png) [@EricGT](https://swi-prolog.discourse.group/u/EricGT)\ **Post date:** [June 30, 2020, 8:51pm UTC](https://swi-prolog.discourse.group/t/building-a-port-scanner/2580/9 "2020-06-30T20:51:06Z")
Thanks. 😀 I figured the next and last step was to use [Thread communication](https://www.swi-prolog.org/pldoc/man?section=threadcom) to pass the status of the port back and remove the format/2 from port\_scan/2. Will add concurrent\_forall/3 to the todo list and see what happens. ---
**Author:** ![EricGT](https://avatars.discourse-cdn.com/v4/letter/e/f1d935/32.png) [@EricGT](https://swi-prolog.discourse.group/u/EricGT)\ **Post date:** [June 30, 2020, 9:23pm UTC](https://swi-prolog.discourse.group/t/building-a-port-scanner/2580/11 "2020-06-30T21:23:50Z")
> [@peter.ludemann](#): > > Or must use `concurrent_forall/3` Looks like this is out. In reading the [documentation](http://140.211.15.118/pack/file_details/xlibrary/prolog/concurrent_forall.pl) > The maximum number of threads defined is the amount of cores available. Since the number of cores is now typically about 8,12 or 16 on laptops and my other example ran cleanly with 50 threads, [concurrent\_forall/3](http://140.211.15.118/pack/file_details/xlibrary/prolog/concurrent_forall.pl) will probably be much slower, but then again Jan W. is known to change the code based on forum post. 😀 * * * **EDIT** There are two versions of concurrent\_forall/3. [One](https://github.com/SWI-Prolog/swipl-devel/blob/master/library/thread.pl#L290-L394) is in the standard Prolog and one is in [Package “xlibrary”](https://www.swi-prolog.org/pack/list?p=xlibrary) ---
**Author:** ![peter.ludemann](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/peter.ludemann/32/48_2.png) [@peter.ludemann](https://swi-prolog.discourse.group/u/peter.ludemann)\ **Post date:** [June 30, 2020, 9:39pm UTC](https://swi-prolog.discourse.group/t/building-a-port-scanner/2580/13 "2020-06-30T21:39:18Z")
> [@EricGT](#): > > The maximum number of threads defined is the amount of cores available. The code seems to say something different – that the # of threads defaults to the # of CPUs but otherwise can be anything. @jan ? ```prolog ( option(threads(Jobs), Options) -> true ; current_prolog_flag(cpu_count, Jobs) ), ``` ---
**Author:** ![EricGT](https://avatars.discourse-cdn.com/v4/letter/e/f1d935/32.png) [@EricGT](https://swi-prolog.discourse.group/u/EricGT)\ **Post date:** [June 30, 2020, 10:51pm UTC](https://swi-prolog.discourse.group/t/building-a-port-scanner/2580/14 "2020-06-30T22:51:10Z")
When I went looking for concurrent\_forall/3 I found it first in [Package “xlibrary”](https://www.swi-prolog.org/pack/list?p=xlibrary) but in searching for your example code found the version Jan W. [created](https://github.com/SWI-Prolog/swipl-devel/blob/ca64f340ac3977ff68e38a5b231d657ae7e62421/library/thread.pl#L290-L394). That explains the differences. Now looking at Jan W. version. 😉 ---
**Author:** ![EricGT](https://avatars.discourse-cdn.com/v4/letter/e/f1d935/32.png) [@EricGT](https://swi-prolog.discourse.group/u/EricGT)\ **Post date:** [July 1, 2020, 11:38am UTC](https://swi-prolog.discourse.group/t/building-a-port-scanner/2580/16 "2020-07-01T11:38:27Z")
While I am reading your replies in this post, for the particular current need of the port scanner there is no need to balance out the threads. Each thread is trying to open a port and return only one of two results, `open` or `closed` and when the port is `open` the response is instantaneous and the thread is available in the pool, when the port is `closed` it is due to a timeout which takes ~5 seconds. As only 3 out of 65536 ports are open, most of the threads wait for the timeout then return to the pool. If you start the program with 50 threads in the pool and 1000 ports you will see the results typically get dumped out in a batch of 50 then pause, a batch of 50 then pause, etc. The open ports get dumped out almost immediately. So there is nothing to balance as the threads are typically completing in sync. I do agree that balancing is a desired trait for threading that should be considered when multithreading. For some other tests that I have to write, they might benefit from balancing and I will revisit your replies. 😀 ---
**Author:** ![EricGT](https://avatars.discourse-cdn.com/v4/letter/e/f1d935/32.png) [@EricGT](https://swi-prolog.discourse.group/u/EricGT)\ **Post date:** [July 1, 2020, 12:59pm UTC](https://swi-prolog.discourse.group/t/building-a-port-scanner/2580/18 "2020-07-01T12:59:54Z")
The next version makes use of [message queues](https://www.swi-prolog.org/pldoc/man?section=msgqueue) to pass the result of the port scan back to the main thread. This also adds debug messages from [library(debug)](https://www.swi-prolog.org/pldoc/man?section=debug). ```prolog start_scan_pool_02(Number_of_threads,Options) :- thread_pool_create(scan_pool,Number_of_threads,Options). stop_scan_pool_02 :- thread_pool_destroy(scan_pool). scan_pool_02(IP_address,Low_port,High_port,Number_of_threads) :- message_queue_create(Result_queue), start_scan_pool_02(Number_of_threads,[]), time( ( findall(Id, ( between(Low_port,High_port,Port), thread_create_in_pool(scan_pool,worker(Result_queue,IP_address,Port),Id,[]) ), Ids), join_all(Ids), gather_results(Result_queue,Port_results), format('~w~n',[Port_results]) % Peter once showed me a better predicate to print structures but I can't recall it at the moment. Found it: print_term/2 ) ), stop_scan_pool_02. worker(Result_queue,IP_address,Port) :- debug(threads, 'Worker: port ~w', [Port]), port_scan_02(IP_address,Port,Port_result), thread_send_message(Result_queue, result(Port,Port_result)). port_scan_02(IP_address,Port,Result) :- catch( setup_call_cleanup( tcp_socket(Socket), ( % Open stream socket based on TCP/IP which uses IP address and port number, i.e. INET socket tcp_connect(Socket, IP_address:Port), Result = open ), tcp_close_socket(Socket) ), error(_,_), Result = closed ). gather_results(Result_queue,[result(Port,Port_result)|Port_results]) :- thread_get_message(Result_queue,result(Port,Port_result),[timeout(0)]), debug(threads, 'Result - Port: ~w - Result: ~w', [Port,Port_result]), gather_results(Result_queue,Port_results), !. gather_results(_,[]). ``` Example run ```prolog ?- debug(threads). Warning: threads: no matching debug topic (yet) true. ?- scan_pool_02('140.211.166.101',75,84,3). % [Thread 4] Worker: port 75 % [Thread 5] Worker: port 76 % [Thread 6] Worker: port 77 % [Thread 7] Worker: port 78 % [Thread 8] Worker: port 79 % [Thread 9] Worker: port 80 % [Thread 10] Worker: port 81 % [Thread 11] Worker: port 82 % [Thread 12] Worker: port 83 % [Thread 13] Worker: port 84 % Result - Port: 77 - Result: closed % Result - Port: 75 - Result: closed % Result - Port: 76 - Result: closed % Result - Port: 80 - Result: open % Result - Port: 79 - Result: closed % Result - Port: 78 - Result: closed % Result - Port: 81 - Result: closed % Result - Port: 82 - Result: closed % Result - Port: 83 - Result: closed % Result - Port: 84 - Result: closed [result(77,closed),result(75,closed),result(76,closed),result(80,open),result(79,closed),result(78,closed),result(81,closed),result(82,closed),result(83,closed),result(84,closed)] % 41,514 inferences, 0.016 CPU in 63.136 seconds (0% CPU, 2656896 Lips) true. ``` @jan For the debug messages `% [Thread 4] Worker: port 75` I was not expecting the thread numbers to be all differenent as this is using a pool of threads. Is the code correctly reusing the threads from a pool, am I reading the debug messages incorrectly or something else? ---
**Author:** ![jan](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/jan/32/4_2.png) [@jan](https://swi-prolog.discourse.group/u/jan)\ **Post date:** [July 1, 2020, 1:01pm UTC](https://swi-prolog.discourse.group/t/building-a-port-scanner/2580/19 "2020-07-01T13:01:30Z")
> [@anon95304481](#): > > A [balance/1](https://stackoverflow.com/a/57044280/502187) predicate would be more useful here, than concurrent\_forall/2. > balance/1 does a distributed generate and test: I don’t really like the name _balance_ much as its relation to concurrency is a bit far away to me and you can balance practically everything, but I have little clue what a balanced conjuction is. I do think the behavior is useful and intuitive. I assume this also puts the generator in a separate thread and gives the main thread the task of collecting results? Although you can bootstrap concurrent\_forall/2 on balance/1 it might not be ideal to do so. To me, it seems balance is considerably more complicated and slower due to the fact that we must return and collect the results. ---
**Author:** ![EricGT](https://avatars.discourse-cdn.com/v4/letter/e/f1d935/32.png) [@EricGT](https://swi-prolog.discourse.group/u/EricGT)\ **Post date:** [July 1, 2020, 1:10pm UTC](https://swi-prolog.discourse.group/t/building-a-port-scanner/2580/21 "2020-07-01T13:10:13Z")
> [@jan](#): > > return and collect the results. With multithreaded code I lean toward having the worker threads knowing next to nothing of the outside world. In my earlier versions the worker threads would use format/2 which in my mind says the workers know about the display device. By passing the results back in a message queue, this removes that dependency/knowledge. In reading your reply it seems that there is another view to be learned. The final goal of this specific port scanner is to collect the results and persistent them to a file using [library(persistency)](https://www.swi-prolog.org/pldoc/man?section=persistency). My plan was to collect the results from the message queue then pass and persist them to a file in the main thread but in reading your reply it seems it should be done in the worker thread. Feedback desired? ---
**Author:** ![jan](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/jan/32/4_2.png) [@jan](https://swi-prolog.discourse.group/u/jan)\ **Post date:** [July 1, 2020, 1:10pm UTC](https://swi-prolog.discourse.group/t/building-a-port-scanner/2580/22 "2020-07-01T13:10:40Z")
> [@EricGT](#): > > For the debug messages `% [Thread 4] Worker: port 75` I was not expecting the thread numbers to be all differenent as this is using a pool of threads. Is the code correctly reusing the threads from a pool, am I reading the debug messages incorrectly or something else? A thread\_pool could be a misleading name. It is a wrapper around thread\_create/3 that limits the number of threads you can create and make subsequent calls to create a thread either fail with an error or wait until some thread in the pool stopped. Thread IDs can only be reused after you join the ended thread. I’d use concurrent\_forall/3 using the threads(Count) option and assert the port status in the database to be collected later. Properly managing a set of threads and ensuring they are properly reclaimed is quite complex. Since some time SWI-Prolog will GC forgotten threads, but it can take a while before it realises they are forgotten. ---
**Author:** ![EricGT](https://avatars.discourse-cdn.com/v4/letter/e/f1d935/32.png) [@EricGT](https://swi-prolog.discourse.group/u/EricGT)\ **Post date:** [July 1, 2020, 3:12pm UTC](https://swi-prolog.discourse.group/t/building-a-port-scanner/2580/25 "2020-07-01T15:12:40Z")
The next version switches to using concurrent\_forall/3. Since this predicate is so new (3 days old) and I run on Windows, I installed the Windows 64-bit version of the [daily build](https://www.swi-prolog.org/download/daily/bin/). `SWI-Prolog (threaded, 64 bits, version 8.3.2-198-gd839164c7)` Note: This code also moved executing `debug/1` on the command line into a Prolog directive `:- debug(concurrent).` which can easily be commented out. If you peruse the SWI-Prolog source code on GitHub you will often find these lines commented out. Also notice how much simpler the code becomes when using concurrent\_forall/3. The hardest part about writing this was trying to understand `concurrent_forall(:Generate, :Test)`, how did `Generate` and `Test` align with my existing code. So instead of trying to understand the code from the top down I looked at the critical predicate common to all of this which is thread\_create/2 and in concurrent\_forall/2 is in the line `maplist(thread_create(fa_worker(Q, Me, Templ, Test)), Workers)` and just figured out which variables I could set and what they needed. The only one that can be set by calling concurrent\_forall/3 is `Test` which needs to be `port_scan(IP_address,Port)`, after that identifying what the rest of concurrent\_forall needed was easy. For this use of concurrent\_forall instead of `concurrent_forall(:Generate, :Test)` I think of it as `concurrent_forall(:Generate unique threads values, :Call thread with unique values)`. ```prolog :- debug(concurrent). concurrent_scan(IP_address,Low_port,High_port,Number_of_threads) :- concurrent_forall( between(Low_port,High_port,Port), port_scan(IP_address,Port), [threads(Number_of_threads)] ). port_scan(IP_address,Port) :- catch( setup_call_cleanup( tcp_socket(Socket), ( % Open stream socket based on TCP/IP which uses IP address and port number, i.e. INET socket tcp_connect(Socket, IP_address:Port), format('Port ~w: open~n', [Port]) ), tcp_close_socket(Socket) ), error(_,_), true ). ``` Example run ```prolog ?- concurrent_scan('140.211.166.101',75,84,3). % [Thread 5] Running test user:port_scan('140.211.166.101',77) % [Thread 3] Running test user:port_scan('140.211.166.101',76) % [Thread 4] Running test user:port_scan('140.211.166.101',75) % [Thread 5] Running test user:port_scan('140.211.166.101',78) % [Thread 4] Running test user:port_scan('140.211.166.101',79) % [Thread 3] Running test user:port_scan('140.211.166.101',80) Port 80: open % [Thread 3] Running test user:port_scan('140.211.166.101',81) % [Thread 5] Running test user:port_scan('140.211.166.101',82) % [Thread 4] Running test user:port_scan('140.211.166.101',83) % [Thread 3] Running test user:port_scan('140.211.166.101',84) true. ``` NB The threads are being reused this time. A more comprehensive example. (debug/1 was commented out.) ```prolog ?- time(concurrent_scan('140.211.166.101',1,65536,8192)). Port 80: open Port 22: open Port 443: open % 196,643 inferences, 0.500 CPU in 174.439 seconds (0% CPU, 393286 Lips) true. ``` 8192 threads checking 65536 ports in ~3 minutes. * * * The comment Jan W. made about > [@jan](#): > > return and collect the results. and that I asked about now makes more sense. If you read the code for concurrent\_forall you will notice that to pass messages back would require adding more complexity to something that is already very complex. So I take it to mean that if you want to use concurrent\_forall then use it as designed, even it is breaking some rules of thumb such as have the threads know as little as possible about the outside world. ---
**Author:** ![jan](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/jan/32/4_2.png) [@jan](https://swi-prolog.discourse.group/u/jan)\ **Post date:** [July 1, 2020, 4:07pm UTC](https://swi-prolog.discourse.group/t/building-a-port-scanner/2580/26 "2020-07-01T16:07:50Z")
Just wondering what it would mean to write this in SWI-Prolog, I got to these two files. [test\_balance.pl](https://swi-prolog.discourse.group/uploads/short-url/sryV8i5kAeMAwHmQqlCFt31YEMu.pl) (1.1 KB) [balance.pl](https://swi-prolog.discourse.group/uploads/short-url/v3j67MTuVYNtCG97q1EdUT9jeII.pl) (4.2 KB) The implementation is rather tricky. I did decide for an arity 2 version for now to make clean what the generator and tester are. Not yet sure what to do with it. When matured and with a proper name, add it to the `library(thread)`, I guess. ---
**Author:** ![EricGT](https://avatars.discourse-cdn.com/v4/letter/e/f1d935/32.png) [@EricGT](https://swi-prolog.discourse.group/u/EricGT)\ **Post date:** [July 1, 2020, 4:40pm UTC](https://swi-prolog.discourse.group/t/building-a-port-scanner/2580/28 "2020-07-01T16:40:42Z")
The next version adds [library(persistency)](https://www.swi-prolog.org/pldoc/man?section=persistency). ```prolog :- use_module(library(persistency)). :- working_directory(_,'C:\\Users\\Eric\\Documents\\Port Scans'). :- persistent port_scan_result(port:integer,result:atom). :- initialization(db_attach('port_scan_result.journal', [])). exists_port_scan_result(Request,Response) :- port_scan_result(Request,Response). add_port_scan_result(Request,Response) :- with_mutex(port_scan_result_journal, assert_port_scan_result(Request,Response)). concurrent_scan_02(IP_address,Low_port,High_port,Number_of_threads) :- concurrent_forall( between(Low_port,High_port,Port), port_scan_02(IP_address,Port), [threads(Number_of_threads)] ). port_scan_02(IP_address,Port) :- catch( setup_call_cleanup( tcp_socket(Socket), ( % Open stream socket based on TCP/IP which uses IP address and port number, i.e. INET socket tcp_connect(Socket, IP_address:Port), ( exists_port_scan_result(Port,open), ! ; add_port_scan_result(Port,open) ) ), tcp_close_socket(Socket) ), error(_,_), ( exists_port_scan_result(Port,closed), ! ; add_port_scan_result(Port,closed) ) ). ``` Example run. NB `halt.` is needed so that the data is written to the file. Until `halt` all of the data resides as facts in the Prolog database. ```prolog ?- concurrent_scan_02('140.211.166.101',75,84,3). true. ?- halt. ``` File: `port_scan_result.journal` ```prolog created(1593621240.60769). assert(port_scan_result(77,closed)). assert(port_scan_result(76,closed)). assert(port_scan_result(75,closed)). assert(port_scan_result(80,open)). assert(port_scan_result(78,closed)). assert(port_scan_result(79,closed)). assert(port_scan_result(81,closed)). assert(port_scan_result(83,closed)). assert(port_scan_result(82,closed)). assert(port_scan_result(84,closed)). ``` A more comprehensive example. ```prolog ?- time(concurrent_scan_02('140.211.166.101',1,65536,8192)). % 196,642 inferences, 0.547 CPU in 175.149 seconds (0% CPU, 359574 Lips) true. ``` 8192 threads checking 65536 ports in ~3 minutes. [Next page](https://swi-prolog.discourse.group/t/building-a-port-scanner/2580.md?page=2)