# Building a port scanner **URL:** \ **Category:** Help!\ **Created:** [June 30, 2020, 12:40pm UTC](https://swi-prolog.discourse.group/t/building-a-port-scanner/2580 "2020-06-30T12:40:46Z")\ **Posts on this page:** 8\ **Page:** 2
**Author:** ![EricGT](https://avatars.discourse-cdn.com/v4/letter/e/f1d935/32.png) [@EricGT](https://swi-prolog.discourse.group/u/EricGT)\ **Post date:** [July 1, 2020, 7:09pm UTC](https://swi-prolog.discourse.group/t/building-a-port-scanner/2580/32 "2020-07-01T19:09:45Z")
AFAIK with\_mutex/2 is not needed in add\_port\_scan\_result/2 because there should only ever be one fact for each port and once the facts are generated after the first time I don’t ever expect them to change. Also only one thread will be adding the new fact for the specific port once, that thread will not be competing with other threads and thus no lock is needed. Since this was created to test the open ports of a specific site if something changes then there is something very wrong somewhere. Since there are so few examples of working library(persistency) code to be found, I am leaving with\_mutex/2 for the write in because people are likely to copy this and if the with\_mutex/2 were missing they would be wondering why on rare days their code does not work. **EDIT** Starting with SWI-Prolog 8.3.3 library(persistency) ([GitHub Commit](https://github.com/SWI-Prolog/swipl-devel/commit/063ac3994577b6fd066d5bfd60f96ed125361cf5)) was made more thread friendly by wrapping the four predicates with with\_mutex/2. So ```prolog add_port_scan_result(Request,Response) :- with_mutex(port_scan_result_journal, assert_port_scan_result(Request,Response)). ``` can be changed to ```prolog add_port_scan_result(Request,Response) :- assert_port_scan_result(Request,Response). ``` From [documentation](https://www.swi-prolog.org/pldoc/man?section=persistency) > This module requires the same thread-synchronization as the normal Prolog database. This implies that if each individual assert or retract takes the database from one consistent state to the next, no additional locking is required. If more than one elementary database operation is required to get from one consistent state to the next, both updating and querying the database must be locked using [with\_mutex/2](https://www.swi-prolog.org/pldoc/man?predicate=with_mutex/2). ---
**Author:** ![jan](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/jan/32/4_2.png) [@jan](https://swi-prolog.discourse.group/u/jan)\ **Post date:** [July 1, 2020, 7:55pm UTC](https://swi-prolog.discourse.group/t/building-a-port-scanner/2580/35 "2020-07-01T19:55:33Z")
The persistency library requires the same locking as normal query/assert/retract: as long as each individual assert/retract moves from one consistent state to the next, no locking is required. Only if moving from one consistent state to the next requires multiple retract/retract operations locking is required, both for the reader and writer. Eventually, transactions will require multiple modifications to be wrapped in a transaction and query to require nothing special. ---
**Author:** ![EricGT](https://avatars.discourse-cdn.com/v4/letter/e/f1d935/32.png) [@EricGT](https://swi-prolog.discourse.group/u/EricGT)\ **Post date:** [July 1, 2020, 7:56pm UTC](https://swi-prolog.discourse.group/t/building-a-port-scanner/2580/36 "2020-07-01T19:56:43Z")
> [@jan](#): > > Eventually, transactions will require multiple modifications to be wrapped in a transaction and query to require nothing special. I don’t know if I am understanding that sentence correctly. My take is that this will be like SQL databases where one wraps a series of statements in a transaction and once the transaction completes or is committed will the updates occur. But the use of the word `eventually` means it does not exist at present. That does make sense. ---
**Author:** ![jan](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/jan/32/4_2.png) [@jan](https://swi-prolog.discourse.group/u/jan)\ **Post date:** [July 1, 2020, 8:33pm UTC](https://swi-prolog.discourse.group/t/building-a-port-scanner/2580/38 "2020-07-01T20:33:58Z")
> [@anon95304481](#): > > [format/3](https://www.swi-prolog.org/pldoc/doc_for?object=format/3) could be atomic if it uses a stream lock. format/3 is atomic, like all the core stream write predicates. ---
**Author:** ![jan](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/jan/32/4_2.png) [@jan](https://swi-prolog.discourse.group/u/jan)\ **Post date:** [July 2, 2020, 7:30am UTC](https://swi-prolog.discourse.group/t/building-a-port-scanner/2580/43 "2020-07-02T07:30:19Z")
Here are the results on the modified code. This is the latest version compiled with GCC 9 using PGO, running on Ubuntu 20.04, AMD 3950X, 16 cores, 32 threads. As you see, up to 16 threads the speedup is close to optimal (14 times). Hyperthreading apparently does very little in this case. The code is attached. ```prolog 18 ?- time(count(N)). % 69,907,410 inferences, 3.583 CPU in 3.589 seconds (100% CPU, 19512169 Lips) N = 78499. 19 ?- time(count2(N,1)). % 4,104 inferences, 0.014 CPU in 3.536 seconds (0% CPU, 287142 Lips) N = 78499. 20 ?- time(count2(N,2)). % 4,132 inferences, 0.014 CPU in 1.786 seconds (1% CPU, 293717 Lips) N = 78499. 21 ?- time(count2(N,4)). % 4,202 inferences, 0.024 CPU in 0.899 seconds (3% CPU, 178360 Lips) N = 78499. 22 ?- time(count2(N,8)). % 4,392 inferences, 0.005 CPU in 0.468 seconds (1% CPU, 937750 Lips) N = 78499. 23 ?- time(count2(N,16)). % 4,950 inferences, 0.005 CPU in 0.250 seconds (2% CPU, 1055806 Lips) N = 78499. 24 ?- time(count2(N,32)). % 6,832 inferences, 0.007 CPU in 0.213 seconds (3% CPU, 1041262 Lips) N = 78499. 25 ?- A is 3.536/0.250. A = 14.144. ``` [primes.pl](https://swi-prolog.discourse.group/uploads/short-url/vvVnEb55cmIwMbHRUN4wOosuNhI.pl) (553 Bytes) ---
**Author:** ![jan](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/jan/32/4_2.png) [@jan](https://swi-prolog.discourse.group/u/jan)\ **Post date:** [July 2, 2020, 7:53am UTC](https://swi-prolog.discourse.group/t/building-a-port-scanner/2580/44 "2020-07-02T07:53:29Z")
Right. Added locks to the basic update predicates. This still requires the user to add another layer of locks if multiple related changes are needed. Updated the docs to explain this more clearly. Thanks for spotting. ---
**Author:** ![EricGT](https://avatars.discourse-cdn.com/v4/letter/e/f1d935/32.png) [@EricGT](https://swi-prolog.discourse.group/u/EricGT)\ **Post date:** [July 2, 2020, 4:12pm UTC](https://swi-prolog.discourse.group/t/building-a-port-scanner/2580/46 "2020-07-02T16:12:39Z")
Today Jan W. released [SWI-Prolog 8.3.3](https://swi-prolog.discourse.group/t/ann-swi-prolog-8-3-3/2588) which included a modified version [balance/2,3](https://swi-prolog.discourse.group/t/building-a-port-scanner/2580/26) named concurrent\_and/2,3. So I gave it a try. The first thing I learned the hard way is that you have to accept each answer, e.g. press the space bar after **true** is displayed. ```prolog concurrent_scan(IP_address,Low_port,High_port,Number_of_threads) :- concurrent_and( between(Low_port,High_port,Port), port_scan(IP_address,Port), [threads(Number_of_threads)] ). port_scan(IP_address,Port) :- catch( setup_call_cleanup( tcp_socket(Socket), ( % Open stream socket based on TCP/IP which uses IP address and port number, i.e. INET socket tcp_connect(Socket, IP_address:Port), format('Port ~w: open~n', [Port]) ), tcp_close_socket(Socket) ), error(_,_), true ). ``` Example run ```prolog ?- concurrent_scan('140.211.166.101',79,82,3). Port 80: open true ; true ; true ; true ; false. ``` Adding `fail` to the end of the query will fix having to press the space bar but this is not the way concurrent\_and/2 is designed to work, e.g. ```prolog ?- concurrent_scan('140.211.166.101',79,82,3),fail. Port 80: open false. ``` If one looks at the test cases in test\_balance.pl one sees the use of setof/3. Incorporating setof/3 leads to ```prolog concurrent_scan_02(IP_address,Low_port,High_port,Number_of_threads,Filtered_results) :- setof( Port-Result, concurrent_and( between(Low_port,High_port,Port), port_scan_02(IP_address,Port,Result), [threads(Number_of_threads)] ), Results ), include(filter,Results,Filtered_results). port_scan_02(IP_address,Port,Result) :- catch( setup_call_cleanup( tcp_socket(Socket), ( % Open stream socket based on TCP/IP which uses IP address and port number, i.e. INET socket tcp_connect(Socket, IP_address:Port), Result = open ), tcp_close_socket(Socket) ), error(_,_), Result = closed ). filter(Port-open). ``` Example run ```prolog ?- concurrent_scan_02('140.211.166.101',79,82,3,Results). Results = [80-open]. ``` A more comprehensive example. ```prolog ?- time(concurrent_scan_02('140.211.166.101',1,65536,8192,Results)). % 135,209,226 inferences, 16.281 CPU in 187.520 seconds (9% CPU, 8304597 Lips) Results = [22-open, 80-open, 443-open]. ``` 8192 threads checking 65536 ports in ~3 minutes. ---
**Author:** ![jan](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/jan/32/4_2.png) [@jan](https://swi-prolog.discourse.group/u/jan)\ **Post date:** [July 3, 2020, 8:22am UTC](https://swi-prolog.discourse.group/t/building-a-port-scanner/2580/49 "2020-07-03T08:22:20Z")
> [@anon95304481](#): > > ever used for something parallel Probably not. On the other hand I don’t think I’d use a lock. Worst case you have a couple of redundant entered/1 clauses. They should not affect the result and it is pretty unlikely there will be so many that it has significant impact on memory usage. [Previous page](https://swi-prolog.discourse.group/t/building-a-port-scanner/2580.md?page=1)