I want to read a stream that contains newlines to end-of-file character

rara · September 5, 2020, 3:37pm

Question on tcp connection between java and SWI-prolog.

I have swi-prolog on the server side.
I have java on client side.

I send a stream of strings from java to prolog containing a newline.

[Read_line_to_codes] is
Reading stops at the newline or end-of-file character.

But I want to read to the end of the file character without stopping at a newline.

This is because the stream I send from the client side to the server side contains some newlines.

I don’t know how to rewrite read_line_to_codes.

I’d like to ask you for your help.

Prolog`s Code_is ↓.Currently this code only reads up to the newline.

server(Port) :-
tcp_socket(Socket),
tcp_bind(Socket, Port),
tcp_listen(Socket, 5),
tcp_open_socket(Socket, In, _Out),
add_stream_to_pool(In, accept(Socket)), 
stream_pool_main_loop.

accept(Socket) :-

tcp_accept(Socket, Slave, Peer),
tcp_open_socket(Slave, In, Out), 
add_stream_to_pool(In, client(In, Out, Peer)).

client(In, Out, _Peer) :-

read_line_to_codes(In, Codes),
close(In),
format('~s  ~n', [Codes]),
format(Out, '~s~n', [Codes]),
format('Send = > ~s~n',[Codes]),
flush_output(Out),
close(Out),
delete_stream_from_pool(In).

Boris · September 5, 2020, 5:48pm

If you want to read everything at once, until end of file, you can just use:

read_string(In, Length, String)

Just a nitpick, “end of file” is not a character, this is the state of the stream once the end of the stream has been reached.

peter.ludemann · September 5, 2020, 7:23pm

How is the data serialized? If you’re using JSON, there’s http_read_json_dict/3, which ends up calling json_read/3. If not, the code in https://github.com/SWI-Prolog/packages-http/blob/4d5ff5575fba6df849e57caa1ef815993c0377ea/json.pl can give you some ideas.

rara · September 6, 2020, 6:10am

Dear,Boris

It’s very much appreciated. I could read it.

But I’m still in trouble.Could you please help me?

now code is

server(Port) :-
    tcp_socket(Socket),
    tcp_bind(Socket, Port),
    tcp_listen(Socket, 5),
    tcp_open_socket(Socket, In, _Out),
    add_stream_to_pool(In, accept(Socket)), 
    stream_pool_main_loop.

accept(Socket) :-
    tcp_accept(Socket, Slave, Peer),
    tcp_open_socket(Slave, In, Out), 
    add_stream_to_pool(In, client(In, Out, Peer)).

client(In, Out, _Peer) :-
    read_string(In,35,String),
    close(In),
    format('~s  ~n', [String]),
    format(Out, '~s~n', [String]),
    flush_output(Out),
    close(Out),
    delete_stream_from_pool(In).

Length of readline argument is intenger.
I have to specify the number of characters to read.
For example, if these two files are randomly sent from java,I try to send from java to prolog.

client_send.txt

SWI-prolog
SWI
prolog
Server
Client

client_send2.txt

SWI-prolog
SWI
prolog
Server
ClientSWI-prolog
SWI
prolog
Server
Client

In case only client_send.txt, read_string(In,35,String) is perfect,

But in case sending client_send2.txt, read_string(In,35,String) is not perfect.
It’s reading only half of client_send2.txt.

I want to be able to read all the characters even if the character streams from both files are sent.
Also, even if the file (client_send.txt or client_send2.txt) is sent multiple times in succession,
I want to be able to read everything using Repeat etc.

In other words, I want to read all characters even if the file is sent many times from java after connecting once with tcp.

How should I write it?
Could you please help me?

rara · September 6, 2020, 6:22am

Dear,peterludemann

Thanks for the advice

not json
Read the contents of the file specified by java with Fileinputstream,
It is the flow to send with the stream by executing (getOutputStream.write) to swi-prolog

Boris · September 6, 2020, 9:41am

Take a looks at the docs for read_string/3.

If you use a free variable in the second argument, it will be unified with the length of the string in the third argument. If you don’t care about the length, you can just use an anonymous variable.

rara · September 6, 2020, 11:12am

Dear Boris

I was reading it.
Perhaps I may not understand the sentence

read_string(In,_,String),
or
read_string(In,X,String),

I tried, but nothing is displayed with format…

read_string(In,Length,String),
It is displayed with format() only when (Length=the number of characters to read or less).

Boris · September 6, 2020, 6:34pm

Can you provide a reproducible example? Maybe without Java? Does read_string/3 behave as expected if you read from standard input?

How are you printing with format?

peter.ludemann · September 6, 2020, 8:22pm

The documentation for read_string/3 says:

read_string(+Stream, ?Length, -String)

Read at most Length characters from Stream and return them in the string String. If Length is unbound, Stream is read to the end and Length is unified with the number of characters read.

So, if you do read_string(Stream, StringLength, String) you’ll read everything into String and StringLength will unify with the length of that string (assuming StringLength is unbound; otherwise see the next paragraph). You can also do read_string(Stream, _, String) to get the same result (if you want the length, use string_length/2).

If you do read_string(Stream, 1000, String), then at most 1000 characters will be read into String. You can also do MaxRead=1000, read_string(Stream, MaxRead, String), to get the same result.

rara · September 7, 2020, 3:07am

Dear Boris
Thank you for your reply

I paste the source code and the execution result.

java’client side is Client01.java

import java.net.*;
import java.io.*;


public class Client01 {

    private Client01() {
    }

    public static void main(String[] args) {

        Socket socket = null;
        FileOutputStream fos = null;
        FileInputStream fis = null;
        try {
            System.out.println("start.");

            
            socket = new Socket("localhost", 2011);

         
            fis = new FileInputStream("client_send.txt");
            fos = new FileOutputStream("client_recv.txt");

          
            int ch;
            OutputStream output = socket.getOutputStream();
            while ((ch = fis.read()) !=-1) {
                output.write(ch);
                System.out.print("ch");
            }
           
            
            InputStream input = socket.getInputStream();
            while ((ch = input.read()) != -1) {
                fos.write(ch);
                System.out.print("ch2");
            }
        } catch (SocketException e) {
            System.out.println("occured SocketException.");
        } catch (Exception e) {
            System.out.println("occured Exception.");
        } finally {
            try {
                socket.close();
                fos.close();
                fis.close();
            } catch (IOException e) {
                System.out.println("occured IOException.");
            }
            System.out.println("end.");
        }
    }
}

java read client_send.txt.and send it to prolog

client_send.txt

                SWI-prolog
                SWI
                prolog
                Server
                Client

swi-prolog side(server)

server(Port) :-
    tcp_socket(Socket),
    tcp_bind(Socket, Port),
    tcp_listen(Socket, 5),
    tcp_open_socket(Socket, In, _Out),
    add_stream_to_pool(In, accept(Socket)), 
    stream_pool_main_loop.

accept(Socket) :-
    tcp_accept(Socket, Slave, Peer),
    tcp_open_socket(Slave, In, Out), 
    add_stream_to_pool(In, client(In, Out, Peer)).

client(In, Out, _Peer) :-
    read_string(In,15,String),
    close(In),
    format('~s  ~n', [String]),
    format(Out, '~s~n', [String]),
    flush_output(Out),
    close(Out),
    delete_stream_from_pool(In).

The execution result is shown here.

In case of read_string(In,15,String),

prolog side

?- server(2011).

SWI-prolog
SWI

Client_side occcur[ Socket Exception’error ]when client try sending the 16th character.

In case of read_string(In,30,String),

    ?- server(2011).

        SWI-prolog
        SWI
        prolog
        Server
        C

Client_side occcur[ Socket Exception’error ]when client try sending the 31th character.

In case of read_string(In,35,String),

      ?- server(2011).

                    SWI-prolog
                    SWI
                    prolog
                    Server
                    Client

Client_side no error.
It’s wonderful.

If there is no fixed number like 35,I hope result.

In case of read_string(In,36,String),
?- server(2011).

It’s no disply of format.

Client_side no error.

I think,Probably Because the prolog is waiting for the 36th character to be sent.

But forced termination the running client side using ^c,↓ is showing.

     ?- server(2011).

                    SWI-prolog
                    SWI
                    prolog
                    Server
                    Client

In case of read_string(In,_,String),
this is same result when read_string(In,36,String),

?- server(2011).

It’s no disply of format.

Client_side no error.

But forced termination the running client side using ^c,↓ is showing.

     ?- server(2011).

                SWI-prolog
                SWI
                prolog
                Server
                Client

I hope representing result whithout forcing termination the running client side using ^c

The above is the execution result.

Boris · September 7, 2020, 4:26am

Sorry if I am not getting your point. It still seems that you are trying to put an integer in the second argument of read_string/3?

The point is, do not put an integer. Put a free variable if you need to know the length of the input, or an anonymous variable if you don’t care about it.

I took your client_send.txt and put it in a file on my computer, looks like this:

$ cat client_send.txt 
SWI-prolog
SWI
prolog
Server
Client

Now, I will read this file from standard input and echo it to standard output. I don’t care about the length of the input so I am putting an anonymous variable in the second argument of read_string/3.

$ < client_send.txt swipl -g 'read_string(current_input, _, Str), format("~s", [Str])' -t halt
SWI-prolog
SWI
prolog
Server
Client

In the next example, I only want to write to standard output the length of the input.

$ < client_send.txt swipl -g 'read_string(current_input, N, _), format("~d~n", [N])' -t halt
36

Am I still misunderstanding your question? There is definitely some confusion somewhere.

rara · September 7, 2020, 6:21am

Dear Boris .
In case open and read file,read_string(current_input, _, Str)
→ success

In case of instream read_string(In,_,String),
→Fail

Server(Port) :-
    tcp_socket(Socket),
    tcp_bind(Socket, Port),
    tcp_listen(Socket, 5),
    tcp_open_socket(Socket, In, _Out),
    add_stream_to_pool(In, accept(Socket)), 
    stream_pool_main_loop.

accept(Socket) :-
    tcp_accept(Socket, Slave, Peer),
    tcp_open_socket(Slave, In, Out), 
    add_stream_to_pool(In, client(In, Out, Peer)).

client(In, Out, _Peer) :-
    read_string(In,_,String),
    close(In),
    format('~s  ~n', [String]),
    format(Out, '~s~n', [String]),
    flush_output(Out),
    close(Out),
    delete_stream_from_pool(In).

If you save and execute this code as it is, it will be using an anonymous variable
This way of writing should work.
Somehow it doesn’t work ,in case of stream

?- server(2011).

Nothing is displayed,

Client_side no error.

But forced termination the running client side using ^c,↓ is showing.

     ?- server(2011).

                SWI-prolog
                SWI
                prolog
                Server
                Client

I hope representing result whithout forcing termination the running client side using ^c

Boris:

In the next example, I only want to write to standard output the length of the input.
$ < client_send.txt swipl -g 'read_string(current_input, N, _), format("~d~n", [N])' -t halt
36
Am I still misunderstanding your question? There is definitely some confusion somewhere.

It is this point that I want you to pay attention to.

When you execute it, 36 will automatically be entered in the anonymous variable.

But In case of instream read_string(In,36,String),
?- server(2011).

It’s no disply of format.

Client_side no error.

I think,Probably Because the prolog is waiting for the 36th character to be sent.

But forced termination the running client side using ^c,↓ is showing.

     ?- server(2011).

                    SWI-prolog
                    SWI
                    prolog
                    Server
                    Client

I feel that the behavior is different when opening and reading a file and when reading from an in-stream.

in case of opening and reading a file ,with read_string(current_input, _, Str)

SWI-prolog
SWI
prolog
Server
Client

It’s perfect.

But in case of reading a instream ,with read_string(In, _, String)

?- server(2011).

Nothing is displayed.

But forced termination the running client side using ^c,↓ is showing.

     ?- server(2011).

                    SWI-prolog
                    SWI
                    prolog
                    Server
                    Client

Actually, client_send.txt swipl -g ‘read_string(current_input, 38, Str), format(“~s”, [Str])’ -t halt
SWI-prolog

     ?- server(2011).

                    SWI-prolog
                    SWI
                    prolog
                    Server
                    Client

In case of Instream,read_string(In, 38, String),
?- server(2011).

Nothing is displayed.
This change read_string(In, 38, String) to read_string(In _, String) is same result.

Boris · September 7, 2020, 6:36am

You need to show the complete code you are using. Loading certain libraries using:

:- use_module(...).

can redirect both the input and the output. At this point, this is not a difficulty with read_string/3, it is something else for certain.

peter.ludemann · September 7, 2020, 6:48am

Because read_line/3 is waiting for the end-of-stream, which it sees only if the client is closed. If you want to see input before end-of-stream, then specify a length, but you have to deal with figuring out how much you want to read – you can do character at a time, if you wish. Or, try read_string/5.

You might want to try using nc -vv localhost 2011 instead of your Java client (might need options -N and -q 1)

Boris · September 7, 2020, 7:00am

Now, on second thoughts, maybe the Java side is not closing the stream?

In other words, yes, @peter.ludemann most probably diagnosed your problem correctly.

rara · September 7, 2020, 3:56pm

Dear peter.ludemann,Boris

That’s what I was in trouble.
I’m glad you guessed it from my poor English.
Thanks so much, thank you very much.

peter.ludemann · September 7, 2020, 5:45pm

In general, network programming is tricky. That’s why it’s best to not use raw sockets but to use something higher level. I haven’t programmed in Java for years, but my understanding is that HTTPClient and HTTPRequest provide similar functionality to JavaScript’s fetch; and on the Prolog side, you can use http_server. (I’m in the process of writing a minimal client-server example, but haven’t finished it yet.)

For testing, utilities such as netcat (ncat, nc) are very useful, because they’ve been used for years and have had their bugs and idiosyncrasies worked out. You can also look at their source code; the code is fairly simple once you get past all the options. (I would also recommend doing your first development on a Linux or Unix system; Windows adds some additional complications.) If you really can’t figure out what’s going on, use Wireshark or tcpdump to look at the packets and flows – it’s quite easy to use and is very instructive.

Topic		Replies	Views
Read_string/3 counts code points, right? Help!	5	443	January 28, 2022
How detect end of file while reading JSON? Predicate	0	314	April 15, 2023
Interfacing with another program -- how do i read in program data (not just strings/codes) Help!	2	365	July 7, 2019
Reading socket input stream until period is encountered Help!	5	367	February 23, 2020
Read a number of bytes from a stream Help!	2	216	October 2, 2023

I want to read a stream that contains newlines to end-of-file character

Related Topics