Client requests besides GET?

I’ve been using GETs with http_open/3 extensively and that’s been working great but I’m a little lost in the documentation about how to do POSTs and DELETEs.

The doc says

method(+Method)
One of get (default), head, delete, post, put or patch. The head message can be used in combination with the header(Name, Value) option to access information on the resource without actually fetching the resource itself. The returned stream must be closed immediately.

If post(Data) is provided, the default is post.

so… I’m not super clear. Am I doing this?

http_open(Endpoint,Resp,[
        authorization(bearer(Key)),
        post(_{username:foo,password:123})
    ]),

Does that work? Will http_open/3 encode a prolog json into a proper http json for the post?

What’s the difference between this and all the other post options like http_post/4?

What about DELETEs? Do I do

http_open(Endpoint,Resp,[
        authorization(bearer(Key)),
        method(delete)
    ]),

and again what’s the difference between this and other options like http_delete/3

Here is some sample code that posts json to a url, and the reply is returned as a Dict. It uses http_post/4.

:- use_module(library(http/http_client)).
:- use_module(library(http/http_json)).
:- use_module(library(http/json)).

json_post(Url,JsonDict,Reply,Opts) :-
   current_prolog_flag(float_overflow, Old),
   setup_call_cleanup(
      % parse float overflows as +inf or -inf
      set_prolog_flag(float_overflow, infinity),
      http_post(Url, json(JsonDict),Reply,
                [ json_object(dict),end_of_file(eof)|Opts ]),
      % return flag to original value
      set_prolog_flag(float_overflow, Old)
   ).

Thanks @swi I’m trying to read about it but I’m not clear on what the float_overflow flag is for at all. Is setting it necessary for making post calls?

Same with the options; If I’m already saying json(JsonDict) in arg2 do I also need an option for json_object(dict)? Is that redundant or do they do two different things?

Ditto for the end_of_file(eof); What’s this for? What file is that referring to? Is it necessary? I see nothing about that option, neither in the doc for http_post/4, nor http_post_data/3 which it uses.

And again, what does post data actually look like from prolog’s side? Was I correct with _{username:foo,password:123}?

I’m so confused :frowning:

no, it is not necessary. It is just there in case the server replies with some number outside of the floating point range. Without that flag an exception is thrown, but that may be okay for your use case.

If we receive from the reply stream of the server a json object then json_read/3 is called (see http_read_json/3). The end_of_file(eof) option tells json_read/3 how to handle the situation when the stream ends. Again, it is not a necessary option, it can be useful in some use cases.

Yes, that is correct, but you need to pass the dict as json(Dict) to http_post/4.

json(JsonDict) is for what you are posting to the server. The json_object(dict) is for the reply from the server, if it happens to be a json object. json_object(dict) is an option for http_read_json/3 (which later on calls json_read/3 as mentioned above).

You can call the predicate above like this:

json_post('http://127.0.0.1:8080/some_api',_{text:'hello how are you'},R,[]).
R = _{reply:'I am fine'}.

EDIT: BTW, http_open/3 is a lower level predicate; http_get/3, http_post/4, http_delete/3 are higher level predicates, so it is better to use those.

Thanks @swi !! Appreciate the explanation. Let me review and test this weekend.

@swi just one follow up, using http_get instead of http_open, it looks like the response comes back as a json, right? How do you convert that into a prolog dictionary? Previously with http_open I was using json_read_dict/2 but looks like that doesn’t work with http_get as it gives me

ERROR: Domain error: `stream_or_alias' expected, found `[json ....

What should I use instead please?

To tell http_get/3 to give you json as a Dict, you need to do two things:

  1. Make sure you call :-use_module(library(http/http_json)) before using http_get/3. When this module is loaded it installs a hook that allows http_open, to handle json data properly (http_open is used internally by http_get).
    1.1 If you do not load this module then http_get/3 will simply return an atom, like:
   '{"reply" : "my json reply from the server"}'
  1. You need to pass the json_object(dict) option to http_get/3. This converts the incoming data into a prolog Dict.
    2.1 If you do not pass the option, you will get an old version of json in prolog like this:
   json([reply='my json reply from the server'])
  • Only if you do both 1 and 2 above, you will get a proper prolog dict, like this:
   _{ reply: "my json reply from the server"}

In summary you need to make your call like this:

:- use_module(library(http/http_client)).   % for http_get/3
:- use_module(library(http/http_json)).     % hook to allow json processing in http_open

get_json(JsonDict) :-
   http_get('http://127.0.0.1:8080/myapi',JsonDict,[json_object(dict)]).

You don’t provide any code, so I am assuming you are either not loading the http/http_json module or did not include the json_object(dict) option.

Thanks @swi . Sorry, the code was

api_call(key_list,Key,Resp) :-
    api_endpoint(key_list,Endpoint),
    http_get(Endpoint,Resp,[
        authorization(bearer(Key)),
        json_object(dict) %just added
    ]).

I did have the library imported so I was just missing the option.

Two remaining questions:

  1. I’m noticing that for responses like Resp = ['$VAR'('_'){id:123},'$VAR'('_'){id:234},'$VAR'('_'){id:345}]. even though they look and smell like a list I’m unable to run list predicates on them like member(R,Resp), which gives me ERROR: Syntax error: Operator expected.
    – If I wanted to do a member check on a response like this for a particlar id, how would I do that? I was hoping with a dict I could do ?- Resp = ['$VAR'('_'){id:123},'$VAR'('_'){id:234},'$VAR'('_'){id:345}], member(R,Resp), 234 = R.id . but running into this error.

  2. You mention without json_object(dict) I would be getting the “old” prolog json like json([reply='my json reply from the server']), and I’ve seen something like that in the documentation too, about “old version” of json but for future reference, if I preferred to work with json for another project, is there a more modern representation of it in swi? How would I get that instead?

Thanks.

You are already using the more modern representation which is Dicts. This is what the json_object(dict) option does.

For the first question you need to post the code snippet you are using to process Resp.

Ah! Ok got it, good to know.

Sure, I’ve got

:- use_module(lib).
:- use_module(library(http/json)).
:- use_module(library(http/http_open)).
:- use_module(library(http/http_client)).
:- use_module(library(http/http_json)).
:- use_module(library(http/json_convert)).
:- set_prolog_flag(verbose,silent).

api_endpoint(key_list,'https://api.redacted.com/api/keys').

api_call(key_list,Key,Resp_dict) :-
    api_endpoint(key_list,Endpoint),
    http_get(Endpoint,Resp_dict,[
        authorization(bearer(Key)),
        json_object(dict)
    ]).

test(key_list,Key_location,Resp) :-
    api_key(Key_location,Key),
    api_call(key_list,Key,Resp).

I load that and run

?- test(key_list,'/path/to/my_key.txt',Resp_dict).

and get a valid/correct response along the lines of Resp_dict = ['$VAR'('_'){id:123},'$VAR'('_'){id:234},'$VAR'('_'){id:345}]. but then when I literally copy and paste that exact unification on its own into swipl I get

ERROR: Syntax error: Operator expected
ERROR: Resp_dict = ['$VAR'('_'
ERROR: ** here **

Let me put it another way:

If you enter X = ['$VAR'('_'){id:123},'$VAR'('_'){id:234},'$VAR'('_'){id:345}]. into the top level, do you get an error? Should that work?

For some reason, member does work when I do this

test(key_list,Key_location,Rmem) :-
    api_key(Key_location,Key),
    api_call(key_list,Key,Resp),
    member(Rmem,Resp).

but not from the top level.

The problem is only with the top level, your code will work fine. You have some setting that is causing the top level to print the variables in that manner, and when you copy and paste you are not really copying the value of the variable, but the display format that the top level is using. That is why you get an error.

You probably have some setting in the prolog initialization file (nowdays in $HOME/.config/swi-prolog/init.pl).

Try staring prolog without an initialization file, and type some Dict and see how the top level displays it:

swipl -f none
?- A = _{k1: 1, k2: 2}.  % this is what you type
A = _{k1:1, k2:2}.  % this is how it is displayed

So there is no problem. Just some setting in your initialization file or somewhere else in your code that changes how variables are displayed (which is why copy and paste will not work).

Ahh jeez, ok. Yeah

?- X = [_{id: 123},_{id: 234},_{id: 345}], member(Xmem,X).

works at the top level. Ok gotcha. I guess it didn’t like that I was using '$VAR'('_') like it does.

Thanks again @swi

@swi sorry, one last thing: Earlier you provided the example

json_post('http://127.0.0.1:8080/some_api',_{text:'hello how are you'},R,[]).

You meant http_post, not json_post, correct?

no, I meant json_post in that example. If you want to use http_post look at the source code for json_post that I provided above in the post.

1 Like

I’m wondering what setting is used by you to get this? Did you modify the answer_write_options flag?

I did not. I didn’t modify or set any settings like that. Simply launched swipl (SWI-Prolog version 9.2.2 for arm64-darwin - installed via homebrew) at my terminal, loaded, ran the predicate and got that response.

And btw I’ve seen that json representation, alongside the _{foo:bar} since I started using swi, though it’s possible this was the first time I’ve tried to actually enter it at the top level myself.

Oh, unless the answer_write_options flag is set by hitting ‘w’ at the toplevel for responses so you can see full output? Hopefully that’s not an issue though because I have to do that for troubleshooting purposes.

Yes, that is what happens. I pushed some enhancements:

  • If the portrayed(true) or numbervars(true) is not in answer_write_options, stop trying to name the variables. This is how it used to be long ago, but at least it avoids getting '$VAR'(N) in the output that is not really there.
  • Change w and p to merely toggle the portrayed(true) flag, i.e., stop modifying any other options including max_depth(Depth).
  • Add new commands + and - to enlarge/shrink max_depth with a factor 10.

Decoupling obeying portray/1 from the max_depth limit makes sense to me. Possibly debatable is how to control the max_depth. Toggling between 10 and infinite as it used to be is rather dubious as it infinite can produce really huge amounts of output that is sometimes hard to stop and surely practically impossible to read. There are of course other options, such as prompt for a value, have an additional command for removing max_depth, etc.

Further suggestions are welcome.

1 Like