Example of validating json in an http handler

Hello All,

I’m trying to learn Prolog and I think it would really help to see a real world example of something that I do frequently. Can someone kindly point me to a bit of code that validates json field names in an http handler? I’ve been looking around, but haven’t see anything like that.

Thanks,
-S

Is there a page which specifies the restrictions on JSON keys? Aside from being a string.

DCG is the obvious choice, in Prolog.

I’m not sure what you mean by “validates json field names in an http handler” but perhaps this answers your question …

I have some code that processes an HTTP request that’s a JSON data structure and the response is in JSON.

json_response/2 (line 553); called from reply_with_json/1 (line 521), which also checks first whether the response has been cached; and the http handler is set with a directive at line 497. (The :- ! rules could be replaced by => single-sided unification)

The corresponding client is pykythe/browser/static/src_browser.js at 7a96675f78a8c09e57ea34f45d1e3000266e76f5 · kamahen/pykythe · GitHub , with calls to fetchFromServer().

I don’t know why my request for a bit more info was marked as the solution - it’s certainly not a solution :grinning:

Hi,

What I meant by “validate json”, is making sure the needed object names exist and that they look like the correct data type. I kind of get the basic introductory prolog stuff with the family tree examples and all. I was just trying to map that to something I commonly do in the real world. I appreciate those who took the time to answer. I have a couple of prolog books, but I find them a bit lacking. I just found “Prolog Programming in Depth”. It looks promising. I can’t wait to get it ;-).

Thanks,
-S

Hi,

I forgot to mention that I’m thinking about creating a swi-prolog add-on. I’m going to start reading the docs about that. It seems that, while probably painful, that may very well force me to connect the dots. Maybe doing the add-on while reading the book will be just what I need. I really like the features of the language. I can already see how I could realize some things that that I’ve wanted to do that are very difficult or impossible to do in other languages. I can’t wait to get up to speed, so I can start making some of those ideas a reality.

Thanks Again,
-S

The easiest way to deal with JSON is to convert it to a dict, e.g. with json_read_dict/3 or atom_json_dict/3. You can check for existence of a field with get_dict/3, which fails if the field isn’t there. You can check the type of the field (that you got with get_dict/3) with is_of_type/2 or must_be/2.
If you don’t like dicts, there are predicates that convert JSON to a list of key=value pairs and you can then use member/2 or select/3 to extract fields.

There’s a lot more here:
https://www.swi-prolog.org/pldoc/man?section=json
https://www.swi-prolog.org/pldoc/man?section=bidicts

Hi Peter,

Thanks for the reply.

How would the overhead of converting json to a dict compare to something like an xpath for json (assuming you could do that without converting to an intermediate representation)? I do this when using a json data store in other languages.

Thanks,
Gerry

I suggest to take a look into the djson pack.
I never used this specific contribution by Michael Hendricks, but in the past I used dcg_util, and found it useful, instructive and entertaining. You know, usually we are limited to ‘pick two of three’…

Roughly the story is that JSON has a standard Prolog representation or can be represented using a SWI-Prolog specific extension called dicts. Dicts are SWI-Prolog’s answer to efficient and comfortable key-value sets. Next you can either do things like

must_be(integer, Data.age)

Now, if you get invalid data, the HTTP server will produce a 500 page. You can turn that into a proper 400 page with a little work, but that is for later :slight_smile:

Or you can use a library that validates the dict against a schema. The djson pack is (I think) one answer to that. The openapi pack provides a swagger interface. It is not very mature and includes an also not very mature JSON-Schema validation.

Probably teh JSON support stuff should be moved out of the HTTP libraries and proper schema support should be added …

I had exactly the same problem ages ago, so I created dict_schema to solve it: https://www.swi-prolog.org/pack/list?p=dict_schema. Another problem was crazy confusion of string vs atom from JSON parser. It’s basically a json schema implementation with some declarative conversion for SWI-Prolog.

Example usage is in a bloggin framework to validate API requests: https://github.com/Blog-Core/blog-core/blob/354a79b3fe1803d97a97dc407285869fe1aad79a/prolog/bc/bc_api_entry.pl#L144

1 Like

I put some some notes on how to import JSON to a SWI-Prolog dict via the web using openweather’s JSON data as an example here:

I also dabbled a bit with openweather’s XML to Prolog, but mercifully don’t deal with XML much.

A nice thing about SWI-Prolog is its dicts are very similar to JSON and moving from one to the other is very easy. Once imported into a dict, testing if the desired keys are in the imported JSON is fairly easy.

I wrote this code about 5 years ago as a complete novice, so it may be out of date and could be done better:

:- use_module(library(http/thread_httpd)).
:- use_module(library(http/http_dispatch)).
:- use_module(library(http/http_error)).
:- use_module(library(http/html_write)).
:- use_module(library(http/http_files)).
:- use_module(library(http/http_unix_daemon)).
:- use_module(library(http/http_open)).
:- use_module(library(http/http_ssl_plugin)).
:- use_module(library(http/json)).
:- use_module(library(sgml)).
:- use_module(library(xpath)).

:- initialization(http_daemon, main).

:- multifile http:location/3.
:- dynamic   http:location/3.
http:location(images, root(images), []).

:- http_handler(root(.), weatherapp_json, []).
:- http_handler(root(xml), weatherapp_xml, []).
:- http_handler(images(.), http_reply_from_files('./images', []), [prefix]).

weatherapp_json(_Request) :-
  URL = 'https://samples.openweathermap.org/data/2.5/forecast/daily?id=524901&appid=b1b15e88fa797225412429c1c50c122a1',
  (  catch(call_with_time_limit(20, get_json(URL, Dict)), Error, true) 
  -> (  var(Error) 
     -> length(Dict.list, Length),
        weather_rows(1, Length, Dict.list, Rows),
        reply_html_page(
          [title("~w's Weather Page"-[Dict.city.name])],
          [table([], [th([colspan('3')],["~w's weather over the next ~w days"-[Dict.city.name, Dict.cnt]])|Rows])])
     ;  reply_html_page(
          [title('Error Page')],
          [p('Oops! '-[Error])])
     )
  ; reply_html_page(
          [title('Failure Page')],
          [p('Oops! Something failed.')])
  ).

get_json(URL, Dict) :-
  setup_call_cleanup(
    http_open(URL, In, []),
    json_read_dict(In, Dict, [default_tag(json)]),
    close(In)
  ).

getday(Idx, DayOfWeek) :-
  get_time(TimeStamp), 
  ComingTimeStamp is TimeStamp + ((Idx - 1) * 86400),
  stamp_date_time(ComingTimeStamp, date(Year, Month, Day, Hour, Minute, Second, UTCOffset, TimeZone, Y), 'UTC'),
  format_time(atom(DayOfWeek), '%A, %e %B %Y',
     date(Year, Month, Day, Hour, Minute, Second, UTCOffset, TimeZone, Y), posix).

html_weather_row(Idx, Day, Tr) :-
  Day.weather = [Weather],
  getday(Idx, DayOfWeek),
  Tr = tr([],[td([],[DayOfWeek]),
              td([],['Day: ~w, Night: ~w, Max: ~w, Min: ~w'-[Day.temp.day, Day.temp.night, Day.temp.max, Day.temp.min]]),
              td([],[img([src('/images/~w.png'-[Weather.icon]), alt(Weather.description)],[])])]). 
  
weather_rows(Length, Length, [Day], [Row]) :-
  html_weather_row(Length, Day, Row), !.

weather_rows(Idx, Length, [Day|Days], [Row|Rows]) :-
  html_weather_row(Idx, Day, Row),
  IdxInc is Idx + 1,
  weather_rows(IdxInc, Length, Days, Rows).
  
weatherapp_xml(_Request) :-
  URL = 'https://samples.openweathermap.org/data/2.5/weather?q=London&mode=xml&appid=b6907d289e10d714a6e88b30761fae22',
  (  catch(call_with_time_limit(20, get_xml(URL, Xml)), Error, true) 
  -> (  var(Error) 
     -> get_xml(URL, Xml),
        term_string(Xml, Str),
        xpath(Xml, //city(@name), Name),
        reply_html_page(
          [title("~w's Current Weather Page"-[Name])],
          [p(Str)])
     ;  reply_html_page(
          [title('Error Page')],
          [p('Oops! '-[Error])])
     )
  ; reply_html_page(
          [title('Failure Page')],
          [p('Oops! Something failed.')])
  ).
 
get_xml(URL, Xml) :-
  setup_call_cleanup(
    http_open(URL, In, []),
    load_xml(In, Xml, []),
    close(In)
  ).