Json float overflow

General
1

I have a server (which I can’t change) that sometimes returns numbers outside of the float range, and causes atom_json_dict/3 to throw an exception:

?- R = '{"num":1.797693134862316e+308}',
   atom_json_dict(R,Di,[null(null),true(true),false(false)]).
ERROR: Stream <stream>(0x562fe6d70a00):1:29 Syntax error: Illegal number

it works fine with a regular number:

?- R = '{"num":1.797693134862316e+307}',
   atom_json_dict(R,Di,[null(null),true(true),false(false)]).
R = '{"num":1.797693134862316e+307}',
Di = _{num:1.797693134862316e+307}.

Given that I can’t change the server code, is there a way to get around this error? Perhaps an option to atom_json_dict/3 that turns these values into +inf and -inf?

2

If there is no solution in SWI, then a pragmatic approach might be to take the raw JSON and pipe it through an external process (e.g. jq) to convert all numbers to strings, and only afterwards try to read it. Then you have control over how numbers are converted.

How to convert all integers in a json file to strings via jq command? - Stack Overflow).

3

I pushed a change to the parser to listen to the flag float_overflow and read too large floats as +/-inf. if this flag is set to infinity. I think that is defensible, although I’m open for better suggestions :slight_smile:

1 Like
4

Thanks Jan! Appreciate it. This works!

The only thing is that we may have to wrap the call to return the float_overflow to its previous value.

This would be needed in multithreaded apps and may have to be done in a mutex to prevent side-effects of the different flag value in other places. Perhaps an option json_float_overflow(error|infinity) might be better, but I am not sure, there may be disadvantages for that also. It would operate much like the json_object(...) option.

5

Simply this will do. Prolog flags are thread-local, where they inherit the initial value from the thread that created this thread (or a specified thread using an option). They are implemented using copy-on-write, where we initially simply share the flag table.

    current_prolog_flag(float_overflow, Old),
    setup_call_cleanup(
        set_prolog_flag(float_overflow, infinity),
        <read>,
        set_prolog_flag(float_overflow, Old))
1 Like
6

Great, this is perfect.

7

@firefrorefiddle, thanks for your reply, Jan fixed it very quickly. As I always say, I think this is one of the best supported open source projects. Thanks Jan and all the contributors!

1 Like