JSON/data file API proposal

Game Talk The Lab
1

Having converted recipes to JSON, I have to say that I’m not impressed with the API that picojson gives us. I find it awkward to use. So, I’d like to propose a wrapper around picojson, intended to be much easier to use (and provide much more condensed code).

In terms of reading via picojson, my two main gripes are that it’s awkward to iterate over lists (I think this is partly a gripe about STL iterators), and it’s really awkward to safely read a simple value (7+ non-bracket lines of not-exactly-short code for something that could be handled in 1 line).

First, an example of how I’m currently reading simple values, like ints:

                    if (iter->contains("difficulty"))
                    {
                        if (iter->get("difficulty").is<double>())
                        {
                            difficulty = static_cast<int>(iter->get("difficulty").get<double>());
                        }
                        else
                        {
                            debugmsg("Invalid recipe: non-numeric difficulty");
                            continue;
                        }
                    }
                    else
                    {
                        debugmsg("Invalid recipe: no difficulty");
                        continue;
                    }

Next, an example of how I’m iterating over lists:

                        if (comp_iter->is<picojson::array>())
                        {
                            ++cl;
                            for (picojson::array::const_iterator inner_iter = comp_iter->get<picojson::array>().begin();
                                 inner_iter != comp_iter->get<picojson::array>().end();
                                 ++inner_iter)
                            {
                                // more stuff was in here
                            }
                        }
                        else
                        {
                            debugmsg("Invalid component for recipe: not an array");
                            continue;
                        }

I do not like having to write this boilerplate code all the time.

So, I propose we introduce a wrapper class, which I’ll call “catajson” for the time being. The goal of catajson is to 1-line as many operations as possible, and make statements not excessively long. The programmer can instantiate a root-level catajson instance by calling catajson::open(game *g, std::string filename). The game pointer is there so that we have access to debugmsg.

I’m also going to assume that data file integrity is the responsibility of the modder. That simplifies error handling tremendously, and actually makes it possible to cut down to 1 line in some cases (whereas otherwise, we’d need 4+ to handle the error).

Here’s how I would like to write code which fetches a single value, assuming that “json_recipe” is of type catajson:

int difficulty = json_recipe.get("difficulty").int();

And then, behind the scenes, catajson would do all the sanity checks, the conversions, etc., and if any errors are encountered, it would spit out a message such as ‘JSON error at data/raw/recipes.json[“recipes”][42][“difficulty”]: value is not an integer’.

Existence checking would still be possible

And here’s how I would like to write code which loops over a JSON list:

catajson comp_list = json_recipe.get("components");
for (comp_list.set_begin(); comp_list.has_more(); comp_list.next())
{
    // do stuff
}

Personally, I would find this API for JSON reading to be much more tolerable. I’d like to make sure that the API to write JSON is about as easy to use, which would probably be possible if we made sure that there are OO ways to turn a given object into a catajson representation.

Having that in-between layer might also make it easier to swap to a different serialization method, as long as there are ways to do lists, string-value pairs, and basic values.

If there are no strong objections, I’ll probably implement this next time I write code that interfaces with data files. It would be nice if TDW, Kevin and/or GlyphGryph approve of the proposal, sure, but mostly I just want to not be annoyed at the data interface :P. And maybe cut down game::init_recipes() to 100 lines or fewer, if I can.

1 Like
2

We’re working with json. Do NOT do this. Robust error handling is the primary reason I’d want what you are describing (see how much error handling I’ve got in my new Item_factory class on my branch, and even that is honestly insufficient!).

People WILL make mistakes inputting json by hand, and they are almost impossible to tease out without good messaging.

Other than that, I like the idea, since I’ve sort of been doing it already to an extent in bits and pieces as needed.

3

Perhaps I should clarify: I meant that we will be giving good error messages, but it will happen at the parsing level. I would like the caller to be able to assume that things just work, and any errors will be picked up by the parser, alerted to the user, and then (hopefully) fixed. If the caller wants to, they can do additional error-checking, of course.

Basically, I want to be able to give parse-level error messages that are detailed enough for people to track down problems, rather than the current (and IMO painful) method I’m using, with error handling at the caller’s level.

4

Ah, okay, that makes sense, yes.

5

Sounds great to me.

have to put some thought into having the code (or a data file) specify the layout/required items/etc for validation.