Spreadsheet Automation of JSON edits

Game Talk The Drawing Board
1

I was poking around in the JSONs, preparing to put together a PR of some new XL items, but basically because I’m extremely lazy, I thought to myself “surely there’s a quicker, easier way than doing all this tedious manual data entry”.

First of all, it seemed to me that adding and modifying items would be much simpler and easier if you had it in a spreadsheet rather than a JSON. Did a bit of googling, found a script (http://blog.fastfedora.com/projects/import-json#ImportJSON) that very neatly pulls things out of a JSON and into a nice spreadsheet. That, by itself, makes things a lot easier, I think.

Now I’m not really a computers person, but even from what I vaguely recall from school IT lessons years ago, it ought to be possible (and I think actually relatively straightforward) to go further than this and do a bunch of automation to speed things up a lot.

So, for instance, I’ve been looking at armor.json, and it seemed to me that you could have a system where you define certain functions relating e.g. material, coverage, and body part to weight, volume, warmth, protection etc. with coefficients variously defined for each material type and so on. So that would allow you, for instance if you wanted to make a set of leather armour, to enter the material and the coverage, and have it automatically generate not only the other properties of the item, but also the properties for each part of the armour set. And you could then fine tune them manually if you thought there should be any exceptions to the general guidelines.

I’ve not yet looked at the code for the recipes, but if it’s in comparable json formatting, it ought also to be possible to use the same system to automatically generate crafting recipes for them. And admittedly I’ve not given the idea that much thought or looked at any of the other JSON files, but it could be that the same idea could have even more uses, but honestly I think just what I’ve outlined already seems labour-saving enough.

I sounded this out on the IRC, assuming that if it was feasible,somebody else must have already had a crack at it, but the response I got was basically, “Yeah, that’s sounds like it could be useful”.

Now, again as I say, I am not exactly tech-savvy - so if somebody more capable wants to have a go at it, I imagine you’d be able to do it much more quickly and get a much better and more elegant end-result.

However, if I’m not mistaken, it’s not actually a very complex project skills-wise, it just requires a fair amount of legwork to get set up. So if nobody else fancies it, I’ll probably have a more detailed look at it tomorrow and see if I can work out how to get it set up. If nothing else, it’ll be useful for me as a vehicle for relearning how to use spreadsheets, even if it turns out to be a deadend mod-wise.

I don’t actually know yet how to output a JSON from a spreadsheet, but if there’s a straightforward script for importing them, I presume it won’t be too hard to find one for exporting them and I’m told it should be relatively simple.

Also, you might think it’d be a lot of pointless effort setting up the necessary functions and co-efficients in the first place and not worth the bother working them out, but again, it seems to me you could figure them out in a relatively easy, automated fashion by just calculating the average ratios between the relevant values of existing items.

Thoughts, comments? Does this sound feasible and useful? Has anybody already done this (if so, do you fancy sharing?)?

EDIT: The other obvious objection that strikes me, is that maybe it’s pointless because at this stage, the data’s already there for most of the game’s items. Now, obviously this system would still make adding new items much faster and easier, but if there aren’t necessarily that many new items going to be added, perhaps it’s not worth the effort. That said, it could still have the benefit of standardising the existing items - now, I don’t know if, for instance, the differences in weights or armour values of e.g. a between different parts of leather armour compared to different bits of fur armour, are organised with any precision or uniformity, or if they’re just guesstimated vaguely?

EDIT2: You could also use this system to automatically create the values for XL versions of items, with predetermined ratios of extra weight and resource requirements and so on (this is what originally got me onto the idea, actually)

2

Horry sheit

3

OH BOY

But seriously if this happens I’ll become a contributor. I’m gonna make a recipe for funnels and like three new mutation categories.

4

[quote=“TheFlame52, post:3, topic:7694”]OH BOY

But seriously if this happens I’ll become a contributor. I’m gonna make a recipe for funnels and like three new mutation categories.[/quote]

I don’t think this would affect mutations because from what I can tell, I think their coding is more complex and requires playing with more than just JSONs.

As to making recipes, have a go already! I’m pretty sure it should be fairly straightforward and not require any actual coding know-how really. https://github.com/CleverRaven/Cataclysm-DDA/blob/master/data/json/recipes/recipe_others.json have a poke around in there - I think the formatting’s fairly straightforward and self-explanatory unless I’m missing something (?)

5

concur with baldamundo.
easy enough to add items:

  1. add the actual item in the correct JSON
  2. add a recipe in the appropriate JSON (optional)
  3. add the item spawn chances and locations to the item_groups JSON (optional)

no need to compile in order to test json changes so its easy enough.

besides this, you just need to learn to use git, but there are fine instructions for that around this forum.

for mutations though, you must not only add them to JSON, but also modify the source code in order for them do actually do something.

6

How an where would I submit the funnel recipe once I make it?

7

the best way for that would be to look here: http://smf.cataclysmdda.com/index.php?topic=1063.0
learn how to make a PR.

the alternative would be to just post the recipes in this forum and wait for someone to make the PR himself.

8

A spreadsheet or other tool* that knows how to read and write our json files is great.
Things to keep in mind:
Formulas are fine as a starting point, but we’re not interested in rigidly locking items to them. The item definitions are flexible in the way they are because that’s how we want them.
If you want changes generated by a tool merged, it will need to respect our formatting conventions. This means using the same formatting as the surrounding code, and not reformatting entire files just to make one change, even if the formatting is improved. Mass reformatting makes merging very difficult, especially in json.
The layout (schema to be specific) of the json is highly subject to change, so we will break your tool frequently.

*Someone wrote an item creator program a while back, so you might also want to look at that, unless you just want to work with spreadsheets.

9

It shouldn’t be very hard to have the spreadsheet compose a list of ‘used schemas’ while it is reading/processing the json files.

That is, depending on how the ss reads in the files in question.

10

yes plz, it’d be nice to have something to manage flagging or setting up stuff

A suggestion, make sure you have an exporter which can put all the stuff back 1:1, or at least in a way which is readable in the same way the original entries are. If you imported/exported certain tags/flags and such, it’d be pretty extensible, so long as the repo and script were up-to-date with each-other.

11

If anyone could advise me on how to adapt this script https://gist.github.com/pamelafox/1878143 so that it properly copies underscores and quotation marks I’d much appreciate it. Trying to have it copy over e.g. the “environmental_protection” category heading results in “environmentalprotection” and trying to copy over e.g. “[“VARSIZE”]” results in “[“VARSIZE”]”.

12

Quoting should be enough shouldn’t it? If not, there should be a certain escape sequence.

13

Sorry, what’s an escape sequence?

14

Baldamundo, the github website should have a ‘show RAW text’ button, then you’ll be able to copy it without formatting getting in the way.

15

Sorry, what’s an escape sequence?[/quote]

An escape sequence in languages is one of the kinds of “special notation”. In C, if one did printf( “Hello World\n” ) it would print “Hello World” but then would put a line in, like you had hit enter after typing it. A lot of languages have special characters like quotes or combinations so you can pass information without parsing or altering it, or like the newline \n, are used for visual organization in the middle of text.