The Birth of the Item Factory - Items to Data File Overhaul - Help needed

[Soron]

Recipes moved to JSON, pull request 451.

I’m annoyed with picojson right now. Surely there must be an easier way to interface with JSON files? Maybe something that’s more optimized for usability, rather than optimized to fit in a header. 300 lines to do something that could be done in 50ish in JavaScript >_<.

I didn’t actually “review” all the recipes, I just wrote a Perl script to convert them to JSON (and made sure that the difficulty 0 ones were sane, of course). Would it help if I shared that script? It would take a little editing to be suitable for items themselves, but might still be handy as a starting point.

[GlyphGryph]

Hey, did you also get an architecture set up? I’m gonna have to look this over when I get time…

Yeah, I was opposed to json for the same reason. It’s not meant to be human writeable. YAML is, and would have been perfect here, but working with shoving in C++ libraries is basically the opposite of good times as we all learned when attempting to do it. The choice generally seems to be “boost or nothing”.

Oddly enough, JSON files are /supposed/ to be written in Javascript. Hence the name. Javascript Object Notation. You’re supposed to write them in javascript, and then save the javascript object as JSON. That’s even how I’ll be doing it for larger edits.

I’ve built a few javascript based json generators in my time, but I’m a web developer, and I’m not sure how useful that would be here. It would, at the least, be trivial to set up a web interface that let’s you edit the item list as part of a form and then spits out the json back to you at the end, if you think that would help.

[DinoCat]

I’d settle for a good CSV to JSON converter. There are some online, but I can’t seem to get the JSON arrays converted correctly with them.

[Soron]

Hah, didn’t build (or use) a proper JSON architecture in Perl, no. All I really did was extract the elements of the RECIPE macro using a regex, store those in a relevant place, and then print 'em in the format that I needed.

Also didn’t get a JSON architecture set up in C++, other than the bare minimum required to load recipes without assertion failures (granted, the bare minimum is too large, IMO).

I think I’ll look into whether I can read and write YAML in C++ with less pain, though…

[akhier_the_dragon_he]

So since people are talking about it can someone point me to where I learn wtf json is because what I have found with google when I checked didn’t really tell me so I probably googled the wrong things.

[Soron]

Hey GlyphGryph, do you have any plans to have the item factory store the items, as well? Like, handling creation and deletion, and all that? I ask because I’m currently having issues with player::sort_inv deleting and re-creating items, hence mangling references, and it would be useful if we could rely on pointers being valid long-term.

[GlyphGryph]

Well, the factory currently handles creation of items, at least for mapgen, but doesn’t track them. I don’t really think that is it’s place, though something that tracks the items might be worthwhile. Could you go into more details about what sort of problems you’re having? Why is sort_inv even deleting or creating ANYTHING? That seems incredibly confusing.

Also, haven’t worked on this in a bit, waiting for it to get merged in before continuing and it’s still not happened! >_<

[Soron]

I was trying to make it so that unloading something doesn’t require wielding it (see issue 503), and I figured it would be a fairly simple case of replacing u.weapon with a passed-in item& or item*. Unfortunately, container selection relies on sort_inv, which completely invalidated my references.

As for why sort_inv deletes and creates items: unfortunately, the inventory stores items as a std::vector< std::vector >. Because it’s storing items directly, any attempt to swap elements has the effect of creating and deleting objects. And sort_inv doesn’t just swap - someone decided the best way to sort the inventory is to copy everything into temporary arrays, clear the inventory, and then re-add everything.

I can hopefully fix my current issue by making the inventory store item pointers (which it creates/deletes when stuff is added/removed), but at best that only means item pointers are valid as long as the item is still in the inventory. We might have reason at some point to track a specific item from the ground to the inventory, or vice versa, at which point we’d have issues.

[PiCroft]

I think I may have attempted to duplicate this system on my own without realizing there is a system under development already. That being the case, is there some way I can assist? I have a copy of the item manager branch but I don’t really know what to work on at the moment in case I try something someone else is doing.

[GlyphGryph]

It’s been included in mainline now, so the best way to assist is make one of the modifications in the first post to assist data feed. We still need a better way to handle:
Colors
Materials (Especially since the material system recently changed. T_T)
and the iuse functions.

[PiCroft]

I have a big bag of JSON files that contain (almost) all of the vanilla items. They are an awful mess and tags, flags etc aren’t correct or present on most of them, but the skeleton is there and is ready for proper formatting.

I can upload them once I get home if you like.

[GlyphGryph]

Sure, sounds good.

[PiCroft]

https://dl.dropboxusercontent.com/u/1409531/catadev%20stuff/item_dec.zip

[GlyphGryph]

Just an update on this: It’s chugging along. Item Factory is functional, A good amount of items fully over to the jsons. All melee, and now guns thanks to Soron. There’s still a lot more to this thing, though!

I still need to -
Get other item subclasses working, including the special pseudo items and undroppables.
Implement the crafting ‘tool-type’ system with tool levels.
Once the last batch of dependencies have been removed and itypedef is deleted, I need to restructure a lot of the logic that exists only to support the current ‘mixed state’ abomination.
Need to figure out how the hell I’m going to do artifacts (which apparently aren’t working at the moment anyway)?
Clean up all the json handling code. Gotta try to figure out catajson, that might help.
Replace all the debug messages with PROPER debug messages.

And then I should be (mostly) done with the basic Item Factory - but not with the item overhaul!
I still want to rework items with fallthrough accessors.
Then I need to implement resultant variants via alternative selectors in item definitions (I’m really excited about this part!).
Introduce item quality/durability.
Make some changes to crafting, have three ‘required’ sections - Tools, Components, and Consumed, rather than the current two.
Introduce a components system that tracks what pieces an item was made of when crafted
Want to teak the ammo implementation and the restrictions thereof, probably as part of…
Removing the subclasses for items, and replacing it with a more dynamic system (maybe)

[kevin.granade]

Re: removing the static type references, what is the benefit of doing this? Is the alternative having a fully unified template type that has every possible attribute? Or somethig else? (I’m not saying I have a problem with it, just that I’m curious where you’re going with it)

One thing that would be EXTREMELY helpful for cleaning up the internals of item handling would be to have some actual polymorphism, right now it doesn’t leverage the language at all, every method that wants to do aomething non-generic to an item has to have checks (often multiple ones) to determine what kind of item it is so it knows how to handle it. I’m thinking what would be great would be a way to push that special handling down into a sub-object (not subclass) that a given item can have multiple copies of.

More concretely, a gun for example, could have a container<ammo_type> member, a firearm member, and possibly other members, for example a wearable member if the gun has a sling. These would be stashed in a container in the item and/or item template, and when an activity is performed on an item, for example “reload”, “fire”, or “wear”, it gets routed to the appropriate sub-object for handling. Then the appropriate method doesn’t have to handle every alternative for when we overload actions. As with your overall concept with fallthrough, an action would check the item for a handler, then check the template for a default one.

[GlyphGryph]

Well, that was part of the “maybe” there - it’s a problem I have to study in more depth. But the implementation right now just seems bad. At the very least I’d like to reduce definition dependencies like “ammo” and the inability to have multipurpose items, like a tool or weapon you can ‘wear’.

It may end up just involving reimplementing a better, more sensible subclass system in the end, but at the very least I want to get rid of the whole “tool” and “ammo” subclasses - any item should be able to be assigned a ‘use’ action (even if it’s just some silly message about how using this like that would be ill advised), and it would be trivial to handle well, and every item should be valid ammo if something really wants to use it for that, without having to change the definition of the ammo item itself. Something that, I’ll note, will mean a lot of headaches with the proposed mod-manager I want to implement if it is not changed.

Subclasses like ‘guns’ could very well stick around when I finish my analysis, but having multiple subclasses makes Artifacts… weird. Because artifacts are weird, admittedly, but it would be better, I think, to have it work in a different way.

I definitely agree about polymorphism and subobjects being potentially valuable. I’ll certainly be taking your suggestions into account, as I’ve not done much here except getting frustrated with the limitations of the system as-is.

[Soron]

[quote=“GlyphGryph, post:27, topic:741”]Clean up all the json handling code. Gotta try to figure out catajson, that might help.
Replace all the debug messages with PROPER debug messages.[/quote]

I want to get proper debug messages in before 0.5 is released, so that modders get feedback when their stuff doesn’t work. It’d be pretty easy, as well, so I’m willing to do it if you don’t get to it first.

And since catajson has debug messages already, want me to merge your JSON-handling stuff with catajson, and then make the item factory use catajson? I’ve already copied your check for parse errors (wish I knew about that sooner!), so may as well see what else you’ve done that I haven’t.

[GlyphGryph]

Sounds good to me, honestly!

[PiCroft]

I could have a look at rejigging items so that an items functionality is not based on specific functions for specific items and huge else-if atrocities looking for specific items for specific functionality. Wow, thats a lot of specifics!

I had been thinking of a system while converting the vanilla items into JSON - using tags like “HEAL” and “PAINKILLER” to grant an item those properties, then providing their potency.

[Shoes]

I haven’t read any posts really, but I saw what you’ve done with the crafting recipes and how you’re starting to do that with a few other groups.

I am tempted to take the responsibility for converting clothing to the new system; I want to make sure everything is balanced, and then I want to make sure the system is flexible enough to add parameters for wind resistance, for example.

However, as you now, my C++ experience is limited. If it is as simple as copying over the code you already have for melee weapons and adapting it to clothing, that I could probably do. Just point me in the direction of the various files, and I might get it done in a reasonable amount of time :3 I would also have it on git so you could see how I am doing and pull/take over if need be.