The State of LUA in DDA

[JT1]

To answer your “essential” question in a perhaps glib way, Lua (or scripting in general) is essential for one thing and one thing only: modification of game logic as part of data-driven design

That is not making sense to me. Dda is very heavily data driven in exactly the way outlined in those articles, the majority of game content is externalized to json. What you seem to be saying is that not only the data, but also the game logic should be externalized to data, but that’s not the actual result of injecting LUA, that externalizes game logic as code, at which point it’s a weird circular argument of “we can’t write code, so add lua so we can write code without writing code”.

In other words, it still just boils down to compilation, the only reason to use LUA over c++ is because you can’t or won’t write your changes in c++ and submit them for inclusion in the main project. As far as I can tell, the only reason LUA modding support thrives is that closed source software throws up artificial barriers to modification of the core game.

Let me clarify, dda is not a game engine, it is a game. “Enable people to change anything about dda without forking the project” is not a development goal, and I’m not interested in spending meager project resources on making it happen.

True to the can’t/won’t (more won’t than can’t, but can’t’s a part) and true to the Lua support being more common among closed-source projects, but the game engine goal is indeed what I’m arguing for. It’s quite possibly one of the stronger survival oriented game sandboxes and it already has everything readily able to be adjusted, save for a robust scripting language. Fallout: New Vegas was also designed to be a game (and bugs withstanding actually is quite excellent without mods), but even then is still far more fun in sandbox gameplay, a function for which it was never intended, especially if the Tale of Two Wastelands content is merged in. Skyrim is not even a contest between the game and the sandbox. In both cases, modding efforts almost always retain the underlying game as the foundational glue, but the unique experiences and customisations that players can introduce then turn it into something better… much better in the latter case. (I couldn’t stomach vanilla Skyrim for more than 11 hours, and those were very generous “well maybe it gets better” hours. Modded Skyrim, I’ve been going strong for 1600 hours.)

Without any non-self-taught programming skill or experience I’m almost certainly underestimating the effort involved, but I do see it as having the foundation already there and the work to add onto it would justify the expense in the long term. I have no proof other than other successful examples of the same in the closed-source world and my allegation that scripters who are non-programmers/non-sprite-artists are an untapped talent pool, however.

I’m not suggesting the whole game be available to logic – just the scripted bits that manipulate the world and the player character, like item-uses, activating biomods/mutations, tile examination actions, map generation, spawning of creatures from rot (pretty much the ideal use of external gameplay logic), creature evolution, etc; the 90% of code execution time that occurs in 10% of the code not only should remain in the engine but simply must.

As long as there’s an ability to tweak the results of anything either by polling-and-overruling or by occasional corrective factors, much of the code is fine in-engine. In the end, tweaking is the desired panacea – direct manipulation of the results that the game produces if an end user doesn’t particularly agree with them, so that they’re not forced to go full redneck and head to the hills whenever they find a feature they don’t like. =) The hard-coding and magic numbers of actual in-game environments like Necropolis, for instance, is a big no-no in my books (even in spite of how awesome Necropolis is – I probably wouldn’t change anything).

I suppose I’m still arguing the point without satisfying the conditions of “what bindings”. I can at least list from personal experience what I’ve wanted in the Gamebryo-based games; the game engines already do a great job of data-based dialogue and missions, with conditional logic on what message should be displayed when choosing a particular dialogue option even without actual scripting to do so. Where they tend to fail is their ability to produce reasonable results for adding gameplay-related features or scanning for interactive objects around the player (something that Cataclysm’s scripting language is also a little sparse on), and especially for direct access to actor variables, object trees, and game physics. FNV’s really functional for direct interaction, like triggering a lever, but requires obscene workarounds like cell scanning to apply tweak-level manipulation of characters, since each actor has only a single “one-to-rule-them-all” script that inhibits inter-mod compatibility unless everything is merged together.

For example, my NPCs-use-ammo mod for New Vegas uses the scripted logic to scan for everybody around the player, get a list of all of the items in their inventory, find the matching base class that corresponds to that inventory item instance, and then set the flags on those base classes to compel NPCs to use ammunition for those weapons (incidentally exposing a couple engine bugs on features they didn’t fully test with NPCs that do use ammo, like revolver reloading). If I had direct access to the object classes as a tree, that effort would be simpler, but that’s not available. This mod could have been done as direct modification of the objects, but then would not be automatically compatible with other people’s mods, as the “rule of one” of modifying a single weapon record’s field means that the entire weapon record needs merging. (I hard-coding a list of most object types for faster execution, but that isn’t mod compatible either, so the script needs to hack everything all the time.)

Cataclysm is both good and bad at JSON-based modifications like this: “copy-from”, “relative”, and “proportional” work wonders, but anything more intensive than that also follows the rule of one.

As a prototype of bindings that I would probably personally use:

• Retrieve array of master object table for fields, NPCs, monsters, objects, vehicles instances
• Retrieve all fields, NPCs, monsters, objects, vehicles of type within radius of singular point
• Retrieve type of tile at point (already in!)
• Get player’s X coordinate and Y coordinate (already in!)
• Get player’s overmap X and Y coordinate
• Find overmap tile of type in radius of player, return coordinates if found or nil if not found
• Get overmap type player is currently in
• Get overmap type at tile
• Replace overmap tile with another overmap tile
• Regenerate terrain for overmap tile
• Respawn monsters for overmap tile
• Get instance of object of type within singular point (can be soft coded if above are available)
• Start internal or scripted task for an NPC or creature (“move to point”, “use furniture”, “travel to overmap”, “find overmap tile of type”, “find item of type”, “attack target to kill”, “flee”, etc.)
• Directly manipulate an NPC, creature, or object instance (teleport, move, collide, edit instance parameters (mostly already in!))
• Directly manipulate an NPC, creature, or object base class (root statistics)
• Accelerate vehicle
• Decelerate vehicle
• Trigger vehicle skid
• Flag vehicle as mobile or immobile
• Apply arbitrary morale modifier to player
• Apply specific morale modifier to player with desired duration
• Add effect to NPC or creature (already in!)
• Force NPC to fire once at coordinate and/or actor in that coordinate
• Trigger scripted autopilot of player character with a brief realtime delay between successive statements, returning control to player after completion
• Start a mission
• Set a new mission target
• Set a mission stage
• Finish a mission
• Restart a mission
• Create a trigger area that will produce events upon entry
• Auto-save
• Pop-up to provide message (already in!)
• Pop-up to select arbitrary item from menu, yielding number selected as event handler
• Attach runtime-created variable to an instance (e.g., create arbitrary variable named “fnord”)
• Set, manipulate, and pull the value of runtime-created variable on instance
• Save or discard runtime-created variable
• Event handlers:
  ** Periodic heartbeat (already in!)
  ** When player or character consumes a consumable (parameters: character, consumable, amount left (0 if gone))
  ** When player activates a furnishing (params: character, furnishing)
  ** When player or character strikes NPC with weapon or bullet (params: NPC, character, weapon used), return value determines actual damage that would be caused (giving ability to modify result)
  ** When player or character strikes NPC with explosion firearm (same parameters and return value effect)
  ** When monster strikes character or player with attack (params: character, monster[, attack used]; if multiple attacks added to game)
  ** When player or NPC picks up an item (character, item)
  ** When player enters line of sight of monster (player, monster; possible to recycle safemode code for this)
  ** When there are no hostile entities remaining around player for several turns (similar recyclability)
  ** When player is killed (params: killing creature/character, method of destruction)
  ** When monster is killed (params: monster, killing creature/character, method of destruction)
  ** When NPC is killed (params: NPC, killing creature/character, method of destruction)
  ** When object is destroyed/disassembled/burned/otherwise deleted (object, method of destruction)
  ** When a new character is loaded (params: player)
  ** When a saved game is loaded (params: player)
  ** When the game is saved (params: player)
  ** When an effect is applied to the player or a character (params: character)
  ** When the player starts a mission (params: player, mission)
  ** When player, character, or creature enters a trigger area (params: creature, trigger area)
  ** When player, character, or creature exits a trigger area (params: creature, trigger area)
  ** When player enters conversation mode with an NPC (params: player, NPC)
  ** When player exits conversation mode with an NPC (params: player, NPC)
  ** When character or creature detects player, character, or creature by sound (params: listener, target, volume of sound, sound heard)
  ** When creature detects player, character, or creature by smell (params: listener, target, intensity of scent)
  ** When character detects sound (params: listener, sound type, approximate origin tripoint)

There’s undoubtedly much, much more that doesn’t come to mind immediately, but that alone is a great start.

Much of that can be done by creating JSON objects that are executed by native code in sequence as if they are scripts, but scripts can cut out the middle man, and some can only be done by scripts (e.g., formula modifiers to damage caused) unless particularly complex design is added to obviate them (e.g., ability to modify a character/monster instance’s damage suffered by weapons with certain flags/keywords both at runtime and as innate vulnerability/resistance).

That line count of procedurally-generated code is somewhat disingenuous: those lines are all dead code until/unless anyone uses them, which means no performance impact on anything other than the compilation process

That is not remotely true, if we were to commit to a truly pervasive LUA modding system, every change to any (internal!) game interface must be scrutinized for regressions to the matching LUA interface. Changes that adjust behavior of LUA interfaces must be made backwards compatible or cancelled, new internal interfaces must be either determined to be inconsequential or have matching LUA interfaces added. This is an incredibly expensive proposition. The compilation overhead is inconsequential in comparison.

Returning to the data-driven design point, the desired state is to allow designers to write declarative content that is loaded by the game. Allowing injection of executable code actively subverts this concept. It prevents the formation of well-defined interfaces for injecting data because it’s much easier to just expose all the internal game state to your injected logic, and let the logic side do whatever it wants.

For the former paragraph, I’m arguing in favour of a fairly pervasive system, but I’m not sure that some 500-or-so gameplay functions would need direct checking against 10000-or-so engine-related classes and features and vice versa. I’m presuming you’re stating “interfaces” in the comp-sci sense, i.e., methods and fields, but isn’t the Lua system already set up just to generate the bindings from plain-text entries? How often do methods get deleted or have their parameters changed and how often do fields get deleted or renamed? If a non-trivial amount, how often is that due to game logic and not due to changes in core mechanics?

Your latter paragraph is a pretty damned good argument, and I would very much love to have the need for scripting obviated by declarative content. =) I’m just not sure how practical it is to keep every declarative feature “in house”, especially with the breadth and scope of some changes that people might want to make. The program code is already crowdsourced, but crowdsourcing both the program code and the scripted logic would give a greater number of people the ability to write their own features independently that are still compatible with (and/or pullable into) the whole.

It could indeed turn out to be the singular worst decision ever made for the project, as it may already seem right now, but I like having faith in community-based design. If nothing else, it’d relieve a lot of forum burden by empowering people to make their own ludicrously stupid gameplay changes themselves instead of incessantly asking for them and having them shot down. (Ahem… like myself, again. =P)

[kevin.granade]

True to the can’t/won’t (more won’t than can’t, but can’t’s a part) and true to the Lua support being more common among closed-source projects, but the game engine goal is indeed what I’m arguing for.

That’s pretty much an impasse, I’m not interested in dda as an engine per se. I’m happy to add data-driven behavior, but only to the extent that it improves dda-the-game.

As a prototype of bindings that I would probably

That’s probably weeks to months of work you just outlined. The basic data bindings aren’t that bad, but all the querying and injection of callbacks and overrides is, and as you say, this is just a start.

I’m not sure that some 500-or-so gameplay functions would need direct checking against 10000-or-so engine-related classes and features and vice versa.

I’m not following you, what I mean is every change needs to be scrutinized for “is this going to change how any of the LUA interfaces work?”. This means that someone intimately familiar with the bindings has to review every code change. Alternately, we don’t do that, and the LUA bindings break constantly.

isn’t the Lua system already set up just to generate the bindings from plain-text entries?

That doesn’t meaningfully change anything, someone still has to manually update those text entries every time a bound method is changed. Someone still has to be watching for those changes in the stream of PRs. Someone still has to write helper methods any time an API can’t be exposed as-is.

How often do…

I don’t have numbers, but it’s enough to be problematic. Also remember that the benefit to the project is roughly 0, so I’m not willing to tolerate much.

I’m just not sure how practical it is to keep every declarative feature “in house”, especially with the breadth and scope of some changes that people might want to make.

You’re arguing for exposing an absolutely massive amount of game state via scripting bindings, that’s somehow totally fine despite a total lack of feedback about what parts are being used, but exposing actual data-driven bindings is somehow too much? The coordination and synchronization problems are present either way, the problem size is similar, the main difference is one you can implement blindly and the other requires design.
The difference is that data-driven design has been working and supplying benefits to the project incrementally. If anything, the existence of LUA bindings has been interfering with it by applying shortcuts.

[Aabbcc]

@JT1, your use of Bethesda as example really, really doesn’t work.

Not only their code is closed, they have a far larger development team, and they have an audience orders of magnitude bigger, a far more mainstream audience which is going to have a larger proportion of people willing to mod it who don’t have coding knowledge.

Besides, most people willing to mess with the script language of their games are already going to be familiar with coding, and for those that arent? There’s tutorials galore. Cata’s documentation and tutorials are flimsy and out of date enough as it is.

Lua also in no way guarantees you’ll skip the usual modding problems of multiple mods trying to touch the same thing, there’s only so much you can mitigate that, and frankly? Modifying the source code and using Git to merge changes makes it far more likely for things to be compatible than anything you could think to implement from the ground up in Lua.

[JT1]

1. Those weeks to months of work of adding external gameplay logic will be needed one way or another when work shifts to mission design and interactivity with the world, regardless of whether it’s done in Lua or in an object-based system, and an object-based solution has the additional disadvantage that statements will not benefit from branching conditions with anywhere near as much readability or flexibility.

2. I’m simply having trouble understanding why every change to methods is going to break Lua support with any frequency at all – i.e., why there is even a scrutiny process at all rather than automated unit testing or just waiting for an actual bug report. The removal or changing of the arguments of bound functions or variables will break support, but additions to code generally won’t. There’d simply be an absence of new bindings reflecting the new features, which in itself wouldn’t be anything worse than unfortunate. If it actually breaks compilation, that’s another matter, of course.

The cost-benefit ratio approaching zero I mentioned implies the denominator is increasing, I might add. The more popular the project gets and the more people contribute their own modifications to the world at large without having to fork the project or submit it for internal review, the higher the benefit that is realised for the same linear cost, as compared to hard code which is lower fixed cost, but also fixed benefit.

3. Data-driven bindings are “too much” on the basis of essentially writing a scripting engine where each statement is an object rather than a black-boxed call and return in an embedded language explicitly designed for that purpose. A system similar to StarCraft’s map editor triggers can work, but scripting would get the same thing done far more effectively, and more quickly, without as many esoteric workarounds like the “minions” in League of Legends, the “cloak spells” in Skyrim, and so on.

Scripting also offers greater power and flexibility for the person doing the scripting (especially when it comes to access to program flow control statements such as for/foreach/while loops and branching conditionals, which in my experience scripting is truly awful without). A singular new “generic” function bound to Lua takes only a bit of effort to write a setter/getter, and runs lean and mean, though it incurs maintenance overhead on the back end – a new data-driven event trigger based on an object incurs a larger memory and CPU footprint on the front end and depending on how non-generic it is, also requires a larger amount of new hand-written supporting code to implement.

Since I am indeed probably wearing Bethsoft games thin, another modern moddable indie game with a robust scripting language is Starbound. (I lost around four months’ worth of work on one of my Starbound mods when my hard drive failed last summer, which was a massive bummer… still haven’t gotten back into it.) Anyway, vanilla Starbound ranges from merely okay to good, but modded Starbound is fantastic. The game is also entirely external logic – all of the gameplay logic proper is external, which is an interesting blend between closed-source and open-source, and it’s quite possible to change the entire game to one’s content. Everyone still only makes tweaks to it: it is still Starbound-the-game, yet the modifications that are there fundamentally change the experience, almost universally for the better, because everyone can pick and choose the features they love the most. (There is indeed a rule-of-one incompatibility there, too, if someone overwrites the same core script. But generally mods strive to avoid that whenever possible by just adding their own scripts.)

All in all, I’m getting as tired of arguing this as you are. Ah well. I tried. =)

[Zhilkin]

You should not have been arguing at all. There is no sense in that. All you need to do is to create some mods which will prove that Lua is useful or make some PRs to flesh out Lua bindings. Kevin complained on several occasions for last year or two that nobody is using Lua in large mods (actually even in smaller ones) - no new mods appeared since that time (save for my examples).

If some new large mod would appear - that way Lua could be saved 2016 or 2017. There were plenty of active mergers at that time. I don’t know if it is even possible now with only me, Rivet and kevin (and very rarely others) working with merges, but you have indefinite time until 0.D.

I am personally all for Lua to stay and expanded, but I am very disappointed that it is not being used by others and I have limited time to contribute to dda-lua myself.

[Arche_Zwei]

manufacturing equipment to mass produce parts like steel plates and re-establish industry (giving a function to every Macguffin machine that appears in factories);

Now THAT I would be interested in doing.

I also wanted to add more climatic zones to the map.
DF draws the mountains first, then similates wind, temperature and rain. Then places biomes. Then places civilizations (or in our case human cities, which then could crumble to their simulated end, collected in legends).

But how does LUA make one achieve this? I guess its better for this thsn scripting sand banks and island cities.

I however know only how to compile from source. And have been looking for a guide to get to change anything more complex than the starting screen.

But when I lived on a small tropical island in the middle of the ocean I dreamt of C:DDA taking place there as well.

Would have loved to set up a tent in the main road junction, cleared of people. And set out to scavenge drinking water from whereever and other stuff from the Shops nearby.

[Robik]

Is it possilbe to write a mod in LUA, which would iterate through all items and set “spoils_in” to 0 (to never spoil) for all items where “spoils_in” > some number ?

Edit: by item, I mean item type, not all items that are existing in the world

[Zhilkin]

All items like where? In player inventory?

[Robik]

All items like where? In player inventory?

item type, not all items that are existing in the world, sorry for messy wording

[Zhilkin]

Won’t be possible right now.

[lispcoc]

Can I make PRs to extend lua bindings?
There is work branch to exntend lua bindings has proceeded by @BevapDin(GitHub). I’m afraid of affecting his work.

[Zhilkin]

Sure, you can create PRs.

[lispcoc]

Thx! I’m planning to expose classes and functions to lua.

[overthrow]

Can I get a good example of how to view time durations on today’s experimental?

When I attempt to convert a time_duration via tostring, I get a result like "time_duration_metatable: 0x[pointer or reference address]. I can’t seem to view them, and I can’t seem to compare them with ints, I just get errors about comparing ints to time_duration_metatables, or more confusingly, time_duration_metatables to time_duration_metatables. Any help?

[Zhilkin]

Use game.get_time_duration or wrapper functions from ./lua/autoexec.lua (TURNS, MINUTES, HOURS and DAYS) - see https://github.com/CleverRaven/Cataclysm-DDA/pull/24140.

[lispcoc]

I suppose there may be no LUA API such converting time_duration to integer. It is necessary that add function definition like
{ name = "to_turns", rval = "int", cpp_name = "to_turns<int>", args = { "time_duration" } },

[lispcoc]

Hmm… I’m trying to expose c++ classes and functions to LUA, and thinking about what functions to be exposed.

1. Expose all functions as possible. (except less game rerated classes)
2. Expose functions I’m planning to use only now.
3. Discuss and decide what functions to be exposed.

1 requires very big changes. 3 may be waste of time. I suppose 2 is the better. What do you think?

[Zhilkin]

There is no sense to expose anything you won’t use.

[TheZoneWizard]

You must be married to an angry woman… wink

[Azure_Soul_Azur]

Seems jap mods use Lua more then others… reminds me of playing roblox and making my mini games… temted to relearn Lua and start making weird and crazy shit here in cata…

If life allows me to…