MOD AUTHORS - Major Script/System Revisions!


(BitVenom) #1

Big Changes - Better Code

As previewed and hinted at in a few other threads, the next patch (super soon) - is loaded with new or updated systems, tweaks, and reorganizations that should allow for a whole new phase of Mod development. These changes also lay the groundwork for some of our internal efforts, prepare for new Mod features, and point toward where (and how) further changes will be made.

Because these edits encompass a LARGE number of very major systems they are going to make future Mod authors squeal with glee, and current Mod authors maybe a bit grumpy. This has been a HUGE effort and is looking quite solid - but as with all large deltas (something like 350 files at this point), it may create new issues (hopefully minor) - and the additional flexibility really demands that Mod authors step back from the code/script and think. The patterns introduced here will likely spread to other systems, so taking the time to get behind our new way of arranging and parsing data is critical to your sanity as a Mod author :wink:

There are a handful of ‘new’ (to this codebase anyway) ideas, things that are repeated any place we’ve done edits. The features and choices that these new structures/systems create a given area vary, but generally the thinking behind these changes is consistent:

  • Folder scans - The game gathers the things it can use at boot by searching for them. When possible, hard-coded ‘list’ lua files are GONE. Yes, race.lua is an example, I am sure you can think of others. The engine used to have some scans, but they weren’t consistent, and didn’t have the connections to related data that the new code utilizes.
    – The idea here is that the old ‘list’ files meant Mods had to edit them, making stacking or minor mods VERY hard. Now, if you love a certain Mod, you can add new stuff for it without affecting almost any of the base files.
  • Flexible data types - Tag (sorta a bool), Number (float), String (eh… string?). Now, when some thing you are working on (a Race, Game Rule, Ship, etc) needs to define most of its data, it will be stored in a sub-folder of that thing, inside a file by type (.tag, .prop).
    – This means that you can edit properties of many things in the world without editing existing files (usually) - and it also means people can add data on top of your mod (or to complement) without worrying about changes from the base Mod clashing. Done carefully, this can be insanely useful, empowering, and also really help clarify where design edits have been applied.
  • Tag filters - Most of the new files loaded by the game can be ‘removed’ from load by simply forcing them to fail a tag filter test. An easy example is one of the base HWRM races - Vaygr. If you just really don’t like them, you can have your Mod apply a filter (details below) to prevent them from loading. As long as nothing else you are loading requires them, the game will run fine, and they won’t appear. Your Mod can contain a special file that defines the tags to use when loading things. So, for a ‘TC’ style Mod, you can filter out all of the base content, and filter in only the content from your Mod. All without having to hack (ideally) any existing files.
    – Again, when you don’t have to hack files, you aren’t at risk of an update breaking your Mod, or somehow having to ‘delta’ a patch from us against your Mod. Sure, features or syntax may need some updates, but knowing that your Mod is your files, and will work is a huge benefit.

So, with the base concepts out of the way, let’s talk specifics:

Goodbye Rubrics

First and foremost ‘rubrics’ are GONE. The game has 3 major modes: Single Player, Skirmish, and MP. But those aren’t used or exposed to scripts/assets any longer. Instead, all modes must have a GameRule (GR). If you know much about the engine, you know this was already the case (sorta) - but the new system is making this the #1 rule.

Hello Game Rules (GR)

GR can ‘filter’ game content in a number of ways:

  • Limit Races available
  • Expand Rule-specific Race properties
  • Limit Levels available
  • Expand Ship-specific Ship properies

GR also have a number of new properties that can be seen and used by assets (.ship files) and in-game (the Rules lua, AI, etc)

  • Tags - essentially ‘yes no’ flags, any you can imagine.
  • Numbers - any you can imagine
  • Strings - seeing a pattern yet?

The properties of a GR aren’t stored in a single file (though an initial single file is used to create a GR)… instead there are a series of folders that are searched for script/properties to add to the GR. This means modding a GR for almost anything is an ‘add a new file’ process - almost NEVER a ‘change this existing file’ or ‘nuke this file’ process.

When a GR is selected the tag filters it has for Races/Levels are applied to what the game has detected, so if a level doesn’t match the GR tags for levels, it won’t show up in any lists - ditto for Races.

Races x10000

Races, like GR, now have properties - Tags/Numbers/Strings - and like GR they are loaded by searching sub-folders. In addition, when a GR filters (and keeps) a Race, an extra set of folders defined by the GR is searched under the Race - allowing GR to actually mutate Races. Maybe you have a ‘Free for all’ GR for DM - and the Races are all balanced - but then a ‘Topple the Empire’ GR for DM where one Race is MUCH more powerful and players are supposed to gang up, etc. This is all VERY easy now. This also means that balance tweaks and Mods become less about hacking, and more about careful planning and properties building.

Unit caps now exist per-race and per-game rule.

AI used to be a ‘per level’ or ‘per game rule’ thing. The AI files were very heavily hard-coded for races as well. Yeah, that’s gone.

AI can still be per-level. But AI is now defined per-race (and yes, GR can override this per-race). AI scripts use the Race properties and GR properties… so instead of ‘if race is Vaygr do X’ you see ‘math value = SelfRace_GetNumber(“important_property”, 1.0)’ - no more hard coding, period.

AI also used to live in a bubble where they couldn’t see much about the world - and while this is still true (though I think we have a bit more to do there) - the new systems to query Race and GR values, on top of AI being able to vary (completely distinct code per Race if you wish!) is going to mean much smarter AI with way less hacking or hard-coding. I am excited not just for what we can do, but for what all of you will do.

If a file used to exist in some random folder (I’m looking at you BuildAndResearch!!!) - it’s now most likely found under a specific Race’s path. Almost every script or config asset has been moved and rebuilt to work properly. In theory adding a new Race will be CRAZY easy.

Oh, and that old ‘Random’ Race select in Game setup? ‘Random’ is now a type of Race - and it can choose to filter the other races. So while ‘Random’ can be a Race which selects any playable (and allowed by the GR) race, it can also apply stronger limits…

  • Random
  • Random HW1 Race
  • Random HW2 Race
  • Random Evil Race
  • Races With Extra Arms

You get the picture. And as Mod authors, you can go nuts - without hacking.

Ship Properties From Outerspace!!!

As with GR and Races, Ships can now have properties that vary by the Rule being activated. Ships can also ask a Rule for properties. The important difference here is mostly name-space. Here’s an example:

A Scout is being loaded. It needs to define the maxHealth for the unit. The script can:

  • 1 - NewShipType.maxHealth = getRuleNum(NewShipType, “health_for_scouts”, 100)
  • 2 - NewShipType.maxHealth = getShipNum(NewShipType, “maxHealth”, 100)

For #1, that value will come from one of the files found in a ‘Props’ folder for the current GR. But for #2, that value will come from ‘Props’ in the Ship’s own folder, or a sub-folder for the current GR. It is local to only that ship.

Oh, and in-game, your code (UI, AI, and Rule LUA) can now ask for anything you define as a Property. You can add and define anything you want. This means you can decide that you wish AI would treat certain ships differently… and ask something like:

local myValue = Sob_GetStaticNumber("Hgn_Scout", "howUglyAmI", 3)
if myValue > 4 then FREAK OUT

In this case the Sob type was hard-coded, but your code may have tables or other symbolic paths to this info. Now, you can author data about ships and change it per GR, all in a single Mod. For a TC mod this is a big deal… you can make a GR for ‘balanced’ play, another GR for ‘aliens vs robots’ where aliens are 5x stronger… or a GR called ‘rice paper hulls’ where ships are super weak and weapons are super strong. Again, without hacking the crap out of every file - and all in a single place.

Smaller stuff

A few random new edits (less systemic, etc)

  • Research and Build items costing a single RU are now invisible - no need to force restrict them everywhere.
  • ‘Attack Run’ has new properties to make it much more natural and useful. Edits for other flight behaviors are in the R&D phase.
  • ATI scripts can now be selected from the UI. Colorblind or just preferential changes are easy and shouldn’t count as a ‘mod’. The toolkit will get edits for this shortly.

Bigger Picture

All of these edits and new systems enable Mods to do things that were simply unthinkable before:

  • Add Races to existing Mods (and the base game!)
  • Add Game Rules to existing Mods (and the base game!)
  • Create ‘balance’ style Mods that work on top of existing files (best done with a new GR)
  • Create Mods that Mix & Match different designs (space goverment vs ancient aliens)
  • Create new AI that has more information and design flexibility than ever before

From Old Mod to New Mod

Depending on the details of what your old Mod had in terms of edits to our base files, there are likely quite a few areas where things no longer function. In most cases this is going to be an issue of simply moving your edits to the newly located files, or adding new files to contain your edits and allowing them to stack on top of the updated structure. Here’s a quick list of the most common issues:

  • Races are now distinct folders/scripts. Virtually any race property before is now in only the folder and data files for that Race. The structure is pretty clean and easy to follow. Race meta-data (name, desc, etc), any scripts, stuff that you used to find all over other files that said ‘if Race == Race_Vaygr’, etc - is now probably just in a single file for that race. Ditto for Unit caps. If a Race used to override or customize a specific cap or family for itself - it’ll be in that Race’s data files.

  • AI and related data (Restricts, Build & Research, etc) - also either cleaned of Race hard-coding and left where it was (LevelData\lib, etc) - or moved into a specific Race.

  • Rubric tests in .ship files - Rubrics don’t exist! You should see properties loading from either the Rule or Ship (as in the example above). Think hard about how you edit ships, and try to make your edits also use external properties - it’ll make future edits easier. As a further note, it is likely that .ship variables (the basic stuff you see as NewShipType.xxx) will go away in a future release… to be replaced by an ‘auto load’ of Props with the same name. The other commands (adding abilities, weapons, etc) will switch to being sub-files of the ship (and yes optionally game rule!) - allowing what defines a ship to be built up by additional scripts. This will hopefully include adding new hardpoints, moving or deleting others, etc.

  • The SinglePlayerOptions.lua file used to be where the SinglePlayer GR lived. No longer, it is now inside a correct folder along with all of the other new GR files. There are a ton of changes, most of them very clear and obvious. If you have issues, please ask if they aren’t answered. A much more detailed discussion of each file and the data inside will be added to this thread shortly.

MY MOD?! What did you do?!

Some older or abandoned Mods will probably need real work to update for this patch. In this case, I urge you to contact the authors of those Mods. We can’t add new systems and improve features without making edits someplace… and Mods no longer being maintained are at risk of causing issues.

On the Radar

Almost certainly some of these edits will have side-effects, we’ll find and kill those as needed. In addition, some of the ‘old’ code that was very hardcoded will relate to Races or other new flexible systems - that has been heavily audited, but some may not be obvious… We’ll nuke old stuff and move more into the Race folders are required. If you, as a Mod author can see a way to change something, or a LUA interface that would help you (and not cause issues, create cheating trouble, etc) - make suggestions. This is still an evolving project for everyone involved!

Lastly

There are a rather large number of new interfaces to define, specs for scripts and meta-data to lay in a more technical manner, etc; so I will do that in the Mod support area. Once the patch is out (VERY SOON) - feel free to ask questions, try this new stuff out, etc. We aren’t done, and these patterns will continue to be refined and cleaned.


Update Rolling Out Aug 13
Latest Update Killed My Mod :(
Configuring SquadronSize the "right" way
Can there be a better way to do Trek-style shields?
(Siber) #2

Exciting stuff, it’s going to take me a while to digest everything there of course, but one question comes to mind that I don’t see addressed directly. In the old system there’s a lot of difficulties with building things cross-race or being able to build from captured ships of another faction, and I wish I could be more specific about the problems but we went to a hacky work-around enough years ago that I don’t remember the details of the problems that motivated the hack. I’m curious if you have any comment on how this area has changed in this overhaul.


(BitVenom) #3

Generally speaking, the initial phase of this reorg/rebuild was meant to retain behavior (in the base game). The experience for players should be almost entirely the same.

That said, there are a handful of reasons that ‘cross race’ builds are trouble. For one, the design/balance of a player and their state (research active, etc) - may not be compatible with a unit from outside of that race. Multipliers and the like could act differently on that unit, cause issues, etc. In addition, there’s the issue of unit caps. One race may have a higher unit cap because their unit is weaker or by design more plentiful. If a captured ship creates units for that cap that are stronger or better - balance suffers. Even if we took into account this new unit’s unique cap, it would still be hard to reconcile. I recall there were a few similar (though small) issues.

This is, however, one area I think we’re going to look at - if changes can be made to allow this when a designer is comfortable with it, well then great.


(Siber) #4

Cross-race build is indeed a tricky area, but isn’t the only possible desired outcome. In HW1C capturing a cross-race production ship meant you could use it to build your own faction’s ships instead of it’s original shiplist, unless my memory deceives me horribly, which makes capturing a production ship valuable without being game breaking.


(Sirshelley) #5

NOICE!
Appreciate your efforts!, we’ll try and find uses for all this restructuring.

Although it’s looking more viable to make a new ‘complex’ mod base for the series considering.
P.S
The list/array override system certainly made me jump for joy!


(Sastrei) #6

The asymmetric balance mutations will be great for vorlon/shadows and borg.

I am ecstatic and can’t wait to sink my teeth in. :smiley:


(Nakamura14) #7

This is going to be great for the custom scripts Rebirth will use.
As already discussed, we still need the ability to put extra stuff on existing ships (eg buildable weapons on some), but this is great news!


(BitVenom) #8

If you check out what we did, and how - and then consider the sorts of structures we’re trying to reinforce, you’ll probably see where the functionality you’re waiting for is likely to appear. There is a (loose) plan for it right now.


(Nakamura14) #9

Exactly, that’s why I’ve been bouncing since seeing this.


(Taiidan Republic Mod) #10

If anyone else is seeing something like this:

Error: Universe::createSquadron: could not find ship type (kus_scout) -- FATAL EXIT -- sobfactory/117:! --stack trace--        

The solution is to remove that “Rubric” bit from the .ship file.


(BitVenom) #11

Exactly - if you are doing ANY Rubrics work, get rid of it. Check out how we resolved those to instead be Rules/Ship properties… where it used to say something like “what is my health in MP?” - now it says “what is my health?” - and in that ship, under SinglePlayer\Props is a file with a different health than the default.


(BitVenom) #12

Another thing to note (more of this will be covered in a more technical post very soon) - is that when a Rule (GR) is selected and things that depend on the GR are loading, they don’t just search a sub-folder if that GR’s pathname. The GR can specify a list of paths to search. So if you have a mod with some basic changes (regardless of GR) - and then GR to add flavor or variance, you can do this:

GR - BaseMod
Race_Paths = “BaseMod”

GR - Weird Combat
Race_Paths = “BaseMod,WeirdStuff”

Then, when a Race or Ship needs to gather properties, it will look in [Race/Ship]\BaseMod\Props*.lua for both. Weird Combat will then also check [Race/Ship]\WeirdStuff\Props*.lua


(Cmd Bob) #13

I’ve seen this in the patch notes for the new patch:

“Added optional DefenseShield sob to ships.”

Does this mean that we can have Star Trek-esque shields on ships? (and if so, how does it work?)


(Cebius) #14

The update is sure interesting, but indeed broke my race mods. I am currently confused on how to make them work again.
I have 3 race mods, Turanic Raiders, Kadeshi and The Derelicts, all using the stock data and content.

If i would, for example, let us say, make my Turanic Raiders mod to work again, what is the process that is required for me to do?


(BitVenom) #15

There is code in there… yes. It is also changing again. So, you can use it. But we aren’t putting any examples of it out yet because already it has been renamed, has new options, etc. It’ll be with you very soon, don’t worry.


(Cmd Bob) #16

I see. Awesome to know, and I suppose that I’ve got a lot to do before I can start adding shields anyway, like reformatting and that!


(Sastrei) #17

Shields…SHIELDS! /sulu


(Cebius) #18

Nevermind on my end. I figured it out. Seems it is alot easier than i thought.
I guess people with brain damage can also figure things out (like me. No i am not kidding)


(Nathanius) #20

Do you feel the love for getting this done? You should be able to :smiley:


(BitVenom) #21

Yeah, we have talked about what it would take to do shields ‘for the fans’ - as the features you want aren’t really ones we need. I can’t make any promises. But certainly, we are aware, have some idea of the effort (not terrible, but not cheap) - and I have some ideas for Examples of it once done that’ll blow minds.