[QUESTION] What Tools do you use to diagnose crashes?

Those of us that Modify (MOD) Homeworld are in constant need of methods to diagnose crashes and game freezing. Among the most common and ubiquitous tools are the Hw2.log and HwRM.log files as well as the -luatrace function in the Homeworld Launch shortcut which allows the appending of trace functions (e.g. aitrace) to the aforementioned logfiles. Of course, the date and time-stamped MiniDump.dmp file comes in handy and even the ErrorLog.txt file can sometimes be of use. However, sometimes the Hw2/HwRM logfile gives us no hint of an error, the MiniDump is unhelpful or not even generated and the errorlog is indecipherable to our fledgling eye.

So, in our frustration we come to the GBX Homeworld Remastered Forums and attempt to inexpertly and hence incompletely explain the last modifications we made in our fumbling experimentations and the sequence of events immediately preceding the damnable crash or freeze that is perplexing us so in the hope that someone with greater experience can decipher the error that evades our investigations and point us toward a solution.

I do not know how many times I have found myself in exactly this predicament. I have learned far more from my mistakes (in both Homeworld Modding and in life in general) than I have learned from my successes, but a large part of what I usually learn is how to more expertly solve those mistakes as I adroitly continue to make new ones.

That having been said, asking for help is not one of my strong suits; I tell myself that it is because I take such joy from the troubleshooting and thus solving of problems on my own, but I cannot do so if I lack the proper tools.

I know that, considering the general brilliance and innovative genius of the Homeworld Community there must a plethora of tools, methods and techniques that have been developed and/or stumbled upon over the years that are of help in diagnosing crashes.

Hence the title of this topic: “What Tools do you use to diagnose crashes?”

Tools, Methods and Techniques contributed thus far:

01.) Version Control – the saving of stable versions of Mods prior to further modding in order to maintain a stable reference.
– Contributed by @Cloaked

02.)

Sorry to say, you’ve mentioned all the ways really (hwlog, errorlog) - so you’re only left with 1 option. Stripping it all out and building your mod back up bit by bit from nothing till it starts crashing again. Sadly something I had to do a couple times :frowning:

HW although is moddable, very very very little effort went into making modding easy as it could be…

1 Like

Although I appreciate your input (albeit as a naysayer) and understand that like yourself, not everyone is aware of such, I have seen other tools, methods and techniques (custom lines of code, Lua files, psychic powers, etc.) made or implemented by the various Mod Teams over the last twenty-three years that are helpful in diagnosing crashes (although I had little use for or understanding of them at the time). I found one such recently and somehow have misplaced it before I could utilize it and foolishly did not note and document which Mod Team had included it in their files.

So, once again I ask: “What Tools, Methods and Techniques do you use to diagnose crashes?”

One tool that I myself use is printdebug.lua (special debug commands, similar to the “aitrace” function which enables collated printing to “HW2.log”) within which I have created commands specific to each of the main lua files (i.e. cpubuild_trace, cpubuildsubsystem_trace, cpuresearch_trace, gamerule_trace, level_trace, etc.) as well as to library lua files (autodock_trace, autoharvest_trace, explosiondamage_trace, sobgroupfunctions_trace, sound_trace, etc.) as well as others (e.g. madstate_trace) with which I can track the progression of new data loaded to the game engine. Admittedly, this has been of very limited help when attempting to understand the root source of crashes.

@Bings By the way: Your two posts on shields are fantasticly helpful and thought-provoking. Keep 'em coming!

1 Like

I am new-ish (relatively speaking anyway ahaha) to the homeworld modding scene and was forced into early retirement by a certain studio so I’m not the most in-the-loop, so these tools are all very possible, but from when I’ve learnt in my short time it’s just pain and then some more pain!

Hopefully you can find some useful tools though! If you do please drop it on the dev wiki for any others who may find themselves in a similar position! Tool Reference · HWRM/KarosGraveyard Wiki · GitHub

1 Like

I did not know that the KarosGraveyard Wiki was yours. I have a amassed an extended collection of HW, HW2 and HWRM mods, tools, maps, hods, luas, tutorials, etc. (amounting to something like 250 GB of data) since 1999 that you are welcome to. Some of it is at “Index of /homeworld” which @radar3301 helped me to set up some time ago.

1 Like

It’s a mix of contributions largely by @Cloaked @EatThePath (and others who I can’t find tags for)! I made some minor contributions but in no way it’s mine nor can I take any credit for it! I’m sure they’ll love to go over your stash of info though!!!

2 Likes

@thomcomstock - Very nice tool collection! I added a link to it on the Karos Graveyard Tools page. @Mikali and @sastrei are some of the other contributors. :slight_smile:

Regarding diagnosing crashes, you already named my methods above. Just test often and use version control like git so you can easily see what changed since the last stable version.

Does anyone have any tips for analyzing .dmp and .mdmp files? It seemed Gearbox was able to analyze these files better than the modding community could, not sure what tools/methods they had at their disposal.

1 Like

@Cloaked

OK, thank you. I am presuming that by git you are referring to GitHub, correct? Version Control is an example of what I am referring to when I say TOOLS, Methods and Techniques. Version control is not something that I have taken the time to do and I have too often paid for it dearly.

So, here we have our first real contribution to the topic: Version Control - the saving of stable versions of Mods prior to further modding in order to maintain a stable reference.

I will be listing all contributions (Tools, Methods and Techniques) and contributors to this topic at the top.

1 Like

Yes, that’s correct. “Simply put, Git is a version control system that lets you manage and keep track of your source code history. GitHub is a cloud-based hosting service that lets you manage Git repositories”

I highly recommend taking the time to setup version control. I personally use the GitHub Desktop application as its easy to use.

1 Like

@Cloaked : I will definitely follow your advice. Thank you.

1 Like

So, no one has any techniques or methodology for analyzing game crashes other than those already mentioned?

@Cloaked I have installed the GitHub Desktop App. I am reading the How-Tos, but I am quite unfamiliar with using version control and its inherent terminology. Do you have any advice as to how to begin using it when developing a HW Mod or an example? I am not asking you to teach me how to use it, however if you have any insight that would be useful to a boot (i.e. noob, noobie, beginner…) it would be deeply appreciated.

I would try editing an existing repository first, such as Karos Graveyard. See the “In GitHub Desktop” section here: Wiki; Contributor Help · HWRM/KarosGraveyard Wiki · GitHub

Try editing the Sandbox page: Wiki; Sandbox · HWRM/KarosGraveyard Wiki · GitHub

Once you get the hang of that, then you can create your own repository for a mod. You could set it up similar to the 2.3 Players Patch repository which is here: GitHub - HW-PlayersPatch/Development: Players Patch development repository

1 Like

That is precisely the advice I was seeking. I do appreciate the help. I am hoping to one day myself become as useful to this community as you have been and continue to be. Thank you.

1 Like

Your welcome and thank you!

So, no one in the entirety of the Homeworld Community knows of any other tools, methods, techniques, tricks, etc that help in any way to log and/or diagnose game crashes and freezing?

I have used version control in the form of my guiding principle: “change as little as possible each time”. That way you know that what caused the crash must be the thing you changed.

That said, there can be strange things that creep in and cause sporadic crashes, the causes of which are very hard to find. The divide by zero bug is a case in point:

At one point I started making a list of things that were known (by me) to cause access violations and the associated last messages in the log file. If I can find it I will post it.

2 Likes

@Dom2

I did find the following some time ago on MODDB:


-- **HOMEWORLD 2 ERROR CODES** --
--
*Posted by siliconworm on Mar 31st, 2013 - Basic Client Side Coding*

When the game crashes, usually the reason for the crash is explained in Hw2.log, but there're some cases that there's simply no reason for the crash. This tutorial aims to record different reasons for crashes that apparently have no reason!

Some HW2 crashes just won't give any reason in Hw2.log, for example:

1. If LoadModel(NewShipType, 1) is put behind StartShipHardPointConfig() function in any ship script, the game will crash without any reason.

2. If LoadSharedModel() function is used for a subsystem script, make sure the target subsystem exists, otherwise the game will crash without any reason.

3. The outer bound of a map could be the reason for crash wihout any reason, increase it until it doesn't crash anymore. This problem could occur only when there's something really big in the map, for example, a planet.

4. The lod levels in ship script should match with the ones in the HOD file, otherwise when you zoom in on the ship, the game will crash without any reason.

5. From my experience, if the HOD file is somehow messed up, the game will crash without any reason.

6. The shader types and algorithm in every HOD model should match those in the shader folder. If there's any mismatch, the game will crash without giving any reason.
Here're some frequently recorded error codes in the ErrorLog.txt or MiniDump.dmp:

00000000 -- general issues, like loading a later game without outer bounds, or when a game has turned too messy and the memory is screwed up somehow. In the latter case, there's no clear solution except making the game less messy. One possible scenario is too many damage sound fx being played at the same time (dmp file shows exception code 0xC0000005), and applying a max polyphony limit to the sound should fix it. Changing the battlescar files (removing some battlescars) and then loading an previous game will cause this crash too during loading screen (dmp file shows exception code 0xC0000005).

0000000c -- related to big objects colliding when moving or rotating. Trying to avoid collisions by removing these big objects or making them unmoveable solves the problem. If there's a singularity or black hole that pulls all ships towards the center of the map, this collision error will definitly occur at some point in the game.

This crash also occurs right after a new game has loaded if a ship HOD contains undefined navlight. Bytes at CS:EIP: d8 51 0c df e0 f6 c4 05 7a 05 d8 71 0c eb 3d d8

00000010 -- This happens when a ship is built but its builder doesn't have "ShipHold" ability (exception code is 0xC0000005 in the dmp file).

This crash can also be produced when using a ship HOD in the background folder (don't ask me why I tried that).

It crashes at game start when there are undefined asteroids in the map.

There is strong evidence that repeated use of SobGroup_SetSwitchOwnerFlag in conjunction with SobGroup_MakeUntargeted function for an ever changing global sobgroup (such as during the spawning of dummy hyperspace gates) will cause 00000010 crash.The safe solution seems to be moving the SobGroup_SetSwitchOwnerFlag function to the beginning where that sobgroup was first created and call it there only once, and also removing all make untargeted commands (this was the bug that haunted my mod for 3 years and took hundreds of games to debug).

When the salvages of a ship don't have HOD models, 00000010 crash will occur upon loading the map that contains this ship. In ErrorLog.txt, there is mentioning of lua_error and lua.dll, except the list is much longer, and the bytes at CS:EIP are: 8b 48 10 83 c4 10 8d 85 98 fe ff ff 50 e8 2a 40.

00000020 -- this error code appears when the save menu says there's some invalid file name and crashes, but it could also have something to do with the hangar management system. In some occassions, accessing the launch menu just after a ship exited hyperspace will crash the game and give this error code, especially when there're supposed to be some instant dockers in the dock (e.g., repairing patchers are programmed to instantly dock when the host enters hyperspace). If the launching menu is accessed 10 seconds after hyperspace is finished, there won't be such a problem.

This code can also be produced when AMD graphics card is used to launch some mod for HW2Classic from the HWR bundle, causing instant crash (the solution is to add " -nopbuffer" to the command line, but at the cost of losing shadows).

00000030 -- I suspect this is related to launching conflict.

00000034 -- this can be caused by corrput debris fx in the events file (crashes during load). It can also be caused by loading ship HODs that have unsupported shaders (like loading HWR ship or asteroid models in HW2).

00000168 -- map object not found in the given type. This can happen when a dust cloud object is added with the addCloud function. Also happens when a weapon fire is not found in the weapon file.

00781000 -- related to visual effect overloading (dmp file shows exception code 0xC0000005). Bytes at CS:EIP:
a5 a5 8b 7d e8 83 45 e8 68 83 c7 20 ff 4d dc 8d

When too many Hiigaran pulsar beams, smoke animations or damage effects (especially dmg_damage_puff_combo and the default subsystem damage fx) are being played at the same time, the game will crash with this error code. Reducing or removing those effects will solve the problem.

0xC0000005 -- this crash happens upon starting the game if the number of skirmish options exeeds the maximum limit (32 is max including hidden). Also happens when exitting the program after a long game (HW2 seems to be freeing heaps of memory that were too large).

0xC0000017 -- fail to save after a long time of playing, reason unknown.

No error code crash -- the code is ■■■■■■ up.
Freezing problems:

Can be caused by infinite loop in the lua code.

Can be caused by a buildable ship without addAbility(NewShipType, "CanLaunch"). When this ship is built, the game would freeze.

Can be caused by a ship wihout the Latitude parameter on the hardpoint for the weapon. This type of freeze typically occurs after the ship gets out of hyperspace. This is a hardpoint problem.

Is the aforementioned similar to the list of which you referred?