It’s a bug with the launcher.
In the launcher, after selecting your MOD, you should be able to add “-mpbeta” (without the quotes) on the ‘Commandline Arguments:’ line to pass this along to the game (this argument is only needed while the multiplayer beta is still in effect). But there is a bug with the launcher where it is not appending the arguments you type into the 'Commandline Arguments:" section when it runs the game.
There’s 2 workarounds for this problem.
The easiest fix for this is to right click on the game in Steam, select ‘Properties’ then 'SET LAUNCH OPTIONS…" then add “-mpbeta” (without the quotes) to the launch options. This will always be used and will interfere with things when you are trying to play the single player campaign (so you would need to remove that launch option when you want to switch back to single player). This will enable multiplayer when running the game with mods.
The other workaround would be to create a shortcut to the executable in the HomeworldRM\Bin folder (HomeworldRM.exe) and then modify the shortcut to add “-mpbeta” to the end of the target line, followed by “-workshopmod”, followed by the path to your mod. This path will be different for different players. The path will start with the folder inside the HomeworldRM\DataWorkshopMods folder (that folder will by your numerical Steam ID) followed by a backslash, followed the folder with your mod’s Workshop ID, followed by a backslash, followed by the name of your mod’s .big file, like this:
-mpbeta -workshopmod 1234567890\12345\MyMod.big
You can use that shortcut to explicitly run multiplayer with your mod. Just be aware that each player will need to modify this path to use their specific SteamID as the first part of the folder name.
I’ll get the launcher fixed to use the ‘Commandline Arguments:’ text properly and in the future, you can just add “-mpbeta” in that box in the MOD section of the launcher.
EDIT: Opps, I left out the “-workshopmod” part of the commandline when using a shortcut.