It’s both.
By Relic, the next Homeworld installment was supposed to be the Dust Wars. It featured things like the Nalthorians, T-Mat, the cores, derelict interaction… and more. Most of what you see of that alternate story is what Relic had intended for Homeworld 2.
There were a few development documents passed around on Relicnews, one of which was a development document dated 11/16/99 (Specifically “Homeworld 2 Story Progress 10 pager Final_bullets”). The “Veyger” are mentioned in it as well. There is no sign of the beast, Somtaaw, or any of the Cataclysm trappings. There are several documents that happened after this to shape Homeworld development… according to Homeworld shipyards, Vijil managed to get a hold of these documents. They seem to be the same ones hosted on Assam Kiith’sid
The biggest and most vibrant shred of this story is their 2001 E3 Trailer:
There are other concept documents out there, which are mostly outlines and plans. From my angle there are several concept maps talking about various things like the Tobari, the core locations, and so on.
For some reason, Sierra contracted Barking Dog to do the Homeworld sequel instead. I don’t know why- my supposition is something between time investment or it was cheaper to do BD instead of Relic. Either way, Relic did have plans for Homeworld 2 but the reins were passed to an outside party. As far as I know Rob and some of the key creatives who made Homeworld happen were not involved with Cataclysm.
I do see something about Sierra cutting Relic’s funding, which may be the point where Relic passed the game to Barking Dog.
Presently, I think BBI still feels the Cataclysm changes to the story were too radical to continue with. I think if there’s gonna be another Homeworld game, as long as BBI has some control over the story content, you’d see it more like the Dust Wars and less like Cataclysm.