[TOOL] HWRM DAE importer for Blender


(Taiidan Republic Mod) #1

DAE importer for Blender. The aim is to import dae files into Blender that are export-ready using the Blender HW Toolkit.

Latest version:

To install, put it in the place where the blender addons live:

Then, enable the addon using File > Preferences

Now, you should have a new option under File > Import > …

Issues

  • The biggest issue currently is that dock paths do not import properly most of the time.
  • RODOH dae files have issues, sometimes related to engine glows, sometimes just issues in general…
  • Some ships crash Blender. If this happens uncheck the “split normals” option in the import dialogue.



Original post

Inspired by @PayDay and the DAE editor, I have delved into the land of python scripting for Blender. Yesterday I got it importing the nodes in the right places and with the right hierarchy. Then I closed Blender, which does not prompt you to save any unsaved text in the text editor, for example the script you have spent hours writing and not yet saved… :’(

So now I have re-written it better, and it is creating dummy meshes as well:

So, baby steps for the time being, but I hope to get meshes and materials importing as well soon!


Blender DAE Exporter for HWRM
[TUTORIAL] Blender - Getting a simple ship in game
A question on using Blender to import DAE files
Beyond Confused (How do I get HODOR?)
Beyond Confused (How do I get HODOR?)
Modding tutorials Master Thread
[TUTORIAL] Blender - MAD Animations
(Christoph Timmermann) #2

Oh wow, it’s awesome to help bringing great projects to life. :slight_smile:

That sucks, but hey, you made it even better after all. :smiley:
Looking forward to this, nice work.


(ajlsunrise) #3

Let me know if I can help anywhere… :slight_smile:


(Taiidan Republic Mod) #4

Meshes importing:

sort of:


(Taiidan Republic Mod) #5

Ok, that one was quite easy to solve:

This is definitely going somewhere now :slight_smile:


(D Kesserich) #6

NICE!

Have you put your source anywhere? I’d love to scan through it and maybe help out if I’ve got the time.


(Taiidan Republic Mod) #7

Sure, here is the code:

# HWRM DAE Importer for Blender
#
# To do:
# - Handle nav names to match better collada exporter (currently truncated)
# - Get image file names & materials from the DAE
# - Get UVs from the DAE
# - Get animations from the DAE
# - Make a dialogue box for the file import

import bpy
import xml.etree.ElementTree as ET

context = bpy.context

print("--------------------------------------------------------------------------------")
print("DAE import script")
print("--------------------------------------------------------------------------------")

def MakeJnt(jnt_name,jnt_locn,jnt_rotn,jnt_context):
    print("Creating joint" + jnt_name)
    this_jnt = bpy.data.objects.new(jnt_name, None)
    jnt_context.scene.objects.link(this_jnt)
    pi = 3.14159265359
    this_jnt.rotation_euler.x = joint_rotation[0] * (pi/180.0)
    this_jnt.rotation_euler.y = joint_rotation[1] * (pi/180.0)
    this_jnt.rotation_euler.z = joint_rotation[2] * (pi/180.0)
    this_jnt.location.x = float(jnt_locn[0])
    this_jnt.location.y = float(jnt_locn[1])
    this_jnt.location.z = float(jnt_locn[2])
    return this_jnt

def CheckForChildren(node,context):
    for item in node:
        if "node" in item.tag:
            if bpy.data.objects.get(item.attrib["name"][0:63]) is None:
                print(item.attrib["name"] + " is a child of " + node.attrib["name"])
                print(item.attrib["name"] + " does not exist...")
            else:
                child = context.scene.objects[item.attrib["name"][0:63]]
                parent = context.scene.objects[node.attrib["name"][0:63]]
                child.parent = parent
                CheckForChildren(item,context) # check for next generation?

def CreateMeshFromData(name,origin,verts,faces):
    # Create mesh and object
    me = bpy.data.meshes.new(name)
    ob = bpy.data.objects.new(name, me)
    ob.location = origin
    ob.show_name = True
    
    # Link object to scene and make active
    scn = bpy.context.scene
    scn.objects.link(ob)
    scn.objects.active = ob
    ob.select = True
    
    # Create mesh from given verts, faces.
    me.from_pydata(verts, [], faces)
    # Update mesh with new data
    me.update() 
    return ob

#DAEpath = "C:/Program Files (x86)/Steam/steamapps/workshop/content/244160/403557412/Kad_Swarmer/Kad_Swarmer.DAE"
DAEpath = "C:/Program Files (x86)/Steam/steamapps/workshop/content/244160/403557412/Tur_P1Mothership/Tur_P1Mothership.DAE"

################################################################################
################################## XML parsing #################################
################################################################################

tree = ET.parse(DAEpath)
root = tree.getroot()

print(" ")
print("CREATING JOINTS")
print(" ")

# Create joints
for joint in root.iter("{http://www.collada.org/2005/11/COLLADASchema}node"): # find all <node> in the file
    # Joint name
    joint_name = joint.attrib["name"]
    print("")
    # Joint location
    joint_location = joint.find("{http://www.collada.org/2005/11/COLLADASchema}translate")
    if joint_location == None:
        joint_location = ['0','0','0'] # If there is no translation specified, default to 0,0,0
    else:
        joint_location = joint_location.text.split()
    # Joint rotation
    joint_rotationX = 0.0
    joint_rotationY = 0.0
    joint_rotationZ = 0.0
    for rot in joint:
        if "rotate" in rot.tag:
            if "rotateX" in rot.attrib["sid"]:
                joint_rotationX = float(rot.text.split()[3])
            elif "rotateY" in rot.attrib["sid"]:
                joint_rotationY = float(rot.text.split()[3])
            elif "rotateZ" in rot.attrib["sid"]:
                joint_rotationZ = float(rot.text.split()[3])
    joint_rotation = [joint_rotationX,joint_rotationY,joint_rotationZ]
    print("For joint " + joint_name + ", rotation = " + str(joint_rotation))
    # Joint or mesh?
    is_joint = True
    for item in joint:
        if "instance_geometry" in item.tag:
            print("this is a mesh:" + item.attrib["url"])
            is_joint = False
    # If this is a joint, make it!
    if is_joint:
        MakeJnt(joint_name, joint_location,joint_rotation,context)

print(" ")
print("CREATING MESHES")
print(" ")

# Create meshes:
for geom in root.iter("{http://www.collada.org/2005/11/COLLADASchema}geometry"): # find all <geometry> in the file
    print("Found <geometry> " + geom.attrib["name"])
    for mesh in geom.iter("{http://www.collada.org/2005/11/COLLADASchema}mesh"): # find all <mesh> in the <geometry>
        print("Found <mesh> " + mesh.tag)
        # Mesh vertices
        for array in mesh.iter("{http://www.collada.org/2005/11/COLLADASchema}float_array"): # find all <float_array> in the <mesh>
            if "POSITION" in array.attrib["id"]:
                print("Found position array")
                #print(array.text)
                vertex_data = array.text.split()
                verts = []
                coord = 0
                this_vertex_coords = []
                for v in vertex_data:
                    coord = coord + 1
                    this_vertex_coords.append(float(v))
                    if coord == 3:
                        verts.append(this_vertex_coords)
                        coord = 0
                        this_vertex_coords = []
            elif "Normal0" in array.attrib["id"]:
                print("Found normal array")
        # Mesh triangles
        trias = []
        for tria in mesh.iter("{http://www.collada.org/2005/11/COLLADASchema}triangles"): # find all <triangles> in the <mesh>
            print("++++++++++++++++++++++++++++++++++++++++++++++")
            print("Triangles:")
            print("++++++++++++++++++++++++++++++++++++++++++++++")
            offset_per_vertex = -1
            for inp in tria.iter("{http://www.collada.org/2005/11/COLLADASchema}input"): # find all <input> in the <triangles>
                print("input: " + inp.attrib["semantic"] + " = " + inp.attrib["offset"])
                if "VERTEX" in inp.attrib["semantic"]:
                    tria_vertex_offset = int(inp.attrib["offset"])
                offset_per_vertex = offset_per_vertex + 1
            print("offset_per_vertex = " + str(offset_per_vertex))
            for p in tria.iter("{http://www.collada.org/2005/11/COLLADASchema}p"): # find all <p> in the <triangles>
                print("Found triangles")
                if p.text:
                    tria_data = p.text.split()
                    this_offset = 0
                    this_vertex = 1
                    this_tria_verts = []
                    for i in range(0, len(tria_data)):
                        if this_offset == tria_vertex_offset:
                            this_tria_verts.append(int(tria_data[i]))
                        elif this_offset == offset_per_vertex: # if this is the last value in the vertex
                            if this_vertex == 3:
                                # triangle definition complete
                                trias.append(this_tria_verts)
                                this_offset = -1 # this will be bumped up to 0 by the last statement in the for loop
                                this_vertex = 1
                                this_tria_verts = []
                            else:
                                # triangle definition not complete, move to next vertex
                                this_vertex = this_vertex + 1
                                this_offset = -1
                        this_offset = this_offset + 1
        # Make a mesh!
        origin = (0.0,0.0,0.0)
        CreateMeshFromData(geom.attrib["name"].rstrip("Mesh"),origin,verts,trias)

print(" ")
print("SORTING HIERARCHY")
print(" ")

# Sort out hierarchy
for child in root:
    if "library_visual_scenes" in child.tag:
        for grandchild in child:
            if "visual_scene" in grandchild.tag:
                for node in grandchild:
                    if "node" in node.tag:
                        CheckForChildren(node,context)

print(" ")
print("DONE")
print(" ")

I have to admit it is a bit messy at the moment… In the fullness of time would you consider including it in your HWRM toolkit for Blender?


(Taiidan Republic Mod) #8

Struggling with materials at the moment. I can parse the material names and images for each mesh from the DAE file, but not quite sure how to get them to apply to the object. Anyone got any ideas?


(Christoph Timmermann) #9

Every geometry has a mesh node which has a triangles node, the triangles node has an attribute called material, that should be your reference to the material in library_materials.

For example:

  • <geometry id="MULT[Ter_Rapier]_LOD[0]_TAGS[DoScar]" name="MULT[Ter_Rapier]_LOD[0]_TAGS[DoScar]">
  • <mesh>
    • <triangles count="7581" material="MAT[terran_frigate]_SHD[shipglow]">

(BitVenom) #10

I think he means within Blender, the correct commands to make that connection…


(D Kesserich) #11

Absolutely. I’d recommend getting a Github account and hosting your script there, it’ll make collaboration and integration a lot easier.

[quote]Struggling with materials at the moment. I can parse the material names
and images for each mesh from the DAE file, but not quite sure how to
get them to apply to the object. Anyone got any ideas?[/quote] I know how to create the material and add it to the object (bpy.data.materials.new(MaterialName) for creating the material, and then object.data.materials.append(thatMaterial), I don’t know applying to the correct triangles. Looking at your script I think there’s going to have to be a bit of a refactor in the geometry building to create the UVs and plugging the materials onto the triangles at the same time.


(Christoph Timmermann) #12

I always thought that Blender treats meshes with multiple materials on them as separate meshes internally, because that’s how most engines do that I think.


(D Kesserich) #13

I don’t think so. I know assimp does that because we were having some issues with it doing that at work, but I don’t believe it’s actually done that way hierarchically in Blender.


(Taiidan Republic Mod) #14

Another baby step - materials created (only with DIFF texture at the moment) and assigned to meshes:

The problem I’m having is that each mesh seems to have multiple (duplicate) materials defined, for instance:

      <node name="ROOT_LOD[3]" id="ROOT_LOD[3]" sid="ROOT_LOD[3]">
        <translate sid="translate">24.765867 0.000000 -0.000000</translate>
        <rotate sid="jointOrientX">1 0 0 -90.000000</rotate>
        <rotate sid="rotateZ">0 0 1 0.000000</rotate>
        <rotate sid="rotateY">0 1 0 -0.000000</rotate>
        <rotate sid="rotateX">1 0 0 90.000003</rotate>
        <extra><technique profile="FCOLLADA"><visibility>1.000000</visibility></technique></extra>
        <node name="MULT[P2Swarmer]_LOD[3]" id="MULT[P2Swarmer]_LOD[3]" sid="MULT[P2Swarmer]_LOD[3]">
            <rotate sid="rotateZ">0 0 1 0.000000</rotate>
            <rotate sid="rotateY">0 1 0 0.000000</rotate>
            <rotate sid="rotateX">1 0 0 0.000000</rotate>
            <instance_geometry url="#MULT[P2Swarmer]_LOD[3]-lib">
                <bind_material>
                    <technique_common>
                        <instance_material symbol="MultiMat_0_ncl1_2" target="#MultiMat_0_ncl1_2"/>
                        <instance_material symbol="MAT[Kad_Swarmer_autogen_1]_SHD[ship]" target="#MAT[Kad_Swarmer_autogen_1]_SHD[ship]"/>
                        <instance_material symbol="MAT[Kad_Swarmer_autogen_3]_SHD[ship]" target="#MAT[Kad_Swarmer_autogen_3]_SHD[ship]"/>
                    </technique_common>
                </bind_material>
            </instance_geometry>
            <extra><technique profile="FCOLLADA"><visibility>1.000000</visibility></technique></extra>
        </node>
      </node>

What is “Kad_Swarmer_autogen_1” and “Kad_Swarmer_autogen_3”? Did you come across this @PayDay? How do you handle it?


(Christoph Timmermann) #15

These are strange materials in the Kad_Swarmer DAE from the Steam Workshop Examples. AFAIK, only the swarmer has them. (I think they are connected to the LOD-models.)

Assimp splits a multimaterial mesh and gives me indices of the materials assigned to them for each mesh. That’s how it works in DAEnerys.


(D Kesserich) #16

I finally started playing around with this (or at least the version you pasted earlier in the thread). For some reason it is garbling the hell out of the triangles when I run it.

It also exposed a massive issue with the DAE exporter that I never noticed that I inherited from the original version that is going to take me a while to completely fix (it’s exporting four vertices for every one in the mesh. I’ve got that sorted, but that had a knock-on effect of messing up the materials. I’m really starting to hate the dude who wrote this thing originally.)


(Taiidan Republic Mod) #17

Works fine for me with the Taiidan Interceptor (Blender version 2.76):

I have tried to put this on github, not really sure how it works though…

The next steps are UV mapping - no idea how that works - and assigning specific materials to specific polygons - I think bpy.ops.object.material_slot_assign() is the way forward for that.


(D Kesserich) #18

Hm. Tried it again and the Taiidan Interceptor comes in fine. I’m getting the garbled tris with anything that came out of the DAE Exporter, though. I can’t figure out why. They’ve always been good enough for HODOR, and they’re opening fine in Visual Studio.


(Taiidan Republic Mod) #19

Yeah, I’m having a similar problem. There must be some small syntax differences between Blender and 3DSmax produced DAEs. I do make a couple of dangerous assumptions, like the first vertex id being 0, which may not hold true in all cases…

EDIT:

A couple of tests:

  1. Example ships = meshes import fine

  2. My own ships from 3DS max = meshes import fine

  3. RODOH DAE = meshes import fine

  4. Blender-generated DAE = triangles messed up…


(Snake_B5) #20

I don’t understand what you’re complaining about…
Blender seems to blend the file pretty good in my opinion :stuck_out_tongue:

^^