Skinned Mesh Automation - Study 3

A study in how to assemble baking meshes and automatically skin and export my Tiger asset all from scripts to reduce iteration time when making mesh changes

The Problem

I was seeing myself spend less and less time adding new things or even fixing old issues on the Tiger mesh, mostly because the complete time it took for me to send the mesh through the "pipe" and into engine was so long and for the most part boring. The process breaks down to something like this:


  1. Make change to mesh
  2. Set up and export exploded baking meshes
  3. Bake textures
  4. Set up and export texturing mesh
  5. Make texture changes
  6. Skin mesh and export
  7. Import into UE4


Highlighted in blue are the parts of the process that actually require human interaction, so I sat down one week in November to see if I couldn't come up with a way of reducing the other parts of the process to a bare minimum. Hopefully encouraging me to work more on improving the asset.



The Plan

I was doing the main bulk of the modeling in Maya, but figured this would be a good time to bust out Houdini for the automation aspect, and use it as the main asset assembly software going forward. Setting this up shouldn't be too hard since both Maya and Houdini have pretty good Python APIs and I don't need any soft skinning, only 'rigid' skinning.


For texture baking scripting is a bit more complicated. Substance Painter does not have a Python API per se, but you can send http requests from a Python script to an internal JS server and run commands using its somewhat limited Javascript API. I've used this before on bigger projects at work, but figured it would be overkill for this single asset project and not really save me much time so I left out texture baking from the automation.


With all this in mind I sat out to reduce the workflow to these points:


  1. Make change to mesh
  2. Run automation script
  3. Bake textures
  4. Make texture changes
  5. Import into UE4


Which would reduce the labourus logistics to just a couple button clicks and some baking waiting time. Great! The automation script in question would now deal with three things.


  1. Assemble and export exploded baking mesh
  2. Assemble and export texturing mesh
  3. Skin texturing mesh and export skeletal mesh


As mentioned, the plan was for all this to happen in Houdini after I've exported out model changes from Maya. However, the skinning process in Houdini I found very hard to proceduralize. It just seems like one of those relics from the past that are still very manual and not really Houdini-esque like almost all other parts of Houdini. After chatting with someone from SideFX I understood changes are coming to rigging but not in the immediate future, so I have to take the trip back to Maya for the skinning.



The Houdini Setup

You might be wondering why I'm so insistent on this custom exploded baking mesh, and the answer is that a lot of pieces on the tank are duplicates, and so share UV space, which all have to be removed. Although Painter has features for making only pieces of the same name affect eachother I felt safer doing the exploding myself so I could do other bakes in Houdini if I wanted to.



Most of the Houdini side is just a bunch of blasts, copies and transforms based on the name that is brought over from the high and low poly FBX exported from Maya. A bit of a pain to set up and not very elegant, but did the job and is quite sturdy. I might spend some time in the future making a mesh exploder that uses piece bounds and what not, because the built in Exploded View is not ideal, since it's just not made for this purpose.


One of the implications of having to go back to Maya for skinning is that I need a way to export geo that keeps both materials and hierarchy out of Houdini. This was harder than I first thought, since the FBX ROP simply does not deal with hierarchy at all. I spent some time coming up with a solution that ended up as my Hierarchical FBX exporter that I've already written a post about. Which, for the record, is not ideal and I'd love for SideFX to support this properly.



Back out in /obj and armed with my exporter I can now just point them to my low poly mesh output nulls and combined with a couple regular FBX ROPs inside the Tiger geo that deal with the high poly and main Tiger body, the scene is all set up. To make it run through without even opening Houdini I've made made two Python scripts and put them in the same folder as my Tiger.hip. ExportMeshes.py which will fire off the entire process and houdini_export.py which is the code that will run inside of Houdini. They both read as follows:


ExportMeshes.py

import subprocess
import os

# Path to Houdini Python interpreter, and all needed paths
houdini_path = "C:\\Program Files\\Side Effects Software\\Houdini 17.5.391\\bin\\hython.exe"
houdini_script = os.path.abspath('houdini_export.py')
hip_file = os.path.abspath('Tiger.hip')

# Runs Houdini mesh export
subprocess.call([houdini_path, houdini_script, hip_file])


houdini_export.py

import hou
import sys


# Grabs filepath from arguments send by ExportMeshes.py and loads that scene
hip_file = sys.argv[1]
hou.hipFile.load(hip_file)

# Execute export for low poly meshes
hou.node('/obj/skin_export').parm('exportbutton').pressButton()
hou.node('/obj/explode_export').parm('exportbutton').pressButton()
hou.node('/obj/Tiger/LP_Body_Out').parm('execute').pressButton()

# Execute export for high poly meshes
hou.node('/obj/Tiger/Exploded_Turret_HP_out').parm('execute').pressButton()
hou.node('/obj/Tiger/Exploded_Wheels_HP_out').parm('execute').pressButton()
hou.node('/obj/Tiger/Exploded_Body_HP_out').parm('execute').pressButton()
hou.node('/obj/Tiger/Exploded_Details_HP_out').parm('execute').pressButton()


Every time I run ExportMeshes.py it starts up a headless version of Houdini and runs the houdini_export.py script which opens the Tiger.hip file, and any new geo is picked up by the various File nodes before the FBX exports are triggered, leaving me with a new set of low and high poly meshes.



The Maya Setup

This is going to be somewhat similar to Houdini. I've put together a base scene containing the skeleton for my Tiger, and nothing else. I looked for a decent way of skinning some geo to specific bones and while I know of other ways I've done it in the past where I skin specific verts to specific bones, it felt a bit intense for what I needed here. Instead I'm simply running a skin mesh operation per geo in need of skinning. This does leave me with a bunch of bind poses, and although I'm not completely sure of the performance implications of these, though I'm sure it's fine for the scale at which I'm doing it.


The script I run simply loads a scene, skins geo to bones based on a dictionary - which I've shortened in this example - and selects the geo that goes together and exports it to a certain location. To run it I add it to the end of the previous ExportMeshes.py script, to ensure it happens after the other meshes are exported out of Houdini, leaving me with:


ExportMeshes.py

import subprocess
import os


# Path to Houdini Python interpreter, and all needed paths
houdini_path = "C:\\Program Files\\Side Effects Software\\Houdini 17.5.391\\bin\\hython.exe"
houdini_script = os.path.abspath('houdini_export.py')
hip_file = os.path.abspath('Tiger.hip')

# Path to Maya Python interpreter, and all needed paths
maya_path = "C:\\Program Files\\Autodesk\\Maya2018\\bin\\mayapy.exe"
maya_script = os.path.abspath('maya_export.py')
maya_scene = os.path.abspath('TigerSkel.mb')
geo_to_skin = os.path.abspath('OUTPUT\\Tiger_LP_static.fbx')
output_folder = os.path.abspath('TO_UNREAL')

# Runs Houdini mesh export
subprocess.call([houdini_path, houdini_script, hip_file])

# Runs skinning in Maya
subprocess.call([maya_path, maya_script, maya_scene, geo_to_skin, output_folder])


maya_export.py

import sys
import os
import maya.standalone
import maya.cmds as cmds


# Launch maya and grab paths from system arguments sent from ExportMeshes
maya.standalone.initialize(name='python')
cmds.loadPlugin('fbxmaya.mll')

maya_file = sys.argv[1]
maya.cmds.file(maya_file, o=True)

mesh_file = sys.argv[2]
cmds.file(mesh_file, i=True)

out_folder = sys.argv[3]


# Only an example, the whole dict is quite a bit bigger
# maps as 'Geometry' : 'JointName'
bone_mapping = {'Turret': 'turret',
                'Gun': 'gun',
                'MG': 'mg',
                'BackHingeLeft': 'l_backHinge',
                }

# Skins each piece of geo to a bone
for mesh in cmds.ls(long=True, geometry=True,):
    objname = mesh.split("|")[-2]
    if objname not in bone_mapping:
        print('Warning, unknown mesh: {}'.format(objname))
        continue
    skin = cmds.skinCluster(bone_mapping[objname], mesh, tsb=True, mi=1)[0]


# A dict of joint hierarchies and their output filename
skeletal_roots = {'rootAccessories': 'SK_TigerBodyAccessories.fbx',
                      'rootWheels': 'SK_TigerWheels.fbx',
                      'rootTurret': 'SK_TigerTurret.fbx'}

# For each joint hierarchy, select skinned geo and export to fbx
for root in skeletal_roots.keys():
    export_list = [root]

    for bone in cmds.listRelatives(root, ad=True):
        export_list.append(bone)

    for rel in cmds.listConnections(root):
        for transforms in cmds.listConnections(rel, t='transform'):
            skin_clusters_all = cmds.listConnections(transforms, t='skinCluster') or []
            for skin_clusters in skin_clusters_all:
                for shapes in cmds.listConnections(skin_clusters, t='shape'):
                    if shapes not in export_list:
                        export_list.append(shapes)

    cmds.select(export_list)
    out_file = os.path.join(out_folder, skeletal_filenames[root])
    cmds.FBXExport('-file', out_file, '-s')


And that's it! You might see the very strange syntax used for the cmds.FBXExport(), which is the result of the less than optimal implementation of the FBX plugin in Maya, and it was quite the adventure trying to wrangle it to my needs. An excellent blog post about this subject and more elegant ways of solving it can be found here: Laziness and cleanliness and MEL.


Maybe there will be more updates to the Tiger going forward. Although, ironically, the time spent on this system is the most time I've spent on the project in a while, hah.