Before posting an official ‘feature request’, I was wondering if there is any way right now to do a “ray cast” on objects for object picking. I need accurate object selection based on the current state of a mesh or skinned mesh. I don’t need physics in my project at all.
I’m currently using Physics.Raycast to do this, but there are two key drawbacks:
When a skinned mesh animates, the attached MeshCollider doesn’t update with it. I currently must re-calculate the MeshCollider whenever an animation moves the skinned mesh (slow annoying).
When adding a lot of MeshColliders, it takes a very long time where Unity is non-responsive. For example, one scene I load has ~150k triangles. It takes approximately one second to load from an Assetbundle. When I add MeshColliders to it, it then takes about 16 seconds to load from the AssetBundle (and I can’t even show a ‘loading…’ progress bar).
I know it may not be fast to use the same method that the Unity editor itself uses, but at least it CAN do selection on objects without colliders and with arbitrary skinned mesh animations.
Here’s the component I use to do the collision mesh update. You just attach it to the object that has a SkinnedMeshRenderer and MeshCollider, and whenever you need to update the collision mesh (triggered by playing another animation, etc.) just set the “forceUpdate” flag on this component. It only works with Unity 2.1 (or later?)
–
David Robinson
Coole Immersive, Inc.
using UnityEngine;
using System.Collections;
public class SkinnedCollisionHelper : MonoBehaviour
{
// Public variables
public bool forceUpdate;
// Instance variables
private CWeightList[] nodeWeights; // array of node weights (one per node)
private Vector3[] newVert; // array for the regular update of the collision mesh
private Mesh mesh; // the dynamically-updated collision mesh
private MeshCollider collide; // quick pointer to the mesh collider that we're updating
// Function: Start
// This basically translates the information about the skinned mesh into
// data that we can internally use to quickly update the collision mesh.
void Start()
{
SkinnedMeshRenderer rend = GetComponent(typeof(SkinnedMeshRenderer)) as SkinnedMeshRenderer;
collide = GetComponent(typeof(MeshCollider)) as MeshCollider;
if (collide!=null rend!=null)
{
Mesh baseMesh = rend.sharedMesh;
mesh = new Mesh();
mesh.vertices = baseMesh.vertices;
mesh.uv = baseMesh.uv;
mesh.triangles = baseMesh.triangles;
newVert = new Vector3[baseMesh.vertices.Length];
short i;
// Make a CWeightList for each bone in the skinned mesh
nodeWeights = new CWeightList[rend.bones.Length];
for ( i=0 ; i<rend.bones.Length ; i++ )
{
nodeWeights[i] = new CWeightList();
nodeWeights[i].transform = rend.bones[i];
}
// Create a bone weight list for each bone, ready for quick calculation during an update...
Vector3 localPt;
for ( i=0 ; i<baseMesh.vertices.Length ; i++ )
{
BoneWeight bw = baseMesh.boneWeights[i];
if (bw.weight0!=0.0f)
{
localPt = baseMesh.bindposes[bw.boneIndex0].MultiplyPoint3x4( baseMesh.vertices[i] );
nodeWeights[bw.boneIndex0].weights.Add( new CVertexWeight( i, localPt, bw.weight0 ) );
}
if (bw.weight1!=0.0f)
{
localPt = baseMesh.bindposes[bw.boneIndex1].MultiplyPoint3x4( baseMesh.vertices[i] );
nodeWeights[bw.boneIndex1].weights.Add( new CVertexWeight( i, localPt, bw.weight1 ) );
}
if (bw.weight2!=0.0f)
{
localPt = baseMesh.bindposes[bw.boneIndex2].MultiplyPoint3x4( baseMesh.vertices[i] );
nodeWeights[bw.boneIndex2].weights.Add( new CVertexWeight( i, localPt, bw.weight2 ) );
}
if (bw.weight3!=0.0f)
{
localPt = baseMesh.bindposes[bw.boneIndex3].MultiplyPoint3x4( baseMesh.vertices[i] );
nodeWeights[bw.boneIndex3].weights.Add( new CVertexWeight( i, localPt, bw.weight3 ) );
}
}
UpdateCollisionMesh();
}
else
{
Debug.LogError(gameObject.name + ": SkinnedCollisionHelper: this object either has no SkinnedMeshRenderer or has no MeshCollider!");
}
}
// Function: UpdateCollisionMesh
// Manually recalculates the collision mesh of the skinned mesh on this
// object.
void UpdateCollisionMesh()
{
if (mesh!=null)
{
// Start by initializing all vertices to 'empty'
for ( int i=0 ; i<newVert.Length ; i++ )
{
newVert[i] = new Vector3(0,0,0);
}
// Now get the local positions of all weighted indices...
foreach ( CWeightList wList in nodeWeights )
{
foreach ( CVertexWeight vw in wList.weights )
{
newVert[vw.index] += wList.transform.localToWorldMatrix.MultiplyPoint3x4( vw.localPosition ) * vw.weight;
}
}
// Now convert each point into local coordinates of this object.
for ( int i=0 ; i<newVert.Length ; i++ )
{
newVert[i] = transform.InverseTransformPoint( newVert[i] );
}
// Update the mesh ( collider) with the updated vertices
mesh.vertices = newVert;
mesh.RecalculateBounds();
collide.sharedMesh = mesh;
}
}
// Function: Update
// If the 'forceUpdate' flag is set, updates the collision mesh for the skinned mesh on this object
void Update()
{
if (forceUpdate)
{
forceUpdate = false;
UpdateCollisionMesh();
}
}
}
First thanks for the code, I was looking for that kind of thing.
Second, I’d suggest you implement your own sphere-casting routine, where you check your ray manually for hitting any of your skinned meshes within a radius, see pseudo code for dist_Point_to_Line :
Then update only the collision skin of the nearest animated mesh for which they ray will hit within a radius. This should save you quite a bit of CPU cycles.
Third, perhaps I’m not versed enough in Unity’s API - but IMHO it would be useful to have a function to test against a single colider and not all of coliders, I suppose you could change layers of objects temporarily to achieve that.
Regarding doing testing on very complex meshes without Colliders (150k tris worth of MeshColliders is definitely not going to end in good things) is to render the scene to a render target with replacement shaders. Give each object a 24 bit ID. Then Render the scene to a RenderTarget with each selectable object being rendered. Set the colour in the replacement shader to be R = the first 8 bits of the ID, G = the next 8 bits, B = the last 8 bits. Then get the pixel that was clicked on in the scene and convert it’s RGB value into a 24bit value (stack the 3 values one after another). Then you can retrieve the object clicked on based on ID. This allows you to have 16 777 216 unique objects (if you need less you can only use a single channel). And will remove the massive weight you are putting on the physics system. It will also reuse the skinning done for rendering the actual frame.
Edit: I’m not sure if Unity supports Multiple Render Targets, but if it does, you should be able to do all this in the original Rendering pass (unless deferred is already using too many RTs).
Try to not raycast against meshes with a large amount of triangles. Unity has to search through all the triangles to find which one was hit. No matter how efficient Unity might be at sorting through them with a quad-tree or whatnot, it obviously will take a lot of time.
Use culling masks to eliminate things you don’t want to raycast against. Be selective as you can. If you’re casting a ray to fire at an enemy player, don’t need to include objects that aren’t important to what you’re trying to accomplish.
To make it work, I added collide.sharedMesh = null;
before your line: collide.sharedMesh = mesh;
And added two classes to the bottom of the script file:
class CVertexWeight
{
public int index;
public Vector3 localPosition;
public float weight;
public CVertexWeight(int i, Vector3 p, float w)
{
index = i;
localPosition = p;
weight = w;
}
}
class CWeightList
{
public Transform transform;
public ArrayList weights;
public CWeightList()
{
weights = new ArrayList();
}
}
Thanks for this code, it really helped. BUT it has several severe performance issues that I fixed. In the following is a “high-performance” version that is apx. a thousand times faster if not more… Tested with Unity 4.0.
It could be further improved by using two vertex buffers and swapping them on each frame, so you would also save the memory allocation… But I didn’t need that since I am just writing a mesh converter.
In the future, please keep in mind, NEVER EVER access Mesh.vertices stuff within a loop. It will create a copy every-time you access it. (First I thought your code crashed Unity, but after a few minutes it was done with my mesh LOL)
void ApplySkinningToTriangles(SkinnedMeshRenderer skin, Mesh mesh)
{
// Make a CWeightList for each bone in the skinned mesh
var vertices = mesh.vertices;
var bindposes = mesh.bindposes;
var boneWeights = mesh.boneWeights;
var bones = skin.bones;
// Start by initializing all vertices to 'empty'
var newVert = new Vector3[vertices.Length];
for (int i = 0; i < newVert.Length; i++)
{
newVert[i] = new Vector3(0, 0, 0);
}
// Create a bone weight list for each bone, ready for quick calculation during an update...
Vector3 localPt;
for (int i = 0; i < vertices.Length; i++)
{
BoneWeight bw = boneWeights[i];
if (Math.Abs(bw.weight0) > 0.00001)
{
localPt = bindposes[bw.boneIndex0].MultiplyPoint3x4(vertices[i]);
newVert[i] += bones[bw.boneIndex0].transform.localToWorldMatrix.MultiplyPoint3x4(localPt) * bw.weight0;
}
if (Math.Abs(bw.weight1) > 0.00001)
{
localPt = bindposes[bw.boneIndex1].MultiplyPoint3x4(vertices[i]);
newVert[i] += bones[bw.boneIndex1].transform.localToWorldMatrix.MultiplyPoint3x4(localPt) * bw.weight1;
}
if (Math.Abs(bw.weight2) > 0.00001)
{
localPt = bindposes[bw.boneIndex2].MultiplyPoint3x4(vertices[i]);
newVert[i] += bones[bw.boneIndex2].transform.localToWorldMatrix.MultiplyPoint3x4(localPt) * bw.weight2;
}
if (Math.Abs(bw.weight3) > 0.00001)
{
localPt = bindposes[bw.boneIndex3].MultiplyPoint3x4(vertices[i]);
newVert[i] += bones[bw.boneIndex3].transform.localToWorldMatrix.MultiplyPoint3x4(localPt) * bw.weight3;
}
}
mesh.vertices = newVert;
}
@Golesy I don’t think his code were made to be placed anywhere. To me, it seems like it’s either to be used as a reference or mostly replace the Start of @Darkrobyn. And, to my brief experiments, just copying the vertices to a local var didn’t bring any performance improvement.
I’m no expert with messing with meshes, but I do know one thing or two and I agree with him that the first script contains many performance issues. So, this is my updated complete version, which is working as fine as I could make it work (in a few moments) on Unity 4:
using UnityEngine;
using System.Collections;
public class SkinnedCollisionHelper : MonoBehaviour
{
class CVertexWeight
{
public int index;
public Vector3 localPosition;
public float weight;
public CVertexWeight(int i, Vector3 p, float w)
{
index = i;
localPosition = p;
weight = w;
}
}
class CWeightList
{
public Transform transform;
public ArrayList weights;
public CWeightList()
{
weights = new ArrayList();
}
}
public bool forceUpdate;
public bool updateOncePerFrame = true;
private CWeightList[] nodeWeights; // one per node
private SkinnedMeshRenderer skinnedMeshRenderer;
private MeshCollider meshCollider;
/// <summary>
/// This basically translates the information about the skinned mesh into
/// data that we can internally use to quickly update the collision mesh.
/// </summary>
void Start()
{
skinnedMeshRenderer = GetComponent<SkinnedMeshRenderer>();
meshCollider = GetComponent<MeshCollider>();
if (meshCollider != null skinnedMeshRenderer != null)
{
// Cache used values rather than accessing straight from the mesh on the loop below
Vector3[] cachedVertices = skinnedMeshRenderer.sharedMesh.vertices;
Matrix4x4[] cachedBindposes = skinnedMeshRenderer.sharedMesh.bindposes;
BoneWeight[] cachedBoneWeights = skinnedMeshRenderer.sharedMesh.boneWeights;
// Make a CWeightList for each bone in the skinned mesh
nodeWeights = new CWeightList[skinnedMeshRenderer.bones.Length];
for ( int i = 0 ; i < skinnedMeshRenderer.bones.Length ; i++ )
{
nodeWeights[i] = new CWeightList();
nodeWeights[i].transform = skinnedMeshRenderer.bones[i];
}
// Create a bone weight list for each bone, ready for quick calculation during an update...
for ( int i = 0 ; i < cachedVertices.Length ; i++ )
{
BoneWeight bw = cachedBoneWeights[i];
if (bw.weight0 != 0.0f)
{
Vector3 localPt = cachedBindposes[bw.boneIndex0].MultiplyPoint3x4( cachedVertices[i] );
nodeWeights[bw.boneIndex0].weights.Add( new CVertexWeight( i, localPt, bw.weight0 ) );
}
if (bw.weight1 != 0.0f)
{
Vector3 localPt = cachedBindposes[bw.boneIndex1].MultiplyPoint3x4( cachedVertices[i] );
nodeWeights[bw.boneIndex1].weights.Add( new CVertexWeight( i, localPt, bw.weight1 ) );
}
if (bw.weight2 != 0.0f)
{
Vector3 localPt = cachedBindposes[bw.boneIndex2].MultiplyPoint3x4( cachedVertices[i] );
nodeWeights[bw.boneIndex2].weights.Add( new CVertexWeight( i, localPt, bw.weight2 ) );
}
if (bw.weight3 != 0.0f)
{
Vector3 localPt = cachedBindposes[bw.boneIndex3].MultiplyPoint3x4( cachedVertices[i] );
nodeWeights[bw.boneIndex3].weights.Add( new CVertexWeight( i, localPt, bw.weight3 ) );
}
}
UpdateCollisionMesh();
}
else
{
Debug.LogError("[SkinnedCollisionHelper] "+ gameObject.name +" is missing SkinnedMeshRenderer or MeshCollider!");
}
}
/// <summary>
/// Manually recalculates the collision mesh of the skinned mesh on this object.
/// </summary>
public void UpdateCollisionMesh()
{
Mesh mesh = new Mesh();
Vector3[] newVert = new Vector3[skinnedMeshRenderer.sharedMesh.vertices.Length];
// Now get the local positions of all weighted indices...
foreach ( CWeightList wList in nodeWeights )
{
foreach ( CVertexWeight vw in wList.weights )
{
newVert[vw.index] += wList.transform.localToWorldMatrix.MultiplyPoint3x4( vw.localPosition ) * vw.weight;
}
}
// Now convert each point into local coordinates of this object.
for ( int i = 0 ; i < newVert.Length ; i++ )
{
newVert[i] = transform.InverseTransformPoint( newVert[i] );
}
// Update the mesh ( collider) with the updated vertices
mesh.vertices = newVert;
mesh.uv = skinnedMeshRenderer.sharedMesh.uv; // is this even needed here?
mesh.triangles = skinnedMeshRenderer.sharedMesh.triangles;
mesh.RecalculateBounds();
mesh.MarkDynamic(); // says it should improve performance, but I couldn't see it happening
meshCollider.sharedMesh = mesh;
}
/// <summary>
/// If the 'forceUpdate' flag is set, updates the collision mesh for the skinned mesh on this object
/// </summary>
void Update()
{
if (forceUpdate)
{
if (updateOncePerFrame) forceUpdate = false;
UpdateCollisionMesh();
}
}
}
Take a flat pink object, place it backwards and forwards along the intended ray line, moving it backwards and forwrads by distances devided by 2, and compare the pixel color of that position every time it moves, and slowly narrow in on the distance that makes the screen pixel change from pink to not pink. then you have the precise position of the object, then you can find the nearest bounding box and the nearest vertex to that position. it could be abit difficult to implement with FOV?