Swap Sprite with another sprite with bones/skinning/weight

Unity Engine
,
1

Hello everyone

Im building a project were a user shall scan his/her drawing of a character with a webcamera and apply that drawing as a spawnable sprite with bones and animation in a 2d world. Im currently have created a prefab with a blank figure were I made bones, skinning and some basic animation, were I’m gonna use that prefab to populate the world. This prefab is gonna use the scanned drawing as a sprite.

Changing the prefab to have the scanned texture is a success, but the animation and weighting are lost. Now I want to try if I can get the prefabs bones and weight and apply to the scanned sprite. I searched the web for how to do this, and came up with this code:

 [SerializeField] private GameObject targetSpriteSkin;
    private Sprite spriteSkin;
    // Start is called before the first frame update
    void Start()
    {
        spriteSkin = targetSpriteSkin.GetComponent<SpriteRenderer>().sprite;
        SpriteRenderer spriteRenderer = gameObject.GetComponent<SpriteRenderer>();
        SpriteBone[] bones = spriteSkin.GetBones();
        NativeArray<Matrix4x4> bindPoses = spriteSkin.GetBindPoses();

        spriteRenderer.sprite = spriteSkin;

        spriteRenderer.sprite.SetBones(bones);
        spriteRenderer.sprite.SetBindPoses(bindPoses);
        spriteRenderer.sprite.SetVertexAttribute<Vector3>(UnityEngine.Rendering.VertexAttribute.Position, positions); // how do I retrieve this?
        spriteRenderer.sprite.SetVertexAttribute<BoneWeight>(UnityEngine.Rendering.VertexAttribute.BlendWeight, weights); // how do I retrieve this?
        spriteRenderer.sprite.SetVertexAttribute<Color32>(UnityEngine.Rendering.VertexAttribute.Color, colors); // how do I retrieve this?

    }

But I’m unsure how I can get positions, weights and color data for setting the last lines for SetVertexAttribute.

Other solution I tried was to use SpriteLibrary to swap sprites, but is hard to at runtime with a scanned texture from a webcamera.

My current solution is I have a 3D mesh that acts as a 2D plannar character. When user scannes his/her drawing, the drawing will apply as a texture for the 3D figure. But it would be good if I use a 2D character since its a 2d project.

Much help is appreciated.

1 Like
2

After having the exact problem and also finding this code snipped of a different user online, I eventually found out how to retrieve the data. I only needed the weights, but I guess you can easily replace it for position and color, too.

using UnityEngine.Rendering; // Add this

[SerializeField] private GameObject targetSpriteSkin;
private Sprite spriteSkin;

    void Start()
    {
        spriteSkin = targetSpriteSkin.GetComponent<SpriteRenderer>().sprite;
        SpriteRenderer spriteRenderer = gameObject.GetComponent<SpriteRenderer>();
        SpriteBone[] bones = spriteSkin.GetBones();
        NativeArray<Matrix4x4> bindPoses = spriteSkin.GetBindPoses();

        spriteRenderer.sprite = spriteSkin;

        spriteRenderer.sprite.SetBones(bones);
        spriteRenderer.sprite.SetBindPoses(bindPoses); // No changes until here
        

        // CHANGES:
        // First we get the the bone weights and convert the output to an array.
        BoneWeight[] boneSource = spriteSkin.GetVertexAttribute<BoneWeight>(VertexAttribute.BlendWeight).ToArray();
        // Now create a new NativeArray with the length of the bone weight array. Note: Array and NativeArray are not the same.
        NativeArray<BoneWeight> weights = new NativeArray<BoneWeight>(boneSource.Length, Allocator.TempJob);
        // Now copy the data from boneSource to the new empty NativeArray weights.
        weights.CopyFrom(boneSource);
        // Finally you can apply the VertexAttribute
        spriteRenderer.sprite.SetVertexAttribute<BoneWeight>(VertexAttribute.BlendWeight, weights);

    }
3

Edit, after some more testing, my previous version was not working all the time. I found this post, which fixed it for me! Look at the solution of “Ted_Wikman” in the “InjectBoneWeights()” function of his code.

4

Thank you Daniel

I solved it using flat 3d models and applied the scanned images as textures to their materials.

But good job!