A short and simple permissive license with conditions only requiring preservation of copyright and license notices. Licensed works, modifications, and larger works may be distributed under different terms and without source code.
(Yes)
P. S. messages below are related to earlier version
You need to blend [how many vertices you have] quaternions and apply resulting quaternions to vertex positions. Which is a hell lot of quaternions :). Doing this on CPU is a total performance killer (i got like 1-2 fps). The good part is that vertices do not depend on each other and can be calculated in parallel. So why not use GPU for this?
I dump vertex positions, bone indexes, bone weights and quaternions into textures, send them to GPU for calculation and retrieve the result. And its infinitely faster than just doing it on CPU.
I could probably use compute shaders but i don’t see any significant advantage and also i did not have time to properly learn them.
One thing that confuses me about compute shaders is that i have to manually select constant number of threads. Which means i could either use less threads than i have available ruining performance or i could request more threads than GPU can provide and my shader fails to run.
Also not every platform supports compute shaders while fragment shaders work everywhere (or almost everywhere).
The biggest problem with my approach (idk if it this problem can be solved using compute shaders) is that i get the result of my calculations as RenderTexture. To get my data back to CPU i convert it to Texture2D and call GetPixels()
It works fine but every GetPixels() call allocates a new array of Color and this forces regular GC.Collect( ). It is still bearable but could be easily prevented if GetPixels() would accept an already allocated buffer for it’s data. It turns out i am not the only one who stumbled upon this problem: https://feedback.unity3d.com/suggestions/garbage-free-texture2d-dot-getpixels
If somebody knows how to deal with this bottleneck i would appreciate your input.
Also the readback from the gpu causes a stall as it has to flush everything it’s doing, making this technique pretty much only useful for movie making, no bad thing!
i’m just speaking theoretically here as i didn’t test it properly yet, but i suppose in general the number of skinned vertices you have would be a lot smaller than the number of pixels on screen and thus shouldn’t be much of a problem
if you know a better way to do this or which potential problems could arise please do tell
also if this script turns out too heavy to replace SkinnedMeshRemderer you could probably use it at least for main character or dynamically enable it based on distance as a kind of LOD
I thought CPU performance would be within an order of magnitude of linear blend skinning, so on CPU no worse than say 3 times slower than doing LBS on the CPU. But it was just the use of a fragment shader that I was question more than wanting to use the GPU (and there’s so much I don’t know about Unity it is/was reasonable it’s my ignorance)
That sounds less than ideal… I thought generally you’d want to keep the data on GPU and render it without the CPU massaging the data further - I figured you might use a custom vertex shader to read the data from the texture, but then you’ve kind of implemented a poor man’s implementation of “transform feedback”/“stream out” (and naively running the transforms in a vertex shader without “transform feedback”/“stream out” as the material would make you pay to re-tranform for every pass)… I’m not sure if Unity exposes “transform feedback”/“stream out” but (or keeps such tricks all to itself)
Well, not all fragment shaders work everywhere, and whether it’s effecient everywhere is another matter, though I’m not certain either case necessarily applies here… but if you do want that kind of portability your textures would be POT dimensions and limited to 2048 pixels wide/high, and limited to 4 8-bit channels, right? (too lazy to double check when that ceased to apply)
Might be worth double checking, but I think compute shaders generally are able to spit out data in a form you can feed directly to be rendered, and I’d guess Unity would probably let you use this? (I think I saw a project using compute shaders for skinning on Github earlier, might be worth seeing how others have approached similar problems perhaps?)
EDIT: Sorry, took so long posting I missed the additional responces
EDIT2: Also as Hippocoder hints at, you really don’t want a “stall”, remember the GPU is usually rendering a frame or two ahead, I believe stall means in this case that the CPU must wait for the GPU to finish every thing it’s batched up to finish later, including but not just your transforms, so it can give you the results (see: https://blogs.msdn.microsoft.com/shawnhar/2008/04/14/stalling-the-pipeline/ ).
Indeed it is. However i struggle to find a better way to do it.
AFAIK you can add custom fields to vertex structure but the Mesh class will not support new fields which kinda defeats the purpose
That’s still better than what i currently have.
However to keep it efficient i would need to use separate UV coords for this. I don’t think using uv4 is that bad - i doubt people ever use that many. But i might be wrong
I’m going to look for info about vertex shaders now as i’m not sure how to make it yet. Is it possible to apply my “skinning vertex shader” while keeping whatever material was assigned to the mesh with it’s own shader? From what you said i suppose two vertex shaders would override each other but can i keep at least fragment shader? Otherwise you would have to manually add “vertex skinning” shader code to every shader you use. Or am i missing something?
Thanks for the link. I was not aware of this
Edit:
Meanwhile i started working on vertex shader and achieved some success. I was able to get rid of Texture2D and reading texture data from CPU completely. My vertex shader takes uses the second set of uv coodrinates and samples a texture to retrieve animated vertex coordinates.
However i discovered that while uv1 and uv2 have float precision, uv3 and uv4 are only half which may not be enough
Also i did not found a way to make my shader “cooperate” with object’s material so if you want to use your own shader you would have to copy the vertex function from my shader.
Also i noticed some worrying artifacts on my modes when using this shader. Im not sure they are related to my animation script but it is possible. Will check it when i finish writing code.
So in the end vertex shader did work. However not without problems
I get some weird artifacts along uv seams. No idea what could cause it.
It seems meshes that consist of multiple parts do not work properly. Will test is later.
My script only works in play mode but shader tries to adjust vertex positions all the time. Making my script work in edit mode is not a good idea (it needs SkinnedMeshRenderer for initial parameters and removes it in Start(). After removing it in edit mode it won’t come back. also many other things could get messed up in edit mode)
I think the best solution would be to provide alternative shader without gpu skinning and switch to skinning shader in Start()
Artifacts look to be cracks in the mesh due to vertices along the seam not having quite the same position (or possibly tessellation) as their neighbour on the other side. This could be an issue with the input mesh (eg. that was being masked by other factors such as precision issues) or it might indicate an issue with the skinning (generally you’d want the same vertex position and bones weights to give exactly the same result) …but that’s just a guess.
Oh I thought you might’ve found a way to get Unity to preform transform feedback and were doing the skinning in a vertex shader… because you have UV coord and the vertex shader is being used to read vertex data, I assume the cracks might be due to blending being performed on the texture samples - you’ll want to set the texture to use nearest neighbour sampling (instead of say bilinear sampling)
Anyway not sure this is the best approach either (or if there is a best approach to be found)… but if you stick with this one, perhaps you could trade the UV coord for “SV_VertexID” see: Unity - Manual: Shader semantics (you’ll still want to fix the sampler) though it will rule out some of the possible targets
Until joint angle deformers are supported in vanilla Unity as they are in ALL 3D packages out of the box - I believe the simplest approach for production is corrective morphs/blend shapes driven by joint angles.
I already tries passing the same skinned vertex positions through different means (apply through mesh.vertices and vertex colors) and it worked without any artifacts. So i believe this is not the source of trouble.
That fixed it! Thanks!
I was setting samplers so nearest neighbor everywhere but forgot it where it was the most important
So i am happy to say: it works.
No artifacts, no GPU stalling, no GC.Collect() issues. Just DQ skinning in all it’s glory.
Now it’s about making this usable. I will add a default shader with vertex skinning and a quick readme on how to adjust your own shader to work with the script. Also i will make an alternative shader that will be used in Edit mode so the model is visible
And yeah, i’ll try to use SV_VertexID to pass uv coords and look for other ways to do it.
As far as I know it was Ladislav Kavan’s papers that popularised dual quaternions for skinning (with approximation for slerp), he seems to have a paper advocating just using more bones with LBS as a solution to the candy wrapper issue https://www.cs.utah.edu/~ladislav/kavan09automatic/kavan09automatic.pdf
From what i read using this method requires some additional setup (training animation). Also AFAIK it can help with candy wrapper artifacts but it’s not the only situation where dual quaternions provide good result. Thighs, knees and elbows IMHO look much better with properly set up dual quaternion skinning than with linear skinning but it’s not a candy wrapper problem.
