I’m working with the Jetset iOS app (a virtual production tool) which produces video footage side by side with motion tracking data which animates a virtual camera to match the movements of the iOS device which recorded the original footage.
I’m currently exporting the Jetset camera-rig containing associated Jetset motion tracking animations from Blender to Unity and importing the recorded Jetset footage into Unity manually. The Jetset motion track and video are synced to Blender geometry in rock-solid fashion, but the exact same motion track animation and footage lacks the same precision sync with Unity geometry.
Within the limits of my current level of understanding, I do not comprehend why the Jetset motion track animation and footage which operates correctly within Blender loses precision sync with scene geometry inside of Unity.
VIDEO EXAMPLES:
Jetset In Blender: https://youtu.be/SU9XnMw2UXc
Blender’s Jetset Export in Unity: https://youtu.be/6FRG5ZRW5iQ
In an earlier post, I raised an inquiry with regard to Unity Engine and received helpful attention from Eliot ( thread viewable here: https://forums.lightcraft.pro/t/autoshot-blender-unity-video-plane-is-180-behind-camera-field-of-view/490)
Since then, I’ve applied a workflow based upon an Ian Hubert Blender Motion Tracking Tutorial ( seen here https://youtu.be/RxD6H3ri8RI?si=noG3XJ5iVQop8ymQ&t=85 ) in an attempt to bridge the gap between Jetset and Unity. This approach has advanced me 95% of the way to successfully implementing Jetset Motion Tracking data inside of Unity.
The workflow I’m experimenting with is outlined below in hopes that anyone may see something I am missing:
CURRENT WORKFLOW:
- Film using Jetset
- Use Autoshot to import Jetset takes into Blender
- Within Blender, delete default camera collection so Blender scene contains only Jetset assets.
- Export Blender’s Scene Collection as FBX including animations (for use within Unity).
- Switch over to Unity
- Import the resulting Collection + animations FBX into a Unity Project.
- Imported FBX contains 4 animations formatted as follows:
- F1B4F343DD_zl_F1B4F343DD_zlAction
- F1B4F343DD_zl_zl_ip_F1B4F343DD_zlAction
- zl_ip_F1B4F343DD_zl_F1B4F343DD_zlAction
- zl_ip_F1B4F343DD_zl_zl_ip_F1B4F343DD_zlAction
- Insert Collection FBX into a new Unity Scene.
- By default, Unity will position the footage plane 180 degrees behind the camera, this is fixed by selecting the Jetset footage plane and rotating it 180 degrees on the Y axis.
- Adopting the Ian Hubert approach, scaling of the footage plane is currently being applied manually using the workflow of parenting the footage plane inside of an empty null with its handle for manipulation being set to Pivot (as opposed to center)
- The empty null can now be used to successfully control the positioning of the footage plane while maintaining the correct scale of the footage plane to the camera’s field of view. (I do know that Jetset contains an animation for this scaling, but I’ve not worked through this aspect yet - so right now it’s a manual process).
- The Collection FBX is animated using Unity Timeline (at the root node ) using the first included Jetset animation ( formatted as F1B4F343DD_zl_F1B4F343DD_zlAction)
- The camera now successfully animates using the motion data captured using Jetset iOS app - but the animation (or video?) is not in precision sync with the surrounding scene geometry.
Is anyone able to contribute insight into why this loss of sync is not translating between Blender and Unity?
NOTES:
- I’ve ensured that Unity Timeline and Video Player are set to the correct frame rate.