Auto-generates depth textures for skyboxes on sentis

Hello, I recently saw this project that transforms an equirectangular panorama to a mesh, I wanted to know if there is any project with sentis with these characteristics or if it would be very difficult to port genesis to sentis.

the project use MiDaS v2 integration.

1 Like

This does the rgb2depth. They provide pre-trained checkpoints that works pretty well as well as a script to export to onnx

Hey, I’m the creator of the linked repo and I was in fact able to update the MiDaS inference for Sentis.

The neat thing is, that with Sentis we can use new MiDaS models now, even the v3.1 dpt_beit_large_512 MiDaS model from here (although with 1.4GB it might not be suitable for all use cases, some more detail here)

Anyway, the branch with the updated inference code is here, but still very much unpolished/experimental: GitHub - julienkay/genesis at sentis-update

To be clear, even though the larger model produces a lot better depth maps with higher resolution, for this specific use case of turning these high-res 360 images into meshes I found that it doesn’t immediately improve the result a lot. Some more work would be needed to actually make good use of them.

2 Likes