collision and a 360 photo sphere

I am stumped on a little problem and i’m certain there is an easy solution.

I have a 360 panorama photo from a couple of locations I want to use in a game. I would like a way to have collision on the panorama photo so i can interact with it. (think like throwing a ball against a wall, or onto a undulating terrain)

In the game Real Flight, the rc planes fly inside a 360 panorama photo but yet they can land on the runway and crash into nearby buildings etc. basically exactly what I’d like to do

This gives an example of what i’m talking about

How would you go about recreating this?

I can use the 360 photo as a skybox and it looks great. I can’t work out a way to easily reproduce the environment in 3d so that it aligns with the photo, so that I can turn it into collision. If I was to trial and error my way through recreating these environments in 3d, it will take a crazy amount of time.

In Real Flight the planes also cast a shadow onto the runway which has me completely stumped, because if they were just using invisible collision over a skybox then I would imagine that it wouldn’t accept shadows.

So I imagine I would need to 3d model the whole world so that I can get the shadows and collision, but then I have the issue of mapping the skybox onto the geometry so that it all looks cohesive. I’ve played around with adding some boxes and planes into a scene and spherical projecting my texture onto them and its a complete mess, nothing like I was expecting

Do you know how it’s done in Real Flight? Do you have any idea on how I could approach this issue?

You can make a material which draws nothing but shadows. You make box colliders for the ground, and quads with these shadow-catching materials on them.

The really important thing you cannot violate, if you use a photosphere as a background, is the camera itself cannot move at all. It can rotate to follow the action, but it cannot move, or the illusion of the photosphere perspective is broken.

I don’t plan on having my camera move, just rotate.
How can I make a material that draws nothing but shadows?

so I can make a box collider for the ground, but what if i want objects in the area? or even if the ground is undulating? how would be best to go about that? just trial and error?

you could try creating Depth from image and then create collider or mesh from that,
but probably not very accurate…

or make custom tools that make it easier to place colliders over image/360 image.

also can use 360 shader that flattens the bottom, so then it looks better
(this is three.js, but shader can be converted to unity)
https://threejs.org/examples/?q=GroundProjected#webgl_materials_envmaps_groundprojected

wow that looks really cool, how would you convert this shader to work in Unity?

that shader source is here,
https://github.com/mrdoob/three.js/blob/4bbcaacc4f41bc1b368fdd318ad432499a6506c2/examples/js/objects/GroundProjectedEnv.js

so the void main() part is same as frag shader in unity,
just need to rename same of those types, vec3 > float3 and so on.