Raycasting onto a navmesh game object

Hello fellow Marsians,

I want to put a 2D game on top of an image marker and am trying to get a raycast onto my navmesh with this simple code from SingleSaplingGames:

using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.AI;

public class PlayerMovement : MonoBehaviour
{
    NavMeshAgent agent;
    // Start is called before the first frame update
    void Start()
    {
        agent = GetComponent<NavMeshAgent> ();
    }

    // Update is called once per frame
    void Update()
    {
        if(Input.GetMouseButtonDown (0))
        {
            RaycastHit hit;
            Ray ray = Camera.main.ScreenPointToRay (Input.mousePosition);
            if(Physics.Raycast(ray, out hit, Mathf.Infinity))
            {
                agent.SetDestination(hit.point);
            }

        }
    }
}

Unfortunatly it doesn’t work.

I know MARS usually looks for planes, but in my case I don’t want MARS to detect any planes, just an image marker and then the game object (navmesh) I spawn on that marker.

Any suggestion. Thank’s a bunch.

Hello @macgregger

So there are a couple of things to check on here:

If you dont want MARS to check for planes and just an image marker; make sure your proxy doesnt have a plane condition. The easiest way to create an image marker proxy is by opening the MARS Panel (Window → MARS → MARS Panel) as shown in the attached screenshot:

Now, with regards to making it work with a normal raycast, I would suggest you create a “Floor” game object as a children of your proxy that has just a Collider (no mesh renderer nor mesh filter), so you will be able to cast a Raycast to something (The floor gameobject) and do your interactions from it

Hope it helps! :slight_smile:

@jmunozarUTech Thank you for your reply.

I’ve set the image marker up like you mentioned, but I have two issues:

  1. Testing in Simulation View does no longer work for some reason. I’ve set up a synthetic image marker. But my content only displays when I’m NOT in simulation mode. As soon as I press play the content disappears. I suspect it might have something to do with the world scale, because it worked before I adjusted it. When i build the App, the content displays like it should.

Would a “normal” raycast even work in the MARS Simulation?


  1. My player Cube keeps disappearing when I test the scene on device, regardless if I render the ground or not. I suspect it has something to do with the NavMeshAgent Component, since the blue Cube appears like it should. (see image below) Do you have any ideas on how to fix this?

hello @macgregger ,

I am discussing with the team on what could that be, for sure this is not normal.

To address problem number 1; could you move backward / forward to see if because of the MARS scale the content shows up?.

With regards to problem number 2:
Seems that the underlining logic for MARS is working and I am also inclined to think that the nav mesh and agents might be the problem.

To figure out what is failing I would try to visualize the collider to see how big / small gets according to the modified scale and also I would visualize / log as much as possible what each agent is doing.

Finally I would suggest to try to run a separate scene that only has agents; check that they are working as you expect and on the other hand have another MARS scene only and check that it works also as you expect so you can figure out which part if failing.

Hello @jmunozarUTech ,

  1. Problem 1 seems solved! I moved into the synthetic image marker in simulation view and then the content showed up and also stayed there after I moved back.

  2. Is there a way to simulate my touch input in Simulation View? In other words: Could I test the movement of my playerCube by clicking on the groundPlane in Simulation View?

  3. Is Play Mode possible with MARS? And if yes, how do I set it up in sync with the Simulation View?

  4. Problem 2: I visualized the Collider in Simulation View (see image below) and it seems fine. Also the player cube is displayed which is also fine. But on device the playerCube disappears.

  1. I tested the agents in a seperate scene without a MARS Session and they work fine.

  2. I don’t know how to write the code to log the agents activities. So I would need some time to look into that.

Hello @macgregger ,

Our answers below:

  1. Sweet!, glad it worked out!

  2. Unfortunately no, BUT, you could just press play and test for input like you would on any Unity app in the game view.

  3. Definitely possible, but I dont follow what you mean by “set it up in sync with the Simulation View”, could you elaborate more?

  4. there could be many reasons why this happens; near clip plane, scale, etc. checkout my answer on the other thread that tries to clarify what might be: Scaling issues with marker based game

  5. Then the issue might be in mars OR in the scene scale.

  6. To visualize what is happening you might want to look into gizmos: Unity - Scripting API: Gizmos

And an intro to them in Creating Custom Gizmos for Development - 2019.2 - Unity Learn but there are many many more tutorials online

hope it helps out

Hello @jmunozarUTech Exciting news! I found out what is causing the problems.

As shown in this picture below the problem seems to be that the navmesh is not transforming along with the cube, but stays at the original position it was backed in.

The red playercube is placed correctly on the navmesh, but I didn’t see it because it was on the floor.

Raycasting also seems to work as the red line indicates.

I will now try to match the transform of the synthetic image marker with the transform of the image marker in the scene view. Then the navmesh should be in the right position and everything should work.

However this is not a longterm fix. Ultimately I need the NavMesh to transform with the groundplane, since I can’t control the transform of the real image marker once the scene is build to device. Any suggestions?

Thanks for the replies. Huge help!

1 Like

hello @macgregger ,

Nice!, progress is always good!.

I am not 100% familiar with Unity’s nav-mesh system and haven’t been following lately but I remember that a couple of years ago it had a limitation that it only ran on the X Z plane (but this might have changed recently!).

Being said that, you might want to look if Unity’s navmesh system can change processing planes dynamically and adapt to the orientation of the image, else you might want to explore other alternatives OR as a last resort consider writing your own.

Another approach that you could try (If the nav-mesh system can be set to a specific working plane) is reset the nav mesh every time you detect a change in plane orientation (like you place the image on the floor/ on a wall, etc), but this approach might not work on tilted image markers (not aligned to an axis).

Most likely the best approach would be to look into somehow set the path finding to a plane regardless of the plane’s rotation. But that will open another can of worms if you want to extend the path finding to the surrounding walls / floor.

Have you tried installing the MARS Nav Mesh Support in the content manager?
It provides an example scene where the navmesh does change dynamically, along with a navmeshagent support script designed to deal with the navmesh changing while an object is on it

1 Like

@unity_andrewm No I have not, since I don’t think it’s the right navmesh for me. I need a navmesh that is parented to the game object I want to display on the marker, since there is always some movement and recalculation of the displayed game object once it is placed on the marker. That is ok, as long as the navmesh and all other game objects move in unison and the relation towards them stays the same.

I have experimented with the A* Pathfinding Plugin and implemented a node-based pathfinding which works great for me, since it is very accurate at a very small scale (Poster Size 50cm x 70cm)

However the initial problem still stays the same. The navmesh is created at the start of the application and stays in world space at (0,0,0) and does not translate once the parent gameobject is placed on an image marker.

I can however test the game in Simulation view if I place the simulated Marker at (0,0,0) aswell and then it works.

There is a possible solution. I could try to recalculate the Navmesh manually once the gameobject is placed on the marker or once I press a Play-Button.

A* Pathfinfing provides this funcion:

// Recalculate all graphs
AstarPath.active.Scan();

Where do you think I should put this script? Somewhere inside the existing MARS scripts after MARS has successfully identified the marker and placed the object? Or someplace else?

I don’t know any C# unfortunatly and I don’t feel comfortable doctoring around in the MARS scripts. So a detailed solution would be most welcome. Thank you.

@jmunozarUTech and @unity_andrewm did you get a chance to look at my last post?

hello @macgregger ,

Sorry we where on a holiday yesterday hence the delay :smile:.

What you mentioned makes sense, I would first match the image marker and then I would recalcualte the NavMesh,

You could easily trigger the mesh recalculation after the image is matched just by adding a MatchAction to your proxy and then create an event that triggers a method that you write on your own that will get called when the condition on the proxy is met!

Checkout: Actions reference guide | Unity MARS | 1.4.1

@jmunozarUTech no worries. You’ve been a huge help already.

So in here would go the custom script, am I right?

Like I said I don’t know how to write something like that. My C# skills are non existent. I’m trying to call this function of the A* Pathfinder.

// Recalculate all graphs
AstarPath.active.Scan();

Graph Updates during Runtime - A* Pathfinding Project (arongranberg.com)

It’s probably super easy for somebody who knows his C#. Do you know where I could ask for somebody to write this for me?

Thanks again.

No worries!, I was in your situation long time ago and I really appreciated the help I got from the forums! :).

Its quite simple, just create a c# script like this:

using UnityEngine;

public class RecalculateGraph : MonoBehaviour {

    public void RecalculateThePathfindingGraph()  {
        Debug.Log("Graph is being recalculated!");
        // Recalculate all graphs
        AstarPath.active.Scan();
    }
}

And Add this script to any game object in the scene (It could even be a new game object). In my case I will add it to a new empty game object by just drag and dropping it. Then where you selected that red square add the reference to the created game object and Search for the function.

8224851--1074339--ezgif-3-38c334f8c5.gif

Note that on the MatchAction If you set it to runtime only, it will only be called when you enter play mode (Not the play button in the simulation view). If you notice, I left it in my case in the Editor and Runtime mode so it gets called when I the hit play button in the simulation view (not enter play mode in the unity editor).

Finally, when you hit play (either in editor or in simulation view) you should get at the very least a log in the unity console (Debug.Log) that says “Graph is being recalculated!”.

I dont know if Aron’s Pathfinding system (which is really good) will work in editor mode, so your best bet would be to keep getting into play mode so you can be sure that the function gets called correctly.

Hope this helps! :slight_smile:

Dear @jmunozarUTech I made a huge step forward thanks to you. Thank you kindly.

1 Like

Awesome!, glad you got unstuck!, looking forward to see this project when finished! :slight_smile: