Isometric tilemap with elevation, slope and shadows and mouse selection

Hello, I just wanted to share my experience as a beginner in Unity and the way I’m using isometric Tilemap with my requirements. Here is the summary of my attempts and struggles.

Prior to this I read in some forum, developing an isometric game as a beginner is difficult, and is even a trap. It could be easier to do it in 3D directly. Beginner Question: Isometric TileMaps - Necessary? Recommended? Why?

So I hope it will help other beginners that are trying to make a 2.5D game. I’m also really interested in feedback from more advanced users of Unity. I’m sure my solution could be greatly improved, please share your knowledge if I can improve it.

My main inspiration is Final Fantasy Tactics, and particularly the “Advance” subseries.

9830532--1413804--upload_2024-5-12_23-45-46.png

Based on this inspiration, my objectives are

  • an isometric map
  • elevation : some tiles on top of others to make mountains or ditches
  • not only cubic tiles : slopes
  • tile/tilemap shadows
  • being able to choose a tile with the mouse or the keyboard

1st attempt : using only one Tilemap with tiles
The basic way to use Tilemap. See documentation Unity - Manual: Creating an Isometric Tilemap

  • Unity 2022 LTS (2022.3)

  • Project type : 2D Core project (Unity’s built-in renderer)

  • Hierarchy

  • Camera (Projection = Orthographic)

  • Grid (Cell Layout = Isometric Z as Y)

  • Tilemap (Mode = Individual)

  • Light 2D (to generate shadows)

Summary

  • Easy to set up to experiment

  • But didn’t find a way to use the mouse to correctly choosing a tile with elevation (z > 0)

  • the tile position returned was always the ground tile (z = 0) even on elevated tiles

  • if I wanted to find the highest tile from this ground tile, I just had to look in my tile list and return the corresponding highest tile (e.g., ground tile = (4,5,0) → if highest z = 2 for this tile, return (4,5,2) ). But this method doesn’t work, because of the isometric offset on z : higher tiles are displayed higher on the screen. The higher the tile to be selected (z), the greater the offset. So even if the mouse is on the highest tile, the selected tile position will be wrong. I could have calculated the offset, but it was for me too complicated to understand how.

Vector3 mousePosition = Camera.main.ScreenToWorldPoint(Input.mousePosition);
Vector3Int tileGroundPosition = tilemap.WorldToCell(mousePosition);
Vector3Int selectedTilePosition = HighestTileFrom(tileGroundPosition); // Wrong. Offset when z > 0
  • Slopes were not selectable at the bottom of the slope with the mouse either : their physical shape by using Tilemap tiles is cubic like other tiles, instead of having the slope shape (see 3rd attempt for the resolution)
  • But no Tile shadow due to built-in renderer Unity project type : universal render pipeline is necessary (see 2nd attempt for the resolution)

9830532--1413819--upload_2024-5-12_23-53-46.gif

2nd attempt : several Tilemap with tiles on URP
After my first attempt, I needed to find another solution to fix at least two major problems

  • shadows on tilemap
  • being able to select tiles with mouse
  • (handling slope also, but with the other problems, I was ready to give up this objective if needed)

I saw there is a way to display shadows for Tilemap by using the universal render pipeline, instead of the Unity’s built-in : Create shadows with Shadow Caster 2D | Universal RP | 17.0.3. You can simulate shadow with a 2D Light, for each game object (Tilemap in my case) with a Shadow Caster 2D.

For the tile selector with mouse, as we saw there is an offset for z > 0 tilemaps in my 1st try, it would mean, I needed to use several Tilemap instead of one, to avoid having the offset.

  • Project type : Universal 2D Core (URP) (the documentation to migrate from a built-in render pipeline Installing the Universal Render Pipeline into an existing Project | Universal RP | 7.1.8)

  • Hierarchy

  • Camera (Projection = Orthographic)

  • Grid (Cell Layout = Isometric Z as Y), and as many Tilemap as need for each layer of elevation

  • Tilemap 2

  • Tilemap 1

  • Tilemap 0

  • Global Light 2D (necessary to avoid having all sprites black)

  • Light 2D (to generate shadows)

9830532--1413810--upload_2024-5-12_23-49-54.gif

Summary

  • By adding a Shadow Caster 2D for each Tilemap, I have shadows ! It was not perfect, but it works well for many cases. But it requires shaping the shadow manually (”Edit shape” in the Shadow Caster 2D component) for each Tilemap. Too tedious.

  • I updated my code to handle several tilemaps : I loop on every tilemap from the higher one, to the lower one. I return the tile found on the tilemap I’m checking, on my mouse position with a zOffset to simulate my mouse hovering each tilemaps.

for (int i = _tilemaps.Count-1; i >= 0; i--)
{
   Tilemap tilemap = _tilemaps;
   Vector3Int zOffset = new Vector3Int(0, 0, i-1);
   Vector3Int selectedTilePosition = tilemap.WorldToCell(mousePosition + zOffset);
   if (tilemap.HasTile(expectedTilePos))
   {
       newCell = expectedTilePos;
       newTilemap = tilemap;
       break;
   }
}
  • I still had a problem for selecting slopes : WorldToCell considers the tile as cubic, and is not aware of the custom shape of a slope. I tried some variants by using Tilemap Collider 2D on each Tilemap, adding a custom physic shape for slope tiles, playing with the sprite pivot center, and using ClosestPoint , but I was not able to make it work.
// Does not work
Collider2D[] hitColliders = Physics2D.OverlapPointAll(mousePosition);
...
for (int i = hitColliders.Count - 1; i >= 0; i--)
{
   ...
   Vector3 hitPoint = hitColliders.ClosestPoint(mousePosition);
   Vector3Int cellPos = tilemap.WorldToCell(hitPoint + zOffset);
   ...
}

3rd and last attempt : use GameObject as tiles
My 2nd attempt, showed me that shaping shadow casters was too tedious, and selecting slope with a mouse was still a problem.

To fix my slope detection, I use the same hierarchy and components, but this time instead of using basic Tile in Tilemap, I use GameObject as Tiles, with a new component CustomTile. And to correctly collide with a slope, I attach Polygon Collider 2D component to each tile prefab. This Collider 2D will take into account my custom shape from the Sprite Editor.

Now I can paint my tile prefabs with the GameObject brush from the Tile Palette, instead of using the default brush. To detect the collision between my mouse and each tile, I can use Physics2D.OverlapPointAll

Vector2 mousePosition2D = Camera.main.ScreenToWorldPoint(Input.mousePosition);
List<Collider2D> hitColliders = Physics2D.OverlapPointAll(mousePosition2D).Reverse().ToList(); // From top to bottom
if (hitColliders.Count > 0)
{
   CustomTile newSelectedTile = hitColliders[0].GetComponent<CustomTile>(); // Only the highest tile, others are hidden
   if (newSelectedTile != null && newSelectedTile != SelectedCustomTile)
   {
       SelectedCustomTile = newSelectedTile;
   }
}

9830532--1413822--upload_2024-5-12_23-55-31.gif

But with this method, we can’t use WorldToCell anymore to get my integer tiles position, as tilemaps are only used to lay out my custom tiles, and are not aware of the cartesian position of our custom tiles. The only information I have is the GameObject transform position in my scene. It’s a problem if I want to set the player on a specific tile easily, or to make operations with decimal numbers with an isometric distribution.


To transform isometric position to cartesian position, I do a linear transformation based on this matrix, that corresponding to my tile width and height (1 unit x 0.5 unit, or 32x16px sprites)

public class CustomTile : MonoBehaviour {
    public Vector3Int cartesianPosition;
    ...
}   

public class MapManager : MonoBehaviour {
    ...
  private void AssignTilePositions()
  {
      foreach (CustomTile tile in _tiles)
      {
          Vector3 position = tile.transform.position;

          // {{1,2,-0.5},{-1,2,-0.5},{0,0,1}}
          // CeilToInt to avoid the -0.5 offset on x and y due to the grid
          int cartesianX = (int)Mathf.CeilToInt(1f * position.x + 2f * position.y - 0.5f * position.z);
          int cartesianY = (int)Mathf.CeilToInt(-1f * position.x + 2f * position.y - 0.5f * position.z);
          int cartesianZ = (int)Mathf.CeilToInt(position.z);  // Z remains unchanged

          tile.cartesianPosition = new Vector3Int(cartesianX, cartesianY, cartesianZ);
      }
   }
}

For the shadow, we saw it’s useful to distribute tiles on several layers (for each z), but shadow caster shape need to be shaped manually. To do it automatically, I used the ShadowCaster2DExtensions from Alex Villalba Script for generating ShadowCaster2Ds for Tilemaps . This required to

  • set “Used by composite” to true on the Polygon Collider 2D component for each tile
  • add a temporary Composite Collider 2D on each Tilemap
  • generate the Shadow Caster 2D in the new Unity tools with the extension
  • and remove the Composite Collider 2D (+ Rigidbody 2D) from the tilemap, otherwise the mouse collider on each Tilemap will not work anymore

As MelvMay said in the same post, in Unity 2023 there is an official solution to reproduce this solution ( Script for generating ShadowCaster2Ds for Tilemaps page-2#post-8707710). I was not able to use it because I’m usign the 2022 LTS for the moment.

It means, now I have

  • an isometric map :white_check_mark:
  • elevation :white_check_mark:
  • slopes :white_check_mark:
  • tilemap shadows :white_check_mark:
  • mouse/keyboard tile selection :white_check_mark:

Voilà, I hope it will help a lot of beginners to understand the basics of using Tilemap and Unity, and that I will have the pleasure to play great games inspired by Final Fantasy Tactics Advance in the following years.

Here is the github repo for the final solution GitHub - Tiboh/unity-isometric: An example how I used Unity to have an isometric scene, with shadows, slopes, elevation and mouse control

Tips and points of attention
To help the more beginners that struggle with the Unity Editor UI, here is the little things I struggled to that can help

  • Great tutorial to begin with and understand the basics https://blog.unity.com/engine-platform/isometric-2d-environments-with-tilemap

  • Use URP when you create your project instead of the built-in if you want to use Shadow Caster 2D, but look at the limitation of the URP compared to built-in to ensure your project fits

  • Concerning isometric tilemap usage

  • set your camera to “Orthographic” and not “Perspective”

  • when creating your grid, set your cell size correctly based on the width and height of your tiles sprites. It’s usually (1, 0.5, 1) meaning the width (1) is twice the size of the height. It’s the case if you have 32x16 pixels tiles like in my examples. Your “Cell Layout” should be “Isometric Z As Y”

  • set your transparency sort axis based on your tile width/height. It’s important if you have several layers. If you are using URP, you can set it in the Renderer 2D Data file (Assets/Settings folder). The value is usually (0,1,-0.26). Set the “Sort Axis” of your Tile Palette to the same values. If you set it correctly, your tile should display correctly, without needed to play with the “Sorting Layer” and “Order in Layer” parameters in your tile Sprite Renderer component. In my case they are both set to “Default” and “0”

  • in your tile palette

  • if you want to use the GameObject Brush, you must install the com.unity.2d.tilemap.extras package in the Unity Package Manager

  • if you want to set higher tiles, you can set the Z position with your keyboard + and - keys

  • For sprites

  • when you select your sprite file, in the inspector tab

  • if you are using Aseprite : do not forget to set your import mode to “Sprite Sheet”, it’s by default “Animated Sprite”

  • set the pixels per unit to your tile width (32 in my case) : it will avoid playing in the editor with ridiculous small numbers in the Unity Editor to set game object positions, or text mesh pro font size

  • In the sprite editor

  • do not forget to “Apply” your change if you change something in this editor

  • for my need, I experimented that letting the default pivot point to center (0.5,0.5) is better to avoid playing with the collider offset in tile game objects

  • do not forget you can use the Sprite Editor dropdown to set the custom physics shape to each custom tiles (slope for example). If you change these values, and you don’t see any changes in the collider of the corresponding tile, try to “Reset” the collider by clicking ⋮ first

  • If you are using URP for shadows

  • don’t forget to add a global light 2D, otherwise your scene will be black

  • in your light 2D component, enable shadows by checking the “Strength” checkbox and add a value > 0, and in your Shadow Caster 2D components, enable “Casts Shadows”

7 Likes

Wow, this is awesome sauce. It feels like it should be a GamaSutra or Medium post. Lots of good questions answered in here.

1 Like

Some things I would like to add:

  • [quote=“thibaulthu, post:1, topic: 947248, username:thibaulthu”]

    [/quote]
    In this scene you seem to have shadow casters that cast shadows into different layers, this will reduce your performance because you have to calculate multiple shadow textures. Source and more info: [Tips] on How to improve 2D lights performance
  • [quote=“thibaulthu, post:1, topic: 947248, username:thibaulthu”]
    I use GameObject as Tiles
    [/quote]
    If all you do is spawning a gameobject and not using any other tile features at all, you are better off just instantiating the gameobjects manually instead of using the “Gameobject” field of the tile. Using the tile system will add excess constraints and performance cost in addition to making it harder and less efficient to make an object pooling system that’ll keep the performance in check with large maps.
  • [quote=“thibaulthu, post:1, topic: 947248, username:thibaulthu”]
    there is an official solution
    [/quote]
    you really should mention this before you talk about how you solved the 2d shadow casters for tilemaps. There is also 2d soft shadows in 2023 which is worth the upgrade imo
  • [quote=“thibaulthu, post:1, topic: 947248, username:thibaulthu”]
    for my need, I experimented that letting the default pivot point to center (0.5,0.5) is better to avoid playing with the collider offset in tile game objects
    [/quote]
    You should be careful when leaving the pivot at center for 2 reasons: A) When you have odd numbered width or height, the pivot of the sprite will be in the middle of a pixel which will cause smooth pixel perfect camera movement issues. This is one of the reasons why you should use the pixels mode(or whatever it is called) when moving the pivot of a sprite. B) you wont be able to sort the tall objects correctly because the pivot is not at the floor of the object.
  • [quote=“thibaulthu, post:1, topic: 947248, username:thibaulthu”]
    do not forget you can use the Sprite Editor dropdown to set the custom physics shape to each custom tiles (slope for example). If you change these values, and you don’t see any changes in the collider of the corresponding tile, try to “Reset” the collider by clicking ⋮ first
    [/quote]
    Custom physics shapes for tilemaps will not work as desired when you have more than 2 levels because when the level changes rapidly(from 0-> >2) higher levels will prevent movement behind them <<< in this case red collider will prevent a player etc. moving behind the white collider(I didnt do a good job explaining this)

Additional tips:

PS: For some reason the lowest slope is leaking some light or something, the brown side is becoming green when light shines on it. There seems to be similar problems with the rectangle boxes too

Cheers for your detailed analysis

2 Likes

Hey,

I found this post on Reddit r/Unity3D by u/InfernoInfernal Reddit - Dive into anything

He did a much better job than me to reproduce the feeling of Final Fantasy Tactics A2, with an isometric tilemap with elevation, slope, shadows and mouse selection. His code is well constructed and he reused Tilemap mecanics.

He detailed all his process in a dev diary on Google Docs. A must-read to any beginner in Unity that want to reproduce a Final Fantasy Tactics like game Tactics A Template Dev Diary - Google Docs

1 Like