TL;DR: If RuleTiles arent working, re-import the 2D Tilemap Extras package folder. And keep doing it until it fixes the problem.
Alright, the problem appears to be some AssetDatabase cache issue according to people who had similar problems over the years.
I’m experimenting with a manual workaround right now.
Workaround: locate your 2d tilemap extras package in the “project” and reimport (not reimport all, just regular reimport" the entire folder inside the editor.
If the editor tile editing tool dies again, reimport the package and it’ll fix it.
If you mistakenly selected the “RuleTile.cs” script in the “no script found”, you will need to manually re-import the asset.
Simply create a new RuleTile asset and, in a text editor like notepad, copy paste the “m_TilingRules:” section all the way to the bottom from the broken asset to the newly created asset. Everything else doesnt matter.
It will most likely break again and just re-import the 2D Tilemap Extras package folder again.
You will get this error “SerializedObjectNotCreatableException: Object at index 0 is null” constantly but, like all good unity programmers, ignore it. The RuleTile will work with the tile palette painter and in-game.
Attempting to manually fix the error by replacing the null file marker in the asset will break the editor like selecting the RuleTile.cs file above, requiring the manual reimport steps described above so dont do that.
Hypothetically, an easier way to fix broken existing assets is to go into the .assets file using a text editor and locate and replace the following lines with the below:
m_Script: {fileID: 0}
m_EditorClassIdentifier: Unity.2D.Tilemap.Extras:UnityEngine:RuleTile
And reimport the 2D Tilemap Extras package.
Reloading editor will also require reimporting the 2D Tilemap Extras package after startup in order for the tile assets to register as RuleTiles. The tilemap will still appear normally with tiles painted on it but editing the tilemap with a tile palette or runtime tile editing will require at least one reimport of the 2D Tilemap Extra folder.