O~K
as always, before unity community answers, found the solution myself. Was a bit walking in dark but made it work.
the solution is multiple folded :
1- in your custom editor, you would need a render texture, a single one would be enough, you instantiate it on window initialization and on enable double check and if null, instantiate , the params for with and height should be 128, this is the with and height of the texture being returned by the AssetPreview, shame that we can’t make bigger pictures but i think it makes sense. the last param is the bytes for the colors, choose 32.
OnEnabled/Initialization ==>
finalTexture = new RenderTexture (128,128, 32);
finalTexture.enableRandomWrite = true;
finalTexture.Create ();
//OnGUI ==> Right before using this, check if it's null and repeat the above code to regenerate it.
2- you need a compute shader like this :
// Each #kernel tells which function to compile; you can have many kernels
#pragma kernel CSMain
// Create a RenderTexture with enableRandomWrite flag and set it
// with cs.SetTexture
Texture2D<float4> Input;
int Gray;
int Gray2;
float Epsilon;
RWTexture2D<float4> Result;
[numthreads(32,32,1)]
void CSMain (uint3 id : SV_DispatchThreadID)
{
// TODO: insert actual code here!
float r = Input[id.xy].r;
float g = Input[id.xy].g;
float b = Input[id.xy].b;
float a = 1.0;
for(int i = Gray; i <= Gray2; i++)
if(abs(r -(i/255.0)) < Epsilon && abs(g - (i/255.0)) < Epsilon && abs(b - (i/255.0)) < Epsilon)
r = g = b = a = 0;
Result[id.xy] = float4(r, g, b, a);
}
This compute shader gets two ints for gray color in. I used int because i can easily define the gray color capturing a screen shot and looking into the color by photoshop and it returns a number between 0 to 255. then every gray color between the two numbers will be removed and the texture will be returned
3- finally, before you draw anything using the returned texture from AssetPreview :
Texture2D texture= AssetPreview.GetAssetPreview(TileSet.Prefabs[prefCounter]);
var shader = Resources.Load ("TransparentBackground") as ComputeShader;
int kernel = shader.FindKernel ("CSMain");
shader.SetTexture (kernel, "Input", texture);
shader.SetInt ("Gray", 82);
shader.SetInt ("Gray2", 90);
shader.SetFloat ("Epsilon", 0.00001f);
finalTexture.enableRandomWrite = true;
finalTexture.Create ();
shader.SetTexture (kernel, "Result", finalTexture);
shader.Dispatch (kernel, texture.width / 32, texture.height/32, 1);
Now, the final texture is ready to be used in any gui object that accepts Texture2D as param to draw. this texture is transparent in the places you had the gray color. normally, the gray color that is returned by AssetPreview is 82 exactly, so, by using a very small epsilon like 0.00001 you would prevent any pixel removal unless it’s exactly the same color.
I had to code it like this because some of the assets i had where not clean and by using the loop between two grays, i could at least look at a cleaner version in the editor.
Remember, if you use more than one render texture in a custom editor gui,… memory leakage and each frame two unused assets being generated, in less than 5 min you would over run 32 gigabyte of ram ! and using unload unused assets is not necessary because we are using a single texture over and over and over.