SetPixel on a sprite texture without changing it globally

Hi, so I spent the last days experimenting with different solutions for a kind of “decal” system for my game. I found that setPixel() would probably be the best for my specific case.

The situation now is that I have a Sprite (in 3D-Space), and I want to change some Pixels on it. In order to set pixels on the sprite I had to make the texture of the sprite writable.
But now everytime I set a pixel, the texture changes on all objects that use it. It even stays like that when I restart the scene.

Now this does not seem like unintended behaviour and at least kind of makes sense, but is there some way to just edit the texture on one specific object?
(And if yes, is it still feasible performance-wise?)

I don’t know if its needed, but here is the code I am using.
The calculation which pixel is supposed to be set is only a placeholder, I’m going to deal with that if I get SetPixel() to work

//C#
SpriteRenderer renderer = hit.collider.GetComponentInChildren<SpriteRenderer>();
Texture2D tex = renderer.sprite.texture;
Vector2 pixelUV = hit.textureCoord;
pixelUV.x *= tex.width;
pixelUV.y *= tex.height;
tex.SetPixel((int) pixelUV.x, (int) pixelUV.y, Color.black);
tex.Apply();

By doing this, you write the decal in your asset folder texture. That’s why restarting keep it on the texture : you modified the texture file itself. You should disable the possibility to write your base texture and use a clone instead. Modifying the clone’s pixels will not affect the original one.