Initial General Feedback on Muse Texture and Muse Sprite

Attention: I believe that both the developers of the Texture Generator and the Sprite Generator should read this post since I used both tools in conjunction. I’m sharing all the information under a single heading to avoid spam.

Hello Everyone!

I’m Murat, living in Turkey. I’ve been involved in the gaming industry for many years, but I’ve been active professionally in the gaming sector for the last 5-6 years. I had a game studio where we produced idle-type games, but my company is currently in the process of closing. I mainly work on 3D modeling, animation, rendering, level design, and game design. Unfortunately, I’m not proficient in 2D. I also have coding knowledge. You might be thinking, “So what does this have to do with us?” but evaluating my feedback on Muse AI from my perspective will be beneficial.

Firstly, thank you for inviting me to the closed beta program, Unity team! With the AI trend kicking off, I’ve been closely following various AI tools and actively using some of them. I find Unity’s integration of AI very positive.

The 3D model I mention below belongs to a mobile game that my team and I developed earlier. Using this model, I prepared a study using only a diffuse map as if I was developing a new mobile game.

I don’t have experience in 2D. I worked with stable diffusion for texture works for a while, but it was quite laborious. The effort decreased with Muse Texture and Sprite. However, I struggled while producing some textures with Muse Texture. Even when I entered detailed and/or negative prompts like in Stable Diffusion, I sometimes couldn’t get the results I wanted. I couldn’t succeed in creating a wall texture from a tree with the Muse Texture Generator. It always produced textures that were very similar to each other with the same motif/pattern.

I wondered if I could make this texture with the Muse Sprite Generator. I decided to try the Muse Sprite Generator and obtained the textures I wanted more easily using the same prompts.

I struggled to produce some textures. I think the Muse Texture Generator either doesn’t fully grasp the prompts or doesn’t have enough sampling. I put a lot of effort into getting a stylized work, but I usually got realistic textures. Maybe I couldn’t enter the right prompts :slight_smile:

I used the Muse Sprite Generator more for texture production. However, I produced a few UI elements for a tower defense game I plan to develop soon, and I was very satisfied with the results. I will now produce UI elements with the Sprite Generator that I definitely couldn’t do! It saved me from a big burden.

In the image below, I wrote which tool I used for the textures I used on the 3D model.

This was my first study. As I mentioned earlier, as if I was designing for a mobile game, I tried to create a design as much as possible with stylized textures and using only a diffuse map.

I also have some feedback about the interface:

-I get confused when switching between the Texture Generator and Sprite Generator. It would be helpful to display the name of the tool prominently.

-Sometimes the animation gets stuck while creating a texture or sprite. It gets fixed when I create a new texture/sprite or click somewhere on the interface.

-A sprite I produced once disappeared. It’s also not in the history records. Yet, I used this sprite in the example I gave above. This problem occurred once. One of the lost sprites is in the image below.

Image_5

-Some of the produced textures or sprites are being created in black.

-Some tools on the Texture Generator screen can accidentally disappear. There should be an option to bring these tools back. This happened to me once, and I don’t remember how I managed it :smiley: I had to start a new study because I couldn’t bring it back.

-It would be a nice feature to want the texture/sprite to be of a certain size.

-The two buttons above seem to have no function at the moment :stuck_out_tongue:

In conclusion, as someone who is not proficient in 2D asset creation, this tool is a great advantage for me. Of course, there are shortcomings in the initial stage, but I believe these will be addressed soon.

I hope this wasn’t a boring post. I will continue to share my detailed feedback after completing my next study.

Note: I conducted these tests with Unity 2022.3.10f1 version.

3 Likes

Great feedback, we appreciate the effort you put in!

We are currently training the next iteration of the texture model with a focus on improving the most common types of materials as well as increasing the variety of the outputs for things like wood. We also got some feedback on color guidance and are working on adding a color picker to help with consistency and control over things like tiles/bricks/shingles/etc.

For stylized textures, the model was trained only on photorealsitic assets and we plan to make the style training system (that is already available on Muse Sprite) available on Muse Texture in the near future.

When you get a solid black image on Muse Sprite this is caused by a safety filter that detects potentially harmful content. It gets some false positives and we are working on improving this.

If you run into any specific concepts on Muse Texture that the model should know but can’t generate, please let us know and we will work on adding the capability in on the next training run.

Thanks again for the great write-up :slight_smile:

so cool, Would you consider turning this case into a short tutorial? I really want to follow along. cool

Thank you. I will continue to share my mistakes and feedback in my next study :slight_smile:

1 Like

Since my company is in the process of closing and I’m planning an individual game project, I’m considering publishing devlogs on my YouTube channel. Given that Unity Muse is in its closed beta phase, I assume the Unity team might not approve of such sharing for now. However, I’d love to create small sample works with Unity Muse and share them on my channel, if it’s not an issue for the Unity team :slight_smile:

1 Like

well, someone already did : https://www.youtube.com/watch?v=Pjf4clvXOmk :slight_smile:
found this one too: https://www.youtube.com/watch?v=OPLQR_hquOg
@YJ_GAHNG is it okay he does some YouTube videos?

1 Like

I’ve watched the content shared by @lucvovan. If you grant permission, I would love to prepare professional and informative tutorials in both Turkish and English to introduce this wonderful tool to the people living in Turkey. I don’t want to take such a step without permission, so I await your approval. Thank you! @YJ_GAHNG

1 Like

anyway, so coool u guys, and I want to your tutorial too. it is so cool