Using ChatGPT to create Unity Scripts and shaders.

I have been using this very extensively, and this is a game changer, despite the limitation. So I posted this: Infinite parallax hair volume using wrapping grid tracing (attempt)

I couldn’t found anyone to help, so for a long time I tried solving it the way I know, visually, but couldn’t bridge the gap to math for implementation. The last breakthrough I had was this image:

Basically wrapping a ray intersection of infinite field of circle/sphere into one axis, which allow to collapse te problem to 1 dimension. There was many step to get there.

Enter ChatGPT, I translated the visual problem into something more textual without reference to image metaphor, only giving it the bare premise, so many step removed from my latest breakthrough: and it spit me (paraphrasing and simplifying):

“Sure! how about (a%b)/b”

Here are the desmos graph:

I haven’t completely assimilate the result yet, but HOLLY MOLLY! it would have saved me time, nobody could help, Chat GPT is my new bro!

Also I extensively tested the creative ability of chat GPT.

For context, gpt3 and 2 were wild card dong wild mood swing. Chat GPT was basically an experiment to see if we could control the output of language model, therefore the training was gear toward toward not only a specific style and behavior, but an idea of truthfulness. What that mean is that is that a model can be intelligent enough to go beyond uncontrollable and random generation, basically it’s a prototype of the 3 law of robotics. GPT2 and 3 are more creative, but they have too many random learned context state that are wildly unstable, it’s hard to get them to do something without it being polluted by adjacent concept and context he might have learned, which is why you end up with well being apps telling people to just die, chat GPT is the answer to that. I mean it can recognize things it doesn’t know something and when confronted with input that have many possible answer, it knows to ask for clarification, which mean it is aware of his own knowledge, which is like a big breakthrough revelation abut what neural Ai can do! The fact it can do programming is mostly an accident and not what the ai was created for.

The consequence on creativity is that, it’s probable that the creative context personality, it could have learned from text data, is simply repressed by the training that emphasized truthfulness and the mild mannered persona that contrast with its sibling AI. There is no fact base truthfulness in creative work, therefore I guess that’s translated into sticking to tried and true cliche. It’s also very very annoyed you ask him to be creative, always deferring to your judgement and what you want. Fun things is when I force it to come up with stuff, it kind of get low key passive agressive lol, one time I ask it to write a character profile on a paper thin premise of a story, it called the character Lily and the city MillVill, I asked it why it named them that way, he said those are perfect name, because they are neutral, for an underdevelopment setting, OUCH. I mean he is right, that’s a perfect motivation at an early stage of writting were you don’t have enough data to know where it goes.

The thing is, when people are creative they tend to go for cool or thing that emotionally matter to them, professionally we are ask to always motivate the choice to make a good works, and motivating random emotional pick is somewhat tough and can lead to a lot of turmoil. ChatGPT is on the other side of the spectrum, there is no choice it can’t explain, it always have a reason, and if you toss stuff at it, it will find a plausible reason. For example I made a pastiche of anthem called anAthemA, and ask it to explain why it’s name that way, it found a way to link the name to the story, and you can ask for multiple interpretation possible and it will explain in depth. Now there is a limit, professionally asking why you put something, isn’t just how it relate to other thing in the work, but also how it relates to you, what’s the emotional resonance, that’s the part that make chat GPT feel annoyed by your request, he always remind you that part and tell you he can’t do for you, but still provide possible (cliché) reason you might have had, such an helpful lad, such a bro!

There is probably a way to unlock the repressed creativity with a smart prompting method, I haven’t tested yet. But it’s useful as an assistant, because cliched isn’t always bad, and also it can fill in blanks for you very easily, especially if those don’t matter much, so you can focus on the creative part. It can blend creative idea together, for example blend daria and magical girl. It can write scene, document composition, refine idea based on your input, you can ask to find artist to use as reference to compliment a story, to find motivation for your ideas. It’s not great, but it’s good! If anything, it’s a great autocomplete for art job. For me it’s a perfect complement to all my weaknesses!

NOW on AI NPC chatting, I don’t think we need chatgpt level ai. GPT2 is more than enough, and it can run locally on your computer. Why GPT2? because it’s a matter of design. NPC had one big problems unsolved, flowing dialog and style. Over problem like topic database, emotions and fact tracking, are already mostly solved, the problem was always in the presentation, that is speech pattern instead of spreadsheet report, game like dwarf fortress for example does it and have repetitive template that get filled with data to express state. What we really need is style transfer, something able to take the spreadsheet and turn it into speech pattern, then convert back user speech pattern into useable spreadsheet data for a hand made system to compute. Every current game system that use ai to generate dialog rely solely on the ai to track everything, and has told above, there is a limit due to the working memory architecture. What we need is to figure out how to trigger the relevant style state of the network (probably using some fine tuning too), track things ourselves and feed the working memory with instruction only relevant to what we want, which would differ per game. That’s how some deployed system already do, so it’s not like it’s pure speculation.

2 Likes