Happy new year !
We have just released LLMUnity, a package that allows to integrate LLMs (Large Language Models) in Unity !
At a glance
Cross-platform! Supports Windows, Linux and macOS
Runs locally without internet access but also supports remote servers
Fast inference on CPU and GPU
Support of the major LLM models
Easy to setup, call with a single line code
Free to use for both personal and commercial purposes
Setup
You can install it as a custom package with the GitHub url (how to): https://github.com/undreamai/LLMUnity
We have also submitted it to the Asset store and hope to have it there soon!
How to use
You can find our simple tutorial at Medium: How to Use LLMs in Unity
and samples that you can try as part of the package (simple setup with package manager): Samples
Let us know if you have any comments / questions / suggestions!
Hello.
Thank you for the work going into this. It’s really good. However I’ve noticed a problem I was wondering if you could help with.
In the Inspector Prompt field I’ve set a persona for the AI character to act as. This works really great, but I’ve noticed if we talk for a while, it forgets everything I set in the persona.
If I were to guess why, I would think that the contents of the Prompt field in the inspector only get sent to the AI on run time once and when the context size is exceeded due to continues conversations, it forgets who it is.
Is that how it’s programmed right now to work?
Traditionally, if you are working with GPT to do the exact same kind of role-playing AI chatbot, you have to send the persona every single time a message is being sent to the AI.
Can LLLUnity be updated to work the same way? This would also help with injecting additional important information into the Prompt during run time that can carry on through our interactions. Such as current variable values.
Thank you for that.
Sincerely, I appreciate the effort gone into putting this together. Having the AI run locally is quite a considerable advantage in countless ways. Not only $$$ cost savings on API calls, but quite simply the speed in response time. There’s also some very entertaining LLM’s out there that are fully open sourced and perfect for role-playing chats. Makes everything much more fun. Thank you.
Thank you for the nice words!
Yes that’s exactly how I see it as well:).
Do you have any examples of good role-playing LLMs? I’d be very interested!
Regarding the issue:
The default behavior of LLMUnity is to always keep the initial character prompt when the context is exceeded.
In more technical terms:
LLMUnity is built on top of llama.cpp that handles the prompts as follows:
The “keep” variable of llama.cpp is set by calculating the number of tokens of the initial prompt (here).
If you do something like this, you can also disable the history so that each message is independent from the other:
_ = llm.Chat(message, HandleReply, ReplyCompleted, false);
I’m not sure if I understood correctly, does this answer your question?
Thanks for the link, I have seen this game it’s very cool!
@Tubbritt I’m also working on a long-term memory for the model based on this solution (embeddings).
It should help the model not forget when it has a large discussion / context.
Sorry for the delay in replying. I was out of the country for a few days. Thanks for the information and continued work on this. Having the model never forget would be great as the role playing appeal to all this is amazing. It adds an excellent dimension to interactions with the LLM.
Yes sure, if you want to try a role-playing LMM, check out the Deepseek LLM.
I find this specific version is excellent for my needs.
deepseek-llm-7b-chat.Q4_K_M.gguf
It’s really good and it has a wicked sense of humour when you get the prompt / persona locked in.
