Welcome to the Sentis beta group! You can use the Sentis Beta category to see all information related to the beta program, share feedback, ask for help, and have discussions about Sentis. We communicate almost entirely through this platform, so please set your alerts to be notified.
Welcome
Sentis is a local inference engine for AI models that leverages the compute on end-user devices, instead of a cloud server. It can run anywhere the Unity runtime can be deployed. It replaces Barracuda and addresses much of the feedback we received while Barracuda was in an early prototype phase. Most importantly, Sentis can run many neural networks performantly at game speed. Sentis is currently in an open beta phase that will last for a few months, with a tentative plan to launch publicly for commercial use on November 16, 2023. Sentis will be free for all Unity users and will be available for unlimited local inferences.
What does Sentis do?
Sentis allows you to import AI/ML neural network models (from sources like TensorFlow, PyTorch, and other AI hubs) in the ONNX format into the Unity Editor, then optimize and run inferences in real-time on user devices across all Unity supported platforms.
You can do this in the Unity Runtime (for gameplay) or in the Unity Editor (for game creation) instead of requiring a cloud server with an expensive GPU. Sentis is fast, avoids cloud costs, and allows for private neural network inferences. Ultimately, Sentis empowers Unity developers to take advantage of the incredible innovation happening in AI that was not previously possible. There are many possible use cases and we are excited to see what you create during the beta. Here are a few examples we have seen to get you brainstorming:
1. Image Upscaling: Upscale a low-resolution image, model, texture, etc.
2. Style Transfer: Translate the look or image effect of a scene to a new style
3. Non-Playable Characters (NPCs): Automate the dialogue of characters
4. Speech Recognition (NLP): Interpret live speech
5. Body/Object Detection: Detect objects using the camera
6. Depth estimation: Detect depth using the camera
7. Image, Video, and 3D Model Classification: Detect on-screen objects
8. Handwriting Classification: Detect handwritten letters, symbols, or numbers
9. Creating one-of-a-kind player experiences: Generate infinitely unique gameplay scenarios
10. Animation: Apply automated pose rig to a rigid body
11. Simulation helper: Approximate a complicated function with a neural network to drive gameplay logic
12. Timeseries: Anomaly detection, forecasting/prediction
Posting topics
Please keep your new topics very specific and try to ensure you are not duplicating an existing topic. Here are some topic themes to guide you:
- Features and workflows: Are there any workflows that are unclear or missing? Are there any features that you expected to see, but didn’t? Can you share a clever workaround for an issue that you think may help others? Feel free to also schedule a live meeting with us if you would like to discuss your idea in detail.
- Ease of use: Is it easy for you to use Sentis even if you may not be an AI/ML expert?
- Performance: Does it perform well with a wide variety of neural network model types? Are there any performance issues on platform/OS combinations?
- Use cases: What are your use cases and what neural network models are you using? Can you share a video demo of what you’re working on? We understand models, games, and other assets may be proprietary, so please do not share any sensitive intellectual property (IP) publicly.
- Bugs: Is there a bug that you reported through the bug reporter that’s worth discussing with the community? (See below for specifics about submitting bugs.)
- Documentation: Is any of the documentation unclear? Would you like more documentation for specific items?
Getting started
Installing the Unity Sentis Package
- Download Unity 2023.2 from the Hub. Sentis 1.3.0 works with 2023.2+, while prior versions of Sentis work with Unity 2021.3+.
- Create a new Unity project or open an existing one.
- Click Sentis Beta Experimental package, or go to Window > Package Manager, then click the + icon, select “Add package by name…” and type “com.unity.sentis”
- Press the Add button to install the package.
Note: For versions 1.0.0 and 1.1.0, Sentis was in a closed beta and required an authorization key. For version 1.2.0 and beyond, Sentis will not require a key.
Documentation and samples
Project sample videos
Reporting bugs
Please report bugs via the Unity Bug Reporter in the Unity Editor, and provide as much context as possible so we can rapidly triage the issue. When submitting a bug, please:
- Link to a hosted image or video that clearly shows or explains the issue, including expected vs. actual results.
- Attach a (stripped) project so we can easily recreate your issue.
- Attach Profiler .data and Profile Analyzer .pdata files where possible.
- After filing your bug, open a Discussions topic if you believe the community would benefit. In the topic name, it helps to use your bug name and issue # to make it easier to identify
Frequently Asked Questions
Where can I find neural network models to run in Sentis?
Here is a list of some of the most popular sources for neural networks. Unity does not formally endorse any of these suppliers or marketplaces: Akira, Caffe Model Zoo, Facebook, Google, Hugging Face, Keras, ModelPlace io, Model Zoo / MXNet, ONNX Zoo, OpenAI / Microsoft, PyTorch Hub, TensorFlow Hub.
When will Sentis be released in general availability (i.e., to the public)?
Sorry, we can’t reveal the exact date, but we plan to announce the date, or actually release Sentis before the end of 2023. Once we set a public date, we will let you know.
I have an idea for how to improve the beta program, how do I share it?
Start a Discussions topic and we will work hard to take your suggestion.
Are there other AI products in beta? More coming? How do I get access to those?
Yes and yes! While we can’t comment on the date for additional users to be added to existing closed beta programs or new product beta programs, we can say that there are more coming. Our closed betas use a targeted approach to invite users that we believe will improve the product for all users. If you have been invited to more betas (ie. Muse), you will see them in the main AI Beta category.
Policies
Please comply with the basic policies we have set up to ensure a smooth beta program for you and for Unity.
- Prototyping & private distribution only: The beta versions of this package are for prototyping and private (internal and non-commercial) distribution only, and your access is subject to our Terms of Service (TOS). This helps protect your work from breaking changes that can occur during the beta cycle. In particular, please review section “13. Evaluation Versions” in our TOS and ensure you understand and can comply with these terms before continuing in the program.
- Confidentiality: You are using Sentis before the general public as it is not publicly available. Don’t forget that your use of Sentis is subject to section “17. Confidentiality” in our TOS.
- Code of Conduct: The general Code of Conduct applies to this category on Unity Discussions. Please familiarize yourself with it to ensure productive and inclusive conversations.
[/details]
Thank you for your excitement and interest in AI @ Unity!
Please continue all the great feedback!
Thanks,