[RELEASED] ✨ AR Simulation ✨ — AR development made easy

I’m very happy to present you AR Simulation, our in-editor XR plugin to speed up development of ARFoundation applications.
AR Simulation is available on the AssetStore, and we have a trial version that you can try for 7 days.


Build AR apps with confidence.
Iterate fast, right in Editor.
Non-invasive, drop-in solution. :sparkles:

AR Simulation allows you to work on and test AR apps right in the Editor. No need for numerous deployments, no need to build complicated mock structures…
Just move around in the Editor, test your AR app, without having to change any code or structure. Iterate faster, test out more ideas, build better apps!

Features

:heavy_check_mark: Plane Tracking
:heavy_check_mark: Touch Input (single touch for now)
:heavy_check_mark: Simulated Environments
:heavy_check_mark: Device Simulator
:heavy_check_mark: Point Clouds
:heavy_check_mark: Image Tracking
:heavy_check_mark: Light Estimation
:heavy_check_mark: Anchors
:heavy_check_mark: Meshing
:heavy_check_mark: Camera Background Rendering

6041507--652910--simple-explanation.gif

Licensing

Using AR Simulation requires you to buy a per-seat license —
please buy seats for your team through Unity AssetStore.
This is a single payment, not a monthly subscription.

You can use it for 7 days for evaluation purposes only,
without buying a license. We trust you. :sweat_smile:

Verified to work with

• Unity 2019.3 • 2019.4 • 2020.1 • 2020.2 •
• ARFoundation 3.x • 4.x •
• Device Simulator •
• Built-In • URP •
• “Old” Input System • “New” Input System • “Both”•

ARSimulation is a custom XR backend, built on top of the XR plugin architecture.
Because it’s “just” another XR Plugin and we took great care to simulate important features, it works with your existing ARFoundation app, new/old input system, XR Interaction Toolkit — zero changes to your code or setup needed! :sparkles:
And if you need more control, there’s a lot of knobs to turn.

For support, please see the Manual and our dedicated Discord channel.
Please report bugs on GitHub.

2 Likes

TLDR: This post explains more about our approach for AR Simulation, why we think an XR plugin is a great solution, and how this compares to other solutions like AR Remote and MARS. For even more details, please refer to the Readme.

Long story short, MARS (Mixed and Augmented Reality Studio), AR Remote and AR Simulation serve different use cases and audiences and each have their own Pros and Cons.

  • MARS is a framework. This means that it wraps around all ARFoundation functionality – all your code and design will have to go through MARS. This also means that MARS has a lot of complexity just because it needs to replicate all ARFoundation features (and extend them in some places), a process that is not complete (yet?). Additionally, MARS introduces lots of additional concepts that are useful for specific types of applications but not others (e.g. constraints, functionality injection, simulated environments, …). At some unknown point in the future MARS will get companion apps (?) that allow you to test remotely on a device, and record data for future editor sessions.

  • AR Remote is a XR plugin for Editor that allows you to stream data from your device, in real-time, to test your app without deploying. This is very useful to reduce iteration times, but still requires you to have a physical device to test your app, and you always start “from scratch” (track some planes, find images, …) for testing. Since it’s an XR plugin, no changes to your existing AR Foundation app are required – for desktop, the remote backend is used, on device, the actual device backend will be used.

  • AR Simulation is also a XR plugin for Editor, and allows you to test your app right in the editor, without the need for a physical device. It supports Device Simulator (for testing various resolutions and target devices), allows to set up simple or complex test scenarios (e.g. what does my app do with 3 tracked images and 2 tracked planes, and suddenly an image loses tracking), and allows for simulated environments, automated testing, and others.

We think AR Simulation has a unique selling point in ease of use – we spent a lot of time integrating it seamlessly with AR Foundation and the Unity Editor – and speed of iteration (literally press play and fly around and interact).

Some features are currently better suited towards AR Remote, such as simulating tracked face data, or tracked skeleton data, were we felt the overhead of building “simulation tools” for was bigger than remoting to an actual device with actual data. However, that might change if enough people think that would be useful for them!

Some additional background:
We were active MARS alpha testers, trying to use it for our own AR applications, and started developing our own solution in parallel. After a while, we stopped using MARS (besides of course testing and giving feedback to new releases), as we felt that for our use cases it was too heavy, too complex, required too many changes to existing projects and approaches, and didn’t have the features we needed (e.g. in terms of interacting with AR worlds).

With AR Simulation, our goal were fast iteration times in the Editor for a range of AR applications we and partner companies build. These usually consist of placing and interacting with objects from different angles. We just needed a way to “simulate” an AR device in the Editor, not a full-blown additional framework!

Fortunately, Unity provides the ability to build exactly that using the XR plugin architecture: a custom XR provider that works in the Editor and Desktop builds.
(Note: we suggested to the MARS teams to implement it as XR backend; still not sure why they didn’t.)

Been looking for a tool like this for a while. MARS just seemed very heavy for getting going with an AR Foundation project to test and rapidly iterate in the editor.

Sounds really interesting. Would you please elaborate on the above? (as in comparison to AR remote and Mars)

2 Likes

I’m sure the asset devs will reply back here soon but the comparisons you need are here in the manual for this asset:

https://github.com/needle-tools/ar-simulation/

1 Like

Thanks @ROBYER1_1 , well appreciated.

I updated the post with an in-depth explanation of how we see the current AR environment with MARS, AR Simulation, and AR Remote:

  • Each has its own respective use cases, and we think AR Simulation is the superfast, supereasy, non-invasive, don’t-want-to-mess around solution that today’s AR designers and developers need.

  • MARS might be the one future AR devs need, but nobody knows yet!

  • AR Remote is better when for cases where simulation will be too far off of real world, such as face tracking and skeletal tracking. For other features such as planes, image tracking, point clouds, environments, testing with AR Simulation is faster.

Please let me know if you’d like to know further details or clarification!

1 Like

I use your plugin in two projects in where I didn't use it before and have to say i love it's simplicity. So thank you very much for your very tough work!

I don't need the examples, but I meet an error while they are present:

BuildFailedException: ARKit requires dimensions for all images. Reference image at index 0 named 'imageTrackingExample' in library 'Assets/Samples/AR Simulation/1.0.2/Getting Started/Assets/ReferenceImageLibraryExample.asset' requires a non-zero width.
UnityEditor.XR.ARKit.ARKitReferenceImageLibraryBuildProcessor+<ResourceGroups>d__0.MoveNext () (at Library/PackageCache/com.unity.xr.arkit@3.1.0-preview.4/Editor/ARKitReferenceImageLibraryBuildProcessor.cs:36)
UnityEditor.XR.ARKit.ARKitReferenceImageLibraryBuildProcessor+Preprocessor.OnPreprocessBuild (UnityEditor.Build.Reporting.BuildReport report) (at Library/PackageCache/com.unity.xr.arkit@3.1.0-preview.4/Editor/ARKitReferenceImageLibraryBuildProcessor.cs:86)
UnityEditor.Build.BuildPipelineInterfaces+<>c__DisplayClass15_0.<OnBuildPreProcess>b__1 (UnityEditor.Build.IPreprocessBuildWithReport bpp) (at <cf7d2c2e83f749e8943578126f9eff4b>:0)
UnityEditor.Build.BuildPipelineInterfaces.InvokeCallbackInterfacesPair[T1,T2] (System.Collections.Generic.List`1[T] oneInterfaces, System.Action`1[T] invocationOne, System.Collections.Generic.List`1[T] twoInterfaces, System.Action`1[T] invocationTwo, System.Boolean exitOnFailure) (at <cf7d2c2e83f749e8943578126f9eff4b>:0)
UnityEngine.GUIUtility:ProcessEvent(Int32, IntPtr)
1 Like

Hi! Thanks for the feedback! Happy to hear it’s useful to you!

Which ARSimulation version are you currently using? This bug should’ve been fixed with version 1.0.3-preview.1 released a couple of days ago. Sorry for the inconvenience :slight_smile:

1 Like

@trnq1ll0 to add to what marwi said above – you can update AR Simulation directly through Package Manager right now, no need to wait for a new AssetStore release.

2 Likes

This looks awesome fherbst! Exactly what I wanted out of a simulator. I’ve been building purpose built simulation scenes for my AR apps and it can be a pain. I’m looking forward to checking it out soon :slight_smile:
I noticed you mentioned you have meshing support- do you have a video clip or gif of what that looks like? Meshing is particularly painful to simulate right now in editor.

Hello @GameArt I’m glad you like the package. Let us know when you get to try it.
About meshing: We have plane tracking/detection support, but the meshing manager API is not yet supported (although we look into it and would like to support it eventually).

Hello, I just wanted to let you know that we’re currently working on adding proper meshing support. You can follow the progress here: https://github.com/needle-tools/ar-simulation/issues/37

Please feel free to let us know if you have any feature requests regarding meshing (ideally on github) or if there is anything else missing for you!

6321687--700863--2020-09-17-162631.gif

6321687--700863--2020-09-17-162631.gif

We recently published version 1.0.4-preview.1 :zap: adding support for ARMeshManager for windows and osx

6339876--704265--2020-09-18-162314.gif

Hi

I recently bought your asset from the store. I think it has some amazing potential and I'm looking forward to its progress.

I left a review and mentioned a bug in there, I decided I'd leave a video to show you what I meant.
You can see that here.
The 2 issues are in the description and below.

Please let me know if there's anything I can do to help debug this.

Thank you.

-Satwant

____YT DESCRIPTION__
There are two issues that you can spot in this video. The first is that the plant keeps getting placed even when I'm not tapping on the game window. I would guess it's reacting to the mouse down and placing at 0,0,0. You can see this at the 1min mark.

The second issue is a little more subtle when I click back into the game window I get some slowdown and freezing. You can see this at the 1min and 20sec mark where I cannot move the game camera around and then it jumps.

You recommended I download version 1.0.3-preview.1 Although its not shown in the video as I couldn't work out how to enable preview packages as it changed in 2020. I worked it out in the end thankfully. The issue still persists, it is also still there in version 1.0.4-preview.1

Im using Unity 2020.1.3f and I have a MacOS Catalina 2019

1 Like

Hello @sskenth Thanks for getting back at us!

Thanks for the video, that's great. I'll try to reproduce it tomorrow when I'm back at the office and have access to osx. I assume issue one and two are related.

Btw if you're using discord feel free to join here: https://discord.gg/U2NVvvd
It is sometimes easier and faster to communicate

Hi @sskenth

Do you think you could send me the project you're using in the video? Im trying to reproduce it but unfortunately I havent been able to so far.
I'll also send you a PM with a change that could potentially fix it for you, would be great if you could try that as well.

Edit: OK sent you a message

Hello everyone!

We just added support for Object Tracking in version 1.0.4-preview.2
The "Getting Started" samples contain an example scene.

As always: please let us know if you run into issues or have any feature requests!

And in case you didnt notice: ARSimulation is currently on SALE with 50% OFF in the AssetStore

6354987--706761--2020-09-27-183124.gif

Version 1.0.4 released fixing bugs with image tracking

Great stuff you are doing! Keep up the good work!

1 Like

Thanks a lot! And thank you for your nice review!