Trying AR in Unreal. (sharing experience)

News & General Discussion
1

I wanted to share the experience of trying to test AR on Unreal 4.

That one’s a negative, I thought it might be a good idea to bring up some things unity got right.

  • Trying to launch AR template results in nothing happening. The template is there, but doesn’t build. Trying to build the template results in a redirect to… a 404 page in unreal documentation. Apparently there was a quickstart page for Android, but it wasn’t updated for 4.27 and displays 404. The actors in editor are helpfully called “Documentation actors”
  • Investigation revealed that Unreal 4 (unlike unity) does not come with packaged android sdk/jdk/ndk, and those need to be installed manually. I try installing android studio.
  • The engine does not detect android studio. Android device is not detected as deploy target. Turns out that I also need to run a bat file within the engine directory. ("Engine\Extras\Android\SetupAndroid.bat ")
  • Running SetupAndroid.bat fails! It can’t find the installation directory of sdk (custom location says hi). Online sources suggest to modify the bat by hand.
  • The bat file tries to run sdkmanager.bat from android. The file, when launched, prints a java exception with a stack trace. (Ha). Investigation of online sources suggest that I need to install OpenJDK 8. Supposedly manual modification of bat file is necessary.
  • It does nothing.
  • And the suggestion turns out to be incorrect. Apparently it is necessary to install command line tools for android, because all the OpenJDK 8 thing is old news, and sdk actually comes with OpenJDK 11 now.
  • Of course SetupAndroid can’t find the new tools, and needs to be modified by hand.
  • It finally works! SetupAndroid.bat, that is.
  • But the engine still doesn’t see Android SDK.
  • I try to specify the sdk location by hand.
  • Lo and Behold, now I can see the device I’m using in SDK manager.
  • Except that I still can’t deploy onto it. It does not appear in “launch” option.
  • Found out that trying to launch requires using “Launch Manager”. Build process starts and…
  • And it fails. Of course. “ERROR: Platform Android is not a valid platform to build. Check that the SDK is installed properly.” No other details anywhere.
  • I try to use “Package project for android”.
  • The engine helpfully opens 404 page in unreal documentation when I attempt to do so . No other error messages. No other information.
  • Lots of confused people online with the same problem and no answer. Some suggest deleting “build” folders that does nothing.
  • After trial and error, turns out ndk path should be set to a specific ndk folder, and not the root with all ndks.
  • It builds now!
  • The build fails. Of course. “Installed Build Tools revision 31.0.0 is corrupted”
  • Online sources suggest reinstallation of build tools.
  • That doesn’t work.
  • Online sources suggest that iti s because a file called d8.bat should actually be called dx.bat. And that should be renamed. People online wonder why the heck they should be renaming files by hand in 2021.
  • That doesn’t work either.
  • Random youtube video suggest downgrading build tools from 31.0.0 to 30.x.x.
  • It now builds!
  • Google play protect (that never interfered with unity deploy) warns that the package is for old android and shouldn’t be installed. Pressing “Install anyway”
  • After the initial deploy the app seemingly freezes on Unreal splash screen.
  • I kill the app, redeploy…
  • It boots up to the UI! However, AR doesn’t work. Pressing “begin scanning” button does nothing. No error messages.
  • I poke around the configuration change things, and it finally startsd working somehow.
  • “Feels faster than on unity”.
  • Trying to bring in the model to unreal.
  • Unreal fails to find any textures. It did import base color, though. Very helpful, thanks.
  • No obvious equivalent of AssetPostProcessor. No response online regarding it either. So, configuration by hand it is.
  • I realize that the static mesh model has been exploded into individual components. Thanks, but, I uh, really needed that hierarchy?
  • Hierarchy can be created by “Import into Scene” and only this way will create a blueprint from fbx (why).
  • But it doesn’t let me import materials as material instances. (I really need the instances, thanks?)
  • Searching how to create instances from existing materials results in “Can’t be done”.
  • Which is a lie.
  • To replace material with instance both needs to be selected, and then you need to do “REplace references”. That works.
  • Yay to configurating 20 materials by hand.
  • The window for selecting texture is too small to fit entire texture name.
  • It finally functions!
  • Except that I can’t move imported hierarchy by its root node. For some reason.

Now, compare that to using ARFoundation in unity.

5 Likes
2

Yeah I wouldnt go for VR or AR in Unreal, nor would I go for mobile or web - My experiences match your own so far (mobile is laughable in unreal, like not even worth the time experimenting with it)!

Right now I find I am using unity for AR/VR, mobile and web work (so contract work, dayjob style stuff), and Unreal for console + pc game development (evening stuff).

Interestingly I have actually ended up organically with a weird pipeline where I will use unity to quickly prototype stuff (scripting in c#) and then when I am happy with my whitebox I “translate” that into unreal and then use unreal from there. I find I am faster in unity for prototyping, but cannot reach the same final level of quality and polish as when in unreal.

It sounds like a crazy time wasting workflow but it works really nicely for me :smile: I think as a unity focused developer that would probably be a nice workflow for you as well potentially, especially as once you have some written c# script its much easier to convert to blueprints, than say coding in blueprints from the beginning.

Anyway thanks for sharing!

2 Likes
3

Well, the way I see it, if it works, then it is not crazy.

Regarding mobile, I think that the AR demo had higher FPS on my device than AR samples from unity, it is just the initial configuration is horrible and extremely painful.

Regarding Unreal in general I’m sort of starting to see a theme there. Some features in Unreal engine as far as I can tell, are implemented in a way that is completely nuts. That probably happened due to historical reasons, mostly. Basically, they did something, it worked, and then remained unchanged, and it is highly likely they did it first, so there was no other approach to compare it to. OR something like that. I’m seeing this in couple of places. Like X-axis pointing forward, static mesh being exploded into separate parts upon import and so on.

3 Likes
4

Unity’s stuff is often slower for baffling reasons, I’ve tried doing apples to apples with Unreal, but I’m guessing these are lower level problems or their basic shaders are quicker than URP’s basic (forward) lit shaders. I don’t see how unless SRP is doing something silly.

Because on the surface when I was doing VR in UE4, it looks as though it would be slower than Unity, but on built test it’s faster (forward, everything disabled).

3 Likes
5

Unreal VR games, in my experience, sometimes have some weird input lag going on that doesn’t happen in unity. Even though the FPS might be smoother. Creed is a good example. I played a demo on Oculus Quest, and it was fine, but on PC/Virtual Desktop it was noticeably lagging (compared to, for example, Thrill of the Fight). On other hand, VR mode is easier to run in editor with Virtual Desktop.

6

I haven’t noticed any lag in my own tests. A game may have smoothing applied, or in the case of a game like Creed, the actions are actually not supposed to be instant… it’s the design of that game.

PC/Virtual Desktop: there is plenty of lag from compressing an image and decompressing it on the quest.

7

That is true, however in most games it is not very noticeable. Unreal games are an exception.

8

Mobile in UE4 is not for the faint of the heart, that’s for sure. There’s a baseline amount of tinkering required, and if you want the best results you should compile the engine from source, for both making the smallest possible builds by stripping away unused engine modules and for the occasional source modification (either your own or by merging one of the many useful pull requests floating on github).

When it works, you can get crazy good results, but the road to get there is long and rocky.

3 Likes