Cannot Connect Any Profilers to Android Builds in Unity 2020 Consistently

Unity Versions Tried: 2020.1/2020.2
Mobile Device: Samsung Galaxy M20 running Android 10.0
Computer OS: Windows 10
USB Connection: Type C

Ever since Unity 2020.1, our project can’t consistently connect builds to the profiler. Unity 2019 was okay-ish (sorta) but Unity’s profiler connections overall have never been rock solid in any version compared to android logcat.
This is especially sad considering one of the changes in 2020.1 that got me excited.

I have been pulling my hair out over trying to get a profiler reading and data dumping via profiler API is NOT an option because our game’s flow is dynamic and I want to test it in realtime while playing through and get profiling data directly on the editor. The build turnaround time is already bad enough, having to add an extra layer of “dump data → copy → open raw file in the profiler” is going to completely kill the dev cycle with far too much turnaround and not enough work done to solve real problems.
We moved to 2020.2 specifically for the new profiler API that will let us show stats on builds directly but that will take a bit more time to create the HUD. I’d appreciate it if someone from Unity could direct me to the docs for this new 2020.2 profiler API because I’m having trouble finding it. (EDIT: Nevermind, found it — Unity - Manual: Rendering Profiler module )
We were also hoping that maybe the profiler connection issues will be resolved. Sadly it’s worse than ever for us.

I was able to get it to work once in a blue moon using the same steps provided below and I can’t recall which step it was but it is far less than consistent. It MIGHT work once every 3 days!!! And after working once if I need to build again, it stops working. This is just unacceptable for development. Even trying the same exact steps that makes it work all of a sudden won’t make it work again. It is completely random. I wouldn’t call this “Stable” and it shouldn’t be so hard just to profile an android build.

Firstly, here is what I’ve always done to connect the profiler:

  1. Set the “Run Device” from build settings to my USB connected Android phone. Hit “Build And Run” with “Development Build” and “Autoconnect Profiler” checked on.

  2. Launch the app and switch to “AndroidPlayer” on Profiler dropdown:
    6639637--757708--upload_2020-12-19_15-7-57.png

Unfortunately, “Build And Run” has a problem that is apparently “By Design”…
This is the resolution note on the issue. Basically using just “Build” and “Build & Run” differs because of BuildOptions.StrictMode preventing “Build & Run” from producing the APK if any errors or exceptions occur. However “Build” works just fine and we do not have time to go hunting down every little issue on builds as long as it succeeds.

This “By Design” issue persists on to 2020.2 so “Build & Run” is out of the picture, I have to use regular “Build”. Fine.

Now on to the main event.
If I could “Build” the android APK, launch it on phone and get the editor to magically find it, things would be okay enough to at least keep developing. But basically it’s next to impossible and works in extremely rare cases as mentioned above!

1. Upon reading through the docs on android profiling:

I came across “Android Remote Profiling”
I disabled mobile data on android device, connected PC + Android to same wifi. Disabled the network adapter for ethernet on PC to be safe. Install APK that was created using just “Build” on android and launch the profiler and try to connect using ADB/IP. Nothing worked. I checked the phone IP (192.0.168.100) which is always the same since no other devices connect to my home wifi (working remotely for our company).
It. Did. Not. Work.
Nothing, tried changing the sequence of the above to make sure I was doing it all the right way, it would never connect.

2. Next I tried ADB profiling. I even configured the adb tunnel manually on CMD using the following command:

adb forward tcp:34999 localabstract:Unity-{insert bundle identifier here}

I inserted our app’s bundle identifier in braces {} and the command worked fine. But even while being connected to USB there was absolutely no sign of profiling data.
Please note the following:
-USB Debugging is turned on
-Android Version 10.0
-Android drivers are installed, I can look at the device storage just fine
-The Build window can see the device as the “Run Device” just fine

3. Assuming it’s a firewall issue I tried the following as suggested by the docs:
Firewall:
I added two new Outbound Rules to Windows Defender firewall to make sure it wasn’t blocking anything. One was for TCP ports 54998-55511 and another was for the Unity 2020.2 application in general for any protocol and any remote port. I tried the first one alone and then added the second one to make sure I covered everything. Still didn’t work.
I also use Avira free edition but it’s for viruses only.
I also always allow the firewall popups that come up when installing Unity.

On profiler, sometimes when switching to “AndroidPlayer(ADB@127.0.0.1…)” I get:
“Connection is no longer valid. Calling auto disconnect.”
And it switches back to Playmode/Editor
I try using the IP address to connect and get the following:
“Failed to connect to player IP: 192.168.0.100:55000-55511,35000,4600,4600. Player did not respond”

I manually enter “192.168.0.100”. It won’t work. I also tried “192.168.0.100:5555” because 5555 is the port android logcat uses, nothing.

4. Android Logcat

I am mentioning android logcat here because it has a sort of SUPER connector where the moment I launch the window it automatically shows Connected in green without having to do a thing, similar to how the Build Settings is able to find the “Run Device” every single time on refresh. Both of these parts of the engine can detect the android device 100% rock solid without failure every single time as long as its connected to the USB.
So WHY can’t any of the profiler tools :
Why can’t Unity borrow a page from the android logcat package and implement something similar for the rest of the profiler suite (Profiler, Console, Frame Debugger, Memory Profiler) so that connections are rock solid every time like android logcat??? Even logcat is able to show realtime graph of heap size/heap alloc in addition to console entries. Disconnect/Reconnect is super straight forward at the press of a button.
And by clicking “Other connection options” on the device selection dropdown on logcat, I can see the IP of the phone + port that it is using in addition to android version, abi, sdk, id, phone model number, EVERYTHING basically. If android logcat is discovering the device so splendidly and is able to gather so much info + show realtime data from the phone + connect with such ease, why on earth is there so much connection difficulty with the more important profiling tools?

It is like instant! I disconnect USB and it shows Disconnected in red, I reconnect USB and don’t have to do anything, it just finds it and starts showing data!
If the rest of the tools could connect as easily as android logcat, gathering performance data would be super easy and development headache would decrease by a huge margin!

In a perfect world we’d have android logcat level connection to profiler + bluestacks level android device emulation directly in the engine so there would be less “device-only” bugs that constantly require us to make builds and test to see what is happening on device and with profiler tool connections being this poor, we are left completely blind making mobile development just completely SUCK :frowning:

Please if someone from Unity could help out we’d highly appreciate it! Also to know if you guys are working on stabilizing these connections the right way like logcat. Without profiling we should might as well just give up mobile development!
We BADLY need android logcat-like connection in the profilers! Please guys lets make this happen!

At the very least please tell me what else we can try or if amongst the above there is one sure fire way to do it every time with an extra step for stability not mentioned in docs???

6 Likes

Hi there nd apologies for the persistence of connection issues. While we solved some of these in 2020.1, this has been a continuous effort that is still on going and will cary on into the next year. Rest assured that we won’t abandon this topic until is solved for good. That said it is rather varied, we have multiple distinct bugs centered around connection issues and have been landing more fixes for them over the past months too, with backports, so keep to the newest versions of whatever major version you are on.

That said the fixes that landed in 2020.1 and one of the bigger chunks of fixes that is currently in flight address the base PlayerConnection connectivity more so than the Profiler specifics (though we are fixing those too). This latest batch, currently held back by Switch issues, will address connection drops, reconnection issues after suspend & resume and zombie connections preventing connections to be established. It doesn’t particularly sound like what you are facing, unless this would be maybe down to zombie connections? More likely it could be some Autoconnect issues. Have you tried turning that off? If it doesn’t succeed automatically, it’s more likely to hinder the establishing of a connection until our fixes for that land.

I’m not 100% on top of which variations of such bugs my colleagues fixed recently and with he holidays, response times might be slowed a bit. But if you’d be able to take the time to file a bug report for this, any additional info here helps us in addressing every facet of this issue. If you ping the issue ID in here I can help it along.

Meanwhile, beyond the Autoconnect suspicion, a weird bunch tells me that you might want to test this with an Editor without the logcat package? Maybe they get in each other’s way?

Note: Documentation fixes are underway but won’t be released until the new year, BUT, that page currently erroneously links to the old Recorder API, which won’t work with the new counters, you’ll want to be using he ProfilerRecorder API for that.

Also, there is a video on this topic, too.

2 Likes

Wait, the reason it doesn’t work for you is an error or exception getting logged. I can’t believe that that log would be By Design. Have you reported a bug on whatever is causing this error? (Additionally to the profiler connection issue)

1 Like

Thanks a lot for the quick reply! I really appreciate it :smile: I am going to try out your suggestions and log the bug reports too!

I understand you guys need to make all the profilers work across the full range of devices you support. But I was just wondering why android logcat is so good at doing what it does while the profiling tools suffer? I will definitely try without the package installed, I had a similar theory but I only disconnected and closed the logcat window.

If at all possible, would I be asking for too much if I asked you to forward the idea that for the profiling tool’s android connection specifically, android logcat’s implementation is observed to see if it could be replicated for all the profilers?
I am going on a limb here and clueless with regards to the technicalities behind the suggestion.
But I’d make a guess and say perhaps android takes a majority share of the profiling tool usage seeing as nearly everyone has one. Even if it meant that the tools worked best with android compared to the other platforms as a result of the convergence of the core connectivity tech of android logcat to make it happen.

I will make another build using “build and run” and go over the errors in detail and log them. I really hope you guys ship out the latest fixes soon and profiler connection issues become ancient history :smile: We’ll stay updated with the 2020.2.xf1 lifecycle.
I’ll reply back here if I discover something with your suggestions! Thanks again!

Android connections in particular is focus of ours. I’m currently not sure how logcat implements the connection but wouldn’t be surprised if it was using PlayerConnection and the ProfilerConnection layer on top is what is causing your issues…

1 Like

Sorry for the late response, got carried away with the new profiler recorder API, it is magnificent. One person already suggested I guess to add static constants in place of hard coded string usage but as a first release this is awesome!

Unfortunately amongst all the profiler tools the one I need most right now is the frame debugger because since 2020.2 there’s a fighter model missing in a scene but exists in the editor. I tried RenderDoc but it was confusing.

The “Build and Run” error is apparently related to a shader that can’t find an include .cginc file. I’ll try and handle it but it still feels strange that this error doesn’t stop “Build” from working correctly.

Because if Build and Run fails, the turnaround is Build → Copy manually to Android over USB → Delete existing app (otherwise install fails for some reason) → Install new copied APK → Launch App → Troubleshoot

with Build and Run it’s simply:
Build → Troubleshoot (since App auto launches)

Not sure if u guys would be open to exposing a setting to the user to change the strict error fail build option that is internal to “Build and Run” while “Build” is using the legacy version as mentioned here:

Rethinking just that even if it was originally “by design” will mean
This:
Build → Copy manually to Android over USB → Delete existing app → Install new copied APK → Launch App → Troubleshoot
Becoming Just This:
Build → Troubleshoot

then strictly speaking if we can’t “build and run” we shouldn’t be able to “build” either
Sorry for getting off topic, just thought it would be helpful suggestion :slight_smile:

The build succeeded after clearing shader error.

But switching to AndroidPlayer even though the editor installed + started the app on device successfully does absolutely nothing

One last thing I will try is to remove the android logcat package and see if it changes anything after restarting editor + rebuilding with build&run

Okay finally I am able to run but it meant losing android logcat, here is what I did:

  1. Remove the android logcat package (1.2.0)
  2. Close all Unity Editor instances ( I usually run two to multitask on separate branches of same project copy while one builds - both were running Unity 2020.2.0f1 )
  3. Delete app from phone
  4. Connect phone to PC via USB
  5. Open just one editor
  6. Hit “Build and Run”

I am not if it was because of the multiple editors being opened or because of android logcat but I’ll try to get to the bottom of this.
Android logcat wasn’t even running though so it’s strange that a package could be installed and affect profiler connections without even running.
Not sure if it was the duplicate editor windows either because if one of them was “stealing” profiler connection data, one of them would have had to show something, but neither did.

Yes, the multiple Editor instances might’ve been the actual issue, confusing the build as to what editor to connect to, which would still be an Autoconnect or ProfilerConnection error (was Autoconnect still on? having to manually choose the connection sounds like it wasn’t or that it still failed?). Could you try again with logcat added and just one editor open, just to eliminate that possiblity? Either way this turns out, a bug report on this with these details would be hugely appreciated :slight_smile:

Also, glad to hear you found a resolution to the shader issue and apparently at least a workaround for profiling, even if that comes with a possible steep premium (I.e. loosing logcat).

Also I’m not enough in the know on he build system and what non-builderror-errors it could throw that should legitimately stop a build. I can forward that feedback though.

Autoconnect Profiler was checked on for all builds (including build and run)

So the weird news is:
The autoconnection appears to be razer sharp now, like as instant as I turn the app on before even the splash screen it connects even if it was set to “Editor” or “Playmode” it switches on its own.

Then I introduced android logcat back in, this time version 1.1.1 which is verified instead of 1.2.0. I kept logcat connected and the autoconnection was still super fast.
Next I upgraded to logcat back to 1.2.0, still kept going strong, okay.

Next I turned back on the other editor I use for multitasking, even then it kept getting connected to the first editor correctly, still instantly.

Then I tried to do another build and run to see if it may fail and still no difference :eyes::eyes::eyes: wut???

I cannot for the life of me reproduce what was going wrong the first time with solid steps. It’s like if the other day my one problem was that I couldn’t get it to connect at all even once, right now it’s like the opposite, I can’t get it to stop connecting!

I recall even seeing the following the other day whenever I tried to connect manually. However today I am seeing it when I close the application from memory (not switch apps or go to home screen, down right shut down the app and only then I see this)
So for what it’s worth perhaps when I was trying to tap “AndroidPlayers(ADB…)” from the dropdown, whatever gets called to throw this error on disconnect when the app quits completely was also getting called just from switching to the profiler dropdown to get it to connect.
6643462--758467--upload_2020-12-21_2-27-23.png

For sanity’s sake I just made a build with just “Build” to see if that has any affect because for one thing I wasn’t using build and run while facing those problems. Autoconnect is still going strong.

For anyone reading this, I’d say the only changes I did were:
-Fix the shader error (although this couldn’t possibly have made a difference because it was building fine and I did get to see profiler data show up albeit very rarely)
-Add the firewall outbound rules as per doc suggestion
6643462--758476--upload_2020-12-21_2-38-48.png
-The main difference from the other day with the issues would be that I have restarted the computer and the editors. That’s all I can pin it on. Perhaps the restart helped clear up any firewall/port issues.

So that would be the takeaway =.= add the outbound rules and restart computer.
Multiple editors/android logcat/using just “build” - seem to be not affecting it now.

Btw I hope I added the outbound rules correctly? The docs only specified the port 54998-55511 but I added a second rule for the editor itself which perhaps was too much. Please let me know! And thanks a lot for all of the help :slight_smile:

I see, I’d highly appreciate it if you could forward this along. I have faced this issue on numerous occasions for random types of errors and it isn’t always feasible to fix the errors to continue developing with the fast turnaround of “build & run”. So I’d highly recommend having at least the developer decide if the build option should be strict/quit on any error if we choose build and run Vs build. The two should work identically and maybe a toggle to have the strict mode for those who want to be made aware of such issues.

Just made a discovery
Apparently it has nothing to do with android logcat nor multiple editors. I just closed all editors and reopened just one from which the build was made that worked yesterday and connected it

The moment I open the app I can see “Autoconnected Player”
But as u can see there are no logs or profile data, nothing.

I think this would be counted as a legitimate bug.
I theorized that perhaps this was by design since when the build is made it enters the subnet values (I think I read on a forum) and so perhaps after a computer restart the values on the editor changes so that’s why it can’t connect. However the theory was disproven by the fact that it was able to show “Autoconnected Player” on its own the moment I launched meaning the editor had a sense that the app was indeed turned on and phone is connected but it made no difference since it can’t read the data even if it knows its connected.

To test my theory I just made a new build using “build and run” and sure enough I got back that blazing fast autoconnect.

So my question is, is this a known issue or should I file a bug report that the editor may misbehave with connections from existing builds after a restart?

1 Like

Yes, please do file a bug report for this. The values stored should help connect to the right machine and editor automatically but not get in the way of connecting non automatically and at a very minimum we want better error messaging on this and ideally better handling. Having a bug on this would help us greatly via the provided hardware stats and project and just generally tracking this facet of the connection issues :slight_smile:

1 Like

This whole conversation was really helpful. Thank you.

I am getting the same “Connection is no longer valid. Calling auto disconnect.” after trying a build yesterday and getting some errors which I am working on fixing and then I close unity and re-open, 2/3 of the original errors resolved, this new one appeared and a really nasty one is still there.

Was reported as a bug?

2 Likes

I don’t think so. Could you please report it as a bug?

I spent some time debugging this today on 2020.3.17f1. I noticed what I think are some bugs:

  1. When using Android + .obb + Patch/Run, the player won’t connect.

If you look at the player logs, you’ll see that it’s reading the connection port from the obb file. But in Patch/Run mode, the obb file is not re-deployed. It seems like this causes the player to use a stale port number when trying to connect and it fails.

This issue is resolved by making a Full rebuild, which is obviously not ideal.

  1. I noticed some really strange code in the TestRunner package. See AndroidPlatformSetup.cs and PlayerLauncher.cs:130.

It seems that Android doesn’t properly support connecting via BuildOptions.ConnectToHost. Instead they use an internal proxy connection method. I’ve had success using this same code via reflection, when connecting on Android.

I’m not sure if this second point is necessary, but whoever wrote the test runner seems to think it is. If anything, it should at least be documented somehow.

I’ve wrote this up on the Uninomicon: EditorConnection [Uninomicon]

2 Likes

Im getting this bug too on Unity 2020.3.18f1

1 Like

I still have this problem with connection. Unity 2020.3.16f1/
I have found a bug on the tracker:
https://issuetracker.unity3d.com/issues/android-connection-is-no-longer-valid-calling-auto-disconnect-dot-error-after-tests-ran
But there are not any information about 2020.3.
Could you clarify any information on this bug and when is the update planned to resolve the issue?7519937--927782--upload_2021-9-24_18-9-20.png


And also I cannot connect via IP

Build is Development and Autoconnect is enable.

2 Likes

That one was supposed to be fixed with a bunch of connection related changes that landed on 2021.2 but magically remanifested in a slightly different but still related way, so it didn’t get verified but reopened. The 2020.2 not in it is misleading however, I’ll see if I can remedy that.

Also afaik a good chunk of those fixes was sadly too involved and risky to backport them (yet?).

is there no way to connect & profile to a development build that was already built? You have to use “Build & Run” each time on Android?

edit also related to this, Unity 2021.1.28f1 will get stuck on “application.shutdown.cleanupengine” forever, when trying to close the Editor if it had attempted to connect to an android device in the profiler.