I encounter a strange issue that:
I will regenerate the path points, and call PolygonCollider2D.SetPath() for my complex shapes.
Most time it works well, but sometimes, the Unity stuck with no response, I debug it with VS Studio and get the main thread stack, as below:
I also bought and installed the Panic Button plugin from asset store, and it can not abort any C# script, so I guess there maybe dead loops in tessMeshMergeConvexFaces ?
BTW, I’m using Unity 5.6.0f3 Personal. Both PC and Mac version has same issue.
We have a bug report for this and will get to it ASAP. What I can you is that it’s a bug somewhere in the 3rd party library libtess2 which is used here to triangulate the paths. It’s getting stuck in an infinite loop and sometimes even crashes. Narrowing this down is the “fun” part as so far it’s only been seen on super complex paths.
Hi, thanks for reply, how can I follow this bug or what can I help? It is too important to my game.
My game is heavily used PolygonCollider2D, the path is really complex, and I have spend long time to simplify the path, right now the issue is not the path point count, you know, most time it runs well, but about every 100 times, it occurs that infinite loop, make my game dead. It is really not “fun”, it make me crazy.
If you could create a super simple reproduction project and create a bug report and give me the case number it might help track down the issue.
The only other thing you could try is firing your paths to the libtess2 yourself in a test project but I understand if you’re reluctant or don’t have time for that.
We effectively do this:
// Create tesselator.
TESStesselator* decomposition = tessNewTess(NULL);
// Add as many paths as you like.
tessAddContour(decomposition, 2, points, sizeof(Vector2), pointsInPath);
// Do the tesselation.
const int tessError = tessTesselate(decomposition, TESS_WINDING_ODD, TESS_POLYGONS, b2_maxPolygonVertices (8), 2, NULL);
The problem seem to come when tessTesselate is called. Right now we have a case of a crash
Here’s the existing case: Unity Issue Tracker - Unity crashes in <tessMeshAddEdgeVertex>, when 2D colliders collide
Good information, I will try to reproduction the issue path, and maybe have a try to debug it directly with libtess2, but i’m not familiar with C/C++, however, for saving my game i will definitely try. 
And of course hope you Unity engineer will have time to focus on this, i have searched the internet, there’s many developers has encountered this issue.
Finally I reproduction it, please check the attached simple project.
For PolyIssueTest.cs
//Note, change the value to 93 or 94, you'll make Unity hanging
private int pathFile = 2;
There’s 3 file stored path points in Assets/data/ folder, the file 2 is fine, you’ll see polygon collider in scene view, if you change to load 93 or 94 (the path will be logged at starting), it will make Unity hanging.
Hope that helps.
BTW, I have reported this bug: https://fogbugz.unity3d.com/default.asp?905608_hospnbkdcvseio2p
Please note that:
The path value in file “2” is fine, and the value in “93” and “94” will cause Unity hanging.
So if you want to see the issue, change the pathFile value to 93 or 94 in the source code, then run.
Thanks, good timing. I have just fixed the crash case. I will see if it solves your issue too.
FYI: I have reported the bug in libtess2 here: tessTesselate infinite loop · Issue #23 · memononen/libtess2 · GitHub
That’s great, thanks for such quick done, I’ll follow it.
Is there a way to replace libtess2 in my Unity or should I wait for next Unity update?
You cannot replace it, it’s part of the native engine.
I’ve got a fix for both the crash and infinite loop bug cases. libtess2 is also used by the the sprite mesh generate so it needs a bunch of testing first before it lands in a release. I’ll try to get that out ASAP.
The fixes have just landed in 5.5.3p3 and 5.6.0p4.
Cool, great fast work, I’ll test it soon in my game. Thank you MelvMay!
Hi there is not 5.6.0p4 released yet, i just saw the newest is p3 here Download Archive
Yes, I never said they were released yet just that they have landed in those versions. I believe the scheduled release for 5.5.3p3 and 5.6.0p4 is Wed 3rd May (tommorrow) or thereabouts.
Thank you, good information, I have watched the Unity Patch Release Thread.
PS, i saw physics2d is improving fast, appreciate the good works, seems my game has selected the best proper engine.