The Unity Search is great in its concept, but it frustrates me the hell out of it.
I don’t know if it is a bug, but if not and it was designed this way, there is seriously something wrong with it.
I am using 2022.3 and I have a really large project (hundreds of gigabytes and millions of assets) so I could benefit from Unity Search a lot. The problem is the search index, it keeps rebuilding constantly. I am under the impression that the search only works properly within one editor session after building the index. It takes around 2-3 hours to be built (i9 10940X, 980 PRO SSD). I’ve done it a couple of times hoping that it will stop rebuilding itself, but after restarting the editor I am getting this:
I’ve tried multiple approaches when it comes to the indexes themselves: I’ve tried options with one assets-only index, and then with different indexes that should, in theory, serve their purposes. The setup that I use currently should be pretty decent. I’ve excluded a lot of assets that I don’t care about, so the size of the index is a lot smaller than it could be.
But still, after a restart, I am getting building/combining of the index. It is cpu heavy so I can either kill it or wait for it to end (again, hours time).
So - how to deal with it? I obviously cannot submit this as a bug request, cause I am unable to submit my entire project that way.
Am I the only one experiencing this? Hopefully not. Please help. As I said before - this tool has great potential, but this flaw is a dealbreaker for me to this point that I would prefer to uninstall it completely.
p.s. I know that the first screenshot indicates “loading the index” but:
even if - the metadata about the index (size, number of elements) should be available immediately. I don’t think that there is a reason for the index to be fully loaded to retrieve this information.
what’s the difference between loading and building the index? I am seeing “Combining XXX” in the background tasks window for each index, but the title for each item says “Building XXX index”
if the index are being (re)built - why there is an active button that allows us to “build” it again (which I believe restart the process).
Thanks for the feedback. That is true if the index is already built but not loaded in memory, but if the index needs to be built that information is not available. We will see if we can fix it.
The wording in the Index Manager should be reviewed, because it should say “Building the index” when it is actually building. “Loading” means it is loading the index from disk into memory. “Building” means we are creating the index files, and it is done in several steps. “Combining” is one of those steps (but can also be done during an incremental update, more on this later), where we combine the indexing results of all assets into a single index file.
That is definitely a bug, it should be greyed out while building.
So, to answer the real issue here: no it shouldn’t rebuild the index every time Unity is open. The way it should work is the following:
Unity opened for the first time → no indexing is done unless Search is opened or requested through the Index Manager.
Indexing is done (by opening Search or Index Manager) → All indexes are built into the Library/Search folder.
Unity is close and reopened without doing any changes to assets, Search is opened → Indexes are loaded from the files, no indexing is done. When loading indexes, the background progress window should show tasks that are named “Loading XXX search index”.
Changes are done to some assets → An incremental update is triggered. An incremental update looks at the changed assets, re-index them, and the indexing artifacts are combined into the index file. When doing an incremental update, the background progress window should show tasks that are named “Updating XXX search index”. When only a small amount of assets are modified, it shouldn’t take long.
Can you look inside your Library/Search folder if you see files named “1feb1f8d72d9b68b91ec9ae4342c2a6f.32805.1.index” for example, or if they are destroyed when you quit the application or during a domain reload?
The best way to submit a bug report in this case would be if you could reproduce the issue with a small sample project, but I know it can sometimes be a pain to create a new project and try to reproduce the issue especially after having to deal with these annoying bugs where you are basically blocked for hours waiting for the indexes to be built. So if you prefer not to, no worries. We will investigate on our end as well.
Sorry you have such a miserable experience with our indexing solution. It is the first time we are aware of aushc a problem: indexing always kicking on when you start unity (or after domain reload). I understand that this is almost impossible to log a bug about this since you cannot attach a terabyte project to the bug.
I am unsure why this happens. Maybe it is because your project is so big that some “ints” variable are overflowing? One thing you could try to help us understand what is happening:
Remove all indexes in your project
Create a single index file into a folder without too many assets
Ensure that indexing is done completely
See that if restarting unity triggers indexing again.
This whole workflow is to help us understand if the problem is happening because of the size/numbers of assets indexed or if it is an external problem (some settings or assets in your projects playing badly with indexation).
The reason why I have 3 indexes now is because I’ve tried something different than one index like I used to have before. I have let the search build the index fully a couple of times but it didn’t stop the constant rebuild. I’ll try with index with a very narrow scope and see what happens.
I let the index to be built. In the Library/Search folder, there was a file like “8b984cac7626fd646adfdf889e4f7640.32805.f.index” with a size of 82MB.
I then closed the editor. The .index file was there.
I re-opened my project and the .index file was gone. I tried to capture a screenshot when it happen and it turns out it was even before editor opened. My best shot was:
At this stage, the .index file was already deleted. The “Search/sample.index” is the path of my “small” index of course. I don’t know if that was exactly the moment it got deleted, some things happened before quite quickly.
I don’t see anything relevant in the Editor.log but I am attaching it anyway. Editor.log is available here: Editor.log - Google Drive since the forums doesn’t allow to upload files with .log extension (which is quite weird).
UPDATE:
after the index was rebuilt once again I left the Unity without focus and I came back after a couple of minutes. When the editor regain focus it reimported “Search/sample.index” asset and the .asset file in Library/Search is again gone and the index is rebuilding.
What you describes is really strange. We will investigate further what can be causing this. We might have a bit more question for you. It is totally unexpected to have any index file in your Library folder be deleted. As soon as we have more info I will get back to you.
Are you using an Asset Server by any chance? All indexing artifacts should be managed by the Asset server. So we wonder if that server workflow could potentially delete the index file.
We will continue investigating and keep you up to date.
we are able to repro your issues locally. We will work on a fix for 23.X and for 22.3. Hoepfully it will land soon(ish). I hope you will have the badnwidth to updatge to the latest 22.3 when our fix will land.
In the meantime, we might have a workaround that we need to test a little more. We will keep you posted.
Holy cow! That’s really great to hear! Cause I really love the idea of the search and the possibilities that it brings. You guys gave me hope once again! Thank you so much. I am on 22.3 and do not use AssetServer if that still matters.
First of all - that’s great news and thank you for your support!
It seems that this workaround does have an effect in my case. This also makes sense since the initial building of the index takes quite a lot of time in my case (considering the number of assets) and I also think that entering playmode / domain reload might interrupt this process. Therefore there is a high chance that I never let it finish the first time.
But - does that mean that I should not create any custom indexes or anything until the bug fix lands?
Am I correct about the entering playmode / domain reload thing?
One more thing (although maybe I should post it in another thread).
As you said the index is loaded when the search is being opened. Would you consider adding an option to the Editor that would make it load at project startup? I kind of get the idea of this being default but in my case, I would totally prefer that when the project is opened everything is ready to go.
When the fix lands, you will be able to do everything you like: create multiple custom indexes anywhere in your project with various settings.
The workaround is only till the fixes lands. In this case you would only have a single index. That said, you can still tweak which folder you want to have indexed.
Some information about indexing:
The first pass of the indexing process is to index each assets in your project and create an artifact file for this asset in your Library folder.
This Indexing happens in another process called the AssetImporter. This means Domain reload doesn’t affect it (though if you close unity this will stop indexing).
When all assets have been indexed, we “Recombine” all those index fragments into a big binary blob file. This step happens in the main unity editor and is thus affected by Domain reload. But this step is fully threaded and generally quick (some seconds at most). Also, if for whatever reasons the recombining steps is not completed, there is no need to redinxed assets you just need to recombine all the index fragments.
The bug we fixed was major: it invalidated all indexed fragment each time you would start unity or trigger domain reload. Sorry about that.
We were ask about this feature. We might implement it
Our rational was that by default we didn’t want user to pay for things they do not use. So if you don’t plan to use the Search Window, no need to index anything.
For my case, the search index fail to build and keep rebuilding anytime I open search window just because in my project i have asset file name with unicode character. For example: điện thoại.mp3.
After exclude those files or folder which contains these unicode characters from search index, the seach index build success.
I’m having this issue in Unity 6.2 (6000.2.6f2) But it has existed for as long as I can remember. If I click ‘save’ it will trigger a re-index, or if I copy an object it will also trigger a re-index. It takes about 3 minutes to complete and is super annoying.
