Re-indexing the Content Library? (Search issues)
Scavenger
Posts: 2,677
Search isn't working quite right in Content Library...it's not finding things I know are there.
Is there a way to have the library re-index itself or something?

Comments
Are you using the file search or the database search? That is set by the button to the right of the search text box - cylinder for database, folder for files.
That did it....what's the difference there?
The database search searches the CMS database, the file search searches the actual file names on disc.
But what does that mean? What is in the CMS databasE?
The CMS database includes any tags on the items -- Smart Content includes tags int he metadata, and you can add your own tags as well. For example, products might be tagged with the name of the vendor, the genre, etc.
I had the same problem, so I created my own solution with two independent indexes: one for quick searches by filename and folder, and a deep scan to inspect and index the contents of DUF files. I make the search tool available for free, and so far the feedback has been positive. Searching my database of about 337,000 assets takes around a second. It also supports logical operators, category filtering, and sorting. You can download the free version from www.prizmet.net. The search tool is still being tested by about 50 people as well as me; so far, there have been no problems, just a few minor fixes. Give it a try—maybe it will solve your problem.
Hello. Thanks for the URL to your web site but www.prizmet.net is incorrect; www.prizmat.net , though, take me to the PRIZMAT site. Is Trove Searcher Lite the free tool that you discuss?
I was surprised that text finds nothing but *text* does. Are Regex supported?
FYI 983,805 assets 40m 57s for the quick scan. I can't imagine how long the deep scan will take.
EDIT: Disturbed by "before scene analysis" since I didn't put any scenes in the quick scan. My scenes are not in the same place as my product library.
EDIT2: Found the text in the image. 10 days are left in the current "stage". TaskMgr shows very little disk activity. And no CPU core is above 60% at any time (the one below is the busiest). Very odd.
As a DEMO. That is important information that isn't made clear unless you visit the PRICING page on the website. There are missing features apparently. Why am I letting this thing run for 11 days? If it's just a DEMO?
Disappointing.
Hello,
First of all, let me clarify something important: Trove Searcher Lite is NOT a demo. It is a completely free edition of the software, with no expiration date, no trial countdown, and no obligation to purchase anything. You can continue using it indefinitely.
Yes, the commercial version of Trove Studio offers additional functionality. But having a free edition and a more advanced paid edition does not make the free one a demo.
That said, I agree that the differences between the two editions should be clearly explained on the download page, not just on the pricing page. I'll make sure to improve that.
Now, regarding your other comments, I'd like to clarify a few things.
First, I'm a little confused by your mention of 11 days. Trove Searcher Lite has only been publicly available for five days, so I'm not entirely sure what you mean.
Second, there is a significant difference between Quick Scan and Deep Scan.
Quick Scan is designed to be extremely fast. It indexes file names and folder paths without analyzing the contents of individual files.
For comparison, on my computer, indexing a library containing almost 380,000 assets takes just a little over one minute.
I've also received feedback from other users confirming that Quick Scan is generally a very fast process. Of course, performance depends on the size of the library and the computer's hardware, but since Quick Scan only processes file names and paths, I honestly can't imagine what would make it take an exceptionally long time under normal circumstances.
Deep Scan, on the other hand, is a much more sophisticated process.
It examines the contents of files throughout the library, including DUF files, and analyzes information that isn't visible from file names alone. It can even examine details such as the relative positions of bones in pose files to help Trove Studio in filtering poses process.
Naturally, processing hundreds of thousands of files at this level takes considerably longer.
However, even Deep Scan should certainly not take 11 days under normal circumstances.
There's also something important you should know: you don't have to wait for the entire scanning process to finish before using the program.
The database is updated incrementally as scanning progresses. Every indexed entry is saved, so closing the application does not erase the work already completed. You can stop the process and resume it later without starting everything from scratch.
Still - the software is functional even without deep scan.
Now, if your Quick Scan takes more than a few minutes, that is definitely unusual and something worth investigating.
I'm not dismissing the possibility that something has gone wrong on your computer. Quite the opposite — if you've encountered an issue that I haven't seen during my own testing, I'd really like to understand what's happening.
Could you please provide a little more information?
Approximately how many assets are in your library?
Are you referring to Quick Scan or Deep Scan?
Is your library stored on an SSD or a traditional hard drive?
Is the scanning progress still moving, or does it appear to be stuck?
A screenshot showing the current state of the application would also be very helpful.
I'm genuinely interested in identifying and fixing any problems users encounter. That's one of the main reasons I made the free version available in the first place.
Thanks for your feedback!
Best regards,
Arctoss
I have an i9/14000 with 128 GB of system RAM. The HDD (SATA) is a 10 TB toshiba. There is a single Daz library on the HDD. It is 3.3 TB. It used to be spread across two 2 TB SSD. But SSDs don't like being more than half full. And about two weeks after moving to the HDD, one of the SSDs died. It was only 19 months old.
The deep scan is being weird. It has been running since I downloaded the app. No system degredation. (Trove isn't even using 1 GB of system RAM according to TaskMgr.) It is very background. And the searches I've done (and bridge to DS) are lightning fast and decent. One issue is search terms are exact. I couldn't remember the name of a script (Shader Rotation). I searched for texture rotate and got nothing. I put texture rotate into the daz shop library search and the Shader Rotation product was the first result.
The time estimate has remained over a week in length. But sometimes it will say, for example, 239 hours left for 10+ hours. And other times it will steadily decrease. So however it's doing its estimate is sensitive to the directory structure in some where. (Much like old progress bars used to be really weird).
I have two other asset libraries (only 110 GB total) but I didn't tell Trove they exist.
I forgot to ask. Where do the indexes get stored and how do I get them off the C: dirive?
Why is there a file updating in the %LOCALAPPDATA%\TroveStudioSearcherLite\promotions every five minutes? You also should be explicit about it being Free With Ads.
Hi jmucchiello,
First of all, thank you for taking the time to provide such detailed information. This is extremely valuable feedback, especially since your library is considerably larger than mine!
And I'm genuinely happy to hear that searching and communication with DAZ Studio are lightning-fast, even with almost a million indexed assets. That was one of my main goals when developing Trove Studio: speed, simplicity, and making everyday work in DAZ Studio easier. I wanted a tool that would become a natural part of the workflow rather than another complicated application demanding attention.
I'm also pleased that Deep Scan isn't noticeably affecting your computer's performance. That was intentional. The idea is to let the scanner work quietly in the background while you continue using DAZ Studio normally.
Now, regarding scanning performance...
Your hardware is certainly powerful, but there's one particularly interesting detail: your library contains approximately 4.7 million files spread across more than 727,000 folders, all stored on a mechanical HDD.
That's an enormous number of files and directories!
With that kind of directory structure, disk access can become a significant bottleneck, especially during Deep Scan, which needs to examine the contents of individual files.
Unlike an SSD, a traditional hard drive has to physically move its read/write heads between locations. When processing hundreds of thousands of small files, this can make an enormous difference.
I suspect this explains at least part of the performance difference between your system and mine.
Of course, I don't want to assume that everything is working optimally simply because you're using an HDD. There may still be opportunities to improve the scanning process.
Regarding the estimated remaining time:
I've noticed that Deep Scan has progressed from approximately 29,000 to 155,000 processed assets between your screenshots. So it's definitely making progress.
However, the estimated completion time appears to be fluctuating quite significantly.
The files being analyzed can vary enormously in size and complexity, so processing times are not necessarily consistent. This makes estimating the remaining time difficult, particularly with such a large library.
I'll take another look at the estimation mechanism to see whether it can be improved or presented more clearly.
Fortunately, as I mentioned earlier, Deep Scan saves its progress incrementally. You can close Trove Searcher and return later without losing the completed work.
Now, about your search for "texture rotate" and the "Shader Rotation" product.
That's a very interesting observation.
At present, Trove Searcher's search mechanism is primarily designed around indexed file information and precise search terms. It doesn't yet provide full semantic search, where the program understands that words such as "texture" and "shader", or "rotate" and "rotation", may be related.
I've been thinking about introducing some form of semantic search or intelligent query expansion. Perhaps I'll be able to implement it in a future version.
The important thing will be finding a way to do that without sacrificing the speed and predictability of the current search engine.
There's also one possibility concerning your particular example.
It's possible — although I certainly can't promise this — that the incomplete Deep Scan is contributing to the problem.
Deep Scan collects additional information from inside DUF files, beyond what Quick Scan can discover from filenames and paths alone. Depending on what information is stored in the files associated with that product, the completed deep index might make it discoverable through additional search terms.
Unfortunately, I'm unable to verify this particular case on my own system.
If you could try the same search again after Deep Scan has finished and let me know whether the result changes, that would be extremely useful information.
And one more thing: you mentioned having two additional, much smaller asset libraries.
If you ever have an opportunity to test Quick Scan on one of those separately, preferably on an SSD, I'd be interested in the result. It could help us distinguish disk-related limitations from possible indexing inefficiencies. But please don't feel obliged to spend extra time testing this.
By the way, I have some news!
I've just made the beta version of the full Trove Studio available!
For now, access is by invitation only, as I'm gradually expanding the group of testers before the public beta release.
If you visit www.prizmat.net, you'll find information about the full version and can request beta access.
And honestly, given the thorough testing you've already done and the incredibly useful feedback you've provided, I'd be delighted to invite you to test the full Trove Studio!
Just send your beta request through the website. When it reaches me, I'll be happy to arrange access for you.
Trove Studio offers considerably more functionality than Trove Searcher Lite, and I'd be especially interested to hear how it performs with a library as large as yours.
One additional advantage: both editions use the same database, so you won't need to start indexing your library all over again when moving to the full version.
Once again, thank you for the detailed reports and for putting Trove Searcher through such a demanding real-world test!
A library of nearly one million assets is exactly the kind of challenge that can reveal things I might never encounter on my own computer.
And I'm particularly glad that, despite the lengthy indexing process, the actual searching and DAZ Studio integration are working so well for you.
Best regards,
Arctoss
Hi jmucchiello,
Thanks for bringing these questions up. They're perfectly reasonable, especially considering the size of your library!
First, regarding the database locations:
I've checked the code, and the default locations are:
Trove Searcher Lite:
%LOCALAPPDATA%\TroveStudioSearcherLiteTrove Studio (including the trial):
%LOCALAPPDATA%\DazSearcherThese directories contain several SQLite databases responsible for the Quick Index, Deep Scan results, pose analysis, folder indexing, and saved scanning progress. They also contain database backups.
Unfortunately, the current versions don't yet provide an option to change the database location through the interface.
However, there is a workaround using a Windows directory junction, which allows you to move the actual database files to another drive while keeping their original paths accessible to the application.
For example, to move the Lite database to drive D:
Close Trove Searcher Lite and DAZ Studio completely.
Make a backup of the entire
%LOCALAPPDATA%\TroveStudioSearcherLitefolder.Copy that folder to your preferred location, for example
D:\TroveStudioSearcherLite.Rename the original folder on C: to something like
TroveStudioSearcherLite_backup.Open Windows Command Prompt and run:
mklink /J "%LOCALAPPDATA%\TroveStudioSearcherLite" "D:\TroveStudioSearcherLite"This creates a junction at the original location, redirecting file access to the folder on drive D.
After confirming that Trove Searcher works correctly and recognizes your existing indexes, you can remove the old folder once you're satisfied that your backup is secure.
Importantly, this should preserve your existing Deep Scan progress without requiring you to index everything again.
The same principle applies to the full Trove Studio, although its default data directory is different.
Please be careful with this procedure, particularly while working with such a large database. I strongly recommend keeping a complete backup until you've verified that everything works.
I'll also consider adding a proper database-location setting to the application in a future update. With libraries as large as yours, I can certainly see why it would be useful!
Now, regarding the
promotionsdirectory:That folder belongs to the notification and promotional banner displayed at the bottom of the application.
The banner is designed to display software announcements, update notifications, and occasional information about other PRIZMAT products.
According to the current code, the application checks for banner updates every 15 minutes, not every five minutes.
There is no intentionally programmed five-minute file-writing interval for this mechanism.
If you're seeing a file modified every five minutes, it could be related to the internal WebView2 cache rather than the banner update schedule itself.
Could you tell me the exact filename you're seeing updated?
I'd be happy to investigate that behavior more closely.
And finally, regarding your comment about "Free With Ads":
That's a fair point.
Trove Searcher Lite is genuinely free, with no expiration date or obligation to purchase the full version. However, it does include a small informational and promotional banner.
It isn't a third-party advertising system. The banner is intended primarily for software announcements, updates, and occasional promotions related to the PRIZMAT ecosystem.
Nevertheless, I agree that its presence should be clearly explained on the download page.
I'll make sure that information is presented more explicitly.
Thanks again for your feedback. I appreciate how thoroughly you're examining the application — you're helping me identify not only technical issues but also things that could be better explained to users.
Best regards,
Arctoss
I'm running procmonl. Minutes go buy without TroveStudioSearcherLite.exe performing a ReadFile operation on the library or a WriteFile operation on its local database (in APPDATA). I hate SSDs. Twice I've had a machine just suddenly die because the C: SSD just died without warning.
What database are you using. Most have natural language search or fuzzy logic search capabilities.
I don't store important files on SSDs. And I wouldn't want to confuse the program by adding stuff and then deleting it later.
What does it mean when it says "left in stage"? Are there more stage of deep scan after this one?
I was planning to do this once it's done. Database on SSD drives, again, not something I like. And yes, I know linked directories, available in Unix since the 70s, give Windows fits sometimes.
I may have just looked at the wrong time. It's the state file that updates. %LOCALAPPDATA%\TroveStudioSearcherLite\promotions\state.json
This site can’t be reached
www.prizmat.net refused to connect.
Try:
ERR_CONNECTION_REFUSED
My phone going through the phone carrier can reach it. But my home service (Optimum.Net) gets the can't be reached error. And other Internet is fine on the desktop. I'm posting this on the desktop.