Re-indexing the Content Library? (Search issues)

Search isn't working quite right in Content Library...it's not finding things I know are there.

Is there a way to have the library re-index itself or something?

Comments

  • Richard HaseltineRichard Haseltine Posts: 111,992

    Are you using the file search or the database search? That is set by the button to the right of the search text box - cylinder for database, folder for files.

  • ScavengerScavenger Posts: 2,677

    That did it....what's the difference there?

  • Richard HaseltineRichard Haseltine Posts: 111,992

    The database search searches the CMS database, the file search searches the actual file names on disc.

  • ScavengerScavenger Posts: 2,677

    But what does that mean? What is in the CMS databasE?

  • fixmypcmikefixmypcmike Posts: 19,740

    The CMS database includes any tags on the items -- Smart Content includes tags int he metadata, and you can add your own tags as well.  For example, products might be tagged with the name of the vendor, the genre, etc.

  • ArctossArctoss Posts: 10

     I had the same problem, so I created my own solution with two independent indexes: one for quick searches by filename and folder, and a deep scan to inspect and index the contents of DUF files. I make the search tool available for free, and so far the feedback has been positive. Searching my database of about 337,000 assets takes around a second. It also supports logical operators, category filtering, and sorting. You can download the free version from www.prizmet.net. The search tool is still being tested by about 50 people as well as me; so far, there have been no problems, just a few minor fixes. Give it a try—maybe it will solve your problem. 

  • jjoynerjjoyner Posts: 882

    Arctoss said:

     I had the same problem, so I created my own solution with two independent indexes: one for quick searches by filename and folder, and a deep scan to inspect and index the contents of DUF files. I make the search tool available for free, and so far the feedback has been positive. Searching my database of about 337,000 assets takes around a second. It also supports logical operators, category filtering, and sorting. You can download the free version from www.prizmet.net. The search tool is still being tested by about 50 people as well as me; so far, there have been no problems, just a few minor fixes. Give it a try—maybe it will solve your problem. 

    Hello.  Thanks for the URL to your web site but  www.prizmet.net is incorrect;  www.prizmat.net , though, take me to the PRIZMAT site.  Is Trove Searcher Lite the free tool that you discuss? 

  • jmucchiellojmucchiello Posts: 2,625
    edited October 25

    Arctoss said:

     I had the same problem, so I created my own solution with two independent indexes: one for quick searches by filename and folder, and a deep scan to inspect and index the contents of DUF files. I make the search tool available for free, and so far the feedback has been positive. Searching my database of about 337,000 assets takes around a second. It also supports logical operators, category filtering, and sorting. You can download the free version from www.prizmet.net. The search tool is still being tested by about 50 people as well as me; so far, there have been no problems, just a few minor fixes. Give it a try—maybe it will solve your problem. 

    I was surprised that text finds nothing but *text* does. Are Regex supported?

    FYI 983,805 assets 40m 57s for the quick scan. I can't imagine how long the deep scan will take.

    EDIT: Disturbed by "before scene analysis" since I didn't put any scenes in the quick scan. My scenes are not in the same place as my product library.

    EDIT2: Found the text in the image. 10 days are left in the current "stage". TaskMgr shows very little disk activity. And no CPU core is above 60% at any time (the one below is the busiest). Very odd.

    Untitled.png
    783 x 131 - 20K
    Post edited by jmucchiello on
  • jmucchiellojmucchiello Posts: 2,625

    Arctoss said:

    I make the search tool available for free

    As a DEMO. That is important information that isn't made clear unless you visit the PRICING page on the website. There are missing features apparently. Why am I letting this thing run for 11 days? If it's just a DEMO?

    Disappointing.

  • ArctossArctoss Posts: 10

    jjoyner said:

    Arctoss said:

     I had the same problem, so I created my own solution with two independent indexes: one for quick searches by filename and folder, and a deep scan to inspect and index the contents of DUF files. I make the search tool available for free, and so far the feedback has been positive. Searching my database of about 337,000 assets takes around a second. It also supports logical operators, category filtering, and sorting. You can download the free version from www.prizmet.net. The search tool is still being tested by about 50 people as well as me; so far, there have been no problems, just a few minor fixes. Give it a try—maybe it will solve your problem. 

    Hello.  Thanks for the URL to your web site but  www.prizmet.net is incorrect;  www.prizmat.net , though, take me to the PRIZMAT site.  Is Trove Searcher Lite the free tool that you discuss? 

     

    Oops! My mistake — apparently I can write a search engine, but I can't type my own website address correctly! ;)

    Thank you for catching that. The correct address is indeed www.prizmat.net.

    And yes, Trove Searcher Lite is exactly the free tool I was referring to. You can download it directly from the website.

    The more advanced version, Trove Studio, is available as a trial (beta), but the Lite version is already available.

    Thanks again, and I hope you find it useful!

  • ArctossArctoss Posts: 10
    edited 5:07AM

    jmucchiello said:

    Arctoss said:

    I make the search tool available for free

    As a DEMO. That is important information that isn't made clear unless you visit the PRICING page on the website. There are missing features apparently. Why am I letting this thing run for 11 days? If it's just a DEMO?

    Disappointing.

    Hello,

    First of all, let me clarify something important: Trove Searcher Lite is NOT a demo. It is a completely free edition of the software, with no expiration date, no trial countdown, and no obligation to purchase anything. You can continue using it indefinitely.

    Yes, the commercial version of Trove Studio offers additional functionality. But having a free edition and a more advanced paid edition does not make the free one a demo.

    That said, I agree that the differences between the two editions should be clearly explained on the download page, not just on the pricing page. I'll make sure to improve that.

    Now, regarding your other comments, I'd like to clarify a few things.

    First, I'm a little confused by your mention of 11 days. Trove Searcher Lite has only been publicly available for five days, so I'm not entirely sure what you mean.

    Second, there is a significant difference between Quick Scan and Deep Scan.

    Quick Scan is designed to be extremely fast. It indexes file names and folder paths without analyzing the contents of individual files.

    For comparison, on my computer, indexing a library containing almost 380,000 assets takes just a little over one minute.

    I've also received feedback from other users confirming that Quick Scan is generally a very fast process. Of course, performance depends on the size of the library and the computer's hardware, but since Quick Scan only processes file names and paths, I honestly can't imagine what would make it take an exceptionally long time under normal circumstances.

    Deep Scan, on the other hand, is a much more sophisticated process.

    It examines the contents of files throughout the library, including DUF files, and analyzes information that isn't visible from file names alone. It can even examine details such as the relative positions of bones in pose files to help Trove Studio in filtering poses process.

    Naturally, processing hundreds of thousands of files at this level takes considerably longer.

    However, even Deep Scan should certainly not take 11 days under normal circumstances.

    There's also something important you should know: you don't have to wait for the entire scanning process to finish before using the program.

    The database is updated incrementally as scanning progresses. Every indexed entry is saved, so closing the application does not erase the work already completed. You can stop the process and resume it later without starting everything from scratch.

    Still - the software is functional even without deep scan.

    Now, if your Quick Scan takes more than a few minutes, that is definitely unusual and something worth investigating.

    I'm not dismissing the possibility that something has gone wrong on your computer. Quite the opposite — if you've encountered an issue that I haven't seen during my own testing, I'd really like to understand what's happening.

    Could you please provide a little more information?

    • Approximately how many assets are in your library?

    • Are you referring to Quick Scan or Deep Scan?

    • Is your library stored on an SSD or a traditional hard drive?

    • Is the scanning progress still moving, or does it appear to be stuck?

    A screenshot showing the current state of the application would also be very helpful.

    I'm genuinely interested in identifying and fixing any problems users encounter. That's one of the main reasons I made the free version available in the first place.

    Thanks for your feedback!

    Best regards,
    Arctoss

    Post edited by Arctoss at
  • ArctossArctoss Posts: 10

    jmucchiello said:

    Arctoss said:

     I had the same problem, so I created my own solution with two independent indexes: one for quick searches by filename and folder, and a deep scan to inspect and index the contents of DUF files. I make the search tool available for free, and so far the feedback has been positive. Searching my database of about 337,000 assets takes around a second. It also supports logical operators, category filtering, and sorting. You can download the free version from www.prizmet.net. The search tool is still being tested by about 50 people as well as me; so far, there have been no problems, just a few minor fixes. Give it a try—maybe it will solve your problem. 

    I was surprised that text finds nothing but *text* does. Are Regex supported?

    FYI 983,805 assets 40m 57s for the quick scan. I can't imagine how long the deep scan will take.

    EDIT: Disturbed by "before scene analysis" since I didn't put any scenes in the quick scan. My scenes are not in the same place as my product library.

    EDIT2: Found the text in the image. 10 days are left in the current "stage". TaskMgr shows very little disk activity. And no CPU core is above 60% at any time (the one below is the busiest). Very odd.




     

    Hi jmucchiello,

    Thank you for the additional information and screenshots. Now I understand what you meant by those 11 days!

    I initially thought you were referring to how long the program had already been running. I can now see that you're referring to the estimated time remaining, which is approximately 256 hours. My apologies for the misunderstanding.

    First, let me clarify one important point: Trove Searcher Lite is NOT a demo. It is a permanently free edition of the software, with no expiration date or trial countdown. The commercial version offers additional features, but everything included in Lite remains free to use indefinitely.

    That said, I agree that the differences between the editions should be explained more clearly on the download page, and I'll improve that.

    Now, regarding the performance you're seeing.

    Almost one million assets! That's a seriously impressive library, and I'm particularly interested in how Trove performs at that scale.

    However, I have to admit that your Quick Scan result of approximately 41 minutes is considerably slower than I would expect.

    For comparison, my library contains almost 380,000 assets, and Quick Scan takes just over a minute on my computer.

    Naturally, performance depends on hardware, storage, and the structure of the library, but your result suggests that something deserves further investigation.

    Deep Scan is a completely different matter.

    It is a much more sophisticated process that examines the internal structure of DUF files and extracts information that cannot be obtained from filenames alone. This includes details such as the relative positions of bones in pose files, which Trove Studio uses for advanced pose filtering.

    Processing hundreds of thousands of files at this level naturally takes considerably longer.

    For comparison, I've received a report from another user indicating an estimated Deep Scan time of approximately 14 hours for a library containing 480,000 assets.

    Of course, that's just one user's result, and processing times can vary significantly between computers. Nevertheless, your estimated 256 hours seems unusually high, and I'd definitely like to investigate why.

    There's also something I'd like to explain about why Deep Scan is included in the free edition at all.

    Originally, I intended to leave Deep Scan out of Trove Searcher Lite entirely and reserve that functionality for the full version.

    However, after considering it, I decided to include it because it significantly expands what the free version can do. Deep indexing provides much more detailed information about assets and opens up additional search possibilities.

    But here's the important part: Deep Scan is completely optional! ;)

    You don't have to enable it. If Quick Index provides everything you need, you can simply use the program without running Deep Scan at all.

    There is something else very important to understand about how Deep Scan works.

    I specifically designed it to operate in the background without interfering with normal work.

    I've received feedback from several users who report that the scanning process is practically unnoticeable while they're working in DAZ Studio or using Trove Searcher.

    This is intentional. Deep Scan isn't supposed to take over your computer or prevent you from doing anything else.

    And most importantly, you don't have to wait for Deep Scan to finish before using the program!

    The database is updated incrementally as scanning progresses. Every completed indexing operation is saved, so closing the application does not erase the results already collected. You can interrupt the scan and resume it later without starting everything from scratch.

    In other words, every hour you spend working with the program means more assets are being analyzed and added to the Deep Index.

    As more information becomes available, the search functionality becomes more powerful. Where detailed Deep Index data exists, the program is designed to prioritize it over the more basic information collected during Quick Scan.

    Trove Searcher remains fully functional even without a completed Deep Scan.

    There's another important point worth mentioning.

    Trove Searcher Lite and the full version of Trove Studio use exactly the same database format.

    The database can be moved, or its existing location can be specified in the application.

    This means that if you eventually decide to purchase the full version, you won't lose any of the indexing work already completed. All previously scanned assets and their indexed information remain available. There is no need to start scanning your entire library again.

    Now, there's one particularly important question I'd like to ask about your scanning times.

    Did you happen to start Quick Scan and Deep Scan at the same time?

    I'm asking because, with a library containing nearly a million assets, running both processes simultaneously could potentially create significant competition for disk access.

    Both processes need to access very large numbers of individual files. When two indexing operations are running at the same time, they may interfere with each other's performance, particularly when dealing with hundreds of thousands of small files.

    This is only a hypothesis at this point, but it's something worth checking.

    If you did run both processes simultaneously, I'll need to investigate whether it would be better to prevent Deep Scan from starting until the initial Quick Scan has finished.

    That might be a sensible improvement, especially for users with exceptionally large libraries.

    Regarding your comment about "before scene analysis", could you clarify exactly what concerns you about this message? I'd like to understand what you're seeing and what you think the program might be doing.

    One thing worth mentioning is that both Quick Scan and Deep Scan are designed to automatically detect and index new assets added to your configured content libraries in real time.

    With that in mind, could you tell me where you normally save your scenes? Do you save them inside one of your DAZ content library folders, or do you use a completely separate directory?

    Also, do you have any Auto Save functionality enabled that might be saving scene files automatically?

    I'm asking because I'd like to understand your setup before drawing any conclusions about what's happening.

    Regarding your question about searching and regular expressions:

    The * character acts as a wildcard, which explains why text and *text* can produce different results. I also need to verify the exact regular-expression support in the released version before giving you a definitive answer on that point.

    Now, I'd really appreciate a little more information about your computer:

    • What processor are you using?

    • How much RAM do you have?

    • Is your library stored on an SSD, HDD, or network drive?

    • Are all those assets located in one library or spread across multiple locations?

    I would also be interested to know whether the Deep Scan is progressing steadily, even if slowly.

    I'm not going to dismiss your results simply because the program performs well on my computer. Quite the opposite — this is precisely the kind of feedback I need.

    I've been testing Trove extensively, but there is no substitute for seeing how it behaves on other computers and with libraries of different sizes.

    And honestly, a library containing almost a million assets is an excellent opportunity to discover potential performance issues that simply wouldn't appear during testing with smaller collections.

    Thank you for taking the time to provide these details. I genuinely appreciate it!

    Best regards,
    Arctoss

  • jmucchiellojmucchiello Posts: 2,625
    edited 5:55AM

    Arctoss said:

    • What processor are you using?
    • How much RAM do you have?

    • Is your library stored on an SSD, HDD, or network drive?

    • Are all those assets located in one library or spread across multiple locations?

    I would also be interested to know whether the Deep Scan is progressing steadily, even if slowly.

    I'm not going to dismiss your results simply because the program performs well on my computer. Quite the opposite — this is precisely the kind of feedback I need.

    I have an i9/14000 with 128 GB of system RAM. The HDD (SATA) is a 10 TB toshiba. There is a single Daz library on the HDD. It is 3.3 TB. It used to be spread across two 2 TB SSD. But SSDs don't like being more than half full. And about two weeks after moving to the HDD, one of the SSDs died. It was only 19 months old.

    The deep scan is being weird. It has been running since I downloaded the app. No system degredation. (Trove isn't even using 1 GB of system RAM according to TaskMgr.) It is very background. And the searches I've done (and bridge to DS) are lightning fast and decent. One issue is search terms are exact. I couldn't remember the name of a script (Shader Rotation). I searched for texture rotate and got nothing. I put texture rotate into the daz shop library search and the Shader Rotation product was the first result.

    The time estimate has remained over a week in length. But sometimes it will say, for example, 239 hours left for 10+ hours. And other times it will steadily decrease. So however it's doing its estimate is sensitive to the directory structure in some where. (Much like old progress bars used to be really weird).

    I have two other asset libraries (only 110 GB total) but I didn't tell Trove they exist.

     

    Untitled.png
    794 x 161 - 19K
    Post edited by jmucchiello at
  • jmucchiellojmucchiello Posts: 2,625
    edited 8:48AM

    I forgot to ask. Where do the indexes get stored and how do I get them off the C: dirive?

    Why is there a file updating in the %LOCALAPPDATA%\TroveStudioSearcherLite\promotions every five minutes? You also should be explicit about it being Free With Ads.

    Post edited by jmucchiello at
  • ArctossArctoss Posts: 10

    jmucchiello said:

    Arctoss said:

    • What processor are you using?
    • How much RAM do you have?

    • Is your library stored on an SSD, HDD, or network drive?

    • Are all those assets located in one library or spread across multiple locations?

    I would also be interested to know whether the Deep Scan is progressing steadily, even if slowly.

    I'm not going to dismiss your results simply because the program performs well on my computer. Quite the opposite — this is precisely the kind of feedback I need.

    I have an i9/14000 with 128 GB of system RAM. The HDD (SATA) is a 10 TB toshiba. There is a single Daz library on the HDD. It is 3.3 TB. It used to be spread across two 2 TB SSD. But SSDs don't like being more than half full. And about two weeks after moving to the HDD, one of the SSDs died. It was only 19 months old.

    The deep scan is being weird. It has been running since I downloaded the app. No system degredation. (Trove isn't even using 1 GB of system RAM according to TaskMgr.) It is very background. And the searches I've done (and bridge to DS) are lightning fast and decent. One issue is search terms are exact. I couldn't remember the name of a script (Shader Rotation). I searched for texture rotate and got nothing. I put texture rotate into the daz shop library search and the Shader Rotation product was the first result.

    The time estimate has remained over a week in length. But sometimes it will say, for example, 239 hours left for 10+ hours. And other times it will steadily decrease. So however it's doing its estimate is sensitive to the directory structure in some where. (Much like old progress bars used to be really weird).

    I have two other asset libraries (only 110 GB total) but I didn't tell Trove they exist.

     

    Hi jmucchiello,

    First of all, thank you for taking the time to provide such detailed information. This is extremely valuable feedback, especially since your library is considerably larger than mine!

    And I'm genuinely happy to hear that searching and communication with DAZ Studio are lightning-fast, even with almost a million indexed assets. That was one of my main goals when developing Trove Studio: speed, simplicity, and making everyday work in DAZ Studio easier. I wanted a tool that would become a natural part of the workflow rather than another complicated application demanding attention.

    I'm also pleased that Deep Scan isn't noticeably affecting your computer's performance. That was intentional. The idea is to let the scanner work quietly in the background while you continue using DAZ Studio normally.

    Now, regarding scanning performance...

    Your hardware is certainly powerful, but there's one particularly interesting detail: your library contains approximately 4.7 million files spread across more than 727,000 folders, all stored on a mechanical HDD.

    That's an enormous number of files and directories!

    With that kind of directory structure, disk access can become a significant bottleneck, especially during Deep Scan, which needs to examine the contents of individual files.

    Unlike an SSD, a traditional hard drive has to physically move its read/write heads between locations. When processing hundreds of thousands of small files, this can make an enormous difference.

    I suspect this explains at least part of the performance difference between your system and mine.

    Of course, I don't want to assume that everything is working optimally simply because you're using an HDD. There may still be opportunities to improve the scanning process.

    Regarding the estimated remaining time:

    I've noticed that Deep Scan has progressed from approximately 29,000 to 155,000 processed assets between your screenshots. So it's definitely making progress.

    However, the estimated completion time appears to be fluctuating quite significantly.

    The files being analyzed can vary enormously in size and complexity, so processing times are not necessarily consistent. This makes estimating the remaining time difficult, particularly with such a large library.

    I'll take another look at the estimation mechanism to see whether it can be improved or presented more clearly.

    Fortunately, as I mentioned earlier, Deep Scan saves its progress incrementally. You can close Trove Searcher and return later without losing the completed work.

    Now, about your search for "texture rotate" and the "Shader Rotation" product.

    That's a very interesting observation.

    At present, Trove Searcher's search mechanism is primarily designed around indexed file information and precise search terms. It doesn't yet provide full semantic search, where the program understands that words such as "texture" and "shader", or "rotate" and "rotation", may be related.

    I've been thinking about introducing some form of semantic search or intelligent query expansion. Perhaps I'll be able to implement it in a future version.

    The important thing will be finding a way to do that without sacrificing the speed and predictability of the current search engine.

    There's also one possibility concerning your particular example.

    It's possible — although I certainly can't promise this — that the incomplete Deep Scan is contributing to the problem.

    Deep Scan collects additional information from inside DUF files, beyond what Quick Scan can discover from filenames and paths alone. Depending on what information is stored in the files associated with that product, the completed deep index might make it discoverable through additional search terms.

    Unfortunately, I'm unable to verify this particular case on my own system.

    If you could try the same search again after Deep Scan has finished and let me know whether the result changes, that would be extremely useful information.

    And one more thing: you mentioned having two additional, much smaller asset libraries.

    If you ever have an opportunity to test Quick Scan on one of those separately, preferably on an SSD, I'd be interested in the result. It could help us distinguish disk-related limitations from possible indexing inefficiencies. But please don't feel obliged to spend extra time testing this.

    By the way, I have some news!

    I've just made the beta version of the full Trove Studio available!

    For now, access is by invitation only, as I'm gradually expanding the group of testers before the public beta release.

    If you visit www.prizmat.net, you'll find information about the full version and can request beta access.

    And honestly, given the thorough testing you've already done and the incredibly useful feedback you've provided, I'd be delighted to invite you to test the full Trove Studio!

    Just send your beta request through the website. When it reaches me, I'll be happy to arrange access for you.

    Trove Studio offers considerably more functionality than Trove Searcher Lite, and I'd be especially interested to hear how it performs with a library as large as yours.

    One additional advantage: both editions use the same database, so you won't need to start indexing your library all over again when moving to the full version.

    Once again, thank you for the detailed reports and for putting Trove Searcher through such a demanding real-world test!

    A library of nearly one million assets is exactly the kind of challenge that can reveal things I might never encounter on my own computer.

    And I'm particularly glad that, despite the lengthy indexing process, the actual searching and DAZ Studio integration are working so well for you.

    Best regards,
    Arctoss

  • ArctossArctoss Posts: 10

    jmucchiello said:

    I forgot to ask. Where do the indexes get stored and how do I get them off the C: dirive?

    Why is there a file updating in the %LOCALAPPDATA%\TroveStudioSearcherLite\promotions every five minutes? You also should be explicit about it being Free With Ads.

    Hi jmucchiello,

    Thanks for bringing these questions up. They're perfectly reasonable, especially considering the size of your library!

    First, regarding the database locations:

    I've checked the code, and the default locations are:

    Trove Searcher Lite:
    %LOCALAPPDATA%\TroveStudioSearcherLite

    Trove Studio (including the trial):
    %LOCALAPPDATA%\DazSearcher

    These directories contain several SQLite databases responsible for the Quick Index, Deep Scan results, pose analysis, folder indexing, and saved scanning progress. They also contain database backups.

    Unfortunately, the current versions don't yet provide an option to change the database location through the interface.

    However, there is a workaround using a Windows directory junction, which allows you to move the actual database files to another drive while keeping their original paths accessible to the application.

    For example, to move the Lite database to drive D:

    1. Close Trove Searcher Lite and DAZ Studio completely.

    2. Make a backup of the entire %LOCALAPPDATA%\TroveStudioSearcherLite folder.

    3. Copy that folder to your preferred location, for example D:\TroveStudioSearcherLite.

    4. Rename the original folder on C: to something like TroveStudioSearcherLite_backup.

    5. Open Windows Command Prompt and run:

    mklink /J "%LOCALAPPDATA%\TroveStudioSearcherLite" "D:\TroveStudioSearcherLite"

    This creates a junction at the original location, redirecting file access to the folder on drive D.

    After confirming that Trove Searcher works correctly and recognizes your existing indexes, you can remove the old folder once you're satisfied that your backup is secure.

    Importantly, this should preserve your existing Deep Scan progress without requiring you to index everything again.

    The same principle applies to the full Trove Studio, although its default data directory is different.

    Please be careful with this procedure, particularly while working with such a large database. I strongly recommend keeping a complete backup until you've verified that everything works.

    I'll also consider adding a proper database-location setting to the application in a future update. With libraries as large as yours, I can certainly see why it would be useful!

    Now, regarding the promotions directory:

    That folder belongs to the notification and promotional banner displayed at the bottom of the application.

    The banner is designed to display software announcements, update notifications, and occasional information about other PRIZMAT products.

    According to the current code, the application checks for banner updates every 15 minutes, not every five minutes.

    There is no intentionally programmed five-minute file-writing interval for this mechanism.

    If you're seeing a file modified every five minutes, it could be related to the internal WebView2 cache rather than the banner update schedule itself.

    Could you tell me the exact filename you're seeing updated?

    I'd be happy to investigate that behavior more closely.

    And finally, regarding your comment about "Free With Ads":

    That's a fair point.

    Trove Searcher Lite is genuinely free, with no expiration date or obligation to purchase the full version. However, it does include a small informational and promotional banner.

    It isn't a third-party advertising system. The banner is intended primarily for software announcements, updates, and occasional promotions related to the PRIZMAT ecosystem.

    Nevertheless, I agree that its presence should be clearly explained on the download page.

    I'll make sure that information is presented more explicitly.

    Thanks again for your feedback. I appreciate how thoroughly you're examining the application — you're helping me identify not only technical issues but also things that could be better explained to users.

    Best regards,
    Arctoss

  • jmucchiellojmucchiello Posts: 2,625

    Arctoss said:

    Now, regarding scanning performance...

    Your hardware is certainly powerful, but there's one particularly interesting detail: your library contains approximately 4.7 million files spread across more than 727,000 folders, all stored on a mechanical HDD.

    That's an enormous number of files and directories!

    With that kind of directory structure, disk access can become a significant bottleneck, especially during Deep Scan, which needs to examine the contents of individual files.

    Unlike an SSD, a traditional hard drive has to physically move its read/write heads between locations. When processing hundreds of thousands of small files, this can make an enormous difference.

    I suspect this explains at least part of the performance difference between your system and mine.

    Of course, I don't want to assume that everything is working optimally simply because you're using an HDD. There may still be opportunities to improve the scanning process.

    I'm running procmonl. Minutes go buy without TroveStudioSearcherLite.exe performing a ReadFile operation on the library or a WriteFile operation on its local database (in APPDATA). I hate SSDs. Twice I've had a machine just suddenly die because the C: SSD just died without warning.

    Now, about your search for "texture rotate" and the "Shader Rotation" product.

    That's a very interesting observation.

    At present, Trove Searcher's search mechanism is primarily designed around indexed file information and precise search terms. It doesn't yet provide full semantic search, where the program understands that words such as "texture" and "shader", or "rotate" and "rotation", may be related.

    I've been thinking about introducing some form of semantic search or intelligent query expansion. Perhaps I'll be able to implement it in a future version.

    The important thing will be finding a way to do that without sacrificing the speed and predictability of the current search engine.

    What database are you using. Most have natural language search or fuzzy logic search capabilities.

    If you ever have an opportunity to test Quick Scan on one of those separately, preferably on an SSD, I'd be interested in the result. It could help us distinguish disk-related limitations from possible indexing inefficiencies. But please don't feel obliged to spend extra time testing this.

    I don't store important files on SSDs. And I wouldn't want to confuse the program by adding stuff and then deleting it later.

    What does it mean when it says "left in stage"? Are there more stage of deep scan after this one?

    Untitled5.png
    797 x 42 - 8K
  • jmucchiellojmucchiello Posts: 2,625

    Arctoss said:

    Thanks for bringing these questions up. They're perfectly reasonable, especially considering the size of your library!

    mklink /J "%LOCALAPPDATA%\TroveStudioSearcherLite" "D:\TroveStudioSearcherLite"

    I was planning to do this once it's done. Database on SSD drives, again, not something I like. And yes, I know linked directories, available in Unix since the 70s, give Windows fits sometimes.

    Now, regarding the promotions directory:

    If you're seeing a file modified every five minutes, it could be related to the internal WebView2 cache rather than the banner update schedule itself.

    I may have just looked at the wrong time. It's the state file that updates. %LOCALAPPDATA%\TroveStudioSearcherLite\promotions\state.json

  • jmucchiellojmucchiello Posts: 2,625
    edited 2:04PM

    Arctoss said:

    If you visit www.prizmat.net, you'll find information about the full version and can request beta access.

     

    This site can’t be reached

    www.prizmat.net refused to connect.

    Try:

    ERR_CONNECTION_REFUSED

     

    My phone going through the phone carrier can reach it. But my home service (Optimum.Net) gets the can't be reached error. And other Internet is fine on the desktop. I'm posting this on the desktop.

    Post edited by jmucchiello at
Sign In or Register to comment.