Loading...
> >

How long does it take you to load G8F?

12346

Comments

  • takezo_3001takezo_3001 Posts: 2,023
    edited July 2021

    My Studio is installed on an SSD, yet my content folder is on an HDD at 5400rpm so about 49.5 seconds... and she's absolutely packed with morphs, both bought and custom made.

    Post edited by takezo_3001 on
  • default1506default1506 Posts: 0

    Here's a script I use to load zipped packages. Whenever there's too many morphs, I just delete my library and run this script to reload the packages I need. Hope this helps someone.

    https://gist.github.com/DozyBird/c3990d605069cf5c0e6e044ffc75a714

  • Richard HaseltineRichard Haseltine Posts: 107,932

    dozybird said:

    Here's a script I use to load zipped packages. Whenever there's too many morphs, I just delete my library and run this script to reload the packages I need. Hope this helps someone.

    https://gist.github.com/DozyBird/c3990d605069cf5c0e6e044ffc75a714

    That has a lot of hard-coded paths that will not work for many users.

  • SeraSera Posts: 1,675
    Don't know if anyone has posted this here yet, but this script should help with load times: https://www.daz3d.com/forums/discussion/505201/long-load-times-this-may-help-morphs-manager
  • TBorNotTBorNot Posts: 374

    The base G8F figure loads in about 3 seconds on a Mac mini Intel Version, Intel integrated graphics.  The number of morphs has nothing to do with load times, it has to do with a bug in the fitting operation which runs (somewhat) asynchronously.  If you get two overlapping objects that need to fit, it could sit there and spin for a very long time as they argue with each other.  Keep your objects parented, but make them fit to NONE, then save as a Scene Subset.  It reloads instantly.  Sure, it's a pain to fit everything again, but it's a lot faster than waiting for a half hour! *

    *Note, there is a bug in shells, so you will need to delete things like hair shells before saving because it doesn't come back nice (all white or other such nonsense, especially on genitals), but, hey, it's a bug.

     

  • algovincianalgovincian Posts: 2,664

    TBorNot said:

    The base G8F figure loads in about 3 seconds on a Mac mini Intel Version, Intel integrated graphics.  The number of morphs has nothing to do with load times, it has to do with a bug in the fitting operation which runs (somewhat) asynchronously.  If you get two overlapping objects that need to fit, it could sit there and spin for a very long time as they argue with each other.  Keep your objects parented, but make them fit to NONE, then save as a Scene Subset.  It reloads instantly.  Sure, it's a pain to fit everything again, but it's a lot faster than waiting for a half hour! *

    *Note, there is a bug in shells, so you will need to delete things like hair shells before saving because it doesn't come back nice (all white or other such nonsense, especially on genitals), but, hey, it's a bug.

    How many g8f figures do you have? I'll bet you do not have a large library. What we're talking about has nothing to do with fitting.

    - Greg

  • AllenArtAllenArt Posts: 7,175

    thenoobducky said:

    AllenArt said:

    1:30:36 in DS 4.15. Smart content says I have 167 characters for G8F installed, including the HDs. Characters are installed on an internal, mechanical HD - probably 5400 rpm

    I have 26 DO morphs installed in addition to the base. I'm uncertain how many PA characters.

    My computer specs are 2-2.8ghz Xeon procs and 64 gigs of ram with 6 gigs of video. Windows 8.1.1

    1h 30min? that sounds horrible. Although if 2.8Ghz is the max boost clock, that is a very slow CPU，and Daz is very much dependent on single-core performance.

    To be honest, I wouldn't hold much hope in the loading performance improving with moving to the new framework. Daz really need to rework the way figure loading process is done but probably won't have any programmer time to do it given they are focusing on porting to the new framework.

    Not one hour and 35 minutes...lol. One minute, 30 seconds and the 36 the stop watch tacked onto the end. LOL Sorry about that :P

  • TheMysteryIsThePointTheMysteryIsThePoint Posts: 3,218

    13 seconds.

    But I'm sure I have but a small fraction of the content library some of you others have. It's that totally not bad architecture at work, you see.

  • GhostofMacbethGhostofMacbeth Posts: 1,734

    Twelve seconds

  • davidtriunedavidtriune Posts: 452
    edited August 2021

    eleven seconds

     

     

    jk, 1 min. that's with a crapload of morphs but on a decent computer using an HDD

    but, i can't pose anything.

    Post edited by davidtriune on
  • PerttiAPerttiA Posts: 10,024

    Is crapload 1000, 10000 or 50000 morphs?

  • davidtriunedavidtriune Posts: 452

    PerttiA said:

    Is crapload 1000, 10000 or 50000 morphs?

    I checked and theres 20k files in my g8f morphs folder

  • thenoobduckythenoobducky Posts: 68

    davidtriune said:

    PerttiA said:

    Is crapload 1000, 10000 or 50000 morphs?

    I checked and theres 20k files in my g8f morphs folder

    You see, not all morphs are made equally. Some morph files can be worse than 100 files with simple morphs. Daz cache file size would be a better metric in my opinion. It is just harder to get people to go find it by going to daz studio->preferences->General->dson cache files and go find the g8f file size.

  • MoogooMoogoo Posts: 136

    G8F around 12 - 14 seconds and G8M 5 - 6, my laptop is getting old and needs a reinstal but uses SSD which I guess is the big time saver.

  • RL_MediaRL_Media Posts: 339

    I remember there was one misspelling on one morph file that like tripled my load time. I forget what it was, but it was a typo of some kind, which made it search forever for a nonexistant file being linked.

  • pjwhoopie4801218pjwhoopie4801218 Posts: 794
     

    You see, not all morphs are made equally. Some morph files can be worse than 100 files with simple morphs. Daz cache file size would be a better metric in my opinion. It is just harder to get people to go find it by going to daz studio->preferences->General->dson cache files and go find the g8f file size.

    On my machine, the size of that data folder is 1.48 gb

  • jbowlerjbowler Posts: 841
    edited September 2021

    Chumly said:

     

    You see, not all morphs are made equally. Some morph files can be worse than 100 files with simple morphs. Daz cache file size would be a better metric in my opinion. It is just harder to get people to go find it by going to daz studio->preferences->General->dson cache files and go find the g8f file size.

    On my machine, the size of that data folder is 1.48 gb

    The file in question on Windows is almost invariably "%AppData%/DAZ 3D/dson/cache/data/Daz 3D/Genesis 8/Female.cache.dsf" because most people don't change the Preferences/General/DSON Cache Files :/Folder : value in DAZ Studio.  The size in bytes of that file is directly related to my load times and is also controlled by the number of product files that contain morphs/poses/etc that can be applied to G8F.

    The latest log messages from 4.15.0.30 imply that approaching three quarters of the load time on my system/content goes in DAZ Studio doing something which the dev who wrote the log messages calls "Creating sum stage formulas...", but there is no break down of the times beyond that.  The only other substantial time sync is "Scanning for addons...".  These steps get logged twice when loading the basic G8F, once for the G8F figure and a second time for the eyelashes.  However the times for the eyelashes only amount to a percent or so of the total.

    In my case I have non-Daz addons which apply to G8F as well as DAZ Productions ones.  It's possible that there are a few rogue addons, or morphs, or other packages which generate an undue amount of "sum stage formulas" but it is possible that the DAZ Studio code to handle those formulae doesn't scale well.  Myself, I suspect the latter but I can see no way of proving it short of spending a very large amount of work gathering statistics with variable amounts of CMS content.  Since I've got my load time down to about 9s by eliminating about most of the characters I have bought from my default set I'm not inclined to attempt that analysis, but it should be easy for DAZ Productions to work out whether more timing messages are needed at this point.

    Post edited by jbowler on
  • WonderlandWonderland Posts: 7,133
    edited September 2021

    It’s at least 20 minutes, maybe more. I go do something else until it finally loads. How are people getting it to load in seconds? 

    Post edited by Wonderland on
  • PerttiAPerttiA Posts: 10,024

    Wonderland said:

    It’s at least 20 minutes, maybe more. I go do something else until it finally loads. How are people getting it to load in seconds? 

    Resolving conflicts and warnings first, and if the loading time is still too long, removing content (morphs for them)

    It took quite a while, but I went through all the characters/morph packs I had installed by dialing them on the "Genesis 8 Female Dev Load" without textures and checked if the results were something I might use at some point... I was stunned to see the amount of character morphs that produced completely unusable results, especially some of the bodies looked like they came out of horror movies.

  • James_HJames_H Posts: 1,091

    I look forward to trying this.

  • jbowlerjbowler Posts: 841
    edited September 2021

    Wonderland said:

    It’s at least 20 minutes, maybe more. I go do something else until it finally loads. How are people getting it to load in seconds? 

    Clear out your DSON cache (F2/General tab, hit the clear button), close DAZ Studio.  Launch DAZ Studio; use 4.15.0.30 - the general relesase from yesterday. Then just add the G8F basic character to the scene (search for "basic" in Smart Content), drink coffee, have a shower, read another chapter of War'n'Peace.  When the character appears use Help/Troubleshooting/View log file...; 4.15.0.30 logs the total time, it will almost at the end of the log file:

    2021-09-02 15:06:57.025 Finished asset load (open): 2m 17.793s - /People/Genesis 8 Female/Genesis 8 Basic Female.duf

    So if that time doesn't show something around, most like over, 20 minutes the suggestions I'm about to make probably won't help.  All the same I'm pretty sure it will be a big number; over the 140s my load takes with my full content set.  Scroll up from that line past this line (this only exists when the DSON cache has been cleared or is out of date):

    2021-09-02 15:06:56.313 Cache update required: /data/Daz 3D/Genesis 8/Female Eyelashes.cache.dsf

    and look for these three lines; the lines above the preceding line are required, not the ones below which correspond to the eyelashes, not the figure itself:

    2021-09-02 15:05:50.627 Creating sum stage formulas...
    2021-09-02 15:06:54.874 Creating multiply stage formulas...
    2021-09-02 15:06:55.098 Processing scene data...

    If there are warnings between the lines then that might be part of the problem, but just delete any intervening lines.  Check that the third line is "Processing scene data..."; for the eyelashes and other add-ons the third line is different.  Work out the elapsed times for the first two; 1m 4s for the first in this case, less than a second for the second.  So that's about 50% of the total load time and it is all in the "sum stage", not the "multiply stage".  If you see 50% or, more likely, much more then you can almost certainly address the problem by reducing the number of G8F characters DAZ Studio can see.  If you see substantially less than 50% check out the time for the "Scanning for addons..." step above those three lines - that's 1m in my case and, with the time to rebuild the DSON cache "Cache update started: /data/Daz 3D/Genesis 8/Female.cache.dsf" accounts for pretty much all the time.

    So assuming you have something that mimics the above breakdown now look at the size of the actual DSON cache file, by default it is "data\Daz 3D\Genesis 8\Female.cache.dsf" in the DSON cache folder displayed in the preferences dialog above.  Mine is something like 360MByte, IRC, but when I reduce my character and morph count substantially it reduces to 68MByte in size and the load time goes down from 140s to under 10s.

    I published some detailed timing figures a while back (March?) on the beta thread (IRC) about how much individual package types, e.g. wardrobe vs morph vs character, affect the load time, but the conclusion was that the most important one to look at are the individual G8F-based characters.

    So in the "Parameters" tab with "Gensis 8 Female" selected in the scene go to "Genesis 8 Female/Actor/Full Body/People" and select that entry (People).  Guesstimate how many rows there are; in my reduced set I see about 60 character dials but in the full set I was nearer 360.  It's also possible to look at all the morphs in "Genesis 8 Female/Actor", I have to hit "pagedown" 93 times to get to the bottom of the list in my reduced set and each page-down scrolls the list up by 21 items, so that's 1974 (94x21) morph dials.  I dread to think how many there are when my full set of packages are visible.  It's obvious that a lot of the morph dials come from morph packages (the things in "Real World" if you have any of them for example) and others come from wardrobe items; any item that the figure might wear gets its morph dials added.  Nevertheless the big gain seemed to come from reducing the "People" count, possibly because the figure morphs are very large.

     

    Post edited by jbowler on
  • WonderlandWonderland Posts: 7,133

    Well I know I have a huge amount of characters and morphs but I want them all available because I mix everything when creating characters, that’s why I purchased so many different characters and morphs so I can create something original. Removing characters and morphs defeats the whole point of buying them in the first place...

  • azoohouseazoohouse Posts: 664

    My thoughts exactly! Load times have become ridiculous! Can't this be fixed within the software?

  • jbowlerjbowler Posts: 841

    Wonderland said:

    Well I know I have a huge amount of characters and morphs but I want them all available because I mix everything when creating characters, that’s why I purchased so many different characters and morphs so I can create something original. Removing characters and morphs defeats the whole point of buying them in the first place...

    So far as I am aware, based on everything I have seen posted so far, the issue is the complexity of handling all those characters/morphs and the only solution using the same hardware (specifically CPU) is to reduce the complexity by removing the characters/morphs from the view of the DAZ Studio UI.  My set of instructions is designed to determine whether someone else's slow load times seem to be broadly the same as mine, if there are significant differences in the log times from 4.15.0.40 (in terms of percentage of total) then you are probably experiencing a different problem.

    Various other suggestions have been made.  There seems to be an obsession with the DSON cache and given that DAZ has repeatedly checked DSON cache status it might be a good idea to ensure the cache is on raided M2. SSDs, but I moved mine from a PCIe 4x mounted M.2 SSD to two raided M.2 SSDs on the motherboard and I saw, at best, a 10% speed-up in the cache creation, saving me 1s out of 140s.  DAZ Studio does access a lot of files, but then the profiling/timing information on my system doesn't show disk access.  It might be worth ensuring the DSON cache is on a physically different set of drives relative to the content, it might be worth splitting the content across multiple SSDs but I haven't tested this because I find it unlikely.

    I do have 64GByte of RAM and that sits at about 40% occupancy so the rest of it is used by Windows as file cache.  I do see a slowdown, maybe as much as 20%, if DAZ has to leave rebuild the DSON cache and DAZ Studio hasn't run for long enough for the file cache to get flushed (other apps, like Chrome, are always actively using it).  Operating system file system caching normally obviates issues of slow disk read in this kind of scenario.

    Apart from those tweaks (more RAM, SSD) which seem to offer only minor improvement, my intuition is that raw single core CPU speed is likely to be a straight (inverse) factor controlling load time.  DAZ Studio does not seem to multi-thread the load, even though it is doing thousands of things that don't affect each other - building the data structures for an individual morph does not require access to the vast majority, or maybe any, of the other morphs.  So forget core count.  Look at level 1 and level 2 cache size and raw CPU speed.  Go for a gamer system with an unlocked CPU that you can overclock, this is likely to require liquid cooling but since it is only the CPU you probably have that already.

    Bear in mind that at least some of this will be down to the performance of specific Qt classes in Qt 4.8.  Moving to Qt5 or Qt6 will likely change the low performance items.  We already know the close-down slowness was addressed, and maybe fixed, by Qt changes.

    So here's my other solution: I multitask:

    "C:\Program Files\DAZ 3D\DAZStudio4\DAZStudio.exe" -instanceName #

    I normally have two or three instances of DAZ Studio running in parallel.  On my system DAZ Studio uses THREE CPU cores not ONE simply because there are three of them.  I just swap from one to another when the UI slows down.

  • GhostofMacbethGhostofMacbeth Posts: 1,734

    13 seconds

  • jd641jd641 Posts: 462

    Finished asset load (open): 1m 1.141s - /People/Genesis 8 Female/Genesis 8 Basic Female.duf

     

    Looking at the top of the Smart Content thingy near the search field, it says 252 g8 figures.

     

    Looking higher in my log file I do get some errors:

     

    2021-09-04 04:43:31.722 WARNING: ..\..\..\..\..\src\sdksource\fileinput\dzassetdaz.cpp(9364): Alias property reference not found: Genesis8Female:/data/DAZ%203D/Genesis%208/Female/Morphs/bluejaunte/Misumi/eCTRLMisumiSignatureSmileHD.dsf#eCTRLMisumiSignatureSmileHD?value
    2021-09-04 04:43:31.722 WARNING: ..\..\..\..\..\src\sdksource\fileinput\dzassetdaz.cpp(1819): Property alias not created!
    2021-09-04 04:43:31.730 Creating sum stage formulas...
    2021-09-04 04:43:33.300 WARNING: ..\..\..\..\..\src\sdksource\fileinput\dzassetdaz.cpp(6975): Formula output property not found: Genesis8Female:/D%3A/Trabajo/DAZ/%21%21Glutes%20Utilities/Dformer%20Glutes.duf#D-Former_1-1?value - in file: /data/DAZ%203D/Genesis%208/Female/Morphs/Soto/Glutes%20Utilities/PBMGlutesUtilitiesUnderCurve.dsf
    2021-09-04 04:43:33.373 WARNING: ..\..\..\..\..\src\sdksource\fileinput\dzassetdaz.cpp(6975): Formula output property not found: Genesis8Female:/data/DAZ%203D/Genesis%208/Female/Morphs/DAZ%203D/Josephene%208/FHMJosephene8.dsf#FHMJosephene8?value - in file: /data/DAZ%203D/Genesis%208/Female/Morphs/SASE/Karma/FHM-SASEKarma.dsf
    2021-09-04 04:43:33.396 WARNING: ..\..\..\..\..\src\sdksource\fileinput\dzassetdaz.cpp(6975): Formula output property not found: Genesis8Female:/data/DAZ%203D/Genesis%208/Female/Morphs/DAZ%203D/Josephene%208/FBMJosephene8.dsf#FBMJosephene8?value - in file: /data/DAZ%203D/Genesis%208/Female/Morphs/SASE/Karma/FBM-SASEKarma.dsf
    2021-09-04 04:43:59.755 WARNING: ..\..\..\..\..\src\sdksource\fileinput\dzassetdaz.cpp(6975): Formula output property not found: Genesis8Female:/data/DAZ%203D/Genesis%208/Female/Morphs/bluejaunte/Sahel/SahelSignatureSmileHDLv2.dsf#SahelSignatureSmileHDLv2?value - in file: /data/DAZ%203D/Genesis%208/Female/Morphs/bluejaunte/Ensley/EnsleySmile.dsf
    2021-09-04 04:43:59.755 WARNING: ..\..\..\..\..\src\sdksource\fileinput\dzassetdaz.cpp(6975): Formula output property not found: Genesis8Female:/data/DAZ%203D/Genesis%208/Female/Morphs/bluejaunte/Sahel/SahelSignatureSmile.dsf#SahelSignatureSmile?value - in file: /data/DAZ%203D/Genesis%208/Female/Morphs/bluejaunte/Ensley/EnsleySmile.dsf
    2021-09-04 04:44:07.107 Creating multiply stage formulas...
    2021-09-04 04:44:07.351 WARNING: ..\..\..\..\..\src\sdksource\fileinput\dzassetdaz.cpp(7041): Formula target property not found: Genesis8Female:/data/DAZ%203D/Genesis%208/Female/Morphs/3D%20Universe/G8FFawne/CTRL3duG8FFawne.dsf#CTRL3duG8FFawne?value - in file: /data/DAZ%203D/Genesis%208/Female/Morphs/3D%20Universe/G8FFawne/MCM3duFawneFitSandals.dsf

    Kinda strange and random.

  • HamEinarHamEinar Posts: 129

    After an extensive pruning of my runtime, G8F now loads in about 3 minutes (down from 7). Irritating to have to remove stuff I've paid for, and I am MUCH more picky with purchases now..

  • jbowlerjbowler Posts: 841

    HamEinar said:

    After an extensive pruning of my runtime, G8F now loads in about 3 minutes (down from 7). Irritating to have to remove stuff I've paid for, and I am MUCH more picky with purchases now..

    Me too; the whole thing is irritating and has certainly caused me to avoid buying new characters.  I don't remove them; I de-installed them with DIM then re-install them to a different CMS directory (G8Figures-Unused) which is not included in my default CMS configuration.  When I have purchased new figures since then I've put them in a third CMS directory, G8Figures-New, which is also not in my standard CMS configuration.  I can add the -New CMS directory without much of a hit on G8F load time, or scene load time, and then experiment.  The new figure ends up with in my core set or in -Unused.  Every now and then I'll take a pass through -Unused, normally by looking through my product library online.  It's very easy to move stuff between CMS directories with DIM.

    Other people who have commented on this thread make a completely separate setup for a given scene containing only the stuff that is in that scene.  Presumably they get a whole lot of errors the first time they load it but Daz removes the unresolved references when the scene is saved.  Removing the available-but-unused morphs is likely to give a small but maybe perceptible improvement to UI response as well; important when working on animations.

  • thenoobduckythenoobducky Posts: 68
    edited September 2021

    I will just repeat this observation I have made a while ago.

    Observations: a large portion of the load time is spent parsing the cache file, cache mainly contains formulas (creating sum stage formulas). 

    Assumption: every formula takes the same amount of time to load.

    Hypothesis 1: by counting how many formulas each product adds to the cache file, we can figure out relatively how much each product impact load speed.

    Hypothesis 2: removing products that adds the most impact will allow the most improvement with the fewest amount of content removed.

     

    What we need is some sort of metrics like:

    keep factor = usefulness of product / no of formulas in a product

     

    Post edited by thenoobducky on
  • jbowlerjbowler Posts: 841

    thenoobducky said:

    Assumption: every formula takes the same amount of time to load.

    I suspect that the time to do the "sum formulas" step in the load depends on something like the square of the number of "sum formulas".  I originally assumed times were simply additive; my original timings were based on adding/removing groups of products and the overall time seemed approximately additive, but it would be unusual for something like this to be strictly linear because lists of things are being built internally.  In particular adding a new character adds the morph dial for that character to every other character.

    Hypothesis 1: by counting how many formulas each product adds to the cache file, we can figure out relatively how much each product impact load speed.

    Hypothesis 2: removing products that adds the most impact will allow the most improvement with the fewest amount of content removed.

    I suspect both are true, but I don't know a suffiiciently efficient way of measuring the first; it's obviously how to do it with DIM and a running DAZ Studio but it's a couple of minutes work plus the G8F load time for each product...

Sign In or Register to comment.