How long does your DIM take to populate?

It might be because I have a ridiculous amount of content or the fact that my Daz library is on an external hard drive, but DIM takes about 20 minutes for anything to even show up... I was wondering how long it takes other people...

Comments

  • TaozTaoz Posts: 10,259
    edited August 2019

    It shouldn't have anything to do with where your DAZ Library is as it doesn't seem to access it before you start installing/uninstalling and whatever you can do with your files. But if your downloaded files are on the removable drive and it's slow (USB 2?) it may have to do with that.

    Mine takes about 2 minutes to update online from when I click the Start button. That's with all DIM data (downloaded files, manifest files, thumbnails) placed on an older fairly slow internal HDD (no SSDs). I have 15580 products and in DIM it looks like this:

    11964 Ready to download
    2441 Ready to install
    6279 Installed

     

    Post edited by Taoz on
  • TaozTaoz Posts: 10,259
    edited August 2019

    BTW, I can't rule out that caching (I have DIM open most of the time) may affect the speed.  I'll try later after a reboot (note that a restart may not clear the disk caches, you need to do a cold boot, preferably with the power plug removed for a couple of minutes to ensure that all is cleared. Pressing the PCs Start button after it's been unplugged  is also a good idea as some PSUs may hold the power for a while, you'll usually see the power LED light up a for a moment if you do that which means there is still some power left.). 

    Post edited by Taoz on
  • TaozTaoz Posts: 10,259
    edited August 2019

    OK, tried again after a cold boot with power off.  Took 3:45 minutes this time with no other things running, so caching seems to be a factor.

    Post edited by Taoz on
  • namffuaknamffuak Posts: 4,409

    I see about six and a half minutes from filling in my password to getting the list of available downloads, ready to install, and installed. My download directory is on an external USB 3 drive (2 TB) and my DIM control and installation manifest files are on my C: drive in the default locations. I have a bit over 13,000 products showing in DIM.

    Items that may influence time:

    1) DIM updates the date/time stamp of every manifest (dsx) file in the download directory

    2) DIM walks the installation manifest files in your appdata directory

    3) DIM scans your Daz product library

    From these three sources DIM fills in the three tabs - available downloads with product updates split out, ready to install (with, again product updates split out) and installed products. (1) is dependent on the size of the download directory and the speed of the disk and the disk interface. My download directory is 1.31 TB, with 26,300+ files on, as I said, a USB 3 drive connected to a USB 3 port. (2) is dependent on the number of installed files and the speed od the disk the manifest resides on; on my laptop, this is an internal 7200 RPM 1 TB drive and I only have 1,000 products installed. (3) is dependent on your network connection and I'm running on Starbucks' Google gigabit wifi.

    I'll time the main system when I get home tonight, but it will be an apples to turnips comparison - the home system has the 'download' directory copied to an internal 2 TB SATA drive and the install manifest (all 13,000+ products installed) is on an SSD - and there's no network connectivity. So I expect startup to be faster.

  • AtiAti Posts: 9,185

    To put the different times in perspective: with the downloaded files being kept in their original location: startup was over 30 minutes. With the downloaded files moved to a different folder, startup time is now 35 seconds.

  • TaozTaoz Posts: 10,259
    edited August 2019
    namffuak said:

    3) DIM scans your Daz product library

    It doesn't seem to do that (as I wrote in my first post).  I've tried to rename the DAZ Library as well as the Connect library and then update DIM online, it didn't complain but updated as normal.  If it does and doesn't warn you if the DAZ Library is missing it's badly coded, but I doubt the DAZ programmers are that careless (that would normally also cause some serious malfunction). 

     

    Post edited by Taoz on
  • tj_1ca9500btj_1ca9500b Posts: 2,057

    For me, it's about 2 minutes after DIM starts up to sort through the downloads and figure out which ones aren't installed yet.  I've been doing manual downloads for about 7-8 months now, as my rendering computer is now isolated from the interweb thingie, to help reduce the risk of nasty bugs.

    Anyways, my setup has the Daz files in the 'Default' DIM location, on a 2TB Samsung 970 EVO NVME SSD.  So the drive access times are pretty fast.  As others have noted, if you are accessing an external drive, etc. this can impact sort times.  Nothing wrong with that approach, just felt the need to mention that access times for external drives over USB is often slower than internal NVME based drives, or SATA drives for that matter...

  • TaozTaoz Posts: 10,259

    My guess from what I read here is that the number of downloaded files matters a great deal, when it comes to update speed.  It seems to update the .dsx files for each downloaded product plus it's probably doing some compare between this and that every time it updates which is probably taking a while. 

  • namffuaknamffuak Posts: 4,409
    edited August 2019
    Taoz said:
    namffuak said:

    3) DIM scans your Daz product library

    It doesn't seem to do that (as I wrote in my first post).  I've tried to rename the DAZ Library as well as the Connect library and then update DIM online, it didn't complain but updated as normal.  If it does and doesn't warn you if the DAZ Library is missing it's badly coded, but I doubt the DAZ programmers are that careless (that would normally also cause some serious malfunction). 

     

    Not your local 'My Library' - the Daz Product Library under your account - and I'm assuming the process is semi-optimized to look at updates since last connect, but I may be wrong. This is how it finds out about new purchases and updates.

    ETA - Startup time on the home system is just under 11 seconds. There is no network connection, so no download of data from Daz AND no update of the dsx manifests in the download directory. 24 ready to install and 13,182 installed.

    From this, the bulk of the startup time is network access and updating the dsx manifests in the download directory.

    Post edited by namffuak on
  • WonderlandWonderland Posts: 7,133
    edited August 2019
    Ati said:

    To put the different times in perspective: with the downloaded files being kept in their original location: startup was over 30 minutes. With the downloaded files moved to a different folder, startup time is now 35 seconds.

    I do have my downloaded files in another folder. I have 17,266 files installed...

    Post edited by Wonderland on
  • TaozTaoz Posts: 10,259
    edited August 2019
    namffuak said:
    Taoz said:
    namffuak said:

    3) DIM scans your Daz product library

    It doesn't seem to do that (as I wrote in my first post).  I've tried to rename the DAZ Library as well as the Connect library and then update DIM online, it didn't complain but updated as normal.  If it does and doesn't warn you if the DAZ Library is missing it's badly coded, but I doubt the DAZ programmers are that careless (that would normally also cause some serious malfunction). 

     

    Not your local 'My Library' - the Daz Product Library under your account - and I'm assuming the process is semi-optimized to look at updates since last connect, but I may be wrong. This is how it finds out about new purchases and updates.

    What is the path to that "Daz Product Library"? 

    There is a DIM folder called "ManifestFiles" which contains modified versions of the .dsx files that accompanies the install zips in the "Downloads" folder, these ManifestFiles contain information about the installed products and DIM uses them to keep track of what is installed and where.  I've moved these and all other product related files that DIM is using to another drive:
     


     

    This is the .dsx file that accompanies the product file(s), and below the modfied version which contains local install data:

    <?xml version="1.0" encoding="UTF-8"?>
    <ProductSupplement VERSION="0.1">
    <ProductName VALUE="Victoria 2.0 Character Pack 1"/>
    <ProductStoreIDX VALUE="8-1"/>
    <UserOrderId VALUE="6717907"/>
    <UserOrderDate VALUE="2011-01-10T19:39:07Z"/>
    <InstallerDate VALUE="2013-02-08T00:05:00Z"/>
    <ProductFileGuid VALUE="5d4b40a5-b6e0-da8e-9a4d-f46b438e95b6"/>
    <InstallTypes VALUE="Content"/>
    <ProductTags VALUE="DAZStudio4_5,DAZStudio4,DAZStudio3,DAZStudioLegacy,Poser9,PoserLegacy,CloudAvailable"/>
    </ProductSupplement>

    -----------------------------------------------------------------------

    <?xml version="1.0" encoding="UTF-8"?>
    <DAZInstallManifest VERSION="0.1">
    <GlobalID VALUE="3835eed4-edb1-04ca-54be-0f94c44b0b0e"/>
    <MetadataGlobalID VALUE="3835eed4-edb1-04ca-54be-0f94c44b0b0e"/>
    <SmartContent VALUE="False"/>
    <ProductName VALUE="Victoria 2.0 Character Pack 1"/>
    <ProductStoreIDX VALUE="8-1"/>
    <ProductFileGuid VALUE="5d4b40a5-b6e0-da8e-9a4d-f46b438e95b6"/>
    <InstallTypes VALUE="Content"/>
    <ProductTags VALUE="DAZStudio4_5,DAZStudio4,DAZStudio3,DAZStudioLegacy,Poser9,PoserLegacy,CloudAvailable"/>
    <ArchiveDate VALUE="2013-02-08T00:05:00Z"/>
    <InstallerDate VALUE="2013-02-08T00:05:00Z"/>
    <UserInstallAccount VALUE="Account"/>
    <UserOrderId VALUE="6717907"/>
    <UserOrderDate VALUE="2011-01-10T19:39:07Z"/>
    <UserInstallDate VALUE="2017-05-29T04:47:42Z"/>
    <InstalledSize VALUE="543613"/>
    <UserInstallPath VALUE="F:/My DAZ 3D Library"/>
    <File TARGET="Content" ACTION="Install" VALUE="Content/Runtime/Libraries/Face/V2 Character Head/Alien.fc2"/>
    <File TARGET="Content" ACTION="Install" VALUE="Content/Runtime/Libraries/Face/V2 Character Head/Alien.rsr"/>
    <File TARGET="Content" ACTION="Install" VALUE="Content/Runtime/Libraries/Face/V2 Character Head/Elderly.fc2"/>
    <File TARGET="Content" ACTION="Install" VALUE="Content/Runtime/Libraries/Face/V2 Character Head/Elderly.rsr"/>
    <File TARGET="Content" ACTION="Install" VALUE="Content/Runtime/Libraries/Face/V2 Character Head/Heavy Torso.fc2"/>
    <File TARGET="Content" ACTION="Install" VALUE="Content/Runtime/Libraries/Face/V2 Character Head/Heavy Torso.rsr"/>
    <File TARGET="Content" ACTION="Install" VALUE="Content/Runtime/Libraries/Face/V2 Character Head/Oriental.fc2"/>
    <File TARGET="Content" ACTION="Install" VALUE="Content/Runtime/Libraries/Face/V2 Character Head/Oriental.rsr"/>
    <File TARGET="Content" ACTION="Install" VALUE="Content/Runtime/Libraries/Face/V2 Character Head/Super Model 2.fc2"/>
    <File TARGET="Content" ACTION="Install" VALUE="Content/Runtime/Libraries/Face/V2 Character Head/Super Model 2.rsr"/>
    <File TARGET="Content" ACTION="Install" VALUE="Content/Runtime/Libraries/Pose/V2 Character Body/Alien Body.pz2"/>
    <File TARGET="Content" ACTION="Install" VALUE="Content/Runtime/Libraries/Pose/V2 Character Body/Alien Body.rsr"/>
    <File TARGET="Content" ACTION="Install" VALUE="Content/Runtime/Libraries/Pose/V2 Character Body/Elderly Body.pz2"/>
    <File TARGET="Content" ACTION="Install" VALUE="Content/Runtime/Libraries/Pose/V2 Character Body/Elderly Body.rsr"/>
    <File TARGET="Content" ACTION="Install" VALUE="Content/Runtime/Libraries/Pose/V2 Character Body/Heavy Torso Body.pz2"/>
    <File TARGET="Content" ACTION="Install" VALUE="Content/Runtime/Libraries/Pose/V2 Character Body/Heavy Torso Body.rsr"/>
    <File TARGET="Content" ACTION="Install" VALUE="Content/Runtime/Libraries/Pose/V2 Character Body/Oriental Body.pz2"/>
    <File TARGET="Content" ACTION="Install" VALUE="Content/Runtime/Libraries/Pose/V2 Character Body/Oriental Body.rsr"/>
    <File TARGET="Content" ACTION="Install" VALUE="Content/Runtime/Libraries/Pose/V2 Character Body/Super Model 2 Body.pz2"/>
    <File TARGET="Content" ACTION="Install" VALUE="Content/Runtime/Libraries/Pose/V2 Character Body/Super Model 2 Body.rsr"/>
    <File TARGET="Content" ACTION="Install" VALUE="Content/Runtime/Support/DAZ_3D_8_Victoria_2_0_Character_Pack_1.dsx"/>
    </DAZInstallManifest>

     

    moved_dim_folders.png
    689 x 186 - 27K
    Post edited by Taoz on
  • NorthOf45NorthOf45 Posts: 5,692

    I think he means at the DAZ3D end, your Product Library.

  • xyer0xyer0 Posts: 6,333
    Ati said:

    To put the different times in perspective: with the downloaded files being kept in their original location: startup was over 30 minutes. With the downloaded files moved to a different folder, startup time is now 35 seconds.

    @Ati, you mean "different folder," right? Not "different drive?"

    I read the other thread you linked. But I was not certain then either. Are you simply moving all the DIM downloaded zips into a different folder on the same drive?

  • TaozTaoz Posts: 10,259
    edited August 2019
    NorthOf45 said:

    I think he means at the DAZ3D end, your Product Library.

    C:\Users\[user]\Documents\DAZ 3D\Studio\My Library\  ?

    I've moved that to F: too.  And can't find anything there that looks like being DIM related.  Neither in the original C: folder which is still there.  In the folder moved to F: there is a data\auto_adapted folder with many subfolders, but they seem to contain .dsf files only.  Maybe Connect related? 

    I find it all quite confusing, DAZ is placing files all over the place on C:, both for DS, DIM and the CMS. 

    Post edited by Taoz on
  • namffuaknamffuak Posts: 4,409
    edited August 2019
    NorthOf45 said:

    I think he means at the DAZ3D end, your Product Library.

    Correct. Step 3 is to parse your Daz store product library - not something on your system. Then, when you download the files with DIM the existing store dsx file gets modified to add your order ID and order date before it is saved in your download directory.

    Post edited by namffuak on
  • NorthOf45NorthOf45 Posts: 5,692
    Taoz said:
    NorthOf45 said:

    I think he means at the DAZ3D end, your Product Library.

    C:\Users\[user]\Documents\DAZ 3D\Studio\My Library\  ?

    I've moved that to F: too.  And can't find anything there that looks like being DIM related.  Neither in the original C: folder which is still there.  In the folder moved to F: there is a data\auto_adapted folder with many subfolders, but they seem to contain .dsf files only.  Maybe Connect related? 

    I find it all quite confusing, DAZ is placing files all over the place on C:, both for DS, DIM and the CMS. 

    Not on your computer, the DAZ3D servers, your Product Library on the web site. They have records of all your purchases, so they check what you have downloaded vs. what is available (new or updates). If you move the accompanying .dsx files in the Package Archive (or change the folder without moving them over), DIM will show them all as "Available for Download" because it thinks you don't have anything. 

  • AlmightyQUESTAlmightyQUEST Posts: 2,006
    Ati said:

    Ah jeez, I used to move the download files to a second drive pretty regularly, once I got a new drive that had a ton of space I hadn't done it in a while. Made a huge difference, so thanks for pointing this out!

  • AtiAti Posts: 9,185
    edited August 2019
    xyer0 said:
    Ati said:

    To put the different times in perspective: with the downloaded files being kept in their original location: startup was over 30 minutes. With the downloaded files moved to a different folder, startup time is now 35 seconds.

    @Ati, you mean "different folder," right? Not "different drive?"

    I read the other thread you linked. But I was not certain then either. Are you simply moving all the DIM downloaded zips into a different folder on the same drive?

    Different folder, same drive. It used to be in "F:\Program Files\DAZ 3D" (for me), and I moved them over to "F:\Program Files\DAZ 3D backup". Now I just need to move the new ones over once in a while, and it should remain fast, while I still have the original zips for backup purposes. (This location is not listed in DIM.)

    Post edited by Ati on
  • TaozTaoz Posts: 10,259
    NorthOf45 said:
    Taoz said:
    NorthOf45 said:

    I think he means at the DAZ3D end, your Product Library.

    C:\Users\[user]\Documents\DAZ 3D\Studio\My Library\  ?

    I've moved that to F: too.  And can't find anything there that looks like being DIM related.  Neither in the original C: folder which is still there.  In the folder moved to F: there is a data\auto_adapted folder with many subfolders, but they seem to contain .dsf files only.  Maybe Connect related? 

    I find it all quite confusing, DAZ is placing files all over the place on C:, both for DS, DIM and the CMS. 

    Not on your computer, the DAZ3D servers, your Product Library on the web site. They have records of all your purchases, so they check what you have downloaded vs. what is available (new or updates). If you move the accompanying .dsx files in the Package Archive (or change the folder without moving them over), DIM will show them all as "Available for Download" because it thinks you don't have anything. 

    Ah, OK.  I've never thought about the Product Library in connection with DIM.  I've written software that pulls user and product data from the Product Library in the user accounts and there's no DIM data there so that confused me.  DIM connects to its own "private" server AFAIR.

  • nonesuch00nonesuch00 Posts: 18,729

    So then DIM is parsing and updating every file in your product list if you save them locally. I can be impatient but it's not like 3 1/2 minutes is an unreasonable amount of time to wait for DIM to parse all 4000+ of those saved download files xml data when DIM's main purpose anyway is to download files that are often gigabytes in size and take hours in doing so (in my case with a 3.5mb/s download speed). DIM took 3 minutes to startup? Excellent. Now be prepared to wait another 40 minutes as it downloads the new 630MB DS PB version.

  • xyer0xyer0 Posts: 6,333
    Ati said:
    xyer0 said:
    Ati said:

    To put the different times in perspective: with the downloaded files being kept in their original location: startup was over 30 minutes. With the downloaded files moved to a different folder, startup time is now 35 seconds.

    @Ati, you mean "different folder," right? Not "different drive?"

    I read the other thread you linked. But I was not certain then either. Are you simply moving all the DIM downloaded zips into a different folder on the same drive?

    Different folder, same drive. It used to be in "F:\Program Files\DAZ 3D" (for me), and I moved them over to "F:\Program Files\DAZ 3D backup". Now I just need to move the new ones over once in a while, and it should remain fast, while I still have the original zips for backup purposes. (This location is not listed in DIM.)

    Thanks so much @Ati. I'll do that. I had gotten accustomed to waiting a half hour or less for DIM to come on.

  • AtiAti Posts: 9,185
    xyer0 said:
    Ati said:
    xyer0 said:
    Ati said:

    To put the different times in perspective: with the downloaded files being kept in their original location: startup was over 30 minutes. With the downloaded files moved to a different folder, startup time is now 35 seconds.

    @Ati, you mean "different folder," right? Not "different drive?"

    I read the other thread you linked. But I was not certain then either. Are you simply moving all the DIM downloaded zips into a different folder on the same drive?

    Different folder, same drive. It used to be in "F:\Program Files\DAZ 3D" (for me), and I moved them over to "F:\Program Files\DAZ 3D backup". Now I just need to move the new ones over once in a while, and it should remain fast, while I still have the original zips for backup purposes. (This location is not listed in DIM.)

    Thanks so much @Ati. I'll do that. I had gotten accustomed to waiting a half hour or less for DIM to come on.

    Yeah, that half an hour waiting time was starting to bother me. I only downloaded my purchases every couple of weeks because of it. Now I can go back to downloading them immediately. And it was a pretty quick "fix," or workaround, or whatever you want to call it.

  • xyer0xyer0 Posts: 6,333

    This is great! I only turn on the internet connection of my Daz computer when I have to download files; so, now I can go from an hour down to a few minutes.

Sign In or Register to comment.