DSF files can be compressed, or... "In 15 hours you can save 200GB or more!"
CypherFOX
Posts: 3,401
Greetings,
For my laptop, with a measly 512GB of on-board storage, I use an external 1TB USB-C drive for My Library. I was out of space on it, and only have around half of my Genesis 3 stuff, and all my Genesis 8 stuff on it. I had already trimmed a lot from it, stuff that I hated to see go. I wanted to put more things on it, since it's my primary device.
I wrote a small Ruby script to go through all the .dsf files in my library, check if they are compressed, and if not, compress them. I ran it more-or-less overnight.
It literally saved me 200GB, out of a 1TB library. I calculated the total size of DSF files beforehand, and it was in the range of 360GB just of DSF files, so it cut them mostly in half. They mostly consist of points in 3D or UV space, and similar data sets, but in plain ASCII form. This is fine, as it lets the points be precisely represented, but it's expensive in disk space terms.
What I'm saying, here, is that if DAZ were to pre-emptively compress DSF files, it'd save a TON of space in our libraries.
-- Morgan
n.b. For libraries that are not multiple TB of data, it probably won't matter as much, but even for those it'll save noticable amounts of space.
Daz 3D is part of 
Connect
DAZ Productions, Inc.
7533 S Center View Ct #4664
West Jordan, UT 84084
Licensing Agreement | Terms of Service | Privacy Policy | EULA
© 2025 Daz Productions Inc. All Rights Reserved.
Comments
you can save them compressed, it is the default AFAIK
Compression has its costs. DS would need to uncompress those files either in-memory or in a temp location to extract any relevant information from them. Say, in order to read just the metadata/header portion of a file, it would need to uncompress the entire gzipped content in a temporary location which may cause additional I/O, cpu cycles or even memory. I would not recommend compressing all dsf files indiscriminately but rather based on some tradeoffs. In my opinion, the only dsf files that may benefit from compression are the ones which need to be read in their entirety at some point such as those that contain a lot of geometry information. On the other hand, dsf files that contain mostly metadata are usually small and would not benefit a lot from compression, yet one would still have to pay the cost in terms of additional I/O. But if one considers hard drive space as premium and willing to accept increased load times...
Interestingly, that brings me to question how optimized even the duf files are in terms of separation of metadata and geometry content (say, in case of morphs)? Do we even have proper segregation between those two to leverage compression in an efficient manner?
Greetings,
.DUF files contain metadata (and can be relocated on disk), .DSF files typically contain vertex information (XYZ, UV, etc.), from what I've seen. I didn't arbitrarily compress .DUF files, although I could. The decompression takes place on load of that data, and typically in-memory. But it doesn't use significantly more memory than loading the data in the first place. It does mean that scene-load is slightly slower, but not enough to make up for 200GB of storage.
And no, @WendyLuvsCatz the ones you get during DAZ Install Manager, or downloading from the store, are not typically compressed on disk, unfortunately.
If you're making them yourself, of course, you can choose to have them compressed...but the ones bought from the store...are more often than not, uncompressed.
-- Morgan
I can only suggest another area that would help get you back gigs of drive space if you have a huge library, and that is in DIM. There is an option at the bottom of one of the tabs--I think it is the Installed tab-- that will delete the downloaded packages once they are installed. It is off by default so that you don't have to download the products again, so there's the tradeoff.
If you do go this route, it only affects that specific product and forward. You would need to remove the others yourself. Check DIM settings for location. Also, because I just came from the thread on Painter's Lights, discontinued products are still available to download and install again through DIM, aren't they? They have to be once you buy them.
Greetings,
Yeah, I always delete the packages after installation. Occasionally I've wondered what it would be like to redownload all of them, and store them off just in case DAZ ever goes down, but I don't think my ISP would like me very much...
The other place that would probably have a LOT of savings would be converting all the TIFF files to PNG files. The TIFF files are usually normals, and so JPEG would be very bad because of the artifacting, but a lossless PNG could be good. It's _possible_ that those are already compressed (TIFF does have a compressed mode) but I believe I've seen a number of uncompressed ones before, and textures take up a f*ton of storage. The problem here is that while compressing DSF files is a no-behavioral-change operation (because DAZ transparently uncompresses them in memory), compressing a TIFF file to a PNG would require altering any files (generally DUF IIRC) which reference it to point to the new file. That's doable, but...not easily.
If I get to the point where I need to rewrite TIFF to PNG references also, I'll probably release it as a proper tool.
-- Morgan
A first step before converting to png might be to batch convert all Tiffs and save as compressed, if it's possible in your image editor. I did look at writing a program to open tiffs myself in Borland C++ Builder when I was using v4, but the standard specified so many different compression methods that could be used & I couldn't understand that I left it in disgust. When I eventually upgraded to 10.1, opening Tiffs was built in & I didn't need to bother.
Is there a reason you wrote a ruby script to do this job rather than use the "Batch Convert" tab inside Daz Studio, which will compress dsf files, along with duf, ds, dsa and dsb files?
.dsb files are already compressed.
Thanks for trying these kinds of experiments. I appreciate it and am looking for solutions to save diskspace. It's been difficult so far, only two things I can do myself are:
- reformat oversized textures or odd formats into normal .jpgs. There's no quality loss, saves oodles of diskspace, only downside is it takes ages by hand. Would love if someone made a script for these things.
- remove oversized HDRI's. 200mb for a single HDRI just eats up diskspace while it's the same quality as a more reasonable 40mb one.
I assume the batch process can uncompress dsb, no? Would it change the file ending if they were uncompressed?
It would depend in any event - some older .dsb files are what would now be labelled .dse, encrypted scripts, and I've not tried Batch Convert on any .dsb files. I was mainly pointing that tey don't need cmpression as they already are.
And yet drives are awfully cheap.
_ alright then _
I've written an app (Windows) that can resize textures in DAZ Install zips, it writes a modified zip with the resized textures, no other files are changed. Currently only PNG and JPG support. Click my signature link, it's at the end of the page.
My fix, 8TB ext HD = $194
Thank you, I will look into it.
Greetings,
Doesn't work as well with a laptop. Both portability and battery would suffer.
I use an external USB-C 1TB external SSD drive for the stuff I'm very likely to use, so that it's not agonizingly slow, and that works okay. My next laptop will have more SSD storage built in, but I got this one in 2017.
The rest of my library is on an achingly slow network drive. It's actually as fast as a (reasonably priced) NAS can be, but not fast enough when loading lots of assets.
-- Morgan