Converting DAZ Scenes to DUF Scenes and auto_adapted

ken_7dac5ac5a2 · May 2013

With the release of DAZ Studio 4.6 I began getting errors reading DUF files related to content in the auto_adapted folder not being found. The issue is more general than just the new 4.6 release, so I am starting a new topic. The more general issue, for me, is what is the most effective way to convert old DAZ scenes to DUF scenes. This would involve some understanding of what is the auto_adapted folder and how is it implemented. It seems reasonable that DAZ should provide some substantive help converting from the old format, as this is a significant change. (I don't dispute the reasons for the change.)

I have searched both the web in general and the forums in particular. There are four forum topics related to this subject; however, I have been unable to get any reasonable understanding of what the auto_adapted folder is and how it is used.
DAZ Discussion Forum | "auto-adapted"
DAZ Discussion Forum | The DUF file size versus DAZ file size ...
DAZ Discussion Forum | Dealing with the data/auto_adapted folder
data auto_adapted showing up in Content Library - Daz 3D

Let me try to formulate some questions:

1. The idea is to get rid of DAZ scenes in favor of DUF scenes. What is the best (and easiest) way to do that?

2. What is the auto_adapted folder and what is it used for?

3. What determines where this (generated) folder is located?

4. It here a way to keep this generated content separate from installed content (e.g. as installed by DIM)?

5. Is there a way to convert old DAZ scenes that avoids creating content in auto_adapted?

6. What changed in 4.6 that causes content in the auto_adapted folder to not be found even though there were no problems in 4.5 with the same content directory lists.?

Thanks for your help.

Bejaymac · May 2013

2) Anything you load into DS that isn't in the DSON format will have DSF asset files created when you save the scene, these go into the "auto_adapted" folder, usually under the old "mesh name_vertice count" system. The same thing happens any time you use the Transfer Utility or autofit, the mesh isn't something DS can use in the format it's loaded in as, so during the conversion by the TU or autofit a DSF asset file is created, the mesh in scene is deleted and the DSF asset file is loaded in it's place.

3) Same as it's always been, the first DS native directory you have listed in the content directory manager, that's where the data folder gets made any time you saved a scene in DS1 to DS4.6.

5) Only if the scene had nothing but Genesis and it's content in it.

ken_7dac5ac5a2 · May 2013

Bejaymac said:
2) Anything you load into DS that isn't in the DSON format will have DSF asset files created when you save the scene, these go into the "auto_adapted" folder, usually under the old "mesh name_vertice count" system. The same thing happens any time you use the Transfer Utility or autofit, the mesh isn't something DS can use in the format it's loaded in as, so during the conversion by the TU or autofit a DSF asset file is created, the mesh in scene is deleted and the DSF asset file is loaded in it's place.
3) Same as it's always been, the first DS native directory you have listed in the content directory manager, that's where the data folder gets made any time you saved a scene in DS1 to DS4.6.

5) Only if the scene had nothing but Genesis and it's content in it.

What you say in 3) doesn't appear to be happening for me. I have one and only one data/auto_adapted directory. It is in a Scenes directory that is outside of all the folders in my content directory list. Apart form the data directory, it contains .daz files (dated prior to 4.5) and another directory Studio4 that contains both .daz and .duf scenes (that I saved from .daz scenes). I don't think it has ever been in my content directory list, but I can't be positive. I don't know why it ever would have been at the top, however. With Studio 4.6, the content in auto_adapted could not be found, whereas it was working OK in 4.5 the last I tried. On first starting 4.6, it was definitely not in the content directory list at all.

I realize a possible explanation is that I was mistaken and it was always at the top but somehow got deleted after I did the last conversion to DUF, but that seems unlikely.

Corrected later: I found a screen dump that indicated the Scenes directory was at the bottom of my DAZ Studio Formats relatively recently and at about the time I saved the DUFs. I might have deleted it after that, which would explain why 4.6 was not finding the auto_adapted content. The situation is still not consistent with making the auto_adapted folder in the top content directory.

Before experimenting, could I ask some more questions.

Does this happen when it is loaded or when it is saved? In particular, if I were creating a scene and loaded, say, Victoria 4.2.cr2, would a data/auto_adapted folder get created (if it wasn't already there)?

There aren't that many items in my one auto_adapted directory. They all appear to be related to DUF scenes files that I saved from opening DAZ scenes. If I look in the DUF files, I do see that the auto_adapted folder is referenced (in many places) as

"parent" : "/data/auto_adapted/..."

Thanks for your help.

ken_7dac5ac5a2 · May 2013

I restored my content directory list to have Scenes at the bottom of my DAZ Studio Formats. (I now believe this is where it was when I saved several DAZ scenes as DUF scenes.) I made a new scene (using 4.6) with the Plated Wolf (Wolf.cr2), something I had not used before. It created Scenes\data\auto_adapted\Wolf_146145.

So it appears the auto_adapted files are created in the last, not first directory in the DAZ Studio Formats list.

And it only creates the auto_adapted folder on save, not load.

This explains my situation (only one auto_adapted directory that is in Scenes) and what happened with 4.6 (I had apparently deleted the Scenes directory from the list for some reason I don't recall.)

This seems to be a strange behavior, but (if you know about it) it provides a way to keep the created content separate from the installed content.

I understand documentation is not DAZ's strong point, but this behavior is important, as well as unexpected. It should be documented IMHO. I do appreciate your help.

Richard Haseltine · May 2013

It may be that the Auto_adapted folder is in the content directory that holds the scene, if it is saved to a content directory.

Another way to avoid the folder is to save the items as an Asset - but that is possible only for TriAx figures, and converting legacy figures to TriAx is not always a good idea.

ken_7dac5ac5a2 · May 2013

Richard Haseltine said:
It may be that the Auto_adapted folder is in the content directory that holds the scene, if it is saved to a content directory.

No, the scene I just saved was a new one, and it was not saved in the directory where the data/auto_adapted directory is located. The behavior is consistent with saving in the last directory in the DAZ Studio Formats list.

The trouble with conjectures and trial and error is that they may be wrong or not have the full picture. Nothing beats having the developers document what they have done. For this issue I think that should be done. There is a lot of confusion, even among normally knowledgeable people, and it is important.

Richard Haseltine · May 2013

It's also consistent with my suggestion - the Data folder is always in the root of the content directory, not a sub folder.

ken_7dac5ac5a2 · May 2013

Richard Haseltine said:
It's also consistent with my suggestion - the Data folder is always in the root of the content directory, not a sub folder.

Yes. In my case the content directory is Scenes and it has a data subdirectory with auto_adapted. That is where the generated content is going. The DUF files are located somewhere else, which is not consistent with your suggestion that the location is determined by the scene files.

ken_7dac5ac5a2 · May 2013

Later: Just to absolutely sure, I saved a DUF file on the Desktop. In that case it created the auto_adapted folder in the first item in my DAZ Studio Formats list. It looks like the algorithm is more complicated than we expected. All the more reason to figure it out and document it correctly ;-)

Also I was partly wrong. The behavior is not inconsistent with your suggestion if you assume that "determined by the scene file" means if the scene file is saved somewhere in a content directory tree, then the created content will be saved in the data directory at the root of that content directory.

The rest of the algorithm may well be "else it is stored in the data directory of the first directory in the DAZ Studio Formats list". But this is another conjecture. ;-) I did check the conjecture by saving a DUF scene in a content directory that was neither at the top nor bottom of the DAZ Studio Formats list, and it did indeed save in the data directory at the root of that content directory.

I don't feel that scenes are content, and thus there is no reason to store them in a content tree. But that may depend on your definition of content.

jestmart · May 2013

3) Same as it’s always been, the first DS native directory you have listed in the content directory manager, that’s where the data folder gets made any time you saved a scene in DS1 to DS4.6.

ALWAYS, that is why I have a "My DS Files" library, otherwise backing up personal files would be a nightmare.

ken_7dac5ac5a2 · May 2013

jestmart said:
3) Same as it’s always been, the first DS native directory you have listed in the content directory manager, that’s where the data folder gets made any time you saved a scene in DS1 to DS4.6.
ALWAYS, that is why I have a "My DS Files" library, otherwise backing up personal files would be a nightmare.

It seems to be more complicated than that. If you save in a subdirectory of a content directory then it goes into the data directory at the root of that content directory. Otherwise it goes to the data directory at the root of the first directory. See the posts just before yours.

kitakoredaz · May 2013

I really need to say thanks Kenneth Evans
and it is important to know clear
when ds make auto_adapted files and where they save these files without telling uesr.

then I hope daz have more good way to manage such auto-adapted files.
I know some of user may say there is not heavy problem,
but I think, the way of ds managing auto-adapted files is not so smart and reasonable way.

then it is my understanding about manage ds duf files and auto-adapted.

1. I must save the scene or many duf files, under the ds contrents directory tree, sub folda.
or simply use defualt direcotrries which daz products regurally used under the ds contents directory.

(I understand,, there is not so strict rule,, but vendor usually keep the rule.
so there is already,, scene, or preset/character etc where we can use to save each duf)

if I hope to use original direcotry for my original files.
I can make sub foldas then name , arrange sub directory as I like, under the root.
but I do not save duf files. in the root ds contents directory.
I must make sub-direcotry under the root, then save it in the sub-direcotry.

2 before I save the files in the new directory out of current ds contents directory (und sub directories)
I must decide another root ds contents directory, then tell it by directory manager to ds.
after that, I save the scene in sub-directory of new root content directory.

eg if I want to save, new.duf out of defult contents directory tree,
first I make new root directory, and tell it by content manager.
so that now I have two ds contents directory (root)

when I save new duf , in one of contents directory,,
ds must save auto-adapted file under the contens directory/data which I choose.

if there is mistake,, please tell me,, I think it is reasonable way for me.

untill daz change their management about auto-adapted files,
for cr2, pp2 , or auto-fit then auto changed triax dsf.

Joe Cotter · May 2013

Ok, my brain just doesn't want to wrap itself around this discussion for some reason... can someone summarize what this is all about in a simple method my brain will accept?

ReDave · May 2013

@Gedd: it's not that complicated: content that isn't in DS Native format will be saved to the auto_adapted subfolder of the Content folder you saved the scene to. That's about it. Older DS content (DS 1&2) will presumably have to be converted yet again. Dynamic clothing, once draped and even if it is for Genesis, appears to go to auto_adapted as well, so old Dynamic content may stop working altogether (Dynamic Cape doesn't load at all for me)

Joe Cotter · May 2013

I thought about formulating a reply to that and decided it wasn't worth it. Thanks for the effort though.

Bejaymac · May 2013

Gedd said:
Ok, my brain just doesn't want to wrap itself around this discussion for some reason... can someone summarize what this is all about in a simple method my brain will accept?

Lets see if I can frazzle your noodle even more :P

Basically from the early DS betas (pre DS1) right up to and including DS4.0, nothing you loaded into the scene was in a format that was native to DS, as you loaded something into the scene DS would convert it into something it could use, and then hold all of that converted data in RAM.
When you then went and saved a .DAZ scene file DS would pick it's way through all of that data looking for specific items, any mesh items would be saved out as DSO files, any morph channels (full or empty) would be saved out as DSD files, and any UV data would be saved out as DSV files.
These files would all go into sub folders in the data folder in the first listed DS native directory, up to the end of DS2's run they were saved in the "data/mesh name_vertice count" folder system, in the case of Victoria 4.2 her DSO/DSD/DSV files would end up in the "data/blMilWom_v4b_68498" folder, the DAZ scene file only held the rig/channel info/surface settings etc.
With DS3 they changed things a little, they added an extra folder into the mix to keep DS2 and DS3 scene data files separate, so V4's files would end up being saved into the "data/3_0/blMilWom_v4b_68498" folder in DS3, they did the same thing with DS4.0.

With 4.5 we actually have content that doesn't need to be converted when you load it, Genesis and the rest of the Tri-Ax WM content, it's all in file formats that are native to DS4.5, ie the user DUF files and the DSF asset files all of which are in the DSON scripting language. When you save a DUF scene with just Genesis content DS doesn't have to go and write anything to the data folder, instead it just adds a reference in the DUF file that points to the existing DSF asset files that are all ready in the data folder, it stops the long save times and keeps the file size of the DUF scene file small.

However we still have the same problem in 4.5 as we have always had, nothing else is compatible, and needs to be converted when it's being loaded, even stuff you use the autofit or Transfer Utility on. Now however it's not saved out as DSO/DSD/DSV files, instead it's all saved as DSF asset files, which end up in the "data/auto_adapted" folder, in the case of V4 her asset files would end up in the "data/auto_adapted/blMilWom_v4b_68498" folder, and due to amount of stuff DS has to write out for this content, saves can take a while, as can loading it back in, and the DUF file can be quite big, even when it's compacted.

Joe Cotter · May 2013

Thank you very much, that was an excellent explanation. :)

I do have a couple questions based on this. I understand why the save takes so long, but loading it back, it's been converted yes? So why so long to reload? Also, if I'm in the process of converting files from a native format to an updated duf format, what exactly do I have to save and what can I delete? I think I could figure it out based on this from testing but you could shorten that process. This is actually very handy as I plan on updating many of my files to a native format with updated textures, cameras, lighting etc...

kitakoredaz · May 2013

I really hope to get option to decide where I save these all auto-udapted files.
and I need which scene file are now using them.

I can strongly recommend you never , do not delete auto-modified data,
even though it is in different ds directly which you save the scenes.

I delete the auto-adapted directory under the test contents, which I used almosot oneyears before.
then now many scene which I made and saved another daz contents directory,
show same erroer.

Now I want to cry. really cry. I do not like auto-adapted files,
so that I hope,, duf scene, duf prop which actually have dsf file as product already.

Bejaymac · May 2013

Long loading is due to having to read, load and organize several thousand files, it takes much longer to load them than it does to load one or two big files.
I've never been one for keeping my scene files, once I've got the renders out of it that I want I delete it, and then go through the data folder clearing it out too, so I'm the wrong person to ask on converting old .DAZ files to .DUF.

However I do know that if you did convert them, that the data in the DSO,DSD & DSV files would be saved into the "auto_adapted" folder as DSF asset files, if you then deleted those old files and you ran into the same issue a few people have reported, and the DUF file can't find the DSF asset files then your DUF scene file is only fit for the trash can, unless you know your way around the inside of the DSON files, in which case you might be able to save them, but only if the asset files are still on the computer.

Joe Cotter · May 2013

Thank you, it looks like I'll be doing some testing in a separate runtime :)

DAZ_Spooky · May 2013

My answer in the other thread on the Autoadapted folder was only partially correct.
The Autoadapted folder is for all content that DAZ Studio has to convert assets, which does include both Poser content and ,.daz files. The difference between the two is that most Poser files can be recreated without the Autoadapted folder (So are share-able) .daz files can not. (So are not share-able.)

Share-able is key to the error mentioned and to moving things around. As long as you have the base assets installed, for example Victoria 4, then it doesn't matter that the asset does not exist in the autoadapted folder, the scene will still work correctly, on your machine, on someone else's machine, etc. whether you move the files around or not. Whether you moved the content directory or not. Even if you delete your entire content directory and start over.

Not share-able means that the scene will not load because it can't recreate the assets if you have moved things around or changed computers.

Now, presuming that you can load your previous .DAZ scenes, and try as we might, there are some versions of DS from which we can't load .daz files. There are two ways to convert a .daz to a .DUF.

The first is to load the .daz and File>Save As>Material/Hierarchial Material Preset. You can then apply that to the Poser version and save it out as a DUF.

The second option is likely to be a bit simpler for most situations though. :) File>Save As>Support Asset> Scene Asset It will ask you where to put the user facing files, which is where you will find it. And you are done. :)

kitakoredaz · May 2013

Can we merge some data/auto-adapted directory as one?:roll:

eg

A. /dazcontents/data/auto-adatpted/aa,bb,cc,
(my default,, all contents I bought are installed "dazcontents",, and save finished scene ,or preset)

B. /mytestsdirectory/data/auto-adapted/dd,ee,ff ,,,
(I use this directory for test only. s ave duf files which I delete or which I do not like much )

C /secretscenes/auto-adapted/gg,hh,,
( important for me,, but,,,,secret duf are hidden here,,, )

I hope to save every strange long name files and strange name folda in auto-adapted,
in A dazcontents/data/auto-adapted.

if it can work,, I can save duf scene or subset to some directory and delete directory as I want.

( in which directory I only save presets and scene duf ,as subset) sometimes I just cut and paste
auto-adapetd directories to "default contents direcotry/data" they will merge one and I can keep safety for-ever!!

I hope so. but I am afraid,, it may break some files when I merge auto-adapted directories.

because I sometimes find,, there are perfectly same name folda and dsf file in diffrent directory/data
but the date of saved is different.

how duf can serch and find correct one from two "data/auto-adapted/samenamefolda/samename.dsf" ?
file path is recorded only relative path,,,I think.

if ds must choose nearest one from current scene directory,, there should be mistake.

overwrite wrong one , when save scene in another directory.
then when load another scene which use same auto-adapted file,, the assets has been changed.

then load other scene files in different directory, which used the over written auto-adapted file break.

DAZ_Spooky · May 2013

You should only have one in the first place. Moving them around causes issues especially with .daz files.

kitakoredaz said:
Can we merge some data/auto-adapted directory as one?:roll:
eg

A. /dazcontents/data/auto-adatpted/aa,bb,cc,
(my default,, all contents I bought are installed "dazcontents",, and save finished scene ,or preset)

B. /mytestsdirectory/data/auto-adapted/dd,ee,ff ,,,
(I use this directory for test only. s ave duf files which I delete or which I do not like much )

C /secretscenes/auto-adapted/gg,hh,,
( important for me,, but,,,,secret duf are hidden here,,, )

I hope to save every strange long name files and strange name folda in auto-adapted,
in A dazcontents/data/auto-adapted.

if it can work,, I can save duf scene or subset to some directory and delete directory as I want.

( in which directory I only save presets and scene duf ,as subset) sometimes I just cut and paste
auto-adapetd directories to "default contents direcotry/data" they will merge one and I can keep safety for-ever!!

I hope so. but I am afraid,, it may break some files when I merge auto-adapted directories.

because I sometimes find,, there are perfectly same name folda and dsf file in diffrent directory/data
but the date of saved is different.

how duf can serch and find correct one from two "data/auto-adapted/samenamefolda/samename.dsf" ?
file path is recorded only relative path,,,I think.

if ds must choose nearest one from current scene directory,, there should be mistake.

overwrite wrong one , when save scene in another directory.
then when load another scene which use same auto-adapted file,, the assets has been changed.

then load other scene files in different directory, which used the over written auto-adapted file break.

kitakoredaz · May 2013

YES SIR that is what I find after lost many secrets scene which made from .daz products

otehr scene which I only use poser type files,
can load assets even though I delete auto-adapted files!!

ken_7dac5ac5a2 · May 2013

DAZ_Spooky said:
Now, presuming that you can load your previous .DAZ scenes, and try as we might, there are some versions of DS from which we can't load .daz files. There are two ways to convert a .daz to a .DUF.
The first is to load the .daz and File>Save As>Material/Hierarchial Material Preset. You can then apply that to the Poser version and save it out as a DUF.

The second option is likely to be a bit simpler for most situations though. :) File>Save As>Support Asset> Scene Asset It will ask you where to put the user facing files, which is where you will find it. And you are done. :)

I have been saving DAZ scenes using File | Save As | Scene, which makes a DUF. This seems to be the most straight forward thing to do. Is there a reason you didn't mention that?

ken_7dac5ac5a2 · May 2013

DAZ_Spooky said:
You should only have one in the first place. Moving them around causes issues especially with .daz files.

There can be more than one auto_adapted directory. As mentioned above, the (undocumented) algorithm seems to be:

If you save in a sub-directory of a content directory then it goes into the data directory at the root of that content directory. Otherwise it goes to the data directory at the root of the first directory in the DAZ Studio Formats list.

So you get an auto_adapted directory in every content directory in which you save a scene in one of its sub-directories and possibly one other in the first directory. This seems to be the part that people are unaware of. It wasn't mentioned, for example, in the excellent summaries by Bejaymac and statements abound that the auto_adapted directory always goes to the data directory at the root of the first directory.

For me, I am saving to what used to just be a directory with scenes, but which now has the generated data/auto_adapted stuff. As long as I always save somewhere in that directory, I should only have one auto_adapted directory. This should make it easier to manage generated content, and is a great improvement IMHO to what used to happen with DAZ files where the generated content is all over the place.

For example, when DIM came out, I removed all my old content and started over just using DIM. I soon discovered I could not open any old DAZ scene files. To solve this problem, I added the old directories at the end of my new content directories, even though I would have have preferred to take them off the computer and even though only the created content is needed for the DAZ scenes and the installed content is duplicated. I haven't found an easy way to separate the created content out. This should much easier now that everything is under auto_adapted.

DAZ_Spooky · May 2013

Kenneth Evans said:
DAZ_Spooky said:
Now, presuming that you can load your previous .DAZ scenes, and try as we might, there are some versions of DS from which we can't load .daz files. There are two ways to convert a .daz to a .DUF.
The first is to load the .daz and File>Save As>Material/Hierarchial Material Preset. You can then apply that to the Poser version and save it out as a DUF.

The second option is likely to be a bit simpler for most situations though. :) File>Save As>Support Asset> Scene Asset It will ask you where to put the user facing files, which is where you will find it. And you are done. :)

I have been saving DAZ scenes using File | Save As | Scene, which makes a DUF. This seems to be the most straight forward thing to do. Is there a reason you didn't mention that?
Because that method leaves you with the issues inherent in the .daz files. That isn't converting them to DUF files.

ken_7dac5ac5a2 · May 2013

kitakoredaz said:
YES SIR that is what I find after lost many secrets scene which made from .daz products
otehr scene which I only use poser type files,
can load assets even though I delete auto-adapted files!!

Hmm. I inadvertently removed my Scenes directory (the one that has auto_adapted) from my content directories, effectively deleting it as far as DAZ is concerned. I was not able to open my DUF files that contained Poser content. I got a dialog about not being able to find assets. The listed assets were all in data/auto_adapted. In fact, these directories were explicitly referenced in the DUF file. Restoring it fixed the problem and the DUF files loaded again.

ken_7dac5ac5a2 · May 2013

DAZ_Spooky said:
Because that method leaves you with the issues inherent in the .daz files. That isn't converting them to DUF files.

I don't understand. Could you elaborate. It does create DUF files that load fine (so far).

Richard Haseltine · May 2013

A .duf file can point to pretty much anything DS can load, what Spooky is talking about is converting the content into DS4 assets instead of DS3 or earlier items so that the .duf is pointing to simple native objects, not objects that need to be imported.

ken_7dac5ac5a2 · May 2013

Richard Haseltine said:
A .duf file can point to pretty much anything DS can load, what Spooky is talking about is converting the content into DS4 assets instead of DS3 or earlier items so that the .duf is pointing to simple native objects, not objects that need to be imported.

Thanks Richard. I saved the DAZ scenes in DS 4.5. Would they not be DS4 files. Are these the files in auto_adapted? If not, what are the files in auto_adapted.

Perhaps I don't understand what gets converted into simple native objects and what needs to be imported.

I just saved a DUF scene created using ClydePuffer. (No DAZ scene involved.) In the DUF I file I see:

 "asset_info" : {
  "id" : "/data/auto_adapted/ClydePuffer25_63029/geometry_7f85c2c4_3f94_6c88_cf15_2e031ebf21f5/ClydePuffer25.dsf",
...
   "legacy_source_info" : {
    "location" : "/Runtime/Geometries/ClydePuffer/ClydePuffer25.obj",

as the only file references to ClydePuffer. Is one of these a native object and one a need-to-be-imported object?

Maybe I should just ask what is a simple native object and what needs to be imported. If the .obj needs to be imported, is that a problem? OBJ is a standard format used by most 3D programs.

BTW I looked up "Support Asset" and "Scene Asset" and found nothing entered yet in the documentation. I don't know what they are. In general, I guess I don't know what the menu items that DAZ_Spooky mentions mean.

I am naturally concerned that the obvious choice for saving a scene is not a good one, especially since that is what I have been doing.

Notifications

Converting DAZ Scenes to DUF Scenes and auto_adapted

Comments

Adding to Cart…