V3D Magic Extract And Append Bundle (Commercial)

2»

Comments

  • V3DigitimesV3Digitimes Posts: 3,416
    edited May 10

    Thank you, this confirmation helps a lot.

    It matches what I am seeing on my side: all files even large files can work in DS4, small/medium DUF files can work in DS6, but larger or more complex DUF files may hang or crash during the initial source analysis step in DS6, before the item list is displayed.

    I am already working on a possible solution, and I have some promising leads. It may be possible to use a safer analysis method for large files in DS6.

    If you have any way to share one of the problematic DUF files with me, that would be extremely helpful (you can PM me). Testing against a real file that triggers the issue would make the fix much more reliable.

    Post edited by V3Digitimes on
  • ArtiniArtini Posts: 10,649

    Interesting scripts. Glad, I am using only DS 4.24 at the moment, but plan to check DS 6 later at some point.

    Hopefully, you will fix most of the issues by then.

    All the best for you.

     

  • barbultbarbult Posts: 26,808

    I emailed you some DS4 information (links to Dropbox) I couldn't get Dropbox to sync the scene file itself, so I only sent screenshot and log info. In DS4 with the large scene, the append script displayed the list and let me select a G9 character. When I continued, the script displayed an error message.

    That is the only info I have at this point.

  • V3DigitimesV3Digitimes Posts: 3,416
    edited May 10

    Artini said:

    Interesting scripts. Glad, I am using only DS 4.24 at the moment, but plan to check DS 6 later at some point.

    Hopefully, you will fix most of the issues by then.

    All the best for you.

     

    Thank you!

    Yes, I hope so. I’m doing my best to fix DS6-related issues as they appear.

    That said, DS6 is still not fully stable, so fixing a bug today does not always guarantee that the same script will behave the same way a few weeks later. This is true for any script and any DS6-related bug at the moment: the target is still moving.

    But I’ll keep following the DS6 changes and updating the scripts when needed.

    All the best to you too.

    Post edited by V3Digitimes on
  • barbultbarbult Posts: 26,808

    I tried the big scene in DS6 again and monitored the RAM usage as the script ran and crashed. There was no big spike.

    Screenshot 2026-05-10 103309.png
    1126 x 638 - 62K
  • barbultbarbult Posts: 26,808

    You might want to consult with Totte (Code66). He has experience with DS6 scripting and lists. I know he is aware of some issues, but I don't know if they are related to what you are doing.

  • V3DigitimesV3Digitimes Posts: 3,416

    Thank you, Barb. The RAM information is useful too. If there was no big spike before the crash, that is one more indication that it may not be a simple memory exhaustion issue.

    About the list itself, I don’t think this is the part involved here. I generated a test list with about 30,000 items without issue. But yes, I may ask Totte if he has already run into instability issues with XXL DUF files in DS6.

    In my own tests, using virtual DUF files, I found that the DS6 issue happens just after the selected DUF file is read, even before the script starts parsing it for analysis. So at this stage, it looks more like a DS6 instability with very large DUF files than a list display issue. I already have two workaround/patch options for that part now.

    Honestly, the only thing really missing for me is a real source file that triggers the crash, or a real source file that does not crash but ends with an error instead. That would allow me to solve the issue based on actual data, not only on hypotheses from artificial test files.

    BTW, I answered your email. Thanks a lot for the extra information and testing.

  • barbultbarbult Posts: 26,808

    The temp file that was created when the error occurred in DS4 was 0 bytes. It was still 0 bytes after I clicked OK on the error message. It was still 0 bytes after I refreshed the folder containing the file. It appears that nothing was written to that temp file.

    I tried again, but this time I selected a different item from the scene (not a G9 character). It was appended successfully. 

    In another attempt to append the G9 character, I monitored the RAM usage. It got very high, close to the limit of what I have. I attached the screenshot. The two sharp pointy peaks is when the error occurred. I don't know if that is the cause of the problem or not. The RAM usage did not get that high when loading the successful non-G9 prop.

     

    Screenshot 2026-05-10 105926.png
    1688 x 957 - 256K
  • V3DigitimesV3Digitimes Posts: 3,416

    barbult said:

    The temp file that was created when the error occurred in DS4 was 0 bytes. It was still 0 bytes after I clicked OK on the error message. It was still 0 bytes after I refreshed the folder containing the file. It appears that nothing was written to that temp file.

    I tried again, but this time I selected a different item from the scene (not a G9 character). It was appended successfully. 

    In another attempt to append the G9 character, I monitored the RAM usage. It got very high, close to the limit of what I have. I attached the screenshot. The two sharp pointy peaks is when the error occurred. I don't know if that is the cause of the problem or not. The RAM usage did not get that high when loading the successful non-G9 prop.

     

    Thank you, Barbult. This is extremely useful information.

    The fact that the temporary file is 0 bytes is very important. It means the problem is probably not that DS4 cannot reload a malformed generated DUF. It is more likely that the script fails before anything is actually written to that temporary DUF, and then DS4 tries to merge an empty file, which naturally ends with a “File Format Error”.

    So that changes the diagnosis a bit: the error message appears at the reload step, but the real failure probably happens earlier, during the temporary DUF creation/export step.

    The fact that another non-G9 prop from the same scene appended successfully is also very useful. It suggests that the scene is not simply unreadable, and that the list/scanning/ part is not the main issue here. The problem seems more specific to extracting that G9 character from this large scene.

    Also, do you remember if there is anything special about that G9 character in the source scene, anything that could make this character different from a more “standard” G9 character? I don’t know yet if this is related, but since the non-G9 prop appended successfully from the same scene, anything specific to this character may help. And even specific technologies on its wearable could matter.

    The RAM peaks are suspicious too. I cannot say yet whether they are the direct cause or just a symptom of the failure, but if RAM gets close to the limit only when extracting the G9 character, it may explain why the temporary DUF remains empty.

    I will add specific checks for this case, so the script can detect a 0-byte temporary DUF and stop with a clearer diagnostic instead of trying to reload it. I am also working on alternative extraction paths / safer workarounds for very large or complex DUF scenes.

    THANKS for your scene, just received it, I'm gonna work on it right now!

     

  • barbultbarbult Posts: 26,808

    The G9 character appends in DS4 if I don't also load children nodes. So maybe one of the children nodes is the problem - maybe the hair? (FE Chic Layered Bob Hair)

  • V3DigitimesV3Digitimes Posts: 3,416
    edited May 10

    Lol, I just came here to say the exact same thing : I am investing the children to see which one fails...
    EDIT : and yes, I cannot load : FE Chic Layered Bob Hair...
    I just need to know why...
    EDIT2 : buying the product right now to be able to analyze it.
    EDIT 3 : stand alone, directly from its product, not simulated, the hair loads properly : loading on a figure, simulating on the timeline 0-6 frames... Creating a new scene: Reloading timeline simulated hair without issue, with its animation (to be confirmed).
     

    Post edited by V3Digitimes on
  • barbultbarbult Posts: 26,808

    Here is a strange thing (DS4). I created a scene with a G9 Dev Load, so no eyes, mouth, etc.. I fit the FE Chic Layered Bob Hair to G9. Then I did a 30 second timeline pose transition of G9. Finally, I did a current frame simulation of the hair on frame 30. I saved the scene. I can append that G9 with children just fine. But if I append without children I get an error message about missing data files for the hair. Since I am not appending children and the hair items are not checked, why is it even looking for hair data files while appending only G9? G9 does get appended in spite of those error messages.

    I'll put the scene on Google Drive for you.

    Screenshot 2026-05-10 130734.png
    847 x 1163 - 96K
  • V3DigitimesV3Digitimes Posts: 3,416

    Thanks, I am on the same path of tests, so this information is crucial..

    Yes, if you append G9 without children, the script should not need the hair data files if the hair nodes are not selected and not included as children. So if DS4 still looks for FE Chic Layered Bob Hair data files, that probably means my extraction filter is still catching some hair-related data somewhere — most likely through animation / simulation / modifier data linked to the fitted hair or to the figure.

    This was actually one of the hardest parts of the development: deciding exactly which pieces of information must be kept or removed depending on the user choices. A DUF scene contains a lot of cross-references, and during development I had to go through thousands and thousands of lines to avoid keeping too much or removing too much.

    So yes, it looks like I missed one case here, but compared to the amount of filtering already done, this should be manageable. I will probably have to review the filter for child-related animation / simulation data, especially for fitted or simulated hair.

    In my own quick test I did not reproduce the same problem yet, but I may not have saved exactly the same simulation state / cached state :  I did not freeze. Your smaller test scene will be very useful, because it should be much easier to analyze than the big scene.

  • V3DigitimesV3Digitimes Posts: 3,416

    I will have to stop for today, but I will continue investigating this tomorrow.

    At this point I have two main directions to check:

    First, how the script should handle simulated hair data, and maybe other simulated items too, especially when the simulation is saved on the timeline or linked to fitted children.

    Second, how to handle very large / complex DUF files more safely in Daz Studio 6. I have already made good progress on that part, but it still needs more development and testing.

    Your smaller test scene will be very useful, because it should help me isolate the hair/simulation filtering issue without having to work only from the huge scene.

    Thank you again for all the tests and information. This is really helping me narrow the problem down.

  • V3DigitimesV3Digitimes Posts: 3,416
    edited May 11

    A small teaser of what’s coming in the next update...


    Preliminary tests are looking good: you should be able to reload assets with or without standard timeline animations, and with or without dForce simulation data for timeline-simulated nodes.

    All combinations are possible.... because apparently one checkbox was not enough. ;)

    More to come soon - for this next update...

    More coming in the next update (1.2, not submitted, developped the 10 and 11/05/2026):

    - Group nodes are now displayed and can be reloaded, either as empty parent nodes (to catch location/orientation for instance) or with their children, depending on the “Also load Children Nodes” option. his is implemented, but still needs to be tested with a wider range of scene configurations.

    - The interface now displays parent/child hierarchies, closer to what you see in the Scene pane.

    - Animation data is now filtered more safely: animations belonging to nodes that are not imported should no longer be kept in the generated DUF.

    - Simulation data from non-imported geometries is now cleaned from the generated DUF file. Said differently : dForce simulation cache data can now be stripped from the generated DUF, without removing the dForce modifiers themselves.This one was on me: I had tested simulated dForce figures, but not timeline-simulated ones...

    - For nodes that were animated in the original scene, you can now decide separately whether you want to keep (or not) standard timeline animations, and/or dForce simulation data.

    For example, if you load an animated figure with timeline-simulated dForce hair, you can now choose between several combinations:
    keep the figure animation but remove the hair simulation, remove the figure animation but keep the dForce hair simulation, keep both, or remove both.

    So yes, you can even end up with a perfectly static figure and animated dForce hair. Which is probably useless, but pretty funny. ;)

    extractor 12 ex1.JPG
    457 x 219 - 22K
    Post edited by V3Digitimes on
  • jjoynerjjoyner Posts: 793

    @V3Digtimes,

    Thanks much for your hard work and persistence in improving all of your products including this one.  I bought the bundle on Saturday and played a little yesterday with extracting props and figures.  I look forward to exploring the three sets of scripts more.

  • franky85franky85 Posts: 155

    I just want to thank you for making this. I don't have much experience with it yet, but I have used it 3 times over the week-end and is a huge timesaver in my workflows.. so thank you so much! The amount of scene assets / scene subsets I've saved over the years because I wanted a specific hair with specific morphs applied, or clothes, or camera, or whatever.. now everything is much faster and just a few clicks away without the constant bouncing back and forth between scene files !

  • This has worked beautifully for me. I'm a little intrigued about why it gave me a message about  not being able to find monks robes when the scene is doctors in scrubs, but it did just what I wanted it to.

  • V3DigitimesV3Digitimes Posts: 3,416

    @jjoyner

    Thank you so much for your kind words, and also for your support. It really means a lot. This bundle took a lot of persistence indeed, because DUF files can be surprisingly deep and tricky, but I am very happy that it is finally out in the world and already useful. I hope the three sets of scripts will save you a lot of time once you start exploring them more.

    @franky85

    Thank you so much! This is exactly the kind of workflow problem I hoped the scripts would help with. In my own 3D development work, I often need to retrieve, compare, or reuse specific elements from previous scenes, scene subsets, or development files, without constantly reopening and navigating through the whole source scene. So I am really glad to read that it is already saving you time in real use.

    @jhogenboom_da69c035a9

    Thank you very much, I’m really glad it worked beautifully for you.

    About the “monks robes” message in a doctors-in-scrubs scene: yes, that is exactly the kind of strange DUF filtering case I am very interested in investigating. Sometimes when I correct a DUF filter, the fix covers a whole family of cases, but DUF files are so complex and nested that some extra node-related data can hide in places where you would not expect it at all.

    So I would be very interested in the DUF file, because it would allow me to confirm whether the fix I implemented yesterday already covers your case, or whether this is a new hidden case that I should handle properly too. Of course, only if you are OK with that and if it is not a problem for you to share it. If you prefer not to, I completely understand, no worries at all.

    If you are OK with sharing it for analysis, we can continue by PM and find the easiest way to transfer it depending on the file size.

  • V3DigitimesV3Digitimes Posts: 3,416
    edited May 12

    Going on with version 1.2 development.

    I am currently testing the new hierarchical interface display with a heavily nested scene, including figures parented to bones of outfits/figures nested inside other figures, already nested in other props/figures/other nodes, where some "fit to" outfits are also children of children props..

    The hyper-nested loading itself already works in the current 1.0 version. The real work here is about the interface: displaying this kind of complex hierarchy in a clear and usable way before loading.

    Of course, the hierarchy can be collapsed and expanded.

    Bones are not displayed as separate items, since they are not useful geometry import targets. However, when an item is parented to a bone, it is displayed under the corresponding figure, and the bone name is mentioned directly in its label, for example: [parented to: Right Hand].

    And yes, what you see here is a typical dev test file: ugly, heavily nested, and intentionally made to stress-test the scripts.


    SUPER NEST.JPG
    612 x 946 - 72K
    Post edited by V3Digitimes on
  • jmucchiellojmucchiello Posts: 1,260

    I finally put this to the test. I didn't explore it fully. For just loading a figure, it seems fine. I haven't dug into the other scripts.

Sign In or Register to comment.