LOD System

2»

Comments

  • TugpsxTugpsx Posts: 819

    BTW the script does not work on DS2025/6 as expected. surprise

    For those on a Mac, this product will not work directly on MacOS unless you are running DS in WineSkin or something that supports Windows exe files.
    The file list shows that there is an executable file required for the scripts function. 

  • keyclapkeyclap Posts: 0

    TiZ said:

    maguseveretus said:

    But then I ran the LOD operation on the scene, and when it was done, somehow the scene size, according to the tool's own measurements, ballooned up to 11541MB!! How in the world did that happen?? And it doesn't seem to be a glitch in how it is calculating the value; the scene won't render in Iray unless I undo it. So it really is growing the scene somehow.

    The reason why it went from 4554MB to 11541MB is because the script searched for original textures first, undoing all the stuff Scene Optimizer did. 

    Are you the author of the script? If so, what justification do you have for undoing the work that Scene Optimizer has already done? Especially without my consent? Like I said before, Scene Optimizer is the reason the scene was fitting in VRAM to begin with. If you want to re-resize the textures that Scene Optimizer made, that's probably a good idea, because most users probably don't have access to a script that conveniently redoes Scene Optimizer's resizing. But going back to original textures without consent and making the scene no longer fit in VRAM is an extremely confounding action to take.

    It's frustrating to know that I'll have to rename my character's textures to prevent you from changing them just because you think you know better.

    I understand your frustration. The reason why the script works this way is to support non-destructive workflow, so it always tries to look for original textures.

    I'll add a toggle that can switch between using original textures and using Scene Optimizer textures.

  • TiZTiZ Posts: 26

    keyclap said:

    TiZ said:

    maguseveretus said:

    But then I ran the LOD operation on the scene, and when it was done, somehow the scene size, according to the tool's own measurements, ballooned up to 11541MB!! How in the world did that happen?? And it doesn't seem to be a glitch in how it is calculating the value; the scene won't render in Iray unless I undo it. So it really is growing the scene somehow.

    The reason why it went from 4554MB to 11541MB is because the script searched for original textures first, undoing all the stuff Scene Optimizer did. 

    Are you the author of the script? If so, what justification do you have for undoing the work that Scene Optimizer has already done? Especially without my consent? Like I said before, Scene Optimizer is the reason the scene was fitting in VRAM to begin with. If you want to re-resize the textures that Scene Optimizer made, that's probably a good idea, because most users probably don't have access to a script that conveniently redoes Scene Optimizer's resizing. But going back to original textures without consent and making the scene no longer fit in VRAM is an extremely confounding action to take.

    It's frustrating to know that I'll have to rename my character's textures to prevent you from changing them just because you think you know better.

    I understand your frustration. The reason why the script works this way is to support non-destructive workflow, so it always tries to look for original textures.

    I'll add a toggle that can switch between using original textures and using Scene Optimizer textures.

    Oh, you're the developer! Thanks for understanding my frustration.

    I disagree with the users that state that these scripts should not be used together, or the implication that having done so invalidates my frustration. These are different tools that do different things, even if the purpose for which they're doing it is the same, and they also convey different information. Scene Optimizer won't tell me how much VRAM a scene takes, but it does tell me what the biggest maps on every node are. So I can be like "Oh, no wonder it doesn't fit, this entire environment is a buttload of 4096px maps". And then it allows me to take a specific action on a given list of nodes without any sort of guesswork or heuristics. I can say "I know that this entire environment will not fit in my VRAM as-is; please halve the texture sizes of everything in it." And it just does that. (Even though the resize quality is bad.) Meanwhile, the heuristics in the LOD System are a very different yet very useful approach. The fact that it can find maps that don't actually convey any useful information and convert them to values is amazing. Combining bump and normal maps is also amazing. And the fact that it uses distances to determine which maps don't contribute much to the scene is very useful too. But the algorithms it uses to do what it does are a bit opaque. Like, if I enable the "extra resize" slider, what is that going to do, on which nodes is it going to do it, and what is it using to make its determinations? Anyways, the point is that these tools don't overlap completely.

    Your stated rationale makes a lot of sense. The implementation toward that end--or the effect that actually occurred--does not accomplish that rationale as it is now, though, in my opinion. I think it makes sense to keep track of what all the textures were before the script was run, but reverting to the original textures from before Scene Optimizer was run is like... going even further back in time, so to speak, and I don't need or want the script to do that. In attempting to be non-destructive, it was unfortunately quite destructive. I would like it to treat whatever textures it sees when it is first started in a given scene as the original textures. So I guess the option I would like to see isn't so much "use Scene Optimizer textures" but rather "consider current textures to be the original textures". The 3072px maps I made for my character were named like Scene Optimizer textures, but I used ImageMagick to make them. Fortunately, your tool only looks for the specific naming pattern of NxN, so I can use a slightly different one to keep it from going to a higher size when I don't want that.

    Since the tool is currently insistent on original textures, I pulled a little trick; I batch-resized the Snow Village's textures into a content directory that takes precedence over the directory where the original textures live. That way, the Snow Village will always be using 2048px textures maximum no matter what, even when loaded in fresh. So when I load that christmas scene and run the LOD System on it, it takes it from 4554MB to 3520MB! 1GB for almost free; that's awesome! So when you consider that the scene with its original textures was 12GB and LOD System wouldn't take it below 11GB with its default settings, it makes perfect sense to use these tools together. 12GB to 3.5GB between the two tools is a seriously massive and impressive VRAM saving.

    If you're currently amenable to more changes: I can see that there is debug info in the logs, but maybe creating a checklist of changes for users to verify before actually applying them would be a good addition.

  • TiZ said:

    Oh, you're the developer! Thanks for understanding my frustration.

    I disagree with the users that state that these scripts should not be used together, or the implication that having done so invalidates my frustration.

    I wasn't trying to invalidate your frustration. I called it "a bit" unwarrented. You used the phrase "without consent" (or the equivalet) several times with bold and other highlighting, for example. That was a bit hyperbolic.

    And as I said, expecting them to work out of the box together wasn't something I'd expect. Since the dev has said they'll make their script aware of the other, that confirms they currently shouldn't be used together because they aren't aware of one another.

  • Can you please make a tutorial video with less memes more instructional for using your product? Ive tried using it on a scene with two characters and all default settings and nothing changed. Double checked with scene optimizer and all textures remained the same. Looking to reduce render times.

    Thanks in advance.

  • mark128mark128 Posts: 1,033

    I did a little benchmark on LOD. The scene uses the PW Lake Mansion with 15 Genesis 9 characters at varying distances from the camera. My computer has a Ryzen 9 9950X3D with 96GB of RAM and a NVIDIA GeForce RTX 5090. The scene was setup in Studio 4.24.0.4 and the LOD script was run in that version also. The rendering was done in 6.25.2025.35308.

    I have a 4K display and without DAZ running I am typically using about 1.5GB of memory on the graphics card to support the display. With DAZ Studio running and the unreduced scene loaded I was using about 6.9GB on the graphics card. While rendering this jumped to 28.8GB on graphics card. 

    I don't completely understand the controls on the LOD script but I ran it with default parameters. With DAZ Studio running the LOD reduced scene I was using 4.8GB or memory on the graphics card. While rendering this jumped to 14.8GB. 

    If I subtract the before render memory usage from the rendering usage, the original scene used 21.9GB and the reduced scene used 10GB. That is a huge reduction in VRAM usage. Note that the texture reduction also appeared to reduce the VRAM usage by the default Texture Shaded display at least in DAZ 6. 

    The reason for this large reduction is that LOD is very aggressive at downsizing maps. The characters near the camera appear to have maps on surfaces other than the head reduced from 4K to 512. On the distant characters they were reduced to 256. On Characters that had LIE makeup the maps on the head were not reduced at all. LOD seems to reduce maps aggressively on clothes and hair also.  

    I could not find any case where the maps on the Mansion had been reduced although I only checked a few. The maps on the Mansion are not huge and seem to have been fairly carefully sized to produce good results. 

    I have attached the render of the original scene and the reduce scene. If you compare them, you can see some differences in the reflections on shinny material like hair and some of the dresses. I'm not sure why this is happening. If I get some time I will see if I can figure out what is going on. 

    Big_test.jpg
    1920 x 1080 - 748K
    Big_Test_LOD.jpg
    1920 x 1080 - 746K
  • ArtiniArtini Posts: 10,460

    Great comparison, @mark128 and your insights.

    Please keep posting.

     

  • TiZTiZ Posts: 26

    mark128 said:

    I have attached the render of the original scene and the reduce scene. If you compare them, you can see some differences in the reflections on shinny material like hair and some of the dresses. I'm not sure why this is happening. If I get some time I will see if I can figure out what is going on. 

    It's possible that the heuristics it uses to determine whether a map is redundant is deciding to replace glossy/specular maps with static values. If you click on the settings cog in the script's main window, you can change the Remove Threshold to 0. You'll probably get less VRAM savings, but it might not affect the shininess of objects as much.

  • kprkpr Posts: 347

    Another thanks to @Mark128 for the info and the comparison images smiley

  • MrRogerSmithMrRogerSmith Posts: 133
    edited February 13

    I was interested in using the script to decrease my render times.  That's not specifically advertised as a benefit of the script but I thought that if the VRAM usage was being lowered then the render time would be faster.  Here are my test results using the tool's default settings.

    I've included a viewport screenshot so you can see that I'm working with a pretty detailed scene.

    Then I've shared a render without the script, and then a render with the LOD script.  As you can you see in the version using the script the glass texture of the background door has been removed, making this an unusable render.

    I've also screen-shotted my GPU performance for you to see the difference.  The default scene uses 15.7 GB of memory, while the script version uses 12.3 GB of memory.

    Finally, I report that the difference in render times for this 1920x1080 image is 20:37 without the script, and 19:46 with the script.  Personally I find this to be neglible, but I suppose it could add up if I was doing an animation.

    I found the PDF that's included with the download to be not helpful.  It briefly describes what the features of the script are without actually explaining how they work or how to use them to improve our results.  For example, my render is missing a texture (the glass) that is critical for the scene, and yet after reading through the PDF I have no idea what setting is responsible for this and how to fix it.  Frankly the PDF is too technical because it's language is confusing and yet not technical enough because it leaves so much information out.

    I further found the YouTube video on this product to be juvenile, obnoxious, and unprofessional.  I could tell that the PA was making fun of the Scene Optimizer script but I wasn't able to actually tell what he was doing with his own script.  At first I thought Scene Optimizer was the product being advertised, but then I got confused once I realized he was actually making fun of it.  Then he was making a bunch of manual changes and I'm not sure if he was still using Scene Optimizer or if he'd switched over to his own product.  I recommend that keyclap remove that video and try again, this time with a video which is actually helpful in showing us how to optimize his script.  I can hear keyclap now saying, "God, this Roger Smith is really an idiot if he didn't understand the video and the PDF."  And yes, that's true, I am an idiot, but I'm also a customer, so you need to learn how to communicate to an idiot like me if you want to sell to an idiot like me.

    Personally, I don't find this product to be worth the money but I would be happy to reconsider if the keyclap addresses the issues detailed here.

    7-2-4 (LOD script).png
    1920 x 1080 - 3M
    7-2-4 (original).png
    1920 x 1080 - 3M
    entire scene.png
    2215 x 1873 - 3M
    with LOD script gpu.png
    3838 x 2083 - 164K
    without script gpu.png
    3838 x 2085 - 167K
    Post edited by MrRogerSmith on
  • mark128mark128 Posts: 1,033

    I could not find any case where the maps on the Mansion had been reduced although I only checked a few. The maps on the Mansion are not huge and seem to have been fairly carefully sized to produce good results. 

    I was wrong about this. LOD did reduce a lot of the maps on the Mansion, just not the once on the outside of the Mansion facing the camera. This set is huge. The house is fully furnished and all the furniture and applicencs in the house have texture maps. Many of the maps from inside of the house were aggressively reduced.

     TiZ said:

    It's possible that the heuristics it uses to determine whether a map is redundant is deciding to replace glossy/specular maps with static values. If you click on the settings cog in the script's main window, you can change the Remove Threshold to 0. You'll probably get less VRAM savings, but it might not affect the shininess of objects as much.

    I don't think this was the cause.  I think it was the reduced resolution on of glossy/specular/top coat maps. Many of those maps got reduced from 4k or 2k to 256 and were so low resolution they didn't really work anymore.

    I fiddled around to try to recrease the downsizing of the maps in the places it was causing problems. I increased the Distance Factor to 12, its max value which seemed to cause it to do less downsizing near the camera. I then increased the Clarity Radius to 500 and that stopped all downsizing of maps on the two characters closest to the camera. I then discovered on the setting panel there is a min resolution which was 256. I increased it to 512 but this had no effect. No sure what this does but LOD continued to downsize most maps to 256 on anything far from the camera. 

    There is a Reset objects to original resolution button (with the blue globe). If you check the Selected Only box, this will reset the maps only on the objects you have selected. Otherwise it does all the maps in the scene. I used that to undo the maps on a couple of dresses. I then rendered the less reduced scene. The orginal and this less reduced scene are attached.

    The orginal scene used 21.9GB of VRAM. The first reduced scene used about 10GB. This less reduced scene uses 12GB of VRAM, still a significant reduction. 

    The less reduced scene still has a few reflection differences on some of the characters far from the camera but fewer than the first try and much less noticable. 

    I wish the minimum resolution actually worked. I could first aggressively reduce the scene, then remove that on some objects and redo with a less aggressive resolution. Right now I don't know if it was a bug that it didn't do what I expected or I just don't understand what it is suppose to do.

    I have not been keeping track of render times, but I don't think LOD is improving render time much at all on this scene.

     

    Big_test.jpg
    1920 x 1080 - 748K
    Big_Test_LOD2.jpg
    1920 x 1080 - 747K
  • TiZTiZ Posts: 26
    edited February 14

    I think that folks trying to use this to reduce render time are barking up the wrong tree, so to speak.

    A render time reduction could occur if one of the changes LOD System makes ends up reducing the complexity of the scene in terms of raytracing, but the way Iray uses VRAM is not like video games. It's not a thing where it will find a way to make it work but slow things down if the scene doesn't fit in VRAM. If it doesn't fit, you get CPU Iray unless you have disabled CPU rendering. In that sense, yes, you can get an improvement in render time just because any supported GPU will always render faster than any CPU does. But reducing VRAM usage on a scene that already fits will not improve your render time.

    Of course, getting a scene that doesn't fit in your VRAM to fit in there is a huge deal. I'm working with 8GB of VRAM myself, so it's especially so for me. So this product is valuable, but not for render time reasons. Some vendors sell environments with the highest-quality textures possible, without themselves providing a way to reduce the resource consumption. So many times have I loaded an environment and found myself unable to render it without crunching down its textures somehow. It's happened so often that I made a shell script (not a DAZ script) that batch-resizes textures to a higher-precedence content directory.

    If you want to improve render times, you have to make compromises to your scene or render quality somehow. Like using less iterations but applying a denoiser, rendering at a lower resolution, removing objects that add complexity to the scene, or for interior scenes, using an Iray cutout plane to let light from an HDRI in.

    Post edited by TiZ on
  • TiZ said:

    I think that folks trying to use this to reduce render time are barking up the wrong tree, so to speak.

    I was going to say this. The only time savings would probably be the time saved moving 7GB to the GPU card instead of moving 12GB to the card.

  • ArtiniArtini Posts: 10,460

    Thanks for comparisons and additional info.

    My GPU on laptop has only 8 GB of video ram, so this system probably does not help me much.

     

  • mark128mark128 Posts: 1,033

    Artini said:

    My GPU on laptop has only 8 GB of video ram, so this system probably does not help me much.

    Actually I think it could help you a lot. With only 8 GB of VRAM many scenes will not fit on your graphics card and will be using CPU rendering which is much slower. Using this or the other similar product (Scene Optimizer) you will be able to get larger scenes to render with you GPU which will make them render much faster.  You aren't going to be able to do big scene like my test scene with 15 Genisis 9 characters but it will allow you to do larger scenes with more characters.

  • mark128mark128 Posts: 1,033

    I decided to try Scene Optimizer on this scene for comparison. In Scene Optimizer you need to select which objects you want to reduce. This scene has maybe a thousand objects to sort though, most of them in the mansion. I unselected the two Gensis 9 characters nearest the camera and their clothes and hair and reduced everything else by a factor of 2x. I then undid the reduction on the water and dock. The original scene used 21.9GB to render. This Scene Optimizer reduced scene used 17.7GB. I have attached the original render of the full scene and the Scene Optimizer reduced scene.

    I could save all the background characters off as a scene subset and reduce them by 4x instead of 2x to save more VRAM, but Scene Optimizer is much slower and more cumbersome to use than LOD. LOD is much faster and the ability to quickly reset the maps to the original on all or selected objects is very useful. I really like the idea that LOD adjusts the amount of reduction based on distance from the camera. The only problem is that LOD seems to be too aggressive in downsizing. The document PDF says the minimum map size is 256 pixel and LOD is using that size for all the Characters except the ones near the camera. By changing the parameters on LOD I was able to stop or reduce the downsizing near the camera but nothing I could do would stop the downsizing on the distant characters to 256. 

    If you press the setup button it brings up a panel with a minimum resolution setting which was set to 256 but changing this has no effect. This option is not mentioned in the documentation PDF so I suspect it is not finished yet.  If I could control the minimum map size to 512 or 1024 I think LOD would be much more useful. At present you can easily undo all the downsizing on selected objects but it is much harder to get reduced downsizing on distant objects. The only way I can see to do it is to save them out as scene subsets and load them into DAZ Studio with a camera much closer to get reduced downsizing, then transfer them back to the scene. 

    I'm not sure if LOD is still being developed but I hope them implement the min resolution feature. 

    Big_test.jpg
    1920 x 1080 - 748K
    Big_Test_SO.jpg
    1920 x 1080 - 742K
Sign In or Register to comment.