Polish Subway Huge VRAM demand
bailaowai
Posts: 44
in The Commons
I bought the subway set from Polish, and I'm finding that it has huge, gigantic memory requirements. I've tried set 1 and set 4 and both fallback to CPU rendering because they won't fit in my measly 6GB of VRAM (980ti). The log indicates that set 1 needs 29GB of texture memory, and set 4 needs 24GB of texture memory. Wow! I've actually not run into this before, even with some of Stonemason's huge sets.
Is there a way to cut off parts of the scene that I don't want to minimize the memory requirement so that it can GPU render?
Daz 3D is part of 
Connect
DAZ Productions, Inc.
224 S 200 W, Suite #250
Salt Lake City, UT 84101
Licensing Agreement | Terms of Service | Privacy Policy | EULA
© 2024 Daz Productions Inc. All Rights Reserved.
Comments
In the render settings, have you tried "Optimize for memory" rather than "Speed"? Not sure if this helps, but it might.
But you are right, this is extreme... I wouldn't even be able to render it in CPU mode.
Is that the new Cyberpunk one? I just ran into that type of memory issue with the Antebellum Mansion, which is gorgeous, but requires 11 GB for textures on full preload.
Strange, looking at the Cyberpunk Subway, it doesn't mention the size at all:
Maybe there are just a lot of objects. Here's a couple of ideas, you've probably thought of these already:
Octane Render
out of core textures
not that I own either set
Daz really needs to start posting requirements on the product pages.
The only solution I can come up with off the top of my head is to delete textures from the material zones that you don't see. That might be enough to knock the amount down to something that can fit on your card? I have a custom textureless single Iray material for just this purpose.
Laurie
Thanks for the information. I was going to buy it but no can do scenes with huge memory requirements
If the material zones are broken up sensibly, you could do stuff with no map/procedural shaders, or plug in other stuff (like Mec4D shaders). Though that's a lot of work, and it rather requires that materials don't have crucial details baked in.
The product information lists 193 (!) texture/bump/normal maps, ALL of which are 2048x2048 or 4096x4096. Just looking at the layouts and objects, I can guarantee that most items in any given render of that scenery would NOT need that kind of resolution.
If the garbage can object is only 50 pixels tall in the final render, having a 2048x2048 texture on it is meaningless. A 256x256 texture would look just as good, and take a tiny fraction of the VRAM compared to the bigger texture.
29GB of textures alone? That's just crazy. Resize those puppies. Esemwy and I (in another thread) are both working on image resizing scripts for just this purpose. Based on this, I think I may have to make my scripts do XtraSmall resizes as well as Medium and Small......
Again, I call out to PAs to make MULTIPLE sizes of your maps for your products. And corresponding material presets. So that this kind of stuff isn't as big of an issue......
(now, if I can just work out that darn transparent PNG resizing issue, mine will be ready to go.....)
edit: Just checked....yep, there is NO nvidia card that can render this scene on CUDA. P6000 Quadro has 24GB available, and at 29GB of textures, this scene can ONLY be rendered (at full texture resolutions) by CPU. And then only if you have 64GB or more RAM, as with the OS/DS/etc., that won't all fit in RAM with only 32GB. Or you'll be swapping to disk with virtual memory, which slows renders to a CRAWL.
I just looked at the Reduce Texture script everyone's been talking about:
http://www.daz3d.com/forums/discussion/137161/reduce-texture-sizes-easily-with-this-script#latest
It appears to be a great solution for Polish Subway and other large scenes. I really can't see any differences in the before/after comparisons of using the script. You can select all objects, deselect anything you want a closeup on, and reduce all textures at once! Assuming it works as well as everyone says, I'm glad PA's include the 4096 maps for closeups (though including multiple sizes woudl be fine too). Thanks to everyone who pointed out this cool freebie!
I have used that Reduced Texture Script to reduce the files multiple times on the same scene and haven't seen any difference in quality. It would be hard to even spot the renders on my art thread where I did it. I know in a couple of the fairy pictures I did that. There's a way to save the reduced texture alongside the regular one, by holding the control button while you run the script (details on the script thread.) Occasionally I have trouble with a face texture loading and just reload that one from scratch. It's worth the effort and makes a lot of scenes more accessible to my small card. I'll be tackling the Antebellum Mansion with it next.
Hey, there was a similar thread to this on the new mansion. I understand the concerns over resource usage, i really do. The numbers being thrown around are misleading however - The mansion shows around 13gb of uncompressed textures, however the actual GPU load is around 3.5gb. I know this isn't much help for anyone with sub 2gb cards, but these scenes are still GPU friendly with any mid-tier card. I'm really not trying to cause any drama, i just wanted to provide some sort of clarity. If anyone has this item, it would be useful to run a GPU monitor (like gpuz) to provide a more accurate load number.
Sorry, I didn't realize that. I love the new info in verbose mode, but you're right, the numbers are misleading if they're being compressed. Sounds like a GPU monitor is worth running specifically for this reason. I don't have the subway set yet. Definitely on the wishlist though.
That is a very nice looking set. Also, having hit the 32GB limit of my pathetic computers DIM slots, I'm not sure that IF it has 3DL mats, the subway would load in my willfully inadequate computer. Somewhere there has to be 16GB or 32GB DDR3 sticks (per stick, not total for the entire kit), just not anywhere that I can locate in non-ECC variety. I sort of wanted that two years ago Dram makers.
Some where in the Beta thread I hinted at why it is such a problem with Iray, akin to the difficulties I had on a 32bit system with only 3GB of system ram back in Studio 3.5 days. With such limited memory, you are stuck with using environment spheres and ad-hock partial sets with limited number of figures for renders. You will never be able to load up a Stargate Atlantis city-ship model and render a fly-by with the usless skimping's of memory nvidia has been putting on there products. If your lucky enough to be able to afford an i7 system with more then four memory slots (capable of more then 128GB of memory), then such incredible sets such as that subway are not unreasonable to work with in 3DL, and even have room for filling such a set with a believable number of figures for a full scene.
Thanks, @KindredArts , having someone with the specs to really test this (a TitanX) helps us understand just WHAT the stats Iray is putting out are. So evidently those are the UNCOMPRESSED memory values, which are then compressed as it loads. But even with similar compression, 29GB uncompressed textures is very likely to be very close to or over the 8GB limit for non-Titan-level VRAM.
Still it would be nice if the 'verbose' iray details gave us actual USAGE in VRAM, rather than the uncompressed size to be transferred to the card. What it is giving us isn't terribly helpful if it's uncompressed size.
Also they need to monitor efficient modeling from their PAs as part of the quality control. There are many ways to model and with complex sets, there really needs to be some efficiency and thought put into not just the design, but the modeling and possible uses from the customer going forward. I have modded for games where the SDK gave you specific limits that you could not go over as not to impact performance in the game, something similiar should be in place with DS also
No problem, i have voiced my concerns many times about resource usage. We are however trying to achieve professional quality rendering on consumer grade hardware, so there's a fairly fine line we have to tread. I'm hoping that high-capacity cards become more mainstream in the near future, to a point that compromises are no longer even a consideration. Fingers Crossed
I'm not seeing anything in the promo renders that really loooks like it demands this degree of resolution to the textures to be honest. Especially given that it is a subway, and they are usually crowded with people, I would imagine adding memory intensive textures would be prohibitive to usefulness for such a large scene.
Guess I'll be returning this set. I can't get it to render on my machine
So this has to end. Most of us are hobbyists and not looking for commercial quality sets with those types of textures. I'd be happy if they offered 2 versions for sale. One for hi quality like the one out now, and one for those of us who can't have the latest and greatest computers/graphics cards and tons of other bling. Guess I"ll be buying older stuff and converting iray like I've been doing when things go on sale.
Again, if you are willing to work at it, I highly recommend using Mec4D's shaders or my free procedural shaders or something similar to rework the surfaces of products like this. If they are laid out sensibly, you can get texture load to VERY low levels without a huge amount of work (maybe).
There's always the reduce textures script
http://www.daz3d.com/forums/discussion/137161/reduce-texture-sizes-easily-with-this-script#latest
I've had a few scenes that I couldn't get to render in Iray either, and this helped a lot.
It's a beautiful set. I'll probably be getting it after I upgrade my card. If I tried it now, I think my computer would blow up. LOL. it's really straining on the Antebellum Mansion, which is not nearly as big.
The set looks a bit sloppy to me. In several promos the trash is floating above the ground and the green aqua lighting looks a bit strange
I think PA's need to realize that not everyone has superduper expensive video cards.. I think also Nvidia needs to take a page out of Otoy's book and have an implementation of dumping overflow onto the CPU and system ram..
Really can not understand this need for super hi def textures 4096x4096 seems a bit of overkill even 2048x2048 seems a bit much..
If you want detail then they have to be that big. Any smaller when you do close ups the texture gets fuzzy and pixilated. 4096x4096 is an industry standard so your going to keep seeing them. We are doing this work to produce quality work and we are going to do what's needed to get that quality. You can take the image into GIMP or Photoshop and reduce the size but you will lose quality.
I agree with frank, especially since there is a script and other mechanisms to easily reduce the textures.
There is nothing for the reverse. If PA's started delivering only 2048x2048 and below, there is no script to magically increase the resolution for close ups.
Maybe Daz can get him to do an update with an optimisation preload.
The problem is that Iray really favors a material definition approach, rather than a texture map approach to design, and a lot of PAs just don't work that way.
Some great tips on here, thanks all. The reduce texture script seems interesting, and I might try to play around with some textures to resize or delete them myself and see if I can make it go. I have the Mecd set also.
I did a CPU only render, my machine only has 16GB of system ram. It does render, and it peaks at 11.7GB of system ram usage, so I guess the verbose mode is reporting uncompressed texture size as noted above. I'm not sure if that's apples to apples or what, seems like maybe a card with 12GB might work then.
Even with this "issue" I think this is an awesome set that I'll poke around and see if I can make use of. I agree with some of the posters above - it's better to have "too big" of textures than to be stuck with pixelated renders for close-ups I guess. Polish does do fantastic work for sure.
Also, I tried memory optimized render, still doesn't fit in my card.
I think memory optimized only affects instances.
...and people wonder why I am designing a duo Xeon render box with 128 GB of physical memory
On my current system this would take almost a week to render without characters or any other effects as it would be in Swap Mode by day one. (I would probably even need to increase the size of the VM partition).