Polish Subway Huge VRAM demand

13

Comments

  • WendyLuvsCatzWendyLuvsCatz Posts: 33,758

    this thread helped me avoid needless expenditures 

  • hphoenixhphoenix Posts: 1,335
    namffuak said:

     <snip  >This is something that a PA shouldn't have do and is double work ......

    in reply to this

     

    How exactly is that done now? Opening a session of GPU-z and writing it down on a piece of paper? That would need to be addressed first, correct? Also if the scene fits on a 4gb GPU and renders, it would fall in with the requirement of DS recommending a 4GB card for DAZ Studio, so that's already the requirement, correct? Sounds like it needs to be thought out more before making any type of requirements on a PA.

    it apparently does not even fit on a 6GB GPU without doctoring of textures by the end user.

    the PA surely has rendered their set at sometime or DAZ QA has so are aware of requirements, the purchaser only becomes aware upon using and possibly subsequently returning.

    An informed choice avoids this needless refund process.

    Seee my post above - I did not doctor anything and ran the scene as loaded. While Iray verbose mode indicated it needed a massive amount of texture memory it actuall renedered a 1920 X 1080 scene using only 958 MB on my 6 GB 980 ti. Plus, of course, the 48 MB that Windows 7 is grabbing on both my cards.

    I'll be playing with it more later today to see why it blew memory on my 4 GB 960m on my laptop.

    Was that JUST the environment, or did you add in a figure or two with hair and clothes?

    Iray also allocates scratch space, based on the geometry and lighting.  It's usually the second largest allocation.  Followed by the geometry, the lighting and the material definitions (in that order).

    While the scene itself may fit within a 6GB (3rd largest consumer-level card memory available, behind 12GB Titans and 8GB 1080/1070 cards) adding in a figure with clothes, hair, and props adds additional overhead......and that's the TYPICAL usage of most environments.....as a place for the figures to be acting/posed in.

     

  • JPiatJPiat Posts: 65

    A quick test using the full presets + V4 and outfit + Titan(s) X GPU.

    According to GPU-Z, the render needs 6903 MB.

    But I use high threshold and there is a lot of "out of the screen" stuff.

    Nota : Titan X maxwell are not so expensive now... At least on ebay.

     

     

     

    Test01.jpg
    1902 x 1055 - 575K
    ZGPU.JPG
    471 x 485 - 59K
    Threehold.JPG
    349 x 210 - 18K
  • marblemarble Posts: 6,851
    JPiat said:

    A quick test using the full presets + V4 and outfit + Titan(s) X GPU.

    According to GPU-Z, the render needs 6903 MB.

    But I use high threshold and there is a lot of "out of the screen" stuff.

    Nota : Titan X maxwell are not so expensive now... At least on ebay.

     

    Which prompts me to ask again, does the "out of the screen" stuff still get loaded into VRAM and, if so, would Iray Section Panels change that?

  • Male-M3diaMale-M3dia Posts: 3,567
    edited January 2017
    marble said:
    JPiat said:

    A quick test using the full presets + V4 and outfit + Titan(s) X GPU.

    According to GPU-Z, the render needs 6903 MB.

    But I use high threshold and there is a lot of "out of the screen" stuff.

    Nota : Titan X maxwell are not so expensive now... At least on ebay.

     

    Which prompts me to ask again, does the "out of the screen" stuff still get loaded into VRAM and, if so, would Iray Section Panels change that?

    Yes and hidden items as well. Delete what you don't see from the scene.

    Post edited by Male-M3dia on
  • namffuaknamffuak Posts: 3,739
    hphoenix said:
    namffuak said:

     <snip  >This is something that a PA shouldn't have do and is double work ......

    in reply to this

     

    How exactly is that done now? Opening a session of GPU-z and writing it down on a piece of paper? That would need to be addressed first, correct? Also if the scene fits on a 4gb GPU and renders, it would fall in with the requirement of DS recommending a 4GB card for DAZ Studio, so that's already the requirement, correct? Sounds like it needs to be thought out more before making any type of requirements on a PA.

    it apparently does not even fit on a 6GB GPU without doctoring of textures by the end user.

    the PA surely has rendered their set at sometime or DAZ QA has so are aware of requirements, the purchaser only becomes aware upon using and possibly subsequently returning.

    An informed choice avoids this needless refund process.

    Seee my post above - I did not doctor anything and ran the scene as loaded. While Iray verbose mode indicated it needed a massive amount of texture memory it actuall renedered a 1920 X 1080 scene using only 958 MB on my 6 GB 980 ti. Plus, of course, the 48 MB that Windows 7 is grabbing on both my cards.

    I'll be playing with it more later today to see why it blew memory on my 4 GB 960m on my laptop.

    Was that JUST the environment, or did you add in a figure or two with hair and clothes?

    Iray also allocates scratch space, based on the geometry and lighting.  It's usually the second largest allocation.  Followed by the geometry, the lighting and the material definitions (in that order).

    While the scene itself may fit within a 6GB (3rd largest consumer-level card memory available, behind 12GB Titans and 8GB 1080/1070 cards) adding in a figure with clothes, hair, and props adds additional overhead......and that's the TYPICAL usage of most environments.....as a place for the figures to be acting/posed in.

     

    Just the scene - like I said, I'm still playing with it. More later.

  • namffuaknamffuak Posts: 3,739
    edited January 2017

    Test on the laptop; I have no explanation, just observations. Note the Iray out of memory error in the log in image1.  Note the gpu-z showing in image2 - a snapshot taken AFTER the image1 snapshot, showing the gpu at 100%, memory maxed out -- but still working. And I've attached the render at the point of cancellation

    Here's the tail end of the log:

    2017-01-18 17:09:17.482 Finished Rendering
    2017-01-18 17:09:21.837 Total Rendering Time: 11 minutes 40.0 seconds
    2017-01-18 17:11:13.276 Saved image: C:\!DAZ\!subway\test run.jpg
    2017-01-18 17:11:25.423 Iray INFO - module:category(IRAY:RENDER):   1.0   IRAY   rend info : Device statistics:
    2017-01-18 17:11:25.423 Iray INFO - module:category(IRAY:RENDER):   1.0   IRAY   rend info : CUDA device 0 (GeForce GTX 960M):      11 iterations, 212.435s init, 266.924s render
    2017-01-18 17:11:25.423 Iray INFO - module:category(IRAY:RENDER):   1.0   IRAY   rend info : CPU:      10 iterations, 195.361s init, 412.474s render

    And, for the record, Studio was using 11.9 GB of the 16 GB on the laptop.

    I resized the render, it was 1920 by 1920.

     

     

    Image1.jpg
    1920 x 1080 - 424K
    Image2.jpg
    1920 x 1080 - 399K
    test run.jpg
    1280 x 1280 - 697K
    Post edited by namffuak on
  • hphoenixhphoenix Posts: 1,335
    edited January 2017

    I'm guessing, based on the observed behavior, that cards that are unable to allocate enough memory continue to try as the render progresses.....as memory requirements can change throughout the processing (unlike some rendering algorithms).  So what is happening is that Iray is trying to allocate what it requires to start initially, fails on one card.  That card keeps everything it DID load, and keeps trying until the main card resolves enough that the memory requirements go down, at which point the other card tries again, and may be able to iterate a few times (and then may need more memory, and fail out again.)

    That would explain why we are seeing cards that couldn't fit a scene initially actually being utilized, and generating some iterations toward the solution.

    (This is just a guess.....without input from DAZ devs or Iray devs, we won't know precisely what is going on here.....)

     

    Post edited by hphoenix on
  • namffuaknamffuak Posts: 3,739

    Possibly. This post was on my laptop, with the 960m and cpu as the only two render devices - and the log indicates a match between them in terms of iterations. I need to throw a couple of figures in the mix and see if I can melt my laptop. :-)

  • kyoto kidkyoto kid Posts: 38,518
    edited January 2017
    namffuak said:

    Test on the laptop; I have no explanation, just observations. Note the Iray out of memory error in the log in image1.  Note the gpu-z showing in image2 - a snapshot taken AFTER the image1 snapshot, showing the gpu at 100%, memory maxed out -- but still working. And I've attached the render at the point of cancellation

    Here's the tail end of the log:

    2017-01-18 17:09:17.482 Finished Rendering
    2017-01-18 17:09:21.837 Total Rendering Time: 11 minutes 40.0 seconds
    2017-01-18 17:11:13.276 Saved image: C:\!DAZ\!subway\test run.jpg
    2017-01-18 17:11:25.423 Iray INFO - module:category(IRAY:RENDER):   1.0   IRAY   rend info : Device statistics:
    2017-01-18 17:11:25.423 Iray INFO - module:category(IRAY:RENDER):   1.0   IRAY   rend info : CUDA device 0 (GeForce GTX 960M):      11 iterations, 212.435s init, 266.924s render
    2017-01-18 17:11:25.423 Iray INFO - module:category(IRAY:RENDER):   1.0   IRAY   rend info : CPU:      10 iterations, 195.361s init, 412.474s render

    And, for the record, Studio was using 11.9 GB of the 16 GB on the laptop.

    I resized the render, it was 1920 by 1920.

     

     

    ...considering I only have 10.7 GB of available memory after Windows & system utilities, just loading the set it would not fit without immediately going into Virtual memory let alone, trying to render it. Even my largest scene with 8 figures and a wet mist effect is 3 GB smaller than that, and when I make a camera move or pose adjustment, it usually means go put the kettle on for tea and a couple premade crumpets in the toaster while I wait for the action to complete.

    Post edited by kyoto kid on
  • namffuaknamffuak Posts: 3,739

    There's no question, this is a heavy set. But I'm doing worst-case to see what Iray does to the gpu(s) here. There are several subset scenes and a good bit can be loaded in just as props.Set 4 is running on my laptop as I type this - Studio is eating 8.1 GB. I could probably drop about one third of the preload for the section I actually have under camera.

  • Male-M3diaMale-M3dia Posts: 3,567
    edited January 2017

    I loaded the main scene with one figure (which is pretty heavy with normals, translucent, SSS and HD) and simply deleted out what wasn't viewable; GPU-z reads about 3.6gb memory utilized on my 980gtx and DS is running about 17gb physical memory. That said, one of the subset scenes provided may work better and i'll probably get more memory for my machine since I tend to work in other programs while rendering. I would probably be in better shape if I turn off zbrush, modo, firefox, chrome and photoshop. smiley The biggest thing that will eat up resources are the emissive surfaces, but it still wasn't a large effort to remove unnecessary items that aren't visible to get the product to render.

     

    Subway01.jpg
    1435 x 788 - 226K
    Post edited by Male-M3dia on
  • TooncesToonces Posts: 919

    Wait a second! That looks like the doctor from the new Netflix series, the OA!

    I won't say if he's a good guy or a bad guy. I don't want to spoil it. :)

    doctor1.PNG
    563 x 532 - 477K
  • Serene NightSerene Night Posts: 17,253
    edited January 2017

    Why not sell optimized texturest? I'm still not getting that. At this point if people have to hide most of it to render they  might as well buy a subway backdrop and use it in their environment tab.

    Post edited by Serene Night on
  • Oso3DOso3D Posts: 14,617

    Because it's work to help a very narrow subset of customers:

    Those willing to do some work at switching texturesets, recognizing when to use one vs. the other, and change settings accordingly... but not so savvy they can't use other measures to manage the issue (like Iray cut planes, texture adjusters, etc).

    And who have machines that are good enough to run decent Iray but not quite enough to run this particular scene.

     

    So you have to have a big enough number of those people. Then, if you have two texture sets, now it's a bigger download and more QA to do (and, thus, potential problems).

     

    So are there enough of those people who would only buy the product if it had multiple texture sets to balance the cost of doing all that?

     

    Dunno, but I can understand why a lot of PAs wouldn't bother.

    (Now, personally, I maintain that going with minimal map solutions in most cases is a general improvement to a product, but people can potentially outline the same argument)

     

  • Male-M3diaMale-M3dia Posts: 3,567

    Why not sell optimized texturest? I'm still not getting that. At this point if people have to hide most of it to render they  might as well buy a subway backdrop and use it in your environment tab.

    That's a subjective question. If I didn't have my fully loaded character it would  fit in card memory, but to reduce render time you should be cutting up the scene anyway so you aren't rendering things out the render, especially any emissive surfaces. It's a large scene so the complete scene just loads everything, not just only the things you need.

  • namffuak said:

     <snip  >This is something that a PA shouldn't have do and is double work ......

    in reply to this

     

    How exactly is that done now? Opening a session of GPU-z and writing it down on a piece of paper? That would need to be addressed first, correct? Also if the scene fits on a 4gb GPU and renders, it would fall in with the requirement of DS recommending a 4GB card for DAZ Studio, so that's already the requirement, correct? Sounds like it needs to be thought out more before making any type of requirements on a PA.

    it apparently does not even fit on a 6GB GPU without doctoring of textures by the end user.

    the PA surely has rendered their set at sometime or DAZ QA has so are aware of requirements, the purchaser only becomes aware upon using and possibly subsequently returning.

    An informed choice avoids this needless refund process.

    Seee my post above - I did not doctor anything and ran the scene as loaded. While Iray verbose mode indicated it needed a massive amount of texture memory it actuall renedered a 1920 X 1080 scene using only 958 MB on my 6 GB 980 ti. Plus, of course, the 48 MB that Windows 7 is grabbing on both my cards.

    I'll be playing with it more later today to see why it blew memory on my 4 GB 960m on my laptop.

    Can you clarify how you make this work?  I have a 980 Ti, and it absolutely will not render without falling back to CPU.

  • bailaowaibailaowai Posts: 44
    edited January 2017

    OK there is more to the story here.

    I typically render my scenes at 4096px on the long edge, which is what I tried when I originally ran into this problem and started this thread.  At 4096 X whatever the scene will absolutely not render in my 980 Ti's 6GB of memory - it falls back to the CPU for sure.

    However I notice one of the posters above said he rendered at 1920x1920.  I loaded the scene and tried a render at that resolution - works.  No problem.  GPU memory peaks at around 4.1GB according to GPUZ.

    Now here is what's interesting.  It also renders at 3000x3000px (peaks around 5.5GB).  BUT, the other thing that's interesting is once I increased the resolution to 4096px and it fails to load on the GPU (falls back to CPU memory), it never goes back to being able to render at lower resolutions.  The GPU memory stays filled, according to GPUZ, at 5.5ish GB, not more not less even after I cancel the render.  The only way to get it back to being able to GPU render is to CLOSE DAZ, re-open, and render at the lower resolution.  If I open Daz, load the scene, do nothing else, render at 4096px, it fails and won't render even if I try lower resolutions.  If I open Daz and render at 1920px it works (CPU render, 99% utilization on the GPU).  No idea if this is scene specific or if this is a bug in Daz.

    I guess bottom line is that for "lower" resolutions (like 1920px) maybe this scene is workable.

    In all cases just now for testing I just loaded the full set "1" (scene 1 or whatever), didn't move the viewport, just rendered.  No characters loaded, just the scene.

    Post edited by bailaowai on
  • TheKDTheKD Posts: 2,597

    There is some kind of issue with iray. Sometimes if I am running render after render messing with lights and whatnot, eventually it fails and goes to CPU, I have to save, close and reopen, and it renders normal again after that. A VRAM leak? Never heard of such a thing, but maybe it exists lol.

  • kyoto kidkyoto kid Posts: 38,518

    ...I wonder if the old purge memory script might help with this.

  • marblemarble Posts: 6,851
    kyoto kid said:

    ...I wonder if the old purge memory script might help with this.

    I would think that works for system RAM but doubt that it includes VRAM?

  • kyoto kidkyoto kid Posts: 38,518

    ...just a thought. There is also a script that cleared shader information as well.

  • If the GPU drops out for lack of memory you will usually need to restart DS, and possibly the OS, to get it back into use.

  • TooncesToonces Posts: 919

    Next time, check your log. If you see any errors for OptiX, you can disable it. I usually keep it enabled, unless I'm dealing multiple renders with a scene that approaches the limits of my VRAM.

    http://www.daz3d.com/forums/discussion/110626/iray-intermittently-stops-utilizing-video-card-s

  • Oso3DOso3D Posts: 14,617

    Under what circumstances is OptiX good to use? I've been keeping it off because I seemed to hit problems a bunch, but it's been a while.

     

  • HavosHavos Posts: 4,915

    Under what circumstances is OptiX good to use? I've been keeping it off because I seemed to hit problems a bunch, but it's been a while.

     

    I enabled it a few months back, having not really given it much thought before, and it considerably speeded up my renders (maybe 20-30% faster), with no issues I have suffered so far, either on my desktop, or the laptop. Others have posted that they have suffered instabilities when it is enabled, so it is likely to differ from system to system.

  • TooncesToonces Posts: 919

    Yes, OptiX speeds up my Iray renders significantly.

  • McGyverMcGyver Posts: 5,958

    Back in the early days they used to include file size in the information on the store page...

    I guess DAZ thinks we all own huge mega render farms and supercomputers and these things no longer matter to us...?

    Is it really THAT hard to just throw that information up there on the store page?... Jeez.

  • TooncesToonces Posts: 919

    If you didn't want to use the reducing mechanisms, couldn't you just look at the map resolutions on the store page while you shop? Just curious. Polish Subway (indicating lots of 4096 texture maps):

    • 193 Texture, Bump, Normal, Specular, and Transparency Maps (2048 x 2048 to 4096 x 4096)

    Seems like that would sufficiently indicate high file size...

  • CypherFOXCypherFOX Posts: 3,398
    edited January 2017

    Greetings,

    So I did a render with this, being equally curious, and I found the same thing...  Not that it was too big, or anything.  It fit okay, with a character on SubD 2, clothes, etc., but that Iray didn't free up memory.  Now here's what happened.  I did a render at ~1024x576 and it worked fine, fast and clean (attached), and appeared to take ~7GB of memory on the card according to GPU-Z.

    I then figured I'd try to render the same scene at 3840x2160.  I wandered off, and when I came back (>2 hours later) the quality was laughable, and I realized it'd done CPU rendering.  I tried used SimTenero's Iray Memory Assistant, removed a bunch of trash cans, and other pieces that weren't in the scene and were unlikely to contribute meaningfully to the light bounces hitting the scene.  I started a render, only to see that it instantly went CPU.  Checking GPU-Z showed that the ~7GB of memory was still allocated.  Promptly saving the scene, shutting down DAZ Studio, reloading it, and opening recent scenes, bringing it back in.  GPU-Z was showing ~800MB used (my normal Windows load), and then I hit Render, and it loaded the scene in and promptly started rendering a frame or so a second.

    I didn't alter the texture compression settings, or a bunch of other things that would typically be my next step.

    The thing I wanted to point out is that there is apparently a bug in Iray; if it fails to allocate video-card memory for a scene, it never lets GO of whatever memory it HAD set up.  This is probably an 'exception' situation, where the allocation isn't unwound when the exception gets thrown.  Closing D|S and reopening it seems to clear it up for me.

    --  Morgan

     

    WIP Kickass Subway.png
    1024 x 576 - 992K
    Post edited by CypherFOX on
Sign In or Register to comment.