Iray render keeps crashing

linoge8888linoge8888 Posts: 62

Hello, 

My renders keep crashing because of the textures quality consuming too much memory. Only two Gen8 characters in that setup (https://www.daz3d.com/teen-bedroom). Is there something else to do (DAZ settings ?) than lowering textures quality ?

My gear is far from being bad : 

OS : Win10 latest version

GPU : RTX 3070 - 8Go

CPU : i7-10700K  - 3.8 GHz

RAM : 64 Go

Log : 

 [INFO] - IRAY:RENDER ::   1.0   IRAY   rend info : Rendering... Iray [INFO] - IRAY:RENDER ::   1.15  IRAY   rend progr: CUDA device 0 (NVIDIA GeForce RTX 3070): Processing scene... Iray [VERBOSE] - IRAY:RENDER ::   1.13  IRAY   rend stat : Geometry memory consumption: 347.893 MiB (device 0), 0.000 B (host) Iray [VERBOSE] - IRAY:RENDER ::   1.13  IRAY   rend stat : Texture memory consumption: 4.542 GiB for 274 bitmaps (device 0) Iray [INFO] - IRAY:RENDER ::   1.13  IRAY   rend info : Importing lights for motion time 0 Iray [INFO] - IRAY:RENDER ::   1.13  IRAY   rend info : Initializing light hierarchy. Iray [INFO] - IRAY:RENDER ::   1.13  IRAY   rend info : Light hierarchy initialization took 0.158s Iray [VERBOSE] - IRAY:RENDER ::   1.13  IRAY   rend stat : Lights memory consumption: 5.963 MiB (device 0) Iray [VERBOSE] - IRAY:RENDER ::   1.13  IRAY   rend stat : Material measurement memory consumption: 0.000 B (GPU) Iray [VERBOSE] - IRAY:RENDER ::   1.13  IRAY   rend stat : PTX code (2.844 KiB) for SM 8.0 generated in 0.013s Iray [VERBOSE] - IRAY:RENDER ::   1.13  IRAY   rend stat : Materials memory consumption: 960.480 KiB (GPU) Iray [INFO] - IRAY:RENDER ::   1.13  IRAY   rend info : JIT-linking wavefront kernel in 0.045s Iray [INFO] - IRAY:RENDER ::   1.15  IRAY   rend info : CUDA device 0 (NVIDIA GeForce RTX 3070): Scene processed in 12.512s WARNING: ..\..\..\..\..\src\pluginsource\DzIrayRender\dzneuraymgr.cpp(359): Iray [ERROR] - IRAY:RENDER ::   1.15  IRAY   rend error: CUDA device 0 (NVIDIA GeForce RTX 3070): Not enough memory for kernel launches (0.000 B (464.133 MiB) required, 0.000 B available). Cannot allocate framebuffer. WARNING: ..\..\..\..\..\src\pluginsource\DzIrayRender\dzneuraymgr.cpp(359): Iray [ERROR] - IRAY:RENDER ::   1.15  IRAY   rend error: CUDA device 0 (NVIDIA GeForce RTX 3070): Failed to setup device frame buffer WARNING: ..\..\..\..\..\src\pluginsource\DzIrayRender\dzneuraymgr.cpp(359): Iray [ERROR] - IRAY:RENDER ::   1.15  IRAY   rend error: CUDA device 0 (NVIDIA GeForce RTX 3070): Device failed while rendering WARNING: ..\..\..\..\..\src\pluginsource\DzIrayRender\dzneuraymgr.cpp(359): Iray [WARNING] - IRAY:RENDER ::   1.15  IRAY   rend warn : CUDA device 0 (NVIDIA GeForce RTX 3070) ran out of memory and is temporarily unavailable for rendering. WARNING: ..\..\..\..\..\src\pluginsource\DzIrayRender\dzneuraymgr.cpp(359): Iray [WARNING] - IRAY:RENDER ::   1.15  IRAY   rend warn : All available GPUs failed. WARNING: ..\..\..\..\..\src\pluginsource\DzIrayRender\dzneuraymgr.cpp(359): Iray [ERROR] - IRAY:RENDER ::   1.15  IRAY   rend error: Fallback to CPU not allowed. WARNING: ..\..\..\..\..\src\pluginsource\DzIrayRender\dzneuraymgr.cpp(359): Iray [ERROR] - IRAY:RENDER ::   1.15  IRAY   rend error: All workers failed: aborting render Iray Render error: Internal rendering error.

 

Post edited by linoge8888 on

Comments

  • PerttiAPerttiA Posts: 9,456

    Your textures are already taking more than an 8GB GPU can handle for textures+geometry combined when running on Windows 10, there is nothing else to do but to reduce the textures.

  • Where do you read more than 8 GB ?

  • cgidesigncgidesign Posts: 433

    You can force a more agressive texture compression - but it might reduce image quality:

    Medium Threshold: Any texture with a resolution below this or equal to this value will not get compressed
    Inbetween Medium and High Threshold: Any Texture with a resolution in this range will get medium compression
    High Threshold: Any texture with equal or greater resolution will get maximum compression

  • I've set the maximum treshold to 2, and it keeps crashing. Is this normal ? Not sure, where to get the exact need for memory in the log ? Oh, both characters have a 1200-frame mocap animation each. Before i tried with a 3600 framed animation and DAZ was totally unstable and impossible to render/run. Could it be linked ?

  • cgidesigncgidesign Posts: 433

    In your log you find lines that have the tag [VERBOSE] in it. Those ones tell you the vram consumption.

    In the one you posted that is:

    • Geometry memory consumption: 347.893 MiB
    • Texture memory consumption: 4.542 GiB
    • Lights memory consumption: 5.963 MiB
    • Materials memory consumption: 960.480 KiB

    So, you are running at around 5 GB.

    There is one more relevant line:

    WARNING: ..\ ... CUDA device 0 (NVIDIA GeForce RTX 3070): Not enough memory for kernel launches (0.000 B (464.133 MiB) required, 0.000 B available). Cannot allocate framebuffer.

    Iray needs to create a so called Frame Buffer but cannot do so because no vram is left.

    Your scene needs around 5.5 to 6 Gigabyte. Your GPU has 8 GB but Windows is taking a big bite of that for itself. And I fear rendering an animation will increase the vram usage because of some memory leaking.

    Regarding the compression:

    Make a test with:

    min: 512 / max: 4096
    and check the log

    then

    min: 256 / max: 512
    and check the log again

    If the latter does not help to get enough vram it gets difficult.

    One tip is:

    https://www.daz3d.com/scene-optimizer

    A tool that helps to reduce texture resolution etc. It is on sale right now (16. Jan 2022).

    Other tips can be found in this thread:

    https://www.daz3d.com/forums/discussion/543101/is-there-a-way-to-render-part-of-a-scene-instead-of-the-whole-thing

  • PerttiAPerttiA Posts: 9,456

    linoge8888 said:

    Where do you read more than 8 GB ?

    That is not how much is available to Iray rendering.

    Attached is a test I made some time ago to see how much RAM and VRAM was used while rendering in IRAY

    Case a) just one lightweight G8 figure with lightweight clothing and hair
    Case b) four similar G8 characters with architecture
    Case c and d) started increasing SubD on the characters to see at which point the rendering would drop to CPU

    "RAM/GB" and "VRAM/MB" taken from GPU-Z, "DS Log/GiB" taken from DS Log, no other programs were running but DS and GPU-Z
    The "DS Log/GiB" is the sum of Geometry usage, Texture usage and Working Space - After Geometry and Textures, there should still be at least a Gigabyte of VRAM available for the Working space => In my case, Geometry + Textures should not exceed 4.7GiB

    Note; Case c) was already using 38GB's of RAM, even though the rendering was done on the GPU, Case d) when rendering on CPU the RAM usage went almost over my 64GB's

    Using RTX 2070 Super (8GB), i7-5820K and 64GB's of RAM on W7 Ultimate

    Since base load with W10 is almost a gigabyte (300MB's on W7), that would leave about 4GiB's available for textures+geometry in your case.

    RenderTST2.PNG
    615 x 574 - 40K
  • functionfunction Posts: 268

    My experience is, can not leave any rendered picture opened in computer background, must save it or close it, the rendered pic consumed too much RAM and VRAM, so the GPU is stop working very quickly.

  • frank0314frank0314 Posts: 13,380

    What all do you have in the scene besides the props. No way in the world the prop textures can take up 4.8GB. I mean if really detailed I guess it could take up 1GB but most look as if they are just plain standard textures that could be 800x800. How big are the maps? If they are above 2048x2048 you definitely need to reduce them. From a look at the promos you can safely trim the map size down to 800x800 or even smaller. Especially if you not doing closeups.

  • function said:

    My experience is, can not leave any rendered picture opened in computer background, must save it or close it, the rendered pic consumed too much RAM and VRAM, so the GPU is stop working very quickly.

    If you leave the render window open then the scene data remains in the memory of whatever device(s) did the rendering, so that rendering could be resumed.

  • linoge8888linoge8888 Posts: 62
    edited January 2022

    Well, thanks everyone for your answers. I've restarted my computer, only opened DAZ and now it works! The figures are quite the same. I precise I make image series, not the single frame mode with a rendering window.  

    2022-01-16 19:34:10.542 Iray [INFO] - IRAY:RENDER ::   1.11  IRAY   rend progr: CUDA device 0 (NVIDIA GeForce RTX 3070): Processing scene...2022-01-16 19:34:10.644 Iray [VERBOSE] - IRAY:RENDER ::   1.8   IRAY   rend stat : Geometry memory consumption: 216.055 MiB (device 0), 0.000 B (host)2022-01-16 19:34:18.428 Iray [VERBOSE] - IRAY:RENDER ::   1.8   IRAY   rend stat : Texture memory consumption: 4.212 GiB for 246 bitmaps (device 0)2022-01-16 19:34:18.428 Iray [INFO] - IRAY:RENDER ::   1.8   IRAY   rend info : Importing lights for motion time 02022-01-16 19:34:18.429 Iray [INFO] - IRAY:RENDER ::   1.8   IRAY   rend info : Initializing light hierarchy.2022-01-16 19:34:18.585 Iray [INFO] - IRAY:RENDER ::   1.8   IRAY   rend info : Light hierarchy initialization took 0.156s2022-01-16 19:34:18.586 Iray [VERBOSE] - IRAY:RENDER ::   1.8   IRAY   rend stat : Lights memory consumption: 5.963 MiB (device 0)2022-01-16 19:34:18.590 Iray [VERBOSE] - IRAY:RENDER ::   1.8   IRAY   rend stat : Material measurement memory consumption: 0.000 B (GPU)2022-01-16 19:34:18.591 Iray [VERBOSE] - IRAY:RENDER ::   1.8   IRAY   rend stat : PTX code (2.844 KiB) for SM 8.0 generated in 0.000s2022-01-16 19:34:18.591 Iray [VERBOSE] - IRAY:RENDER ::   1.8   IRAY   rend stat : Materials memory consumption: 806.262 KiB (GPU)2022-01-16 19:34:18.670 Iray [INFO] - IRAY:RENDER ::   1.8   IRAY   rend info : JIT-linking wavefront kernel in 0.023s2022-01-16 19:34:18.670 Iray [INFO] - IRAY:RENDER ::   1.11  IRAY   rend info : CUDA device 0 (NVIDIA GeForce RTX 3070): Scene processed in 8.127s2022-01-16 19:34:18.672 Iray [INFO] - IRAY:RENDER ::   1.11  IRAY   rend info : CUDA device 0 (NVIDIA GeForce RTX 3070): Allocated 64.051 MiB for frame buffer2022-01-16 19:34:18.672 Iray [INFO] - IRAY:RENDER ::   1.11  IRAY   rend info : CUDA device 0 (NVIDIA GeForce RTX 3070): Allocated 1.544 GiB of work space (1807k active samples in 0.000s)2022-01-16 19:34:18.672 Iray [INFO] - IRAY:RENDER ::   1.11  IRAY   rend info : CUDA device 0 (NVIDIA GeForce RTX 3070): Used for display, optimizing for interactive usage (performance could be sacrificed)2022-01-16 19:34:18.938 Iray [INFO] - IRAY:RENDER ::   1.11  IRAY   rend info : Allocating 1-layer frame buffer

     

    Post edited by linoge8888 on
  • linoge8888linoge8888 Posts: 62
    edited February 2022

    Well, it seems I can't render a scene where the characters have a 2 minute long complex animation (.fbx from mocap). I have to split the animation in half (1500 frames). The tinier the animation is, the lesser DAZ crashes. More intriguing : I render a single frame from a heavy animation scene. OK. Then if I launch another render on the same scene, it will irremediably crash. I have to restart my computer to get successful series of renders, each time. 

    My hardware : 

    GPU : RTX 3070 - 8Go

    CPU : i7-10700K  - 3.8 GHz

    RAM : 64 Go

    The question is : if I buy a GPU with more memory, will the crashes be fixed ? Here is the logfile : 

    Successful render : 

    Iray [INFO] - IRAY:RENDER ::   1.0   IRAY   rend info : Using OptiX version 7.1.0Iray [INFO] - IRAY:RENDER ::   1.0   IRAY   rend info : Initializing OptiX for CUDA device 0Iray [INFO] - IRAY:RENDER ::   1.0   IRAY   rend info : Rendering with 1 device(s):Iray [INFO] - IRAY:RENDER ::   1.0   IRAY   rend info : 	CUDA device 0 (NVIDIA GeForce RTX 3070)Iray [INFO] - IRAY:RENDER ::   1.0   IRAY   rend info : Rendering...Iray [INFO] - IRAY:RENDER ::   1.12  IRAY   rend progr: CUDA device 0 (NVIDIA GeForce RTX 3070): Processing scene...Iray [VERBOSE] - IRAY:RENDER ::   1.11  IRAY   rend stat : Geometry memory consumption: 268.818 MiB (device 0), 0.000 B (host)Iray [VERBOSE] - IRAY:RENDER ::   1.11  IRAY   rend stat : Texture memory consumption: 4.212 GiB for 246 bitmaps (device 0)Iray [INFO] - IRAY:RENDER ::   1.11  IRAY   rend info : Importing lights for motion time 0Iray [INFO] - IRAY:RENDER ::   1.11  IRAY   rend info : Initializing light hierarchy.Iray [INFO] - IRAY:RENDER ::   1.11  IRAY   rend info : Light hierarchy initialization took 0.158sIray [VERBOSE] - IRAY:RENDER ::   1.11  IRAY   rend stat : Lights memory consumption: 5.964 MiB (device 0)Iray [VERBOSE] - IRAY:RENDER ::   1.11  IRAY   rend stat : Material measurement memory consumption: 0.000 B (GPU)Iray [VERBOSE] - IRAY:RENDER ::   1.11  IRAY   rend stat : PTX code (2.844 KiB) for SM 8.0 generated in 0.088sIray [VERBOSE] - IRAY:RENDER ::   1.11  IRAY   rend stat : Materials memory consumption: 810.117 KiB (GPU)Iray [INFO] - IRAY:RENDER ::   1.11  IRAY   rend info : JIT-linking wavefront kernel in 0.027sIray [INFO] - IRAY:RENDER ::   1.12  IRAY   rend info : CUDA device 0 (NVIDIA GeForce RTX 3070): Scene processed in 11.876sIray [INFO] - IRAY:RENDER ::   1.12  IRAY   rend info : CUDA device 0 (NVIDIA GeForce RTX 3070): Allocated 64.051 MiB for frame bufferIray [INFO] - IRAY:RENDER ::   1.12  IRAY   rend info : CUDA device 0 (NVIDIA GeForce RTX 3070): Allocated 1.474 GiB of work space (1724k active samples in 0.003s)Iray [INFO] - IRAY:RENDER ::   1.12  IRAY   rend info : CUDA device 0 (NVIDIA GeForce RTX 3070): Used for display, optimizing for interactive usage (performance could be sacrificed)Iray [INFO] - IRAY:RENDER ::   1.12  IRAY   rend info : Allocating 1-layer frame bufferIray [INFO] - IRAY:RENDER ::   1.0   IRAY   rend progr: Received update to 00001 iterations after 12.496s....

     

     

    Failed render : 

     Iray [INFO] - IRAY:RENDER ::   1.0   IRAY   rend info : Using iray core convergence estimate. Iray [INFO] - IRAY:RENDER ::   1.0   IRAY   rend info : Allocating 1-layer frame buffer Iray [INFO] - IRAY:RENDER ::   1.0   IRAY   rend info : Using batch scheduling, caustic sampler disabled Iray [INFO] - IRAY:RENDER ::   1.0   IRAY   rend info : Initializing local rendering. Iray [INFO] - IRAY:RENDER ::   1.0   IRAY   rend info : Initializing OptiX for CUDA device 0 Iray [INFO] - IRAY:RENDER ::   1.0   IRAY   rend info : Rendering with 1 device(s): Iray [INFO] - IRAY:RENDER ::   1.0   IRAY   rend info : 	CUDA device 0 (NVIDIA GeForce RTX 3070) Iray [INFO] - IRAY:RENDER ::   1.0   IRAY   rend info : Rendering... Iray [INFO] - IRAY:RENDER ::   1.12  IRAY   rend progr: CUDA device 0 (NVIDIA GeForce RTX 3070): Processing scene... Iray [VERBOSE] - IRAY:RENDER ::   1.11  IRAY   rend stat : Geometry memory consumption: 268.818 MiB (device 0), 0.000 B (host) Iray [VERBOSE] - IRAY:RENDER ::   1.11  IRAY   rend stat : Texture memory consumption: 4.212 GiB for 246 bitmaps (device 0) Iray [INFO] - IRAY:RENDER ::   1.11  IRAY   rend info : Importing lights for motion time 0 Iray [INFO] - IRAY:RENDER ::   1.11  IRAY   rend info : Initializing light hierarchy. Iray [INFO] - IRAY:RENDER ::   1.11  IRAY   rend info : Light hierarchy initialization took 0.158s Iray [VERBOSE] - IRAY:RENDER ::   1.11  IRAY   rend stat : Lights memory consumption: 5.964 MiB (device 0) Iray [VERBOSE] - IRAY:RENDER ::   1.11  IRAY   rend stat : Material measurement memory consumption: 0.000 B (GPU) Iray [VERBOSE] - IRAY:RENDER ::   1.11  IRAY   rend stat : PTX code (2.844 KiB) for SM 8.0 generated in 0.013s Iray [VERBOSE] - IRAY:RENDER ::   1.11  IRAY   rend stat : Materials memory consumption: 810.117 KiB (GPU) Iray [INFO] - IRAY:RENDER ::   1.11  IRAY   rend info : JIT-linking wavefront kernel in 0.022s Iray [INFO] - IRAY:RENDER ::   1.12  IRAY   rend info : CUDA device 0 (NVIDIA GeForce RTX 3070): Scene processed in 11.763s WARNING: ..\..\..\..\..\src\pluginsource\DzIrayRender\dzneuraymgr.cpp(359): Iray [ERROR] - IRAY:RENDER ::   1.12  IRAY   rend error: CUDA device 0 (NVIDIA GeForce RTX 3070): Not enough memory for kernel launches (0.000 B (464.133 MiB) required, 0.000 B available). Cannot allocate framebuffer. WARNING: ..\..\..\..\..\src\pluginsource\DzIrayRender\dzneuraymgr.cpp(359): Iray [ERROR] - IRAY:RENDER ::   1.12  IRAY   rend error: CUDA device 0 (NVIDIA GeForce RTX 3070): Failed to setup device frame buffer WARNING: ..\..\..\..\..\src\pluginsource\DzIrayRender\dzneuraymgr.cpp(359): Iray [ERROR] - IRAY:RENDER ::   1.12  IRAY   rend error: CUDA device 0 (NVIDIA GeForce RTX 3070): Device failed while rendering WARNING: ..\..\..\..\..\src\pluginsource\DzIrayRender\dzneuraymgr.cpp(359): Iray [WARNING] - IRAY:RENDER ::   1.12  IRAY   rend warn : CUDA device 0 (NVIDIA GeForce RTX 3070) ran out of memory and is temporarily unavailable for rendering. WARNING: ..\..\..\..\..\src\pluginsource\DzIrayRender\dzneuraymgr.cpp(359): Iray [WARNING] - IRAY:RENDER ::   1.12  IRAY   rend warn : All available GPUs failed. WARNING: ..\..\..\..\..\src\pluginsource\DzIrayRender\dzneuraymgr.cpp(359): Iray [ERROR] - IRAY:RENDER ::   1.12  IRAY   rend error: Fallback to CPU not allowed. WARNING: ..\..\..\..\..\src\pluginsource\DzIrayRender\dzneuraymgr.cpp(359): Iray [ERROR] - IRAY:RENDER ::   1.12  IRAY   rend error: All workers failed: aborting render Iray Render error: Internal rendering error. Iray [INFO] - IRAY:RENDER ::   1.0   IRAY   rend info : Device statistics: Iray [INFO] - IRAY:RENDER ::   1.0   IRAY   rend info : CUDA device 0 (NVIDIA GeForce RTX 3070): 	 0 iterations, 11.837s init, 0.000s render

     

    Post edited by linoge8888 on
Sign In or Register to comment.