GPUs cut out
Myds
Posts: 106
OK. I just want to say that I read as many threads as I could before writing, and the solutions there didn't seem to be the problem I'm having.
I have 32 Gb ram on an intel i7. Windows 10. My DAZ version is 4.10
I have two GTX 1070 cards (8 GB), not in SLI configuration.
With a simple scene setup, a G2 this room: https://www.daz3d.com/dream-street-bedroom-suite (I mention this because I really doubt my scene has over 8 GB of data to process)
I added an overhead ghostlight, because Iray likes more light.
For some reason, I've been having slow IRAY renders lately.
Tonight, I found out that it was rendering via the CPU, even though I only have the GPU's checked in the settings.
Somtimes, it will tell me it's starting with the CPU; if I close DAZ and restart, it might use my GPU's for a few minutes, then change over to CPU.
One time, I noticed the screen went black jsut as it switched to CPU.
I was concerned my cards were overheating, but I'm using afterburner to keep them cool--never much over 50 C.
This is really vexing, as I invested in the NVIDIA cards to make rendering faster, and now I'm getting slow CPU renders.
Here's a code I found in the log, if it helps: "2018-04-08 04:56:05.152 WARNING: dzneuraymgr.cpp(307): Iray ERROR - module:category(IRAY:RENDER): 1.7 IRAY rend error: CUDA device 0 (GeForce GTX 1070): unspecified launch failure (while de-allocating memory)"
Daz 3D is part of 
Connect
DAZ Productions, Inc.
7533 S Center View Ct #4664
West Jordan, UT 84084
Licensing Agreement | Terms of Service | Privacy Policy | EULA
© 2026 Daz Productions Inc. All Rights Reserved.
Comments
Here's more log, in case it helps. I tried to go back to where the problems begin:
2018-04-08 05:14:26.873 Iray INFO - module:category(IRAY:RENDER): 1.0 IRAY rend info : Updating geometry.
2018-04-08 05:14:26.943 Iray INFO - module:category(IRAY:RENDER): 1.0 IRAY rend info : Using built-in ray tracing (1.0.0).
2018-04-08 05:14:26.943 Iray INFO - module:category(IRAY:RENDER): 1.0 IRAY rend info : Importing scene graph.
2018-04-08 05:14:26.943 Iray INFO - module:category(IRAY:RENDER): 1.0 IRAY rend info : Importing geometry.
2018-04-08 05:14:28.150 Iray INFO - module:category(IRAY:RENDER): 1.0 IRAY rend info : Geometry import (1 object with 3285k triangles, 1 instance yielding 3285k triangles) took 1.206087
2018-04-08 05:14:28.152 Iray INFO - module:category(IRAY:RENDER): 1.0 IRAY rend info : Updating materials.
2018-04-08 05:14:28.195 Iray INFO - module:category(MATCNV:RENDER): 1.0 MATCNV rend info : found 547 textures, 0 lambdas (0 unique)
2018-04-08 05:14:28.210 Iray INFO - module:category(IRAY:RENDER): 1.0 IRAY rend info : Emitter geometry import (25 light sources with 40201 triangles, 1 instance) took 0.00s
2018-04-08 05:14:28.210 Iray INFO - module:category(IRAY:RENDER): 1.0 IRAY rend info : Updating environment.
2018-04-08 05:14:28.212 Iray INFO - module:category(IRAY:RENDER): 1.0 IRAY rend info : Updating lens.
2018-04-08 05:14:28.212 Iray INFO - module:category(IRAY:RENDER): 1.0 IRAY rend info : Updating lights.
2018-04-08 05:14:28.213 Iray INFO - module:category(IRAY:RENDER): 1.0 IRAY rend info : Updating object flags.
2018-04-08 05:14:28.213 Iray INFO - module:category(IRAY:RENDER): 1.0 IRAY rend info : Updating caustic portals.
2018-04-08 05:14:28.213 Iray INFO - module:category(IRAY:RENDER): 1.0 IRAY rend info : Updating decals.
2018-04-08 05:14:28.216 Iray INFO - module:category(IRAY:RENDER): 1.0 IRAY rend info : Allocating 1 layer frame buffer
2018-04-08 05:14:28.221 Iray INFO - module:category(IRAY:RENDER): 1.0 IRAY rend info : Using batch scheduling, architectural sampler disabled, caustic sampler disabled
2018-04-08 05:14:28.221 Iray INFO - module:category(IRAY:RENDER): 1.0 IRAY rend info : Initializing local rendering.
2018-04-08 05:14:28.221 WARNING: dzneuraymgr.cpp(307): Iray WARNING - module:category(IRAY:RENDER): 1.0 IRAY rend warn : Non-physical gamma value 1.00 used in tonemapper.
2018-04-08 05:14:28.247 Iray INFO - module:category(IRAY:RENDER): 1.0 IRAY rend info : Rendering with 2 device(s):
2018-04-08 05:14:28.247 Iray INFO - module:category(IRAY:RENDER): 1.0 IRAY rend info : CUDA device 0 (GeForce GTX 1070)
2018-04-08 05:14:28.247 Iray INFO - module:category(IRAY:RENDER): 1.0 IRAY rend info : CUDA device 1 (GeForce GTX 1070)
2018-04-08 05:14:28.247 Iray INFO - module:category(IRAY:RENDER): 1.0 IRAY rend info : Rendering...
2018-04-08 05:14:28.247 Iray VERBOSE - module:category(IRAY:RENDER): 1.2 IRAY rend progr: CUDA device 0 (GeForce GTX 1070): Processing scene...
2018-04-08 05:14:28.248 Iray VERBOSE - module:category(IRAY:RENDER): 1.3 IRAY rend progr: CUDA device 1 (GeForce GTX 1070): Processing scene...
2018-04-08 05:14:30.760 WARNING: dzneuraymgr.cpp(307): Iray WARNING - module:category(IRAY:RENDER): 1.7 IRAY rend warn : RTKernel: please check the input geometry (degenerated/inefficient RT hierarchy)
2018-04-08 05:14:30.792 Iray VERBOSE - module:category(IRAY:RENDER): 1.7 IRAY rend stat : Geometry memory consumption: 98.5177 MiB (device 0), 0 B (host)
2018-04-08 05:14:30.795 Iray VERBOSE - module:category(IRAY:RENDER): 1.6 IRAY rend stat : Geometry memory consumption: 98.5177 MiB (device 1), 0 B (host)
2018-04-08 05:14:50.718 Iray INFO - module:category(IRAY:RENDER): 1.6 IRAY rend info : Initializing light hierarchy.
2018-04-08 05:14:50.718 Iray VERBOSE - module:category(IRAY:RENDER): 1.6 IRAY rend stat : Texture memory consumption: 2.31497 GiB (device 1)
2018-04-08 05:14:50.720 Iray VERBOSE - module:category(IRAY:RENDER): 1.7 IRAY rend stat : Texture memory consumption: 2.31497 GiB (device 0)
2018-04-08 05:14:51.093 Iray INFO - module:category(IRAY:RENDER): 1.6 IRAY rend info : Light hierarchy initialization took 0.38s
2018-04-08 05:14:51.098 Iray VERBOSE - module:category(IRAY:RENDER): 1.6 IRAY rend stat : Lights memory consumption: 6.81389 MiB (device 1)
2018-04-08 05:14:51.099 Iray VERBOSE - module:category(IRAY:RENDER): 1.7 IRAY rend stat : Lights memory consumption: 6.81389 MiB (device 0)
2018-04-08 05:14:51.262 Iray VERBOSE - module:category(IRAY:RENDER): 1.6 IRAY rend stat : Material measurement memory consumption: 0 B (GPU)
2018-04-08 05:14:51.263 Iray VERBOSE - module:category(IRAY:RENDER): 1.6 IRAY rend stat : Materials memory consumption: 801.66 KiB (GPU)
2018-04-08 05:14:51.293 Iray INFO - module:category(IRAY:RENDER): 1.3 IRAY rend info : CUDA device 1 (GeForce GTX 1070): Scene processed in 23.046s
2018-04-08 05:14:51.295 Iray INFO - module:category(IRAY:RENDER): 1.3 IRAY rend info : CUDA device 1 (GeForce GTX 1070): Allocated 20.0037 MiB for frame buffer
2018-04-08 05:14:51.298 Iray INFO - module:category(IRAY:RENDER): 1.2 IRAY rend info : CUDA device 0 (GeForce GTX 1070): Scene processed in 23.051s
2018-04-08 05:14:51.306 Iray INFO - module:category(IRAY:RENDER): 1.2 IRAY rend info : CUDA device 0 (GeForce GTX 1070): Allocated 20.0037 MiB for frame buffer
2018-04-08 05:14:51.349 Iray INFO - module:category(IRAY:RENDER): 1.3 IRAY rend info : CUDA device 1 (GeForce GTX 1070): Allocated 1.65625 GiB of work space (2048k active samples in 0.054s)
2018-04-08 05:14:51.360 Iray INFO - module:category(IRAY:RENDER): 1.2 IRAY rend info : CUDA device 0 (GeForce GTX 1070): Allocated 1.65625 GiB of work space (2048k active samples in 0.054s)
2018-04-08 05:14:51.362 Iray INFO - module:category(IRAY:RENDER): 1.2 IRAY rend info : CUDA device 0 (GeForce GTX 1070): Used for display, optimizing for interactive usage (performance could be sacrificed)
2018-04-08 05:14:57.773 WARNING: dzneuraymgr.cpp(307): Iray ERROR - module:category(IRAY:RENDER): 1.5 IRAY rend error: CUDA device 1 (GeForce GTX 1070): Kernel [1] failed after 3.015s
2018-04-08 05:14:57.773 WARNING: dzneuraymgr.cpp(307): Iray ERROR - module:category(IRAY:RENDER): 1.6 IRAY rend error: CUDA device 0 (GeForce GTX 1070): unspecified launch failure (while launching CUDA renderer in core_renderer_wf.cpp:832)
2018-04-08 05:14:57.776 WARNING: dzneuraymgr.cpp(307): Iray ERROR - module:category(IRAY:RENDER): 1.5 IRAY rend error: CUDA device 1 (GeForce GTX 1070): unspecified launch failure (while launching CUDA renderer in core_renderer_wf.cpp:807)
2018-04-08 05:14:57.776 WARNING: dzneuraymgr.cpp(307): Iray ERROR - module:category(IRAY:RENDER): 1.6 IRAY rend error: CUDA device 0 (GeForce GTX 1070): Failed to launch renderer
2018-04-08 05:14:57.776 WARNING: dzneuraymgr.cpp(307): Iray ERROR - module:category(IRAY:RENDER): 1.5 IRAY rend error: CUDA device 1 (GeForce GTX 1070): Failed to launch renderer
2018-04-08 05:14:57.778 WARNING: dzneuraymgr.cpp(307): Iray ERROR - module:category(IRAY:RENDER): 1.2 IRAY rend error: CUDA device 0 (GeForce GTX 1070): Device failed while rendering
2018-04-08 05:14:57.778 WARNING: dzneuraymgr.cpp(307): Iray ERROR - module:category(IRAY:RENDER): 1.3 IRAY rend error: CUDA device 1 (GeForce GTX 1070): Device failed while rendering
2018-04-08 05:14:57.778 WARNING: dzneuraymgr.cpp(307): Iray ERROR - module:category(IRAY:RENDER): 1.2 IRAY rend error: CUDA device 0 (GeForce GTX 1070): unspecified launch failure (while initializing memory buffer)
I tried killing off that large number of lights and rendering again. (I also changed the gamma value, since it complained.) After 7 minutes, it swqitched over to CPU, then crashed to desktop when I canceled the render.
I am not an expert, so I have no definite answers -- and I may not be saying anything that your other sources haven't addressed. However....
I wouldn't say that you have a "small" scene. With 2.3GB of texture memory consumption, something in your scene is spending memory fairly liberally. I notice that the room advertises its use of displacement maps -- those can vastly increase the memory requirements if the implementor casually bumped the subdiv layer to anything higher than 5 -- and I don't believe the logs ever mention the post-subdivision memory cost of the geometry. This gets especially bad if you have any old renders still sitting in background windows waiting to "resume" -- I don't believe that subdivided geometry is shared between renders, so multiple copies could bite into your memory capacity further. I've certainly had single scene elements blast through all the memory in my 8MB GTX 1070.
I'm assuming that you've already tried multiple version of the graphics card driver but, if not, a driver issue could certainly cause the issues you are seeing.
Yes, I did a driver update today, as I conveniently hadn't for a while, as a test to troubleshoot the problem. There was a new version. I suspect you may be onto something about the displacement maps. I've used them myself, but wasn't thinking of them in this particular issue, as the scene was set up some time ago. Thanks for the tip!
Stop stabbing in the dark to solve your issue. :)
Can you render a simple scene when the reported error?
If so:
Save each part of the scene seperately as a subset.
Save the empty (yes delete everything not lights and cameras) scene with lights and camera so you can easily create what you need. Consider deleting any HDRIs too if you're using. Just note which it is/
Merge in items one at a time. Save successful attempts.
Eventually, you should find the culprit.
Guessing the problem, and we can get lucky; but often takes lots longer to effectively solve an issue.
You're wondering if memory issue; is there a memory issue reported? I'm not seeing one from logs you posted.
2.3GB isn't that much as IIRC that is uncompressed size.
@Myds
If you use GPU-Z (https://www.techpowerup.com/gpuz/) you won't have to guess at your GPU parameters while rendering.
Thanks for all the advice I downloaded that program and will try both these things!
The most useful thing I found so far from TechPowerUp's program is that my performance cap is caused by "reliability voltage." I had been running afterburner already, so I knew about temp, could control fan speedc, etc.
I'm doing renders with most of the scene cut out. For example, this one had no walls in it. It did well for 11 minutes, then:
2018-04-08 19:24:56.395 Iray VERBOSE - module:category(IRAY:RENDER): 1.0 IRAY rend progr: 79.45% of image converged
2018-04-08 19:24:56.402 Iray INFO - module:category(IRAY:RENDER): 1.0 IRAY rend info : Received update to 00518 iterations after 678.050s.
2018-04-08 19:24:56.581 WARNING: cloud\dzcloudtasknotifier.cpp(178): recv failed errno=10054
2018-04-08 19:25:06.886 Iray INFO - module:category(IRAY:RENDER): 1.0 IRAY rend info : Received update to 00528 iterations after 688.533s.
2018-04-08 19:25:20.471 WARNING: dzneuraymgr.cpp(307): Iray ERROR - module:category(IRAY:RENDER): 1.3 IRAY rend error: CUDA device 0 (GeForce GTX 1070): Kernel [9] failed after 0.992s
2018-04-08 19:25:20.471 WARNING: dzneuraymgr.cpp(307): Iray ERROR - module:category(IRAY:RENDER): 1.9 IRAY rend error: CUDA device 1 (GeForce GTX 1070): the launch timed out and was terminated (while launching CUDA renderer in core_renderer_wf.cpp:832)
At that point, it switched to CPU.
Another render, with some of the parts that weren't in that last one, but pretty much everything in that last one removed. I'm hoping someone knows what these error-codes mean. Also, I'm wondering, if it were just that the room product was too big, too big for my 2 GPUS with 8 GB video ram each, then who can render it?
2018-04-08 19:38:39.391 WARNING: cloud\dzcloudtasknotifier.cpp(178): recv failed errno=10054
2018-04-08 19:39:05.798 Iray INFO - module:category(IRAY:RENDER): 1.0 IRAY rend info : Received update to 00107 iterations after 355.290s.
2018-04-08 19:39:10.723 Iray INFO - module:category(IRAY:RENDER): 1.0 IRAY rend info : Received update to 00112 iterations after 360.214s.
2018-04-08 19:39:24.876 WARNING: dzneuraymgr.cpp(307): Iray ERROR - module:category(IRAY:RENDER): 1.4 IRAY rend error: CUDA device 0 (GeForce GTX 1070): Kernel [9] failed after 2.075s
2018-04-08 19:39:24.879 WARNING: dzneuraymgr.cpp(307): Iray ERROR - module:category(IRAY:RENDER): 1.7 IRAY rend error: CUDA device 1 (GeForce GTX 1070): Kernel [9] failed after 0.545s
2018-04-08 19:39:24.879 WARNING: dzneuraymgr.cpp(307): Iray ERROR - module:category(IRAY:RENDER): 1.4 IRAY rend error: CUDA device 0 (GeForce GTX 1070): unspecified launch failure (while launching CUDA renderer in core_renderer_wf.cpp:807)
2018-04-08 19:39:24.879 WARNING: dzneuraymgr.cpp(307): Iray ERROR - module:category(IRAY:RENDER): 1.7 IRAY rend error: CUDA device 1 (GeForce GTX 1070): unspecified launch failure (while launching CUDA renderer in core_renderer_wf.cpp:807)
2018-04-08 19:39:24.882 WARNING: dzneuraymgr.cpp(307): Iray ERROR - module:category(IRAY:RENDER): 1.4 IRAY rend error: CUDA device 0 (GeForce GTX 1070): Failed to launch renderer
2018-04-08 19:39:24.882 WARNING: dzneuraymgr.cpp(307): Iray ERROR - module:category(IRAY:RENDER): 1.7 IRAY rend error: CUDA device 1 (GeForce GTX 1070): Failed to launch renderer
2018-04-08 19:39:24.882 WARNING: dzneuraymgr.cpp(307): Iray ERROR - module:category(IRAY:RENDER): 1.9 IRAY rend error: CUDA device 0 (GeForce GTX 1070): Device failed while rendering
2018-04-08 19:39:24.884 WARNING: dzneuraymgr.cpp(307): Iray ERROR - module:category(IRAY:RENDER): 1.8 IRAY rend error: CUDA device 1 (GeForce GTX 1070): Device failed while rendering
2018-04-08 19:39:24.884 WARNING: dzneuraymgr.cpp(307): Iray ERROR - module:category(IRAY:RENDER): 1.9 IRAY rend error: CUDA device 0 (GeForce GTX 1070): unspecified launch failure (while initializing memory buffer)
Taking out the ceiling and all those lights, IRAY rendered it (in ghostlight only) in udner 4 minutes. I suspect that its' the point lights, their sheer number, and/or the combination of them with lots of very reflective surfaces. Of course, I removed the G2 Also for that render, but it can render G2 alone very fast, so I don't think that was it.
Are you leaving multiple render windows open? Each one is going to eat up huge amounts of VRAM in a complex scene which will force it to switch to CPU rendering. Keep track of how many render windows you have open and also the viewport if it's set to IRAY will count as one more render window. I have a 1080ti with 11gb VRAM and I've had scenes that would be thrown to CPU rendering if I had even a single render window already open.
I agree with Etheral. My practice is to do a smallish (~500x750) trial render to check lighting, pose, etc until I am happy with the scene. Once i'm happy with the result, i will close all but one trial render and render at the final size (typically 4000x6000). I have a GTX980ti.
This practice can usually give me final renders that avoid having to further optimize. Then again, I already know i won't be able to GPU render a full Stonemason set with a dozen G8 figures,
@Myds
I wnt ahead and bought the package to try and render. I loaded the full scene then rendered via Camera 4 (1800x1200). The scene loaded on the GPU is 2063MB. It rendered for 9 minutes (11 iterations) when the GPU cut out (black screen briefly) . GPU temp was 70C. GPU came back andI termianted the render.
This hasn't happened before. I conclude that there is something going on with this product since there was nothin unusual with the GPU parameters and I have previously rendered 3-5 hour renders. Irecently rendered a similarly sized scene with Stoenmason's Private Garden ( https://www.daz3d.com/the-private-garden ) without problems.
I suggest you try a different scene to see what happens.
I have been having a similar issue recently where my display will black out briefly, and the render will drop back to CPU only. I also get a windows error saying the display driver stopped working and has recovered, but iray will not utilise my GPU’s without a full windows restart. I found this https://www.drivereasy.com/knowledge/solvednvidia-display-driver-stopped-working-and-has-recovered/ article, and changed my power settings accordingly, and fingers crossed the issue has not returned yet.
I just posted yesterday about power settings with my GPU, and Mart1n71's link is exactly what I tried (but his link is the easiest step-by-step). Not saying that it would account for the OP's issues with that particular scene, but it did seem to account for why my screen was blacking out mid-render. So I can vouch for the idea that changing power settings in Windows doesn't hurt, and it might help. It makes sense that with the default power configurations, especially those profiles set up by the computer builders (mine is an HP), they don't consider the user who would start a render and walk away from their computer for a few hours and still expect it to be running the whole time. You basically have to tell Windows to stop trying to help.
"Device failed"... The card stopped responding to any input and could not be "seen" by the OS.
You either have corrupted drivers, or your over-clocks are failing, or your cards are overheating, or you have failing hardware, or you have an inadequate power supply setup.
Normally, the card will attempt to reboot, if it can, and windows will try to recover it... (Use it without a need to reboot.) However, anything being processed will not recover.
Your card may just be decaying... Or the borderline settings you have, for overclocking, are too close to the border. (Daz doesn't use your card like a game does, so it may end-up riding hotter than expected, at clock-speeds that are too fast, and unstable. While a game, at that same activity-level, would be overheating the card and the OC settings would taper-down.)
Thanks to all who are responding. I managed to render a giant scene, with something like 15 G2's, in a complex building setup with shiny surfaces and maybe 18 point lights, some emissive surfaces and and HDRI setup. It rendered in about 40 minutes, without the graphics card going out.
I can't imagine my cards are overheating, if afterburner is telling me the truth. For renders, I put the fan up high (sometimes 100%) and have the max temp set down from default 83 to 75, but they never get hotter than about 65.
I don't generally leave multiple render windows open. I am lucky to leave one open to curtail the beginning render calculations. Once it gets into showing the picture on screen, I close that one as well.
Thanks to Fastbike1 for checking out the product adn replciating my results. It's a beautiful room, but odd that it gives me so much trouble. I'll read the links about drivers and power now.
@Myds
I'm now pretty convinced that the Dream Suite product is the issue since I have essentially replicated your issues on my machine.
That product / render managed to corrupt my driver to the extent that Studio couldn't see it, nor could the Windows 7 desktop, which defaulted to a standard driver. I needed to reinstall the 388.43 driver to get the gtx980TI recognized by everything. Needless to say, there is a new PA added to my Do Not Buy list.
It's a shame, too, because it has such a nice look.
You mentioned the large number of point lights.
How many are there? ...and will the scene render if that number is reduced?