Bugged network rendering (C8.5v149)?
thoromyr
Posts: 452
In order to speed up render times on a project I tried to use a render node and the result was less than optimal. Times are good, but the result is bad and I'm trying to verify that it is a bug. The master is on OSX and render node on WinXP. I've done other network renders (possibly with opposite arrangement) that worked fine. I *think* the difference here is caustics are enabled, but at present I'm not 100% sure and render times are too long for me to do quick testing.
Of the two snippets, the one that shows rendering tiles exhibits the problem (I aborted the render when I noticed it). This is the second render attempt in a row that exhibited the problem in just the same spot. (I assumed at first it was just an artifact and restarted the render.)
The other clip is from an earlier test and shows the lack of a horizontal line through the glass at the edge of the render tile.
Any confirmation as to the issue? Perhaps someone with a faster rig? (My systems are years old and if I'm right about it being caustics it takes a long time to render -- and right now it is busy on the render).
Daz 3D is part of 
Connect
DAZ Productions, Inc.
224 S 200 W, Suite #250
Salt Lake City, UT 84101
Licensing Agreement | Terms of Service | Privacy Policy | EULA
© 2024 Daz Productions Inc. All Rights Reserved.
Comments
This may be a bug that goes back at least to C7. I usually get it when I use procedural shaders with a randomizer, sometimes with GI or caustics and sometimes with volumetrics when rendering across a network. I'm not sure, but there may be a bug report for C8. You don't mention your version, so I'm not sure if that's relevant to you. I'm still stuck on C7 Pro. so that ship has sailed as far as further development and bug fixes.
I would try seeing where the issue lies. Try turning off cuastics and see what you get. If the problem goes away, then you know it's abug and you can either start a bug report or add to an existing one. The link to the bug tracker (note: you will need to create a new login separate from the forums): https://bugs.daz3d.com/login_page.php
Thanks for the reply! I guess I didn't make the version clear (C8.5v149) -> Carrara 8.5 build 149. The current render set may complete in another day or so (which is why I was really trying to get a render node in to reduce the time). Once that is done I'll try without caustics and compare.
I'm going between platforms and it occurred to me that it might be a difference in how they are calculated between platforms -- if that is the case the same scene could be expected to render differently between the two platforms, not just with network renders.
I'm fairly certain I've seen what looks somewhat like this same issue before on C7. What I recollect was a bit different, though, and the entire tile was a different gamma, not just where a glass object was. But its been quite a while (at least in terms of what my memory can handle) so its a bit fuzzy for me.
This is an example of a randomization bug in C7 rendered using an Intel Mac node, a G4 node and a G5as the host. I eliminated each network node in turn and still had the issue until I used just my host to render. Volumetric effects and GI on the other hand disappeared once the Intel Mac was not part of the render. The G4 and G5 seemed in sync.
http://www.youtube.com/watch?v=6cFfLocjHQU
It goes back to C5.
I've always suspected it's a problem in the Batch renderer on the primary (not the nodes).
Do the nodes match:
a) primary (not batch renderer)
b) primary batch renderer
c) neither, nodes do not match any render from primary
actually, it takes long enough to get to the point where the problem is I've never managed to catch it there to know if its the master or a node that renders which way. And because I've tweaked things between renders up to this point I can't be 100% certain until I get a chance to do some more testing. Good points on what to look for, though. Thanks!
My problem too....
We probably need a set of "control renders" to compare the miss-matched squares:
Primary renders on Computer A and Computer B
Batch Renders on Computer A and Computer B
The issue is *probably* some chunk of code that Carrara is using that calculates differently due to the processor (that's what we all assume anyway), something like the bug Fenric hit with the Mac version of his Ivy Generator plugin that flipped the leaves.... BUT it could just as easily (in my cluless opinion) be a bug in the batch renderer....
I didn't realize there was a bug affecting the ivy generator. I'm switching more and more to rendering on OS X due to 64 bit capability (I only have WinXP 32 bit) and I use the ivy generator. Was the bug fixed?
and I think something has gone wrong with my render. Its still going (and progressing) but incredibly slowly. blech. At this rate I'm not sure when I'll get a comparison.
Yes, Fenric added code that fixed the Mac version of Ivy Gen. The latest version is all good, afaik. I didn't understand the issue, but he explained it briefly on the old forum. Something on Mac sorted things the wrong way around (presumably a result of the compiler software, not because a chip is installed backwards or whatever).
Can you tell from a partial render whether the solo render is the brighter or darker version? In otherwords have you established how it looks when it is "correct"?
I'm probably talking out of my hat, since I don't know how caustics are calculated.... I assumed they are like refraction and based on rays passing through geometry. I *think* there will be a difference in the way Distant Lights and Spot Lights are calculated to create the caustic. I'm just trying to play detective and build a list of suspects. If we run to report there is a problem with "random noise in Caustics in render node" but the real problem is something like "Spot Lights are multiplied in the batch renderer".... Well, I am happy I am not a developer, LOL. The problem could be attached to the lights, the shader (transparency, reflections,caustic), in the calculation of 3D "space" distance (scene scale)....
Despite Fenric's valiant efforts, the Macintosh ivy generator bug was never completely resolved. You can read the thread discussing the problem in the old archived forum:
http://forumarchive.daz3d.com/viewtopic.php?t=161539
Unfortunately, none of the many example images are viewable in the posts. Instead of an image, a line of text will appear such as this:
To view the image you must copy the portion that reads
postimages/origimage_1_2851733.jpg
and add
http://forumarchive.daz3d.com/
to the beginning of it so that it looks like this:
http://forumarchive.daz3d.com/postimages/origimage_1_2851733.jpg
Paste that line into the address area of a new web browser window to view the image. (A royal pain that I hope DAZ will eventually sort out.)
As it stands, the best Mac version of Fenric's ivy plug-in is the last one he issued before making Mac specific changes.
Reversing the normals of the base Carrara object on which the ivy is growing, seems to work around the problem.
Unfortunately, I don't believe that particular version of the ivy plug-in is available to download now. The last time I checked, the current version was dated July 3, 2011, but the best version is dated March 27, 2011.
Thanks for the further information about fenric's ivy plugin. I'll have to do some digging to see what versions I have.
With respect to my original issue (network rendering woes) the delay hasn't just been my getting busy on other things, I finally gave up on getting a render to complete on my macbook and switched back to a winxp desktop. I do have some incomplete renders that are, nevertheless, complete enough to show what I believe are the relevant parts sufficiently to draw some conclusions.
Basically, caustics render differently on OS X vs Windows. I have a suspicion that this may have to do with the gamma values for caustics not being identical between platforms as the rest of the render is identical. Basically, the "washed out" tiles in my original pictures match the WinXP render node's results when it is the only node involved in a render while the bluer tiles are what I get from rendering solely on OS X.
So, regardless of the exact nature of the underlying flaw, my conclusion is that rendering with caustics cannot be done with cross platform render nodes.
Something I should point out (having re-read earlier posts) is that my computers are all x86 systems -- I don't have any non-intel macintosh systems. So this isn't an intel vs G4/G5 type issue.
The WinXP system is running on an AMD dual core (64 bit, but with 32 bit WinXP) while the mac is an macbook pro (core2 duo). Unless the caustics are using special chip features that other parts of the renderer aren't (noting that I'm not using GI) I don't see how it could be an AMD/Intel thing. I *may* get a chance to try a Windows render on an Intel CPU to compare (with the added advantage of it being a newer system so the render shouldn't take days to complete).
Other than OSX/Windows and AMD/Intel I believe that I've ruled out all other variables.
Agree it doesn't seem to be processor related. It is either OS related or a bug in the Batch/node renderer (related to scene scale ?)
I spoke too soon. While the originally observed misbehavior followed platform... So I started rendering the scene on my Windows box, mostly because I hate abusing my laptop with multi-day renders. It took 4 days to render out... Then I decided to update my render node -- it runs in a vm on my linux box so I got that running, updated to the latest beta, added it and... the first network tile finally completed and it has one corner in a region affected by caustics. In just that region there are sharp lines demarking how that tile rendered the caustics. The rest of the tile is fine, just not the area with caustics.
Its a VM so "processor" is not quite the same as real hardware, but this is kvm (linux kernel virtual machine) with only a very thin layer separating it from the hardware. Both the windows and linux machines are Athlon x2 processors. The windows box, being 32 bit, is running the CPU in 32 bit mode. The linux, being linux, is running the CPU in 64 bit mode (and a modest 8GB RAM to go with that). 64-bit mode opens up more registers and I don't remember what all else. I don't see how it would impact rendering, however, unless something like SSE worked differently in 64 bit mode.
But this has two Windows 32-bit instances participating in the batch render giving different results for the caustics. It'll probably be four or five more hours at least before another tile completes (currently all are working on regions that are predominately affected by caustics. When that happens I'll be able to save the result to separate network from local rendering of caustics. Because it has already hosed the caustics, I'm tempted to pull the windows render node out and fire up the OS X render node just to see how it plays.
What I really need, though, is a monster rig with 24 cores :) -- I've been tempted by the (now discontinued, I think) 12-core (not the bulldozer design, true 12-cores) AMD processor. For what it was $1000 each wasn't bad with $400 for a dual-CPU motherboard that could go to 64/256GB. I just haven't had (and still don't have) the money for such a beast. But the pressure to do network rendering would drop off significantly with such a rig... that last 4 day render would have been ~8 hours? wheeee!
I wanted to close out this thread properly, so here goes. The short version is this, always keep track of the resources and requirements for your project and ensure they are being provided.
What this turned out to be was the glass shader was using a capability provided by a DCG product (ATM I don't recall which one) and the render node lacked the extension. Consequently it was unable to handle the shader correctly and did not produce the same result as the master (which did have the extension).