Dual GeForce Recommendations

I'm planning to build a new system that can run Daz3D Studio really well. (I tend to build my own PC's. I can't remember the last time I bought a pre-built PC.) I have some questions about the graphics cards, which will probably be one of the Nvidia GeForce cards. I see from the System Recommendations article that dual (or more) cards are supported for rendering. Should these cards be connected with SLI bridges? Do I want to do an SLI setup? A related question, I think: how does the software control splitting the GPU resources between the rendering job and displaying the output?

I see also that it is better to run the GPU cards at PCIe x16. I realize I would have to get one of the higher-end cpu's that provide 44 rather than 28 PCIe lanes. Do you have any recommendations for CPU/MB combinations that will support 2 graphic cards, each running at x16, plus 2 M.2 SSD's (for the OS and user data), which will each use x4? Do I want to look at MB's based on the X299 chipset, or would the Z270 or Z370 be better?

One more question unrelated to graphic cards. Will Daz3D studio use multithreading for the calculations it makes with the CPU rather than the GPUs? Can it take advantage of multiple cores?

Comments

  • It can use multi-threading for CPU usage.

    In my experience, SLI did not improve render times, and in fact rendering the same scene with and without SLI enabled indicated that it renders much faster without SLI enabled. IIRC, the scene I set up took 25 minutes with SLI and 12 without SLI.

    It can use all CUDA-based cards in the system without needing to be connected in SLI. However, while it will divide the workload between all cards, it is still limited to the VRAM of your lowest card, which means if you have a 4GB card and a 12GB card, and your scene uses 6GB, your cards will be ignored and the render will be dumped off to the CPU. As well, if one of those cards is your primary display driver, that function will consume extra resources for that task, and if the scene exceeds the remainder, it will be dumped off to the CPU entirely.

    As for how the program splits the load: both cards are used for the rendering calculations by pooling the CUDA cores. Your primary display handles the display functions as it normally would, which doesn't require the use of CUDA cores. It's like a separate function the card is capable of.

    VRAM is not pooled, as that's a limitation of the current Geforce line. The new Volta series is supposed to be capable of pooling the ram, so if you have 2 12GB cards with 3500 CUDA cores, you have 24GB of VRAM and 7000 CUDA cores available.

  • JamesJABJamesJAB Posts: 1,760

    Correction about multiple GPUs.

    When you have multiple GPUs with varying amounts of VRAM, each GPU that has enough VRAM to fit the scene will participate... 
    For example lets say that you have the following GPUs installed in your system.
    1x GTX 1080 ti 11GB
    1x GTX 1070 8GB
    1x GTX 1050 ti 4GB.
    Your Render takes 6.5GB of VRAM
    In this case the GTX 1050 ti will not participate in the render, but the GTX 1080 ti and GTX 1070 will.

    SLI : Iray does not use SLI, having the bridge installed for Gaming will not effect Iray one way or the other.
    PCIe lanes are not important for Iray rendering as each GPU acts as it's own render node and does not rely on the PCIe bus.  The initial load to prepare for a render to start is the only place where you may see a speed boost with more PCIe lanes.

    Nvidia has designed the Heatsink/Housing on the Titan V in such a way that the NVLink ports are blocked.  Sorry no memory pooling on the Titan V
    As of right now the only GPU that supports NVLink is the Quadro GP100 16GB (this does not take non-GPU Tesla cards into account)

  • JamesJAB said:

    Correction about multiple GPUs.

    When you have multiple GPUs with varying amounts of VRAM, each GPU that has enough VRAM to fit the scene will participate... 
    For example lets say that you have the following GPUs installed in your system.
    1x GTX 1080 ti 11GB
    1x GTX 1070 8GB
    1x GTX 1050 ti 4GB.
    Your Render takes 6.5GB of VRAM
    In this case the GTX 1050 ti will not participate in the render, but the GTX 1080 ti and GTX 1070 will.

    Is this from a recent update? Before my GPU cluster went down a few weeks ago I had a 4GB 980 in the mix, and if I exceeded that, the Titan Z (6x2), 1080ti, 780ti, and Titan X (Pascal) were ignored.

  • fastbike1fastbike1 Posts: 4,074
    edited January 2018

    @DrNewcenstein "Is this from a recent update? Before my GPU cluster went down a few weeks ago I had a 4GB 980 in the mix, and if I exceeded that, the Titan Z (6x2), 1080ti, 780ti, and Titan X (Pascal) were ignored."

    No, that's the way Iray has always worked in Studio.

    https://forum.nvidia-arc.com/showthread.php?15566-How-is-VRAM-used-on-multiple-GPUs

    Your issues may be more of a system problem running that many gpus. FWIW, I don't see why the 980 would drop out before the 780ti/

    Post edited by fastbike1 on
  • I was using the 980 as primary, so it started with reduced resources.

  • RaymandRaymand Posts: 62

    Thank you very much for this detailed information.

    Assuming you have one or more graphics cards that can be used for rendering, will having more CPU cores provide a further performance benefit? That is, aside from offloading work to the GPU's, does the software use multithreading for the work done on the CPU's?

  • OZ-84OZ-84 Posts: 128
    edited January 2018
    JamesJAB said:

    Correction about multiple GPUs.

    When you have multiple GPUs with varying amounts of VRAM, each GPU that has enough VRAM to fit the scene will participate... 
    For example lets say that you have the following GPUs installed in your system.
    1x GTX 1080 ti 11GB
    1x GTX 1070 8GB
    1x GTX 1050 ti 4GB.
    Your Render takes 6.5GB of VRAM
    In this case the GTX 1050 ti will not participate in the render, but the GTX 1080 ti and GTX 1070 will.

    SLI : Iray does not use SLI, having the bridge installed for Gaming will not effect Iray one way or the other.
    PCIe lanes are not important for Iray rendering as each GPU acts as it's own render node and does not rely on the PCIe bus.  The initial load to prepare for a render to start is the only place where you may see a speed boost with more PCIe lanes.

    Nvidia has designed the Heatsink/Housing on the Titan V in such a way that the NVLink ports are blocked.  Sorry no memory pooling on the Titan V
    As of right now the only GPU that supports NVLink is the Quadro GP100 16GB (this does not take non-GPU Tesla cards into account)

    That is not correct and DrNewcenstein is is right about this ...

    If you have only one of your cards that cant fit the data that is needed to render the scene, everything falls back to CPU render mode. Thats what i observerd on my old systems and that is what i observe on my 4 x 1080ti system. One of my cards is driving a 4k display that reservers more memory in Windows 10 than the other cards do on smaller res displays. If the card that drives the 4k display cant fit the data every card is dropped and falls back to CPU render. 

    Thats the reason why i am looking for a "cheap" Titan XP to change it for one of the other cards ... :-(

    Post edited by OZ-84 on
  • JamesJABJamesJAB Posts: 1,760
    OZ-84 said:
    JamesJAB said:

    Correction about multiple GPUs.

    When you have multiple GPUs with varying amounts of VRAM, each GPU that has enough VRAM to fit the scene will participate... 
    For example lets say that you have the following GPUs installed in your system.
    1x GTX 1080 ti 11GB
    1x GTX 1070 8GB
    1x GTX 1050 ti 4GB.
    Your Render takes 6.5GB of VRAM
    In this case the GTX 1050 ti will not participate in the render, but the GTX 1080 ti and GTX 1070 will.

    SLI : Iray does not use SLI, having the bridge installed for Gaming will not effect Iray one way or the other.
    PCIe lanes are not important for Iray rendering as each GPU acts as it's own render node and does not rely on the PCIe bus.  The initial load to prepare for a render to start is the only place where you may see a speed boost with more PCIe lanes.

    Nvidia has designed the Heatsink/Housing on the Titan V in such a way that the NVLink ports are blocked.  Sorry no memory pooling on the Titan V
    As of right now the only GPU that supports NVLink is the Quadro GP100 16GB (this does not take non-GPU Tesla cards into account)

    That is not correct and DrNewcenstein is is right about this ...

    If you have only one of your cards that cant fit the data that is needed to render the scene, everything falls back to CPU render mode. Thats what i observerd on my old systems and that is what i observe on my 4 x 1080ti system. One of my cards is driving a 4k display that reservers more memory in Windows 10 than the other cards do on smaller res displays. If the card that drives the 4k display cant fit the data every card is dropped and falls back to CPU render. 

    Thats the reason why i am looking for a "cheap" Titan XP to change it for one of the other cards ... :-(

    When I upgraded from a 6GB GTX 1060 to a 11GB GTX 1080 ti, I ran some tests with both cards installed and can say for a fact that when the scene was too big for the 1060 it would just not participate.  The GTX 1080 ti would do the render alone. (If you have some kind of a mid-render event that changes avalible VRAM, that may trigger a failsafe drop to CPU mode for the entire render job.)

  • TheKDTheKD Posts: 2,674

    Yeah, I have a 960 and 1070, sometimes just the 1070 only renders if it's over the 4gb vram load.

  • OZ-84 said:
    JamesJAB said:

    Correction about multiple GPUs.

    When you have multiple GPUs with varying amounts of VRAM, each GPU that has enough VRAM to fit the scene will participate... 
    For example lets say that you have the following GPUs installed in your system.
    1x GTX 1080 ti 11GB
    1x GTX 1070 8GB
    1x GTX 1050 ti 4GB.
    Your Render takes 6.5GB of VRAM
    In this case the GTX 1050 ti will not participate in the render, but the GTX 1080 ti and GTX 1070 will.

    SLI : Iray does not use SLI, having the bridge installed for Gaming will not effect Iray one way or the other.
    PCIe lanes are not important for Iray rendering as each GPU acts as it's own render node and does not rely on the PCIe bus.  The initial load to prepare for a render to start is the only place where you may see a speed boost with more PCIe lanes.

    Nvidia has designed the Heatsink/Housing on the Titan V in such a way that the NVLink ports are blocked.  Sorry no memory pooling on the Titan V
    As of right now the only GPU that supports NVLink is the Quadro GP100 16GB (this does not take non-GPU Tesla cards into account)

    That is not correct and DrNewcenstein is is right about this ...

    If you have only one of your cards that cant fit the data that is needed to render the scene, everything falls back to CPU render mode. Thats what i observerd on my old systems and that is what i observe on my 4 x 1080ti system. One of my cards is driving a 4k display that reservers more memory in Windows 10 than the other cards do on smaller res displays. If the card that drives the 4k display cant fit the data every card is dropped and falls back to CPU render. 

    Thats the reason why i am looking for a "cheap" Titan XP to change it for one of the other cards ... :-(

    No, JamesJAB is correct - Iray will use any card the scene fits into, dropping only those that can't handle it. If you are seeing different behaviour then something else is going on.

  • OZ-84OZ-84 Posts: 128
    OZ-84 said:
    JamesJAB said:

    Correction about multiple GPUs.

    When you have multiple GPUs with varying amounts of VRAM, each GPU that has enough VRAM to fit the scene will participate... 
    For example lets say that you have the following GPUs installed in your system.
    1x GTX 1080 ti 11GB
    1x GTX 1070 8GB
    1x GTX 1050 ti 4GB.
    Your Render takes 6.5GB of VRAM
    In this case the GTX 1050 ti will not participate in the render, but the GTX 1080 ti and GTX 1070 will.

    SLI : Iray does not use SLI, having the bridge installed for Gaming will not effect Iray one way or the other.
    PCIe lanes are not important for Iray rendering as each GPU acts as it's own render node and does not rely on the PCIe bus.  The initial load to prepare for a render to start is the only place where you may see a speed boost with more PCIe lanes.

    Nvidia has designed the Heatsink/Housing on the Titan V in such a way that the NVLink ports are blocked.  Sorry no memory pooling on the Titan V
    As of right now the only GPU that supports NVLink is the Quadro GP100 16GB (this does not take non-GPU Tesla cards into account)

    That is not correct and DrNewcenstein is is right about this ...

    If you have only one of your cards that cant fit the data that is needed to render the scene, everything falls back to CPU render mode. Thats what i observerd on my old systems and that is what i observe on my 4 x 1080ti system. One of my cards is driving a 4k display that reservers more memory in Windows 10 than the other cards do on smaller res displays. If the card that drives the 4k display cant fit the data every card is dropped and falls back to CPU render. 

    Thats the reason why i am looking for a "cheap" Titan XP to change it for one of the other cards ... :-(

    No, JamesJAB is correct - Iray will use any card the scene fits into, dropping only those that can't handle it. If you are seeing different behaviour then something else is going on.

    Well :-|

    All Systems i owned rendered on all cards ... or on CPU. There was nothing between. Maybe something is wrong here. 

    What i would wish for is, that DAZ wouldnt fall back to CPU mode at all. Its simply a waste of everything since the time and iterations settings set in the scene fit a faster GPU render. When Daz falls back to CPU the render will be grainy and useless. I see no potential benefit in this behaviour...

  • JamesJABJamesJAB Posts: 1,760
    OZ-84 said:
    OZ-84 said:
    JamesJAB said:

    Correction about multiple GPUs.

    When you have multiple GPUs with varying amounts of VRAM, each GPU that has enough VRAM to fit the scene will participate... 
    For example lets say that you have the following GPUs installed in your system.
    1x GTX 1080 ti 11GB
    1x GTX 1070 8GB
    1x GTX 1050 ti 4GB.
    Your Render takes 6.5GB of VRAM
    In this case the GTX 1050 ti will not participate in the render, but the GTX 1080 ti and GTX 1070 will.

    SLI : Iray does not use SLI, having the bridge installed for Gaming will not effect Iray one way or the other.
    PCIe lanes are not important for Iray rendering as each GPU acts as it's own render node and does not rely on the PCIe bus.  The initial load to prepare for a render to start is the only place where you may see a speed boost with more PCIe lanes.

    Nvidia has designed the Heatsink/Housing on the Titan V in such a way that the NVLink ports are blocked.  Sorry no memory pooling on the Titan V
    As of right now the only GPU that supports NVLink is the Quadro GP100 16GB (this does not take non-GPU Tesla cards into account)

    That is not correct and DrNewcenstein is is right about this ...

    If you have only one of your cards that cant fit the data that is needed to render the scene, everything falls back to CPU render mode. Thats what i observerd on my old systems and that is what i observe on my 4 x 1080ti system. One of my cards is driving a 4k display that reservers more memory in Windows 10 than the other cards do on smaller res displays. If the card that drives the 4k display cant fit the data every card is dropped and falls back to CPU render. 

    Thats the reason why i am looking for a "cheap" Titan XP to change it for one of the other cards ... :-(

    No, JamesJAB is correct - Iray will use any card the scene fits into, dropping only those that can't handle it. If you are seeing different behaviour then something else is going on.

    Well :-|

    All Systems i owned rendered on all cards ... or on CPU. There was nothing between. Maybe something is wrong here. 

    What i would wish for is, that DAZ wouldnt fall back to CPU mode at all. Its simply a waste of everything since the time and iterations settings set in the scene fit a faster GPU render. When Daz falls back to CPU the render will be grainy and useless. I see no potential benefit in this behaviour...

    What you are asking for is not something that Daz has control over.  Iray is a rendering engine that is developed and coded by Nvidia, so any changes to the way it handles "out of memory" operations is Nvidia code, not Daz3d code.

  • IsaacNewtonIsaacNewton Posts: 1,300
    edited January 2018

    Is it not possible for DAZ to include a checking algorithm in DS (maybe once every 50 iterations or so during a render) to check if the GPUs are being used and provide a warning message if they are not?

    Post edited by IsaacNewton on
  • JamesJABJamesJAB Posts: 1,760

    Aside from what Iray sends to the Daz Studio log file, I'm not sure if Studio even knows what Iray is actively using as a render device.

  • fastbike1fastbike1 Posts: 4,074

    @OZ-84   "When Daz falls back to CPU the render will be grainy and useless. I see no potential benefit in this behaviour..."

    Only if your settings are time limited. XX iterations will look the same regardless of whether it's a GPU or CPU render. Remember that the user controls the settings  so if the render is unsatisfactory it's not a render engine problem. 

Sign In or Register to comment.