Loading...
> >

Triple GPU efficiency

colcurvecolcurve Posts: 171
in The Commons

Hi, when I render with 3 GPU, i notice that they do not run on 100% load, but rather idle every few seconds for a short time.

With 2 GPU active its always 100% load.

Anybody here using 3+ GPU and could share a comment on the monitored efficiency?

Thanks

Comments

  • richardandtracyrichardandtracy Posts: 7,314

    What is your CPU load - could the CPU be unable to feed 3 GPU's fast enough? Or maybe memory bus speed is the throttle.

  • SevrinSevrin Posts: 6,313

    What motherboard are you using?  What GPUs?

  • Ghosty12Ghosty12 Posts: 2,080
    edited July 2020

    Yes it would help to know what the GPU's you have are.. The thing is that consumer grade Nvidia cards from Pascal on will only NVlink/SLI in groups of two now.. So the days of consumer grade multi GPU systems are long gone..

    Post edited by Ghosty12 on
  • nicsttnicstt Posts: 11,715
    edited July 2020

    I really with folks would provide information when asking for help; having to drag it out of them is time-consuming. We aren't even sure they mean Nvidia, they must from the the description of performance.

    Cards

    CPU

    Motherboard

    RAM

    OS

    Rendering engine

    Studio version

    An idea of scenes

    Does it happen with all scenes? Only some?

    Post edited by nicstt on
  • colcurvecolcurve Posts: 171

    2x 980ti

    1x 2080ti

    rampage iv formula board 

    it seems the cpu is quite well loaded but I have to see whether it maxes out. It seems all cores are working but with very mixed load. 
    i have not mentioned the hardware because I heard before that 3+ gpus had some performance penalty. In the idle Phase I observed the vop performance cap in gpuz monitor 

  • Ghosty12Ghosty12 Posts: 2,080
    edited July 2020
    colcurve said:

    2x 980ti

    1x 2080ti

    rampage iv formula board 

    it seems the cpu is quite well loaded but I have to see whether it maxes out. It seems all cores are working but with very mixed load. 
    i have not mentioned the hardware because I heard before that 3+ gpus had some performance penalty. In the idle Phase I observed the vop performance cap in gpuz monitor 

    Well there is your issue right there, you have two completely different generations of GPU's in your system and probably be why you are having issues.. As Daz Studio, iRay and your operating system could be having a dummy spit deciding which device to send the relevant information too.. 

    But if you are determined to keep that configuration your only option then, is from the Render Settings you will have to deselect either the two 980Ti's or the 2080Ti in the hardware tab of the iRay advanced tab.. So that Daz Studio and iRay know which cards to send the data to..

    Post edited by Ghosty12 on
  • colcurvecolcurve Posts: 171

    what mobo / cpu would you recommand nowadays? hope the 2080ti will drop in price once the next generation is launched

  • outrider42outrider42 Posts: 3,679
    The issue could be your power supply or motherboard. You are running 3 cards that use a lot of electricity, your power supply and mobo need to be able to handle them.

    Running different generations of GPUs is not a problem for Iray at all. But remember this, VRAM does not stack. The scene must fit in each GPU's own VRAM for that GPU to run. If the scene exceeds the 980ti 6g, then only the larger 2080ti will run.

    The most likely issue you are having is that they are throttling because of heat. Again these are power hungry cards, so the generate a lot of heat and need to be cooled. Running 3 of these will absolutely raise the heat in the case. Your case needs to be able to cool these cards. Are you monitoring their temps?

    That's why all of this information is important so people can help you.
  • Ghosty12Ghosty12 Posts: 2,080
    edited July 2020
    outrider42 said:
    The issue could be your power supply or motherboard. You are running 3 cards that use a lot of electricity, your power supply and mobo need to be able to handle them.

     

    Running different generations of GPUs is not a problem for Iray at all. But remember this, VRAM does not stack. The scene must fit in each GPU's own VRAM for that GPU to run. If the scene exceeds the 980ti 6g, then only the larger 2080ti will run.

     

    The most likely issue you are having is that they are throttling because of heat. Again these are power hungry cards, so the generate a lot of heat and need to be cooled. Running 3 of these will absolutely raise the heat in the case. Your case needs to be able to cool these cards. Are you monitoring their temps?

     

    That's why all of this information is important so people can help you.

    Ahh k didn't know that iRay does not worry about what generation of GPUs are in a multi GPU system (I was wrong there..) laugh But true on the vram not stacking (not sure this applies to Quadro cards though) so having the two 980Ti's would be pointless.. Considering the 2080Ti has nearly the same amount of ram the two 980Ti's have, and completely forgot about power draw and heat..

    Post edited by Ghosty12 on
  • Richard HaseltineRichard Haseltine Posts: 108,799
    Ghosty12 said:
    colcurve said:

    2x 980ti

    1x 2080ti

    rampage iv formula board 

    it seems the cpu is quite well loaded but I have to see whether it maxes out. It seems all cores are working but with very mixed load. 
    i have not mentioned the hardware because I heard before that 3+ gpus had some performance penalty. In the idle Phase I observed the vop performance cap in gpuz monitor 

    Well there is your issue right there, you have two completely different generations of GPU's in your system and probably be why you are having issues.. As Daz Studio, iRay and your operating system could be having a dummy spit deciding which device to send the relevant information too.. 

    But if you are determined to keep that configuration your only option then, is from the Render Settings you will have to deselect either the two 980Ti's or the 2080Ti in the hardware tab of the iRay advanced tab.. So that Daz Studio and iRay know which cards to send the data to..

    There should be no issue mixing GPU types or generations, a llot of people do. Unless you have nVLinked GPUs (which do have to match) each GPU is a saparate entity as far as Iray is concerned.

  • colcurvecolcurve Posts: 171

    Thanks Richard. I'll try to find out if the idling is correlated to spikes in cpu load.

    Otherwise, do we have some workstation thread, discussing which motherboard/cpu is ideal for 3+ gpu usage?

  • outrider42outrider42 Posts: 3,679
    I have 2 GPUs, and if I leave them be, they can get really warm. The longer a render goes, one of them may start throttling. Nvidia sets up the cards so that they step down their clockspeed as they hit certain temps. Exactly when and how much changes between generations, but controlling heat is vital to keeping them running faster. You have 3 flagship models, and that will create a ton of heat.

    So the first thing you need to do is check your temps. If they are really high, then you have your answer. With 3, you might need to consider better cooling methods, like water. You at least need to set up a fan curve to control the fans better. Your PC might sound like a jet engine, but at least the GPUs are not getting fried.

    And your power supply canny be overlooked. You need at least 1000 Watts for this, if not more. If your PSU is several years old, they do lose capacity over time. This would certainly explain your issues.
  • pwiecekpwiecek Posts: 1,598

    Does your CPU have enough lanes to fully support 3 PCIe x16 slots?

  • joseftjoseft Posts: 310

    have not seen a statement of the CPU model. If the CPU is an older model that was not in the high end of its generation, i can easily see that being a bottleneck here.

    i have run 3 mixed GPUs often (recently a maxwell titan X, a 2060 and a 2080ti) and have not run into this issue

    also have not seen a statement on the power supply wattage. But i think that is less likely to be the issue, if there is not enough power being supplied during the render i think the resulting behaviour would be a lot more agressive/unstable than just some dips in usage. 

  • joseftjoseft Posts: 310
    pwiecek said:

    Does your CPU have enough lanes to fully support 3 PCIe x16 slots?

    even if it doesnt, that should not cause random dips in usage. i run 3 mixed GPUs, 40 lane CPU so its running at x16/x16/x8 and dont have this issue

    as long as the pci slot is not dropping way down to something like 1x, the only time the pci speed should affect a render is the pre-render stage when all the scene data is being loaded on to the GPUs

  • kenshaw011267kenshaw011267 Posts: 3,805

    It is very rare that a motherboard has a PCIE layout to support 3 x16 slots even if it has 3 full length slots. My x370 board has 3 full length slots but if all 3 are populated they work as x8, x8 and x4. I use 2 and populat ethe top and bottom and get x16 and x4. I've never notice any performance problems as described here.

  • Ghosty12Ghosty12 Posts: 2,080
    pwiecek said:

    Does your CPU have enough lanes to fully support 3 PCIe x16 slots?

    It is hard to know since the OP never states what the CPU and motherboard is that they have.. If the system is as old as the 980Ti's then probably not, if the system is as new as the 2080Ti then who knows what the problem is..

  • colcurvecolcurve Posts: 171

    asus rampage iv formula with i7-3930k

    i do not believe its temperature related because the idle-valleys tend to get less as render takes longer. but still if you render like 5-10 seconds 100% and 1 second idle you lose 10-20%

  • joseftjoseft Posts: 310

    i think its your CPU holding you back. 3930k is almost 9 years old, you are asking a lot of it putting it in a system with 3 big GPUs, all of which are far newer than the CPU

    That will also mean its an expensive problem to fix, because that is a discontinued socket. You cant just change your CPU to anything newer without also changing motherboard + RAM

  • Ghosty12Ghosty12 Posts: 2,080
    edited July 2020

    Thoguh both seem decent https://ark.intel.com/content/www/us/en/ark/products/63697/intel-core-i7-3930k-processor-12m-cache-up-to-3-80-ghz.html and https://www.asus.com/Motherboards/RAMPAGE_IV_FORMULA/specifications/ but as richardandtracy mentioned, it depends if the CPU is handling it well and as such could be one of the reasons..

    Post edited by Ghosty12 on
  • richardandtracyrichardandtracy Posts: 7,314

    The reason I asked about CPU was that I have a 7yo quad core Xeon and find that simply feeding one GTX1060 it's running at 25%. Feeding the hunger of a 2080 would probably take my cpu much closer to 100% and then adding a couple of extra cards would likely cause the CPU to be the choke point.

    One simple and easy way of checking would be to run task manager on a Win based PC and simply look at the numbers as it goes through. Oh, and let us know so this thread could reach a natural close-out if that's the case.

    Regards,

    Richard.

     

     

  • outrider42outrider42 Posts: 3,679
    Iray is not a video game, so the things that would typically bottleneck performance do not apply. The number of PCI lanes do not effect performance. This has been tested and proven.

    I have doubts that the CPU is really effecting performance that much here. As I mentioned, I have two 1080tis, so not 3 cards, but my CPU is an aging 4690 with just 4 cores. In gaming that CPU is a bit of a bottleneck with a 1080ti. But for Iray my render speeds 100% match anybody else running two 1080tis. And there are numerous dual 1080ti benchmarks. And remember folks, two of the GPUs being used by the op are 980tis, which are about as old as that i7 is now. The 2080ti is the only newer component in the setup. Back in the day, sickleyield's bench scene was being used by everybody. When the 980ti was still king, numerous users were sporting several of these beasts, and with very similar setups to what our op is using. If you wish, you can browse that old thread and skim through its marks. People using 3 or 4 cards is not that uncommon, and some used older i7s.

    So I still believe we have a power delivery issue of some kind. Your PSU may technically offer enough Watts to cover all 3, but there is more at play with 3 big power hungry cards. You also need to have 3 separate 12 Volt rails that meet the demands of every card. Also, just judging by the components listed so far, it sounds like most of this machine is fairly old. If the PSU is the same vintage as the 980tis then it may be time for an upgrade to it. You need to carefully look at all these specs.

    Also, if your motherboard is aging, look closely at its capacitors for any bulging caps. This could effect things, too. Capacitors dry out over the years, and this effects their ability to store a charge...power.
  • colcurvecolcurve Posts: 171

    Its a seasonic gold 1300w PSU i am using.  would gpuz tell if the cards complain about voltage issue?

  • colcurvecolcurve Posts: 171

    i wonder for 3 gpu workstation, what setup is recommended, is everyone going for AMD threadrippers? i chose a vintage intel cpu because I thought it would only be the gpus doing the hard work.

  • outrider42outrider42 Posts: 3,679

    The CPU generally does not matter. When you start using multiple GPUs, it can matter a bit more. I stress a bit, it is not much. Puget did some testing on multiple GPUs, using up to 4 Titans in two different machines. One had a i7 and the other had Xeon. This test was several years ago, before Ryzen existed. They did find that at 3 and 4 GPUs the i7 setup was a little slower than the Xeon set with the same GPUs. Again I stress a little. At 3 GPUs it very small, it was only at 4 GPUs that the difference was even noticeable.

    But it is important to note they never mentioned any stuttering in performance like you are experiencing.

    Their conclusion might be off, as it seems like simple core counts may be what sets the CPUs apart. Basically, I think a modern Ryzen like a 12 or 16 core would be perfectly fine for multple GPUs.

    But you have several things to consider here. Like how you have 980tis. These are old now. Your best upgrade would be replacing them both with a single 2080ti or an upcoming Ampere. A 2080ti is much faster than the two 980tis combined, and surely the future 3080 and 3080ti will be even faster still.

    As I showed with my own machine, my CPU is not bottlenecking my twin 1080tis. My CPU is almost as old as your, and I only have 4 cores with no hyper threading at all. You have 6 cores with hyper threading. So if I can run 2 GPUs, then you certainly can, too. And that is also why I think you can run 3 GPUs without much issue from your CPU. There may be a small advantage to running a high core count CPU with 3 or 4 GPUs, but this advantage is small, and like I already said, doesn't explain the stutter.

    Actually, you said that things calm down as the render goes on. I think this may prove it to be a power issue. As your GPUs run, they get hot, and like I said, they will throttle. They throttle to reduce heat, and of course, throttling also reduces power draw...meaning that as the render goes on, they actually draw less power because of this throttling. And since they are drawing less power, the PSU is now able to keep up with the demand better.

    I feel this is the best explanation for your issue. I think 2 GPUs would be best here, for several reasons. The power draw is one, heat is another, and you get slightly diminishing returns. You would have to build an entirely new machine to run 3 GPUs at the same, and for what...the benefits are not that huge with 980tis. Heat becomes a serious issue with 3 and 4 GPUs, and you will have to design for that or you will still lose time due to throttling. That is why I think you would be far better off ditching those 980tis (I know that hurts) and going with a single new card in their place.

    Plus if you go with a 2nd 2080ti, you would have the option of using Nvlink, provided your motherboard supports it. With Nvlink you can combine your VRAM, which would give you almost 22GB of VRAM, plus much faster rendering, plus using less power and having fewer headaches. And you can buy a 2080ti without needing to build a whole new machine, so it should cost less.

    There is another thing. Maxwell is on borrowed time. Kepler is going to be killed off with the next release of Iray. And Maxwell will be next! This is listed in the help log. Go ahead and read your help log from a starting point. There should be a note in their stating that Maxwell is marked for end of support and will be deprecated in a future release. This is from the help log of a Kepler user:

    "compute capability 5.0: deprecated (for technical reasons), iray will retire CUDA compute capability 3.X (Kepler) in the next release. 5.0 (Maxwell) will follow in the next major release."

    Again that comes straight from the help log itself, this is from the Iray development team. You probably have time, since Kepler will get killed first. You probably have at least a year, maybe two. But regardless you should be thinking about this while you decide on what to do. In the wake of RTX, Maxwell is aging rapidly.

  • nonesuch00nonesuch00 Posts: 18,781
    edited July 2020

    You should get a PCIe 4 Motherboard for 3 GPUs.

    Post edited by nonesuch00 on
  • pwiecekpwiecek Posts: 1,598
    joseft said:
    pwiecek said:

    Does your CPU have enough lanes to fully support 3 PCIe x16 slots?

    even if it doesnt, that should not cause random dips in usage. i run 3 mixed GPUs, 40 lane CPU so its running at x16/x16/x8 and dont have this issue

    as long as the pci slot is not dropping way down to something like 1x, the only time the pci speed should affect a render is the pre-render stage when all the scene data is being loaded on to the GPUs

    40 lanes is rather high end

Sign In or Register to comment.