PCIe Lanes? (6700K + 2x 980 ti)
colekush
Posts: 11
Hi there, I was wondering if someone could better explain PCIe lanes to me, as I am starting to confuse myself.
I just purchased parts for a new full-time Iray animation work rig (specs below), and I left enough psu wattage for my plan to eventually add a second 980 ti, when I get the extra cash, as I don't think any of my scenes will need more than 6GB vram for the work I do. Though now doing some more research I am fearing that the 6700K might not be optimal for 2x 980 ti because of the limited 16 PCIe lanes? will this impact full functionality? I clearly am not fully understanding PCIe lanes, and would appreciate any clarification here.
Also I understand that when I have 2x 980 ti that Iray will only use the 6gb vram from one card which is fine, but it will use the full additional 2816 CUDA cores correct? for a total of 5632 cores, which theoretically should be double the render speed?
And if I do end up eventually needing more vram, would it be possible to add a titan X to use as the primary for the 12gb vram, and then have the additional 2816 cores from the 980 ti on top of that for a total of 2816 + 3072 = 5888 CUDA cores?
Here are my specs:
- Intel Core i7-6700K 4.0GHz Quad-Core Processor
- Cooler Master Hyper 212 EVO 82.9 CFM Sleeve Bearing CPU Cooler
- Gigabyte GA-Z170X-Gaming 7 ATX LGA1151 Motherboard
- Crucial 64GB (4 x 16GB) DDR4-2133 Memory
- EVGA GeForce GTX 980 Ti 6GB Superclocked+ ACX 2.0+ Video Card
- Corsair Carbide Quiet 600Q ATX Mid Tower Case
- Corsair RM 1000W 80+ Gold Certified Fully-Modular ATX Power Supply
- Samsung 850 EVO-Series 1TB 2.5" Solid State Drive
Thanks for the help here

Comments
Iray treats each card's memory individually...so if you have 2 cards you won't have 12 GB of RAM, but the 6 GB on each card. If you have a 4 GB card and a 6 GB card, as long as the scene fits on both cards (4 GB) both cards will be used. If it is over 4 GB then the smaller card will be dropped.
Now, with that limitation in mind, all the CUDA cores of USABLE cards will be used to render a scene. So, in the case you've outlined above...2 6 GB 980s rendering a scene that's smaller than 6 GB in size will use the full 5600+ cores. But, it won't quite be 2x as fast...close, though.
Thanks for the response.
Do you know why it wouldn't be quite 2x as fast? Is it related to the fact that the second card's vram is not being used?
close...meaning like 90% as fast as 2x speed?
No, the vram on both cards gets used. They are completely independent. And you won't quite get 2X speed because they are independent. Iray copies everything to card 1 and starts it; then it copies everything to card 2 and starts it. And then it monitors the cards and writes the composite output to the viewport. So there's a slight bit of interference between the cards, especially at startup. Instead of 2X I'd imagine the net to be more like 1.95X. I don't have a second 980 ti yet. so I'm just guessing here :-) - but I do have years of experience in high-end multiprocessing/multitasking systems.
It's something in the range of 1.90 and up...depending other factors, but it's generally close.
Ahh ok that makes sense.
So does anyone know if an i7 6700k would be appropriate to get that 1.90 - 1.95X dual 980 ti speed?
or do i need more PCIe lanes?
I have the Intel i7 6700K and two Zotac 980Ti AMP Extreme video cards on a ASUS Z170 Deluxe. I dont appear to have issues with it. I will say that running two cards is much faster then one in Iray. I have not benchmarked it for exact values
your original question was more about your skylake lga1151 cpu and its 16 pci lanes - none of the previous responses have really answered that so i will attempt to do so
the PCI lanes are basically the amount of bandwidth that your cpu and mobo will allow for installed PCIe devices (video cards in your case)
When looking into the specs of your motherboard, you should see somewhere under its PCI slot features, what speeds it supports in different configurations, but even with those specs you need to keep in mind the CPUs limitations. So your CPU has 16 PCI lanes, which means you can run 1 video card and it will use all 16 PCI lanes (assuming your motherboard also supports this - which it does). If you look on the gigabyte website at the specs for your motherboard, it says this under expansion slots: "1 x PCI Express x16 slot, running at x16 (PCIEX16)".
If you were to add the second video card, since you have a total of 16 PCI lanes, the two video cards have to share those 16 lanes - so they will then drop to x8 mode each. If you then added a third video card, that same concept applies - you would have one card running in x8 mode, and the other two in x4 mode.
Just to be clear, if you have two identical cards, but one running in x16 mode and one running in x8 mode, this does not mean that one is twice as fast as the other. Think of it more like this: if those PCI lanes are the highway in which the data has to travel - x16 mode is a wider road than x8 mode, so traffic flows more and does not suffer from congestion. In order for that difference between x16 mode and x8 mode to really be noticed, you would need a huge amount of 'traffic' to be flowing through. Most people agree that x8 mode is plenty for 98% of normal mainstream requirements, its only when you are using extremely high end video cards (like the big Quadro cards) under some extreme loads when x16 mode becomes necessary.
That ended up a little more long winded than i intended, so to summarise that a bit - running two cards with your setup should be perfectly fine. Adding a third card will mean two of them will drop into x4 mode, and if one of those is a Titan, you might start bottlenecking a little. That is where the lga2011-3 setups are ideal, because they support 40 PCI lanes.
A note on the PCI lanes. What is more important in this configuration when it comes to PCI lanes is the mainboard. The Gigabyte Gaming 7 is nice for single graphics (I have that one myself), but not best choice you can get for dual configuration, as the second PCI X16 slot shares bandwidth with the first one, means when you install a second card both will run at X8 , while a single card in slot one runs at X16 (the third slot is only X4, sharing bandwidth with the M.2 connector). So you can surely run two cards, but both will not run at full speed, though that might only affect data transfers from/to memory, not the GPU speed. There are boards that have two full PCI X16 slots available too (like the G1, but that's E-ATX form factor). Just sayin'..
Usually I don't recommend waiting for Pascal GPU cards, but if you really think about Titan or second 980TI, it might be better to stay with what you have now for half a year and then get the new Pascal card with more memory (8GB or higher).
Joseft - thank you very much for the detailed response and clearing this up - makes sense to me and happy to hear that.
bad4u - thanks for the response. Do you mean wait for the Pascal as a replacement for my current 980 ti? Or as a second card? if the latter, I am unclear why the Pascal is more suitable as a second card for my setup as opposed to another 980 ti / titan... aside from having more CUDA cores - would it utilize the 16 lanes in a different way?
Thanks