Daz 3D is part of 
Connect
DAZ Productions, Inc.
224 S 200 W, Suite #250
Salt Lake City, UT 84101
Licensing Agreement | Terms of Service | Privacy Policy | EULA
© 2024 Daz Productions Inc. All Rights Reserved.Daz 3D is part of
Connect
DAZ Productions, Inc.
224 S 200 W, Suite #250
Salt Lake City, UT 84101
Licensing Agreement | Terms of Service | Privacy Policy | EULA
© 2024 Daz Productions Inc. All Rights Reserved.
Comments
Thanks GLE.
I'm curious, when you say "The card driving the display uses about 1,2 GB more RAM than the other, so I'm planning to add a 1070 or 1080 to gain 33% performance and the correct memory headroom. "
I thought people were saying that Windows grabs a lot of memory overhead from BOTH GPU's in a 2-GPU system. Are you saying it only takes it from one?
I'm real confused on that whole issue. It seems like the only time a big memory overhead is taken is when Studio is set to render Iray with my GTX-1070 (not the CPU's), and after I start up Studio. I originally thought it was independent of the application, and solely a Windows thing. Do you have any insight into this ?
Thanks.
Another quick update on this thread - AMD Threadrippers are out, and for ultimate performance with 4 GPU's that would be the way to go with the x399 boards. 60 or 64 PCI lanes, with 4 cards you'll get x16 x8 x16 x8.
...lestee....
First step: Win the Megabucks Lotto.
Next: Follow instructions in this thread.
Nah, I don't think so. At least for the majority (vast majority?) of Studio users. We both know that the one thing you never hear in these discussions is the REAL cost vs. benefit. It's always assumed that more is better. For many of us techies its more about cool hardware than it is an objective evaluation of what real benefits you'll get in your particular situation, and whether those benefits are worth the cost. I even posted a GIF earlier in this thread showing a single GTX 1070 can give iray viewport performance that's not much different than the OP's monster machine with a bunch of GPU's. And I recently upgraded to an 8 core, 16 thread CPU, but how often do I really use it? Almost never.
So don't waste your money on lottery tickets, cuz you won't win. Maybe if you were running an animation business and time was money, then maybe you can make a case for all of this power.
But I'm with you....putting water inside a metal box with electronics is just asking for trouble.
...I'd still need to win a Megabucks (or get some kind of windfall) just to build any type of new system. Crikey getting the money up for just a memory and OS upgrade to my current system (without a high VRAM GPU) is a "big ticket" expense as I am on a fixed income with rent alone taking over half what I get each month (and I'm in "low income" housing).
Rendering in the super slow lane is becoming more and more discouraging..
Meanwhile I did the same thing and I've been using it almost nonstop since.
I'm curious, what apps you use that take advantage of all the cores?
The only one I've been running heavily is Terragen.
It's because you ask something to be universally quanitifed that isn't universally quantifiable.
Individuals value things differently. For some, a single card for Iray use in Daz Studio is good enough. For others, it isn't. The former don't place much value on a monster rig, as they don't feel the need. For the latter that either feel their computer is impacting their productivity (UGH, this computer is SLOW) or it is impacting their productivity (ugh, my computer can't keep up and my render queue is growing) they value a monster rig more.
...exactly.
When I have to wait for countless hours to upwards of a day for a render process to complete, which makes my system completely useless until it is done. Or, when even test renders take up a major amount of my production time, I realise need a better solution than I currently have. I am not an animator so I don't need my own "personal Multi GPU VCA" that will crush times down to a under a minute, but would like to achieve better performance than I am currently experiencing as like I mentioned, it is becoming somewhat discouraging. As I also work in Carrara, GPU rendering is moot in that respect so I have to weigh the cost out as with that programme the more CPU threads and memory you have the better. So to get the best performance I can out of both programmes would mean a fairly expensive system that has both a high VRAM GPU and as many CPU cores along with as much memory as I can get..
....or making a compromise that hamstrings one of them.
I have one Titan X Maxwell (Ebay around $500 us dollars) and 3X 1070's all new Newegg . My specs are in my signature. I liked the Alienware area 51 r2 case so I bought one on Ebay and installed my own X79 motherboard in it. The case was brand new I pealed plastic off surfaces and there was no scratches. I did have to take off the cool color changing side panels that worked due to cooling issure with Iray but the system works! I could run 4 video cards but I've found that my Titan X with a 1070 is good enough for my stills. I do animation work but not in Iray or Daz Studio. I run a Xeon E5 8 core processor at about 3.2ghz. I started out with a Intel i7 3930k 6core but when the server market upgraded so did I, got a really good deal on a matching pair of 8 core Xeon E5's but did not use them on a dual board because for my use a single 8 core Xeon processor meets all my need. So I built a second system with the extra Xeon and it only has 32gig of ram and 2x 1070's SLI for VR and gaming. From time to time I use the other box to net render Carrara with Grid but most of the time 16 treads of my main system is enough. Most of my home built systems comes from Ebay. I don't need the latest and the greatest technology but I like to get the most bang for my buck that will not bankrupt me.
To the best of my knowledge the memory overhead by Windows is on the display card. Has been for years, but Windows 10 (I don't have it) uses more than say Windows 7. But since I use a separate 780 ti for my rendering, I usually don't have any problems unless the scene is too big of course.
If all goes well, all the remainder of parts will arrive today. I did order a separate card for driving the display. I have a spreadsheet of older scenes, I'll see if there is a difference.
The problem is two-fold. Due to "enhancements" in Windows 10 there needs to be a video buffer for every port on every card; without the buffer, if you hot-plug a display into an unused port windows 10 will crash. The second part of this is that the old buffers (8.1 and previous) were basic vga 1024 X 768 - but now they're designed to support 4K displays at full resolution and color depth because some (many?) 4K displays can't do old VGA format. So if the card supports 5 connections you get 5 buffers and there is no current way of fixing this.
Exactly. And that's why when someone says "oh, you need this monster rig cuz it's awesome and it's a monster and it's better and faster", I ask those people to step back and understand that individuals value things differently. And not everyone needs a monster rig.
As I've said before, the response should be something along the lines of "it costs this much (including all these other factors, and the benefits are these". So people can look at their own situation and see if those costs for those benefits make sense. But instead it's usually a bunch of techie stuff that's irrelevant to many or most users.
I don't ask for "universally quantifiable", I just ask for a simple cost/benefit discussion.
Some day, Scott, I'll finally get you talking in terms of cost and benefit...some day...
I think you were looking for more specific benchmarks, which don't exist.
Like I'm building the Threadripper system (UPS guy, where are you?). I figured I could go down two paths:
1. Threadripper system with the two Titan X Pascal cards (already owned). Third card to run the display. Benefit here is all the cards for rendering are running at x16, so renders start rendering the fastest.
2. Get a 4x SLI board with my Intel CPU, or possibly a new CPU & motherboard. Add 2 more Titan X Pascal cards. Benefit here is that I have more rendering power (faster over the long haul.) Drawback will be that it will take longer for renders to start, due to the limited PCI lanes.
Benchmarks for these sort of setups using Iray in Studio? They don't exist.
Nobody is forcing you to build a quad GPU rig. If you don't wanna, don't.
ETA: Cost/benefit? It's expensive.
But if your render times are killing you, it's worth it.
Has anyone tested if there is any real performance hit, for rendering, if using 4x PCIe speed on a card?
I know some bios settings will let you select the speed. Most use auto-select.
I realize that using 4x on a single card will potentially impact live renders, but on the second, third, fourth... etc...
The whole 8x minimum, is ONLY for SLI. Which I have no intentions of using.
I know, for a fact, that cards can run on 1x PCIe lanes, because I ran 8x Radeon 7970's and others have done the same with nVidia's too. (For bitcoin/alt-coin mining) However, there, we were not trying to force-feed 12GB of graphics into the cards. I realize it may slow-down loading of a scene, but the output of the processed image data is barely a few MB of data.
If this can be done on only 4x, then any 16-lane CPU will work, as long as there are 4 PCIe slots available. (Using extender cables to space them apart. Since standard ATX boards can't fit 4 double-wide cards. (You normally have to get an E-ATX, Extended-ATX board and case.)
I have no problem building my rig in a milk-crate again. They hold 8x double-wide GPU's without much effort. However, I am only shooting for 4 Titan-X's, not 8. I'd have to wire a plug for 220v and get a 3200-watt PSU, or two 1600-watt PSU's.
I'd rather buy a $40 system-pull CPU, with a low TDP and a cheap $40 motherboard and slower/cheaper RAM, if it still rendered nearly as fast. (Minus the time to load the cards with the images.)
I don't have any links handy, but from what I can remember, I don't think you'll see anything noticable..no. Certainly not compared to the speed increase additional GPU's will give you.
I figure, it would take, respectably, 2x longer to fill the cards with the project. (Using 4x vs 8x. I know 16x doesn't matter, spare for a few seconds.)
Then again, most actually "share lanes", they don't always sustain a constant 16x connection, That is usually a physical limitation of the number of simuletanious lanes available, which SLI demands for loading. I am not sure how daz loads the cards. But, being non-SLI, implies each is loaded in turn, or with "free shared lanes".
Maybe I can be the experiment. I can just buy my cards first and try to cram them in this board, seeing what I get for results. (I am going to buy a super-board later, but this would make a nice remote-render-farm sister-computer. I only need one super-fast computer, that is power-hungry and needs a silver-spoon and bottle.)
Smething like this, possibly for a future upgrade. A 13-GPU motherboard!
http://cryptomining-blog.com/8860-we-got-our-hands-on-an-asrock-h110-pro-btc-13x-gpu-mining-moitherboard/
NOTE: Windows doesn't like more than 8 of the same card manufacture in the same computer... (Dreams crushed!)
Hello,
I read this interesting topic, I'm only not sure if it is still active due last entry from Nov 2017. But I will try to write.
Currently I'm working on Dell T7810 workstation with 1x E5-2630 v3 (8 cores/16 threads), 16GB RAM and Quadro M2000. OS Win 10 Pro and SW (including DAZ 3D) installed on NVMe SSD, DAZ 3D content is on normal HDD. I have intent to upgrade to Quadro M6000 ideally 24GB version. I just done first step by upgrading PSU from 685W to 825W that is necessary for so power GPU. Firstly I was been thinking about K6000. But now when prices decreased I think that M6000 is better choice due 60% outperforming of K6000 and existence of 24GB version (K6000 has only 12GB). K6000 can be bought for about 500USD, M6000 12GB 700USD and M6000 24GB for about 1000USD. But with some luck it is possible also M6000 24GB for 800USD.
I'm using this workstation not only for rendering but also for scientific computations.
Into near future I'm also thinking about some computer that is able to use multiple GPUs. T7810 has limitation only 300W for GPU. So I can use only one GPU with power requirement of M6000. Maximally K6000 in combination with current M2000. K6000 requires 225W, M2000 75W.
But I have question if in current days is still optimal way to build own system? This means to purchase chassis, to purchase motherboard, PSU, etc. For example servers as Dell T630 or HPE ML350 Gen 9 can be purchased in used condition from eBay for about 1000USD or less. If you have luck you can have for same price newer T640 or ML350 Gen 10. These servers can operate up to 4 300W GPUs including passive cooled Tesla cards. For example very interest me Tesla K80 - you can purchase it in used condition from eBay for about 300 or 400 USD and it is power GPU with 24GB of memory. Advantage is that all these servers are designed to perfect performance - you don't need to worry about HW compatibility, cooling adequacy, etc. Of course there can be problem that all these servers use only air cooling, so water cooling will be necessary to implement by self. What do you think – still build own system or purchase some completed?
I have also one additional minor question. Is it possible to operate PCIe x16 GPU in x16 slot wired as x8? My current situation regarding cards in T7810 is that I have only 2 x16 slots that are wired really as x16. In first I have M2000 GPU and in second NVMe SSD host card by Dell. This host card can hold up to 4 1TB SSD and requires x16 slot. I'm not sure if it will work in x16 wired as x8 in case of presence of maximally only 2 SSD. Maybe yes, because this is dividing x16 into 4x x4 - each x4 for one SSD. I'm asking for this for situation that I will purchase K6000 that I'm able to operate together with M2000, but only problem is that I don't have free any x16 slot. So I can only release one by moving some card to x16 wired as x8.
Thanks
It looks that the topic is definitely dead. What a pity.
In meantime I learned that I'm able to operate PCIe Gen3 x16 nVidia Quadro M2000 in PCIe Gen3 x16 slot wired as x8. But there is another problem - this desired slot is only 25W and the GPU requires 75W. Of course by nVidia SMI I able to limit power to by the slot provided 25W. In matter I have intent to limit M2000 power to meet 300W total limit of Dell T7810 (this means for example for 250W GPU to limit M2000 to 50W). But only 25W looks from my point too low. M2000 power limit can be set from 25W to 75W. By measuring I also learned that M2000 never go significantly below 25W and during rendering it takes about 50W. I expected that I will set limit to 50W (I don’t seen any Quadro or GeForce card taking more than 250W and also Tesla Kepler cards with active cooling are all under 250W) that will be fine for many operations including rendering together with another GPU but now I know that I have accessible only 25W.
But other had it will be better to operate M2000 on 25W only for standard displaying tasks rather then left it lay in table socket. I learned that during rendering standard display operations is limited by heavy load of the GPU and in opposite display operations also limiting rendering or general computing.
Uhm... just one thing. If you have just 16GB of RAM, getting a 24GB card is a moot point, because you can only create scenes up to 14, maybe 15 GB RAM.
I would also say that benefits vary; you can site them as a feature, and something that was a benefit to you, but others (as you say) value things differently.
Take Amazon, they keep telling me I'm missing out on a Prime benefit, namely: music. I don't listen to music that often, so to me, it's an unused feature, not and never will be a benefit.
First VRAM is not the be all and be all of CUDA rendering. Actual CUDA count matters as well and these Maxwell Quadro cards have absolutely terrible CUDA counts. The M6000 has 3072 CUDA which, due to generational improvements, puts it on par with, if not worse than, RTX 2080 or RTX 2070 super. Both of which can be gotten for $500 or $550 new. As to the M2000? That compares unfavorable to the GTX 1650 which is the worst new Nvidia card on the market. I get that these used workstations are on the market for next to nothing but that doesn't mean they should be viewed as any more than parts, or at most a barebones for building a better rig. Buying a m.2 add on card and multiple 1Tb SSD's for this thing? That's a lot of money that could be better used elsewhere.
If you're really blowing past 8 or 11Gb on a regular basis I strongly recommend getting Scene Optimizer, which will cost far less than trying to make these big scenes work.
I know. RAM upgrade is also planned. I have intent to purchase 2nd CPU and start to fill empty RAM slots. In matter I have intent to gradually equip T7810 to full 256GB RAM. This is not of course for rendering but for other tasks (FEA etc.) that typically requires tens GB of RAM.
As I seen benchmark RTX 2080 overperforms M6000 for about 60 – 70 %. But as I wrote I use T7810 not only for rendering. In matter of fact use for rendering is minor. Mayor use is for scientific computing, engineering work, etc. Problem is that many SWs supports only Quadro or Tesla cards. For rendering can be adequate up to 12GB, but for many of my task can be 24GB very good, maybe necessary. Other question is about durability. GeForce cards are not intended to use in situation that it will runs on very high power tens of hours.
In matter of fact in meantime from my 24 August message I started to hesitate if to purchase Quadro (due cost reason this reduces to M6000, because newer P6000 is now too expensive) or some GeForce that can overperform M6000 for near to 100%. In this decision also plays part my intent to build GPU server with 4 probably Tesla K80 that are each one more powerfull than M6000 and its price is about 30 - 40 % of M6000 24GB. But this intent may not be filled for various reasons of course – this is only potential future. Other part in this decision is also that I prefer to maximally utilize currently existing workstation, so T7810. If I will be using T7810 only for rendering it will be clear - I will purchase GeForce. But as I wrote above - I'm not using it only for rendering.
Of course for rendering you not need multiple 1TB SSD. Main reason why I purchase this holder card is that this is product of Dell and is directly supported by T7810. Also with compare to other M.2 holder cards this is very well done. This is equiped by cover with fan similar like are on GPUs that keeps SSD cold. Temperature of my SSD holds about 40C no matter on workload or environment temperature. For example in my notebook is not problem to go up to 55C. So fact that it can hold up to 4 1TB SSDs is for me rather bonus then necessity. In matter I will be happier for version that can hold only 2 SSDs. This holder will be shorter and for PCIe x8. This version also exist, but I not seen it on eBay.
OK, back to my above idea. Is still better to build custom GPS server or to use some eBay bargain and purchase Dell T640 or HPE ML350 Gen10, or maybe older T630 or ML350 Gen9 and equip it by Tesla K80 cards (or any other passively cooled Tesla cards)? Does anybody ever used Tesla K80 for rendering?
Unfortunately I'm not able to find some benchmark of K80 with some GeForce or Quadro card. But it has 4 992 CUDA cores, but clock is only 562MHz with 824MHz boost. Regarding cooling - there is question if water cooling will be necessary in case of passive cooled Tesla card. I never seen water cooling on any GPU server equiped by Tesla cards.
For simulations and the like the E5-2630 is horribly obsolete. You'd be far better off, for roughly the same cost, getting something like a R7 2700 with the same core count but far higher clocks. If you're on a super tight budget then sure get one of the ancient quadros for SW that requires it, and deal with simulations that run for weeks. But if this is for a job get them to pony up for a Pascal rather than the Maxwell dinosaurs. The P2000 and P4000 are both available on the used market for very reasonable prices.
Beware of buying K80's. They are not meant for deployment in the quantity you want in a workstation. You should check with nvidia how many Tesla's your motherboard will support and which ones. Someone bought 2 K80's and could not get them working in the workstation he was using.
M.2's are one of the hardest things to manage heat wise. The actual NAND chips run better hot but the controller chip don't but also don't need to be super cool. By and large the PCIE gen 3 ones are not considered to need any sort of active cooling.
Regarding R7 2700 - as I know Dell and HPE workstations and servers uses Intel Xeon processors, so AMD is impossible. Also I'm not sure, but R7 2700 is not server processor. For simulations when your CPU runs on 100% tens of hours you need server processor like Xeon. Normal processor it will probably kill, their are not intended for so heavy load.
P2000 or P4000 are not solution due low VRAM. This is only 5 and 8 GB. Of course for some task it can be adequate. But it is same as rendering - if task will be bigger than VRAM computing will fall back to CPU and it will be much slower then with old Maxwell card. Unfortunately to use Maxwell card is tax for limited budget.
I don’t want to operate K80 in workstation. T630, T640, ML350 are servers that all supports Tesla cards with passive cooling (you need additional fans kit). For example T640 supports up to 4 300W double wide GPUs and directly is supported K80M (now I don't know difference between K80 and K80M). Also some Supermicro servers supports K80.
Regarding SSD cooling. Yes, what you are writing can work in computer in normal operation. But what in situation of heavy load for long time? I expect that fan speed is controlled and hold SSD temperature on adequate level - not too cold, not too hot.
No. Running a CPU at 100% 24/7 will not kill a CPU. It won't even shorten the lifespan. MTBF is MTBF. As to not getting the R7 because you have a crappy Xeon board, A decent B450 is $100. An R7 2700 is $180. If you paid less than $300 for the refurb workstation good for you but you got what you paid for.
What simulation SW are you using that falls back to CPU if the whole sim won't fit in VRAM and are they only CUDA? If not get off Nvidia and get a FirePro or the like which are far cheaper per Gb of VRAM.
There is no such thing as a K80m. Nvidia appends m to graphic cards to say its a mobile card. Passive cooling in a server rack? Not possible. You hopefully mean the Tesla doesn't need a fan, but it doesn't have one. The server will definitely need fans and server cases are built for a lot of fans pushing air through the obstructed insides. T640's generally, all the ones I've seen but who knows there could be a model my company never used, have/need 8 fans. If you're buying refurb make sure it comes with the fans or be prepared to buy them seperately. T630 is nearly identical so I assume it uses 8 as well. We've never vbought from HP bought I'd be stunned if they weren't built for 8 fans. That's pretty standard for racks.
No, what I told you is how m.2's work. We don't have a ton of them yet but the ones we have have not failed unusually frequently.