Carrara not using all of my render nodes! Solved!

1567911

Comments

  • JonstarkJonstark Posts: 2,375

    The z800's are pretty sweet too, like Jay said they are a little bit bigger than the z600 and the main advantage is that they have more RAM slots.  Also they have this nifty heat separator plastic part between the 2 cpu heatsinks, so that effectively the hot air blowing out the back of the 1st heatsink is 're-routed' so it doesn't blow into the 2nd cpu heatsink, which is a nice design.

    However, while I've been watching/searching the z800s on eBay, I haven't seen any tremendous deals yet (well, I mean they are pretty great deals, but considering the z600 has just as much horsepower and usually goes for quite a bit less, there hasn't been any z800s that have popped up yet that I thought were 'must have').  This seems to hold true for the z820s too; meaning that they are always noticeably more pricey than the z620s that are listed, so I imagine when the time comes that the next generation becomes more affordable, I'll probably wind up going for the z620 instead of a z820.

  • JonstarkJonstark Posts: 2,375

    You know, I saw a listing yesterday for 15 Z600's, for $1300 plus shipping. Thye were all single CPU configurations, but I was THIS close to doing it. When I got home in the evening, it had sold (it was you by any chance, was it, @Jonstark...?)

    LOL!!! Yeah jonstark! Are you buying up stores of Z600s?

    I think you meant $130, not $1300, lol.  $1300 would be waaay out of my price zone  smiley   I did see a couple of z600s go up in successive days for $125, and I greedily bought the first one, but reported the 2nd one I saw here for anyone else who might be interested, then I realized that even at a steep discount I had really bought something I didn't need, so I swore off buying more, so even though I saw some z600s going up for super low prices I promise it wasn't me that grabbed them up lol.

    I do love it when a company dumps a whole load of the same device type on the market at once, artificially depressing the prices, and btw looks like in the near future some dell t5500 dual xeon machines are going to be available early next week.  I don't have any dell servers yet, so... no, what am I thinking, must resist these impulses!   ...  

  • JonstarkJonstark Posts: 2,375

    Funny you are mentioning the Z800's my boss wants me to bring home 4 Z800/Z820's to work on at home to figure out issues at work as we have a few that they were getting ready to get rid of and I indicated that they would be better the the really old server he was going to give me.

    So with the grid license it it 100 Threads or cores or CPU's? I know the documentation indicates CPU's but back then they only had single cores.

     

    I was running into a 84 render core limit when running Carrara on my main Windows 10 machine, but Daz help suggested running my main Carrara app on one of my servers instead; when I did I wasn't running into the same cap, and seemed to get past that previous cap, though I didn't have enough render cores to test whether I could get all the way up to 100 render cores.  I suspect Carrara network rendering plays nicer with Windows 7 then it does Windows 10, or at least that's my best guess.  The official cap that Grid documentation gives is 100 cores, and that should be for the render nodes, not the counting the main machine.  So if I had enough nodes and used my 24 core z600 as the main machine, I think theoretically I should be able to get 124 cores for network rendering (but I haven't tested this yet).

  • GreymomGreymom Posts: 643
    Jonstark said:
    Greymom said:
    Jonstark said:

     

    Just a thought - if you decide you want to get rid of the X5670's, I will buy them from you for whatever you paid for them.  I have a X8 server blade MB they will work in (part of my ongoing LuxRender project).   I hate getting stuck with parts I later find out I can't use (anyone need a half-dozen socket 771 server boards : )    ?).

     

     

    I appreciate the offer, but I actually have 2 z600's and the 1st z600 was the one that I bought the X5670s for, and they worked fine, so I have that one up and running with 24 cores.  The X5570s that I bought came in today, and later tonight I'm going to try switching them into the 2nd z600 (I don't anticipate any problems, but I'm possibly jinxing myself by stating that ahead of time...)

    Yea!  Glad you got them working.  Good luck and happy rendering!

     

  • GreymomGreymom Posts: 643

    I can see that I should have just waited to buy used full machines rather than used parts to build my own, but I do like to build stuff (I keep telling myself).    But now surplus LGA 2011 V1/V2 blade server boards are showing up on ebay for a tempting prices, and the E5-V2 10-12 physical core chips (not ES) are starting to list also!   Sigh.  Going to build what I got - all the power I will need.

    By the time I get everything built and set up, there will probably be a graphics board that has 5000 GPU cores and 32GB of VRAM for $200   cheeky

  • JonstarkJonstark Posts: 2,375

    Just a quick update, wanted to confirm I put the X5570s into the 2nd z600 and it's up and working great, so that's 16 more cores added to the mini render farm.  smiley

  • I am so happy for you, and taking my own notes, of course.  Will you have enough renders to feed the beast? 

  • DartanbeckDartanbeck Posts: 14,285

    Fantastic!!! Again, thanks for all of this! So awesome!!!

  • JonstarkJonstark Posts: 2,375
    edited November 2016
    diomede said:

    I am so happy for you, and taking my own notes, of course.  Will you have enough renders to feed the beast? 

    Lol, nowhere near, I've been so obsessed with building out my render nodes I still haven't fully moved my main Carrara app to the z600 I intend it to run on yet, and haven't done much rendering at all.  I tell myself I'm preparing (and soon there will be no excuses) as I need to get back to actually rendering again now that I have more than enough horsepower.  I'm like one of those guys who's obsessed with souping up his car but never drives anywhere smiley

     

    Btw, had another thought for outside-the-box thinking on render nodes, and there's a whole 'nother field of potentially excellent render nodes to tap that I hadn't thought to look at, which is single-Xeon workstation, like for example the Lenovo S20, Dell T3500, or HP Z400.  I've been so focused on dual-Xeon systems that the single-Xeon servers completely escaped my attention, but these things are usually much less expensive than a dual-Xeon server yet you can easily upgrade the cpu to be a hex-core, which essentially gives you 12 render cores in one box.  Considering that most of the dual-Xeon systems come with quad cores, which is 16 render cores, and as 12 cores is not that many less than 16, and still more than any i7 desktop which will give only 8, it seems these single-Xeon servers also ought to be considered as an even more inexpensive way to add some render node muscle to the workflow...

    Post edited by Jonstark on
  • can something like this be used with Carrara?

    http://www.pickles.com.au/general/item/-/details/HP-BL460c-G6-Blade-Server/22057737

    thats for Auction in my city

  • DartanbeckDartanbeck Posts: 14,285
    edited November 2016

    I'd bet that it can - beautifully. 24 cores for your native render engine - but being a Blade, you may want to keep it in another room - well insulated against sound.

    I've seen these things used in dust-free air-conditioned rooms all hooked up in racks and they get accessed via router/network/switch... that stuff. But that doesn't mean that it wouldn't work solo.

    But I'd wait for someone more knowledgeable to chime in ;)

    Heh... soon we'll all be Rockin' with Carrara Monster Engines!!! We'll take the world (and it's Power Companies! LOL) by Storm!!!! Ahhhhh!!!! devil

    EDIT: for myself, I'm looking for something a bit more conventional - but I have to admit that if I ran across this and I had the cash for it, it would come home with me!

    Post edited by Dartanbeck on
  • I have no geeky knowledge to figure out how to use such things but I might be able to get that cheap

  • DartanbeckDartanbeck Posts: 14,285

    For example, I wish I had the cash in my hands to just click the Buy Now button for this thing:

    HP Z800 Workstation w/2 x 2.93GHZ HexaCore Xeon, 64GB RAM, 2TB SATA, Win 7 Pro

    or this one:

    HP Z600 Workstation 2x 2.66GHz Six Core X5650 12GB RAM 250GB HDD WIN7 Pro

    here's the specs for the X5650 in this second one

    Pretty sweet!

    ...and for just a bit more than the first one,

    HP Z620 Workstation 2x 2.60GHz 8 Core E5-2670 8GB RAM 500GB HDD

    here's the specs on its E5-2670! Two Eight-Core Hyper-threaders giving a whopping 32 core total render Monster - all by itself!

    Truth be told, any of these fine beasties - including that one that's being auctioned by you, would make me tingle with glee!

    Gotta find some cash!!! :)

  • JonstarkJonstark Posts: 2,375
    ToeJam said:

    can something like this be used with Carrara?

    http://www.pickles.com.au/general/item/-/details/HP-BL460c-G6-Blade-Server/22057737

    thats for Auction in my city

    While there is no doubt that this thing could be run Carrara like a beast, I still wouldn't recommend it, since these blades don't seem to have internal fans and have a strange power connector (it's not as simple as buying a power adapter at your local electonics store); instead they are designed to be slotted into a blade enclosure, which mates up with the blade itself and which provides cooling and power to the blade.  Not to mention there don't appear to be any consumer outputs for the blade, like slotting in an ethernet to connect to your network or a hdmi or video port to connect a monitor to it.  I'm not an expert, but I'm pretty sure you should view this as a component to a larger blade server enclosure and not easily used as its own standalone machine, so unless you happen to have a blade enclosure that matches up to that specific blade type, it probably isn't the best way to go.  I know we have some guys on the forums who actually work with these things professionally, so they can probably answer a lot better than I can (all I did was watch a few youtube vids and pull up a spec sheet for this model to take a look at it); hopefully someone more expert will give a better and more detailed answer than I can.

  • TangoAlphaTangoAlpha Posts: 4,315

    For some reason, Carrara has stopped seeing my render nodes. The nodes are deinitely running, and the machines can see each other on the network, but they don't show up in the render nodes list, and they don't get used in the render. Network log is empty. Any ideas?

  • yes this might be more complicated than its worth

  • JonstarkJonstark Posts: 2,375

    For some reason, Carrara has stopped seeing my render nodes. The nodes are deinitely running, and the machines can see each other on the network, but they don't show up in the render nodes list, and they don't get used in the render. Network log is empty. Any ideas?

    Alarmed to see you encounter this, but so far I haven't run into this.  Not sure what the answer would be, and I'm assuming you already tried shutting Carrara off and turned it back on again.

     

  • DartanbeckDartanbeck Posts: 14,285
    ToeJam said:

    yes this might be more complicated than its worth

    Agreed. I didn't know about the power thing - or I'd have brought that up. Sorry, my friend! :(

    I should have, because it was just mentioned a while back and I read it... but forgot :(

     

    For some reason, Carrara has stopped seeing my render nodes. The nodes are deinitely running, and the machines can see each other on the network, but they don't show up in the render nodes list, and they don't get used in the render. Network log is empty. Any ideas?

    Are you connected wirelessly?

    While jonstark mentions that his works wirelessly, mine doesn't. Won't see the nodes at all untill I plug directly into the router. It's been reccomended by others that it's always suggested to run Nodes via a wired network connection to the router.

  • TangoAlphaTangoAlpha Posts: 4,315

    Yes, I've tried restarting Carrara and restarting both machines - no joy. All wired connections - Wifi is not very reliable around here - too much overcrowding and overlapping signals.

  • GreymomGreymom Posts: 643
    ToeJam said:

    yes this might be more complicated than its worth

    Blade servers can be a real bear to adapt to home use.  I don't recommend it unless you have experience with them.  Particularly in light of the deals that Dartanbeck has posted above!   It cost me more for parts to build the equivalent to the Z620 above then to buy the one listed, even with used CPU and RAM!  Mine do have a lot more ram, and I could put multiple graphics cards in them (like I could afford that) but I am amazed by these deals!  Same with the Z600 and Z800 deals.  I got a Supermicro X16 dual blade server like new (no CPU or ram or disks, but with power supply)  originally $12,000, for $300 shipping included.  With 4 X5660's, the twin blades run LuxRender like a rocket, but if you total up the parts I had to buy and the time I put into it, I just should have bought a couple of those used Z600's.

    One thing I am concerned about with these HP machines is if the cooling they come with is adequete for the near-100% CPU duty cycle that rendering utilizes.   That would be something to watch.

    Happy Rendering!

     

     

  • yes as people at work pointed out, I could possibly load Windows to the harddrive but I would not be able to see it boot up without a monitor and if no way to connect one.

  • DartanbeckDartanbeck Posts: 14,285
    Greymom said:
    ToeJam said:

    yes this might be more complicated than its worth

     

    One thing I am concerned about with these HP machines is if the cooling they come with is adequete for the near-100% CPU duty cycle that rendering utilizes.   That would be something to watch.

    That's one thing that I am really pleased with on my current machine (built from scratch)

    When I was researching my AMD eight-core, deciding on which after-market cooler to get, I've seen many people commenting on how great the stock cooler was - and AMD warranties cpus which use the stock cooler - so I used it.

    Amazed! I mean... I beat this thing. I render and render, and then I render some more.

    I do animations. So I design several scenes with several different camera angles to shoot, and run them through the Batch Queue - let 'er snap!

    That wonderful Zambezi (older version - no longer available) holds wide open very well without issue.

    ==========================================================

    I have a feeling that they do quite well.

    I recall animation render folks talking very highly of these machines when they came out. But initial reports can often be a lot micer than reviews after a good, healthy dose of wide-open abuse! Those poor, tiny chips! LOL

  • GreymomGreymom Posts: 643

    Note on HP 460c G6 blade server:

    Hmmm.   I was curious, since I have never seen a blade board without a basic video output for testing/troubleshooting (although they are obviouly not used when operating in server arrays).   The HP specs indicate that there is an integrated ATI RN-50 video chip with 64 MB vram.   This may not be for all configurations, but it is listed as part of the basic specs. https://www.hpe.com/h20195/v2/GetPDF.aspx/c04282677.pdf.  But there is no obvious video output port.   And where are the USB ports it is supposed to have?

    Ah! You need a special breakout connector to get the video output and the USB!  http://content.etilize.com/User-Manual/1015365293.pdf see page 11.    Looks like the connector costs about $20.

    So, this could be useable.

     

     

  • DartanbeckDartanbeck Posts: 14,285
    ToeJam said:

    yes as people at work pointed out, I could possibly load Windows to the harddrive but I would not be able to see it boot up without a monitor and if no way to connect one.

    Wow. And back when I was looking into building mine, I was imagining blades as the same thing as a PC, but built more compact. It makes a lot of sense that they're not!

    The big reason why I finally and forever turned away from the idea of running a compact design is that 'Lots of internal Room' is the best thing for a cpu (and gpu, RAM, SSD, HDD, etc) that are to be run on full cycles for extended periods of time.

    Once I get my HP Z-series, I'll report back - 'cause I'll be getting it as hot as it goes almost immediately after purchase. It's not my fault - entirely. Addiction is addiction... and I'm addicted! I'm just grateful to be addicted to this as opposed to some of the other things folks get hooked on these days!

  • well I have 16 hours still, I might still do a cheeky $10 bid at 9am, worse case senario its scrap metal.

  • GreymomGreymom Posts: 643

    I will be very interested to hear about everyone's experiences with the z-series.   I lucked onto a used HP compact desktop with a I7-2600K cpu for less than the typical cost of the used CPU.   Cooling is inadequete for high-duty-cycle.  No room for anything added but liquid cooling (mounted outside the case, and costing more than I paid for the unit).  So, I will probably swap the chip for a spare I5, put the I7 in a bigger spare MB and case, and someone can use the HP for less intensive stuff.

    Also interested in your AMD 8-core build .   I built one with a Vishera CPU (not the 220 watt!).   With standard cooler would not run flat out without overheating.  Replaced with a bigger one.   Still have problems sometimes.  About to add one of the tall air-cooled radiators with the heat pipes.

    Addiction to technology is much better than addiction to most other things!   My sons are instructed to intervene if I try to buy any more hardware until I finish building and setting up what I have.

    "But the X10 LGA 2011 blade boards are really cheap!"  "Dad!  Close the EBAY window and back away from the computer!"  Sigh.

  • GreymomGreymom Posts: 643
    ToeJam said:

    well I have 16 hours still, I might still do a cheeky $10 bid at 9am, worse case senario its scrap metal.

    Looks like even Amazon has the connector:

    https://www.amazon.com/HP-Bladesystem-c-Class-409496001-409496-001/dp/B00BL2OWLO

    Ebay has them too, for less.

    http://www.ebay.com/itm/NEW-HP-409496-001-Local-I-O-Diagnostic-Cable-SUV-Bladesystem-2xUSB-VGA-NR-/381414317167?hash=item58ce109c6f:g:QTkAAOSw9r1WAwO4

     

  • Greymom said:
    ToeJam said:

    well I have 16 hours still, I might still do a cheeky $10 bid at 9am, worse case senario its scrap metal.

    Looks like even Amazon has the connector:

    https://www.amazon.com/HP-Bladesystem-c-Class-409496001-409496-001/dp/B00BL2OWLO

    Ebay has them too, for less.

    http://www.ebay.com/itm/NEW-HP-409496-001-Local-I-O-Diagnostic-Cable-SUV-Bladesystem-2xUSB-VGA-NR-/381414317167?hash=item58ce109c6f:g:QTkAAOSw9r1WAwO4

     

    oooh coolness,

    I will see how I go at this Auction first but thats good to know, I even have a USB floppy disc drive believe it or not if I have to go that route and there is always the Linux mint bootup flashdrive I made for my old craptop, I will try to crack this baby if I get her, one way or another or find someone who can.

  • DartanbeckDartanbeck Posts: 14,285
    Greymom said:

    I will be very interested to hear about everyone's experiences with the z-series.   I lucked onto a used HP compact desktop with a I7-2600K cpu for less than the typical cost of the used CPU.   Cooling is inadequete for high-duty-cycle.  No room for anything added but liquid cooling (mounted outside the case, and costing more than I paid for the unit).  So, I will probably swap the chip for a spare I5, put the I7 in a bigger spare MB and case, and someone can use the HP for less intensive stuff.

    Also interested in your AMD 8-core build .   I built one with a Vishera CPU (not the 220 watt!).   With standard cooler would not run flat out without overheating.  Replaced with a bigger one.   Still have problems sometimes.  About to add one of the tall air-cooled radiators with the heat pipes.

    Addiction to technology is much better than addiction to most other things!   My sons are instructed to intervene if I try to buy any more hardware until I finish building and setting up what I have.

    "But the X10 LGA 2011 blade boards are really cheap!"  "Dad!  Close the EBAY window and back away from the computer!"  Sigh.

    Right. I can totally understand how easily one could get addicted to technology - especially the passion for trying to build better, faster computers... it's just darned Fun!

    However, it's not the technology that I'm addicted to - it's Carrara Pro

    Optimizing Content 

    I can't help but try to make really nice shaders. It started with making simple ones that look great in renders, which to me was reducing the highlight channel to zero. Then I decided that doing that is absurd, but it does make for a quick fix to things that otherwise don't look right. So I started digging in the textures folders that come with products - sure enough, most of them DO come with a Spec map of some sort - which goes into the Highlight channel to control what is shiny, and by how much. Then I can mix that with something or adjust its brightness to alter its behavior.

    I also like to tweak on the mesh a bit - sometimes. Especially when it comes to clothing or even the characters themselves. Hair too.

    Then I get them their lighting needs and save them to my browser.

    Optimizing Scenes

    I like to take my environmental content - buildings or forests or swamps, etc., and create nice, full 360 degree scenes with them - optimizing them as above, and optimizing their lighting to make their own specific features look right, while also making sure that the lighting I've attached to me Actors and such work nicely in conjunction with them - while having natural looking environmental lighting to cast shadows from these pre-lit actors and other elements onto these environments.

    It all took a while to come up with a system that works right for me. I want my renders done quickly (shooting for around one minute per frame to render), while still looking decent in the ending animation. So... yeah... it took a lot of testing.

    Then it comes time to actually combine elements I've saved to the browser into a single photo shoot for a film. So I start with the scene I need, bring in the actors for the main characters and optimize their individual lighting rigs to match shadows correcly - takes a few minutes - tops. Then I bring in their adversaries and do the same. Any other elements come in next, and then I save with everything contained and begin animating everything. I like to have that main scene saved - so I can always grab a fresh start if I really botch things up - plus it makes for a perfect start for my next scene using the same area and actors, etc.,

    I also add several camera angles - some with the camera in motion, others are stationary. So when I load them into the Batch Queue, I'll be "Adding" the one file several times - as many times as I have filming cameras assigned - give their renders each their own file name with my own nomenclature I've developed to keep the whole process simple in the editing room.

    It all works very slick - and I have to say that I'm entirley addicted to this whole process. It's one of my favorite things to do - though my wife is my all time favorite thing to do. But this is a close... okay... nothing can ever come close to hanging out with Rosie... but I'm still addicted to this whole process... I am! :) But I'm addicted to my Rosie even more heart

  • GreymomGreymom Posts: 643
    ToeJam said:
    Greymom said:
    ToeJam said:

     

    oooh coolness,

    I will see how I go at this Auction first but thats good to know, I even have a USB floppy disc drive believe it or not if I have to go that route and there is always the Linux mint bootup flashdrive I made for my old craptop, I will try to crack this baby if I get her, one way or another or find someone who can.

    While I have no experience with this particular server, they are usually pretty similar for a given support chip set.   I will be glad to offer advice.   You may need to change the heat sinks and/or add cooling fans, depending on what it looks like inside.  I had to add three fans each to my X8 blades (one each for the CPU's and one for the 5500 support chip), but fans are cheap.  You may need to upgrade the BIOS to the most recent, but most server farms tend to keep up this.   If you want to install windows, you will need Win 7 Pro 64-bit, or equivalent with Win 10.   Upgrading from Win 7 to "free" Win 10 was a challenge on my X8 and X9 boards due to the drivers for the on-board graphics, but there was a simple work-around.  Have not tried to install Win 10 directly on a server board yet.

    I wish you luck on your bidding!

     

Sign In or Register to comment.