Digital Art Zone

 
   
1 of 3
1
Rendering with Multiple Cores
Posted: 01 February 2013 02:22 AM   [ Ignore ]
Addict
Avatar
RankRankRankRank
Total Posts:  6135
Joined  2007-12-04

Rendering complex scenes with eight cores, I couldn’t help but to notice that in an intense area, a single core could get bogged down - working all by itself on that vast portion. I figured that, if all of my cores could help with that spot, the whole process should work faster. So I cranked the rendering tile size down all the way - which is 16. This setting is at the very bottom of the rendering tab in the render room, under the network settings.

I haven’t actually timed which is faster, but it only seems to make sense to me.

 Signature 

Dartanbeck @ Daz3d          Check out the Carrara Cafe          ►►►  Carrara Information Manual   ◄◄◄

Profile
 
 
Posted: 01 February 2013 05:35 AM   [ Ignore ]   [ # 1 ]
Power Member
Avatar
RankRankRank
Total Posts:  1387
Joined  2011-01-03

Absolutely! My final renders are nearly always done at 16 size bucket—especially if hair or intricate shadows are involved. The whole idea of multiple cores is to divvy up a large amount of work into tiny pieces.

There is one major threading enhancement I’d like to see (if it is even possible). Once most of the threads reach the end of the image (which annoying reports 100% complete even if there are threads still working…), any threads that are still working re-evaluate their bucket and divide it up further.

For example, with the 16 size bucket, the bits you can actually see complete are 4x4 in size and there are four of them. Using numbers for threads, if thread 2 were working on an area and it was taking a long time (hair commonly) it would look like:

*2
22 

The asterisk marks the current 4x4 area being processed. It would be awesome if other threads could jump in to help. We’d end up with:

24
57 

That’s just a former code-jockey dreaming…there’s plenty of smaller, easier tweaks that Carrara needs first…

 Signature 

- Garstor
Australian-Canadian currently trapped in Texas (maintaining sanity with doses of Carrara, LightWave and PhotoShop)

My 3D art Flickr page for final or near-final images
My 3D art Flickr page for work-in-progress or experiments

Profile
 
 
Posted: 01 February 2013 05:36 AM   [ Ignore ]   [ # 2 ]
Power Member
Avatar
RankRankRank
Total Posts:  1828
Joined  2004-08-24

HI Dartanbeck smile

It does depend on the area that’s being rendered, .. Sometimes,,. in a simple scene, the Big Bucket size will cover an area of “relatively” empty scene space,  quite quickly,. while it can take a little longer for smaller buckets (rendering blocks) to cover the same area.

BUT,.. the opposite also applies,. Big sized render buckets will take longer to work on a complex area.

It would be really nice it there was a flexible and automatic adjustment to the size of area being rendered.

For example,: If most of the image has been rendered, but one core is still working on a complex area, and there are some inactive cores available, which could be allocated to rendering the next available sections of that area. 

in other words ,. Smart rendering where inactive cores, go back and help out the struggling core.

just a thought smile

Profile
 
 
Posted: 01 February 2013 06:12 AM   [ Ignore ]   [ # 3 ]
Power Member
Avatar
RankRankRank
Total Posts:  1387
Joined  2011-01-03
3DAGE - 01 February 2013 05:36 AM

It does depend on the area that’s being rendered, .. Sometimes,,. in a simple scene, the Big Bucket size will cover an area of “relatively” empty scene space,  quite quickly,. while it can take a little longer for smaller buckets (rendering blocks) to cover the same area.

Right…the tricky bit is in predicting those complex areas in advance.

Hmmm…caffeine is kicking in now…come to think of it, the problem is not all that difficult. Each thread captures a timestamp when it starts working on a bucket. After a certain period of time - even make that value configurable - it sends out a “Please help me!” signal that other idle threads can detect and react to.

As noted in my original reply, my biggest beef is that the progress reports 100% prematurely.

 Signature 

- Garstor
Australian-Canadian currently trapped in Texas (maintaining sanity with doses of Carrara, LightWave and PhotoShop)

My 3D art Flickr page for final or near-final images
My 3D art Flickr page for work-in-progress or experiments

Profile
 
 
Posted: 01 February 2013 06:47 AM   [ Ignore ]   [ # 4 ]
Active Member
Avatar
RankRank
Total Posts:  262
Joined  2009-03-09

Yeah, and adaptive bucket sizing would be nice.  There are certainly a few ways to detect how bad a given block will be - count up the objects in the zone, look for hair or plants, and do a probe of the shaders across the bucket looking for transparency (whether real or alpha).  Give weights to them and add them up, and let us set the threshold at which it subdivides the buckets.

It would require some tweaking because the rendering system appears to allocate all of the buckets up front.

 Signature 

Come visit my store!

http://www.daz3d.com/fenric
http://fenric.com/wordpress/store/

Profile
 
 
Posted: 01 February 2013 01:00 PM   [ Ignore ]   [ # 5 ]
Addict
Avatar
RankRankRankRank
Total Posts:  6135
Joined  2007-12-04

So… are you gonna make a new plugin for this?
LOL
C’mon you guys!
Carrara, as you know, is the fastest render monster we have.
As for those large areas of relatively blank scene… I love watching the blur! Those little 16x16 blocks whiz by so fast! But 3dage is probably right, as always… it likely takes slightly longer. But that’s nothing compared to the time savings I think I’m getting from smaller tiles allowing all eight cores to solve every render issue.
I’m feeding each core 2 GB of RAM. Do you think it needs 4? My motherboard can handle 32 GB and I have 16. I couldn’t afford it then, but I’ll look at the upgrade. I bet it’s pretty reasonable. But if you don’t think it would help, I’ll save the cash for more Daz products.

Say Fenric. What’s the deal now. Is it against TOS for us to advertise your site here? Any plans on coming back to Daz, at least with a big bundle of all of your plugins?
I want each and every one of them. Aside from MDD exprot, I have all of your Daz offerings - and I’m going to sign in by you and buy all of yours too… but I’m working on that What is Carrara and How to Use It thread, which is intended to be like a user manual that lives. so you can ask and respond… etc.,
I wanted to do a write-up on you and link to your sight. To me, you and your tools are essential to Carrara. perhaps that’s why I’m always happier than the average Carrara user? But I don’t want to get in trouble with the admins - I am a Daz Artist, and want to keep that title for my whole life.
Come back?:grrr:

 Signature 

Dartanbeck @ Daz3d          Check out the Carrara Cafe          ►►►  Carrara Information Manual   ◄◄◄

Profile
 
 
Posted: 01 February 2013 01:11 PM   [ Ignore ]   [ # 6 ]
Addict
Avatar
RankRankRankRank
Total Posts:  6135
Joined  2007-12-04

Another thing that we Carrara artists can do to help our cores is this:
Take a look at everything in the scene - everything.
Not all at once either. Analyze everything - and look for how YOU can help that render. Personally, I realized that it’s really cool that this incredible texture set I bought for Rosie is one beautiful set of high resolution images and UV placement. I just would never want to try making these so nice - they put a LOT of work into optimizing these textures. For most purposes, I don’t need to see the pores in her skin. I took all of her texture, specular, and bump maps into photoshop and reduced them to 500 x 500, which may be too small for some folks here, but you’ve all seen my Rosie renders - she looks great, I think. I try very hard to enhance models in ways that I can eliminate alpha. It seems to help. When I make a scene that focuses on something in the water, I naturally leave refraction ON in the render… otherwise I leave that off. I’ve also found that where I’ve been using reflections, what I really wanted was a nice blend in Highlight vs Shininess - which creates a very similar, yet much faster render.
Often times you really DO ned reflections - but many cases are as I’ve said.
I know you’ve all heard this before, but duplication and replication is essential if you want to keep resources down. But if you take the time to resize texture files from 4200 x 4200 down to 300 x 300, you’ll save a lot of time, and 300 pixels on a small item int he scene is plenty - with a nice image file.
Just some thoughts for you to consider.

 Signature 

Dartanbeck @ Daz3d          Check out the Carrara Cafe          ►►►  Carrara Information Manual   ◄◄◄

Profile
 
 
Posted: 01 February 2013 01:55 PM   [ Ignore ]   [ # 7 ]
Active Member
Avatar
RankRank
Total Posts:  262
Joined  2009-03-09

I can only advertise my site and non-DAZ brokered products in the “Commercial” sub-forum, not here.  You can put whatever you want, wherever you want it, as long as you don’t break any other rules smile

I’m not ruling out ever putting anything in the store here again, but it depends rather a lot on what DAZ announces this month (if they actually announce anything at all).  I’m certainly not going to rely on them for distribution of updates ever again.  If I put anything else here, it would be more like Inagoni where DAZ would have whatever version Spooky accepted through QA for initial downloading, but you would need to check my site for updates.

 Signature 

Come visit my store!

http://www.daz3d.com/fenric
http://fenric.com/wordpress/store/

Profile
 
 
Posted: 01 February 2013 03:16 PM   [ Ignore ]   [ # 8 ]
Addict
Avatar
RankRankRankRank
Total Posts:  6135
Joined  2007-12-04

My nerves are rattled at the way you mention “The Announcement”.
Damn… what a scary little phrase she dropped on us, eh? I sure hope it isn’t a Bad news announcement.
All I can do now is to stay optimistic. And I am.

I was going to reply to your “Come visit my store!” before I realized that this is your sig. But I’m deciding to reply to it anyways:
I like your store. I like what you do, and I hope to see you continue - no matter Carraras future. Like Holly says, just because they stop changing it doesn’t mean anyone has to stop using it. I know I surely won’t. and if products take some new direction making them incompatible with Carrara, that would be the time I quit buying and call my purchased collection complete. I truly have had enough stuff to complete several types of movies for years now. I mainly collect them because I like to - it’s fun opening a shiny new box. I never buy xbox games because I don’t have an xbox console.

So what updates are you talking about? Do my Fenric plugins need updates?

 Signature 

Dartanbeck @ Daz3d          Check out the Carrara Cafe          ►►►  Carrara Information Manual   ◄◄◄

Profile
 
 
Posted: 01 February 2013 03:34 PM   [ Ignore ]   [ # 9 ]
Addict
Avatar
RankRankRankRank
Total Posts:  5291
Joined  2006-08-27

You can also speed up rendering times by turning off unneeded render functions in the renderer. For instance, if I don’t have anything with refraction, I’ll disable that in the render settings. If I don’t use the DOF, then I turn that off. Even if you don’t have any of those functions in your scene, but leave the options enabled, Carrara still looks for those uneeded functions. I remember Sub7th (I wish he still posted here) said once he would disable the bump as well, unless it was absolutely needed.

 Signature 

I find it somewhat liberating not to be encumbered by accuracy.

Profile
 
 
Posted: 01 February 2013 03:55 PM   [ Ignore ]   [ # 10 ]
Active Member
Avatar
RankRank
Total Posts:  371
Joined  2006-12-10

Bucket size (in any rendering engine) is always a bit tricky. As 3DAge mentions, the most common way I determine what size I want to use is based on how many objects are in the scene. Scenes with many objects tend to render faster with smaller sizes. Especially if they have advanced surfaces with reflections, refraction, transparency, etc. The main issue with hair is that the fine grained tranparency maps are computationally expensive to render. On the other hand scenes with only a few simple objects in them are going to render much more quickly with large bucket sizes.

I’ve always wished that the engine could be smart enough to sub-divide rendering buckets once there were idle threads (cores) available. Pre-calculating bucket sizes for each section of a large image may be counter productive, but it would be nice if when 2 threads finish, they could look for other threads that are still working on their buckets and offer to take half of what’s left there to render.

 Signature 

Feel free to read my 3D Tutorial collection, focusing mainly on the technical aspects of 3D rendering.

Profile
 
 
Posted: 01 February 2013 04:33 PM   [ Ignore ]   [ # 11 ]
Addict
Avatar
RankRankRankRank
Total Posts:  6135
Joined  2007-12-04
JonnyRay - 01 February 2013 03:55 PM

Bucket size (in any rendering engine) is always a bit tricky. As 3DAge mentions, the most common way I determine what size I want to use is based on how many objects are in the scene. Scenes with many objects tend to render faster with smaller sizes. Especially if they have advanced surfaces with reflections, refraction, transparency, etc. The main issue with hair is that the fine grained tranparency maps are computationally expensive to render. On the other hand scenes with only a few simple objects in them are going to render much more quickly with large bucket sizes.

I’ve always wished that the engine could be smart enough to sub-divide rendering buckets once there were idle threads (cores) available. Pre-calculating bucket sizes for each section of a large image may be counter productive, but it would be nice if when 2 threads finish, they could look for other threads that are still working on their buckets and offer to take half of what’s left there to render.

These words always intrigued me. I used to monkey around with code in Neverwinter Nights - kind of a bastardized C language. I wonder if I could play with this:
from the SDK download page

SDK for Developers

The Carrara SDK is a set of headers and libraries that allow third party developers to create royalty-free plugins for Carrara. It comes with full documentation of the Carrara API as well as many samples of code so that you can get started quickly.

Carrara is unique in that it has been designed from the ground up to be a completely open architecture, enabling powerful third-party extensions to be created and seamlessly plugged into the application.

With the SDK you can create new shaders, renderers, light sources, atmospheric effects, modelers, animation effects, primitives, constraints, importers/exporters, and much more.

The SDK includes complete cross-platform support (Macintosh and Windows), in the form of a cross-platform GUI building application, UI specific interfaces which give access to all aspects of the UI elements, and UI support utilities which allow the developer to manage a custom User Interface.

Fenric would be more the one to perform such a task,
I wonder if Fenric could make something for that.
What do you think Fenric?

 Signature 

Dartanbeck @ Daz3d          Check out the Carrara Cafe          ►►►  Carrara Information Manual   ◄◄◄

Profile
 
 
Posted: 01 February 2013 05:24 PM   [ Ignore ]   [ # 12 ]
New Member
Avatar
Total Posts:  10
Joined  2009-01-10

I would love to see a render optimization plugin from Fenric.

Profile
 
 
Posted: 01 February 2013 06:06 PM   [ Ignore ]   [ # 13 ]
Addict
Avatar
RankRankRankRank
Total Posts:  6135
Joined  2007-12-04

Yeah Fenric, could you teach our cores how to swarm the project at hand and efficiently churn out an even faster render?

 Signature 

Dartanbeck @ Daz3d          Check out the Carrara Cafe          ►►►  Carrara Information Manual   ◄◄◄

Profile
 
 
Posted: 01 February 2013 06:19 PM   [ Ignore ]   [ # 14 ]
Power Member
Avatar
RankRankRank
Total Posts:  1387
Joined  2011-01-03
Dartanbeck - 01 February 2013 06:06 PM

Yeah Fenric, could you teach our cores how to swarm the project at hand and efficiently churn out an even faster render?

I’m having a lot of fun imagining the looks washing over Fenric’s face as he reads this thread… hmmm shut eye sick shock

 Signature 

- Garstor
Australian-Canadian currently trapped in Texas (maintaining sanity with doses of Carrara, LightWave and PhotoShop)

My 3D art Flickr page for final or near-final images
My 3D art Flickr page for work-in-progress or experiments

Profile
 
 
Posted: 01 February 2013 06:47 PM   [ Ignore ]   [ # 15 ]
Addict
Avatar
RankRankRankRank
Total Posts:  6135
Joined  2007-12-04

Heh… I sure love that little fox! He just rock the world with code, doesn’t he?
Darn it, Garstor… you have me watching little buckets as render now! Arghhhh!!!
lol
It is interesting how quickly electricity can fathom my imagination, isn’t it? I’m barely able to follow dome of them, and some I can’t at all. There’s even some that I don’t think I see at all! And I also like to turn to my progress panel before rendering. Did you ever see how many rays get launched within, like, half a second? Geez!

 Signature 

Dartanbeck @ Daz3d          Check out the Carrara Cafe          ►►►  Carrara Information Manual   ◄◄◄

Profile
 
 
   
1 of 3
1