Digital Art Zone

 
   
1 of 2
1
Memory Consumption - Is this normal?
Posted: 27 June 2013 01:59 PM   [ Ignore ]
New Member
Avatar
Total Posts:  9
Joined  2003-10-09

Hello everyone,

I started to work on a scene which is bigger than the usual ones I was creating in past. While adding details to the scene I hit a point where the work with the scene slowed down to a crawl. I realized that all of the 32GB of RAM are consumed by DAZ Studio (4.6.0.18 Pro 64bit) and that the system is swapping badly.

I checked the Scene Info tab and I found out that the whole geometry has 6.8M faces total (Scene Info). My first thought was that all the RAM gets eaten by preloaded textures, so I created a duplicate of the scene file, removed all textures from all characters and objects, saved the scene, quit the DAZ Studio and after launching it again I loaded this untextured scene. Unfortunately the memory footprint after the scene finished loading was again exceeding physical RAM and the system was swapping (Memory Utilization with textures compared to same scene without textures).

Then I quit the Studio again and created a sphere primitive with 4096 segments and 2048 sides. That produced a scene with higher face and quad count than the previous one, however the memory consumption was very tiny compared to the real scene (Sphere Scene Info and memory consumption)

Question is - is this normal, or am i facing some kind of a memory leak within DAZ Studio?

Also when I close the normal scene that consumes the 32+GB of RAM the application window closes almost immediately and about 8GB of RAM is released as free but in process list the DAZ Studio is still listed as running and consuming ~18GB RAM…

So my question as already shown in the topic is - Is this Normal?


Zdenek

 Signature 

Zdenek

Profile
 
 
Posted: 27 June 2013 02:36 PM   [ Ignore ]   [ # 1 ]
Power Member
Avatar
RankRankRank
Total Posts:  2401
Joined  2007-12-07

Out of physical memory is the worst form of bottleneck as it feeds back into all parts of the system, so you have my sympathies.  I’ve noticed that DS4.6 seems to be consuming more memory than previous versions, but that may just me having had it catch my eye.  I’ve constructed, and rendered, a couple of tolerably complex scenes in my 32GB of RAM.  What HAS hammered the computer was a very poorly (IMHO) put together set of overlapping materials and custom shaders that seemed to gobble memory like it was going out of fashion during the render.

As it sounds like you’ve not even got to the render part yet I can only suggest deconstructing the ‘problem scene’ doing memory utilisation checks at key points along the way to see if you can spot an obvious culprit.  I’d be curious to hear what you find out if you do this.

 Signature 

Some writings by a friend:
http://www.lizardsofthehost.co.uk/11.html
http://www.lizardsofthehost.co.uk/13.html
http://www.lizardsofthehost.co.uk/7.html

Profile
 
 
Posted: 27 June 2013 09:08 PM   [ Ignore ]   [ # 2 ]
Administrator
Avatar
RankRankRankRank
Total Posts:  16477
Joined  2009-01-22

Hmm… 6.8 million total faces. I sort of think that is the issue. Its not the SIZE so much as DS dealing with all the mesh in 3D space. Never tried a file that large. I lag at 3.5 Million myself. Which is rather HUGE. And that does EAT Ram for the Calculations just so the Viewport can respond, when it does.

 Signature 

Exile, Drows Walk: A Tale of Jaderail starts here. Free pulp fiction at its Fantasy best, Updated 2/3/14.
Proud Member of the Bald Wizards Club My stuff at DeviantArt

Profile
 
 
Posted: 27 June 2013 10:37 PM   [ Ignore ]   [ # 3 ]
Addict
Avatar
RankRankRankRank
Total Posts:  4815
Joined  2005-08-03

I can’t even get mine to use more memory. I have 8GB installed and DS never uses more than 3.5+ (never even hits 4GB), but it’ll eat my CPU (I have an overclocked i7-2600K to 4.7GHz- hyperthreading is enabled). This is while rendering.

How can you see the face count in DS?

 Signature 

Wearing duck crap doesn’t make you a duck - Floki
@Deviantart | @ShareCG
Tutorial: V4 Transmap Brows on Genesis via DS only.
Bulk convert V4/M4 mats for Genesis2

Profile
 
 
Posted: 27 June 2013 11:08 PM   [ Ignore ]   [ # 4 ]
New Member
Avatar
Total Posts:  9
Joined  2003-10-09
Jaderail - 27 June 2013 09:08 PM

Hmm… 6.8 million total faces. I sort of think that is the issue. Its not the SIZE so much as DS dealing with all the mesh in 3D space. Never tried a file that large. I lag at 3.5 Million myself. Which is rather HUGE. And that does EAT Ram for the Calculations just so the Viewport can respond, when it does.

Well 6.8M faces is a lot but as you can see with the testing sphere primitive which has 8M faces the GUI and work in the Studio slows down but the amount of memory consumed is really tiny.

Also When I quit the Studio with the testing sphere loaded it quits immediately and returns all the allocated memory. When I quit it with the other scene that goes over the top with the RAM consumption the window closes, about 8GB of ram is returned to the system as free but in the task list the DAZ Studio is still shown as running and using all the remaining memory and I just have to kill the process to free the RAM.

I also tried to collect the scene for rendering to see how big the scene really is as this collects all the geometries and textures and puts them into one folder. This produces a folder containing roughly 7GB of data. Which is basically the amount of memory the DAZ takes on the intial load of the scene before the RAM consumption starts skyrocketing and goes through the roof.

Vaskania - 27 June 2013 10:37 PM

How can you see the face count in DS?

There is a tab that can be opened called “Scene Info” it gives you basic information about the geometry/poly count in your scene and selected object.

SimonJM - 27 June 2013 02:36 PM

As it sounds like you’ve not even got to the render part yet I can only suggest deconstructing the ‘problem scene’ doing memory utilisation checks at key points along the way to see if you can spot an obvious culprit.  I’d be curious to hear what you find out if you do this.

Well… It seems that I will have to do that which scares me as there really is a lot of objects in there. Fortunately I have them all organized into groups and parented to nulls but it is going to be a time consuming process nevertheless as it takes the Studio about 10 minutes to get into a state where I can start working with the scene after it is open.

Another funny bit is that after I did the collecting part and got the whole scene nicely packed for rendering on an external 3delight engine I downloaded the standalone Linux version of the 3Delight, installed it onto a box where i normally run Luxrender and let it crunch on the RIB file. During the render the Linux system (which has 16GB of RAM) consumed just about 5GB of it for the render.

So this makes me believe that the issue is somewhere in the Studio instead of the render process as also when I hit render in the studio the scene gets rendered and after the render is finished about 7GB of ram is then freed which sort of corresponds with the real size of all the resources in the scene.

—-
Zdenek

 Signature 

Zdenek

Profile
 
 
Posted: 27 June 2013 11:54 PM   [ Ignore ]   [ # 5 ]
Administrator
Avatar
RankRankRankRank
Total Posts:  16477
Joined  2009-01-22

Do you have any VERY old 3D mesh objects in the Scene file? I have loaded a few older items I collected around the web into DS4.6 and had issues. Never saw my Ram Ramp to full use but did notice one Freebe Tree I loaded jumped my ram use WAY up. I had planned to populate the scene with it but two would have killed all my ram. I even tried a Instance and that crashed DAZ Studio. I removed that item from my content folders. It is a DS4+ issue because I can use it fine in my DS3A, but I’m trying to move forward not backwards so DS3A is not a option in my eyes any longer unless I can not figure out how to do it in DS4+. Mostly I can so far.

I’m thinking that could be what you’ve run into. Or even just a BAD mesh, it might not need to be that old. Wish I could offer better help but So far I’ve not seen any mem leaks in 4.6. I do keep meters running on my System just because I’m a Nerd and like to know whats going on under the hood.

 Signature 

Exile, Drows Walk: A Tale of Jaderail starts here. Free pulp fiction at its Fantasy best, Updated 2/3/14.
Proud Member of the Bald Wizards Club My stuff at DeviantArt

Profile
 
 
Posted: 27 June 2013 11:58 PM   [ Ignore ]   [ # 6 ]
Active Member
Avatar
RankRank
Total Posts:  458
Joined  2012-08-17
Vaskania - 27 June 2013 10:37 PM

I can’t even get mine to use more memory. I have 8GB installed and DS never uses more than 3.5+ (never even hits 4GB), but it’ll eat my CPU (I have an overclocked i7-2600K to 4.7GHz- hyperthreading is enabled). This is while rendering.

How can you see the face count in DS?

This sounds like you are using the 32bit version. The 64bit version can use more available memory.

 Signature 

My DAZ Gallery


http://vanguard-studio.deviantart.com/

Profile
 
 
Posted: 28 June 2013 12:02 AM   [ Ignore ]   [ # 7 ]
New Member
Avatar
Total Posts:  9
Joined  2003-10-09
Jaderail - 27 June 2013 11:54 PM

Do you have any VERY old 3D mesh objects in the Scene file? I have loaded a few older items I collected around the web into DS4.6 and had issues. Never saw my Ram Ramp to full use but did notice one Freebe Tree I loaded jumped my ram use WAY up. I had planned to populate the scene with it but two would have killed all my ram. I even tried a Instance and that crashed DAZ Studio. I removed that item from my content folders. It is a DS4+ issue because I can use it fine in my DS3A, but I’m trying to move forward not backwards so DS3A is not a option in my eyes any longer unless I can not figure out how to do it in DS4+. Mostly I can so far.

I’m thinking that could be what you’ve run into. Or even just a BAD mesh, it might not need to be that old. Wish I could offer better help but So far I’ve not seen any mem leaks in 4.6. I do keep meters running on my System just because I’m a Nerd and like to know whats going on under the hood.

I will probably kill the weekend by trying to dissect the scene and try to look for the cause. But even if it is caused by ‘old’ mesh then it is an issue in the Studio as it fails to handle it properly. The scene renders just fine without issues whatsoever…

—-
Zdenek

 Signature 

Zdenek

Profile
 
 
Posted: 28 June 2013 04:24 AM   [ Ignore ]   [ # 8 ]
Power Member
RankRankRank
Total Posts:  1009
Joined  2012-02-04

Another thing to check for: how many objects have subdivision enabled?  4.6 changes the default SubD algo to Catmark, and that is more sensitive to mesh quality/construction than Catmull-Clark was.  And who knows what bugs might be in it.  So if there are a bunch of SubD objects, you could try switching them all to Catmull-Clark, save and reload and see if the problem persists.

Note that 4.6 also has a bug with switching SubD algo: if the figure has any confirmed items with smoothing & collision enabled, switching the SubD algo will hang the mesh smoother.  The workaround is to switch the figure to Base resolution, change the algo, then switch it back to High resolution.

 Signature 

My deviantART Homepage
Set Focal Distance script for DAZ Studio

Profile
 
 
Posted: 28 June 2013 12:46 PM   [ Ignore ]   [ # 9 ]
Addict
Avatar
RankRankRankRank
Total Posts:  4815
Joined  2005-08-03
MadBirdCZ - 27 June 2013 11:08 PM

There is a tab that can be opened called “Scene Info” it gives you basic information about the geometry/poly count in your scene and selected object.

Thanks!

Vanguard - 27 June 2013 11:58 PM
Vaskania - 27 June 2013 10:37 PM

I can’t even get mine to use more memory. I have 8GB installed and DS never uses more than 3.5+ (never even hits 4GB), but it’ll eat my CPU (I have an overclocked i7-2600K to 4.7GHz- hyperthreading is enabled). This is while rendering.

How can you see the face count in DS?

This sounds like you are using the 32bit version. The 64bit version can use more available memory.

Well, I got it after some testing. It WILL use more ram, but apparently what I’ve been doing hasn’t pushed my PC to the point of needing it. I loaded some of my FB sets and props, used AdamR’s Insane Quality render settings, went to a 4000x4000 image, and voila. Just under 5-6GB being used.

I just figured even with a good SSS mapped figure on insane and studio lighting that a slow render would get boosted a bit by using more ram, but my PC doesn’t seem to agree. lol Unless I’m misunderstanding the relationship between CPU and Ram where rendering is concerned (I read somewhere iirc that cpu ‘collects’ the data and ram pushes the pixels out, or something- so maybe it’s my CPU holding my ram back?).

 Signature 

Wearing duck crap doesn’t make you a duck - Floki
@Deviantart | @ShareCG
Tutorial: V4 Transmap Brows on Genesis via DS only.
Bulk convert V4/M4 mats for Genesis2

Profile
 
 
Posted: 28 June 2013 01:00 PM   [ Ignore ]   [ # 10 ]
Active Member
Avatar
RankRank
Total Posts:  757
Joined  2012-02-12

Subdivision eats memory. Try putting all your mesh to base resolution to see how much the scene takes

 Signature 

My Deviant Art

Profile
 
 
Posted: 28 June 2013 01:22 PM   [ Ignore ]   [ # 11 ]
New Member
Avatar
Total Posts:  9
Joined  2003-10-09

OK… So the results are in…

I selected everything in the scene and reduced mesh resolution to Base. After that I switched the Subdivision Algorithm to Catmull-Clark (Legacy) and saved the scene. After quitting and re-opening the DAZ studio I loaded the scene and memory consumption dropped from 35GB (commit charge) to 28GB of physical RAM used out of 32.

Then I selected everything again and returned the base mesh resolution to High and repeated the save, restart and open process while leaving the SubDivision still set to Catmull-Clark. The memory utilization after the scene fully loaded and DAZ Studio became responsive is 28GB.

So it seems that mesh resolution does not really matter and the reason for the 7 more GB of RAM being eaten is the Catmark SubDivision algorithm.

Also the Base resolution of the figure meshes is not really useable as it generates a huge amount of poke-through issues.

Now however, the question still stands - is it normal or ist there some bug in the Catmark?

 Signature 

Zdenek

Profile
 
 
Posted: 28 June 2013 02:03 PM   [ Ignore ]   [ # 12 ]
Power Member
Avatar
RankRankRank
Total Posts:  1336
Joined  2006-03-14

I’ve had WAY more than 6.8M faces in a scene.

This scene has 26M verts in it and oly uses about 22GB of RAM during render.  About 9 on load iirc.

http://adamtls.deviantart.com/art/Urban-Centaur-372197828

What’s happening to you is subdivision ballooning.  When you subdivide, oh, say, Victoria 4… you’re going from about 68K verts to something like 1.2M at full subdivision.  This definitely uses quite a bit more RAM even when the “display” resolution is set to “base” or subdivision level is set to 0.

 

 Signature 

My Tutorials and Freebies Index

Profile
 
 
Posted: 28 June 2013 02:24 PM   [ Ignore ]   [ # 13 ]
New Member
Avatar
Total Posts:  9
Joined  2003-10-09
adamr001 - 28 June 2013 02:03 PM

I’ve had WAY more than 6.8M faces in a scene.

This scene has 26M verts in it and oly uses about 22GB of RAM during render.  About 9 on load iirc.

http://adamtls.deviantart.com/art/Urban-Centaur-372197828

What’s happening to you is subdivision ballooning.  When you subdivide, oh, say, Victoria 4… you’re going from about 68K verts to something like 1.2M at full subdivision.  This definitely uses quite a bit more RAM even when the “display” resolution is set to “base” or subdivision level is set to 0.

Nice scene.

In the one I’m working on I already have roughly 30 characters (V4 / Genesis) and SubDivision was for all characters left on default values (Mesh quality set to High and SubDiv level set to 1 with Catmark SubDiv algorithm). Then there are quite a few objects there.

Does the scene geometry info take the SubDivided surface poly count into account?

 Signature 

Zdenek

Profile
 
 
Posted: 28 June 2013 02:35 PM   [ Ignore ]   [ # 14 ]
Power Member
Avatar
RankRankRank
Total Posts:  1336
Joined  2006-03-14

No, Scene Info only counts original mesh density.  I’ve got a bug report on that somewhere.

 Signature 

My Tutorials and Freebies Index

Profile
 
 
Posted: 29 June 2013 10:30 AM   [ Ignore ]   [ # 15 ]
Active Member
Avatar
RankRank
Total Posts:  329
Joined  2008-04-09

To answer the OP’s original question, yes, it’s normal in the sense that ANY subdividing algorithm is going to increase the amount of RAM used for a scene.  As you figured out, the difference is in which algorithm is being used.

As for the RAM not being released, that’s normal for Windows.  It can take a few minutes for a program to completely shut down and release the RAM, especially if it was actively processing data.

Profile
 
 
   
1 of 2
1