Digital Art Zone

 
   
1 of 2
1
Help understanding Metadata
Posted: 22 November 2012 02:02 AM   [ Ignore ]
Member
Rank
Total Posts:  49
Joined  2004-12-22

I need help understanding Metadata, how it works and why it isn’t working for me.  In order for it to work, what needs to happen, where do the files need to be.  For example, by default, I know the metadata files reside in the Support folder.  Where does the file need to be?  Must it reside only in the Studio\My Library\Runtime\Support folder?  Must the files associated with it reside in the same Runtime folder?  Will it only work if the associated product files are not moved from the default libraries?  I usually install my files to a temporary directory adjust the files to my own structure before placing them in the final directory (I prefer to put all the files for an item in one folder for example - lights, camera, mats, poses… in the same folder as the base item).  I also have multiple runtimes to work with.

Profile
 
 
Posted: 22 November 2012 05:54 AM   [ Ignore ]   [ # 1 ]
Active Member
RankRank
Total Posts:  901
Joined  2009-12-07
Wildlyfe - 22 November 2012 02:02 AM

I need help understanding Metadata, how it works and why it isn’t working for me.  In order for it to work, what needs to happen, where do the files need to be.  For example, by default, I know the metadata files reside in the Support folder.  Where does the file need to be?  Must it reside only in the Studio\My Library\Runtime\Support folder?  Must the files associated with it reside in the same Runtime folder?  Will it only work if the associated product files are not moved from the default libraries?  I usually install my files to a temporary directory adjust the files to my own structure before placing them in the final directory (I prefer to put all the files for an item in one folder for example - lights, camera, mats, poses… in the same folder as the base item).  I also have multiple runtimes to work with.


Hi Wildlyfe

Sure is an interesting topic. I recommend you follow the link in my sig to the metadata toolkit. There are three detailed tutorials with step by step guides explaining everything about metadata and showing you how to make it, use it and share it.

You might also want to check out this thread on using Smart Content http://www.daz3d.com/forums/viewthread/8724/  and this one ont he discussion around metadata guidelines: http://www.daz3d.com/forums/viewthread/7542/

 Signature 

PoseMaster for DAZ Studio and PoseMaster People Pack now sold at Rendo.

Profile
 
 
Posted: 22 November 2012 05:55 AM   [ Ignore ]   [ # 2 ]
Administrator
Avatar
RankRankRankRank
Total Posts:  14445
Joined  2003-10-09

When you install content with metadata a script is placed in the Run Once folder (in the DAZ 3d\Studio 4\ folder in your application data folder) to place the metadata file in the main \Runtime\Support folder - so it doesn’t matter whether the file is installed to the main folder or not. You are then prompted to add the item to the CMS then next time you start DS or process the queue, at which point the data is processed and added to the CMS database and the file in the Support folder is not needed again 9unless you reset the database). Moving files around will break the metadat unless you do it after the content has been processed and use the Content Library pane to do the moving.

 Signature 

DAZ Studio Frequently Asked Questions

Index of free DAZ Studio scripts and plugins list

Profile
 
 
Posted: 22 November 2012 06:12 AM   [ Ignore ]   [ # 3 ]
Active Member
RankRank
Total Posts:  901
Joined  2009-12-07
Richard Haseltine - 22 November 2012 05:55 AM

When you install content with metadata a script is placed in the Run Once folder (in the DAZ 3d\Studio 4\ folder in your application data folder) to place the metadata file in the main \Runtime\Support folder - so it doesn’t matter whether the file is installed to the main folder or not. You are then prompted to add the item to the CMS then next time you start DS or process the queue, at which point the data is processed and added to the CMS database and the file in the Support folder is not needed again 9unless you reset the database). Moving files around will break the metadat unless you do it after the content has been processed and use the Content Library pane to do the moving.


I’ll add one comment to Sir Richard’s. For homebrew metadata or manual installation (i.e. without the installer) you can copy the three metadata files to the Runtime/Support folder. DAZ Studio won’t know about when it starts up (unless you place a copy of the DSA file in the RunOnce folder), but if you choose Content Library/Content DB Maintenance/Reimport Metadata it will be in the list and you can import it manually.

 Signature 

PoseMaster for DAZ Studio and PoseMaster People Pack now sold at Rendo.

Profile
 
 
Posted: 22 November 2012 10:01 AM   [ Ignore ]   [ # 4 ]
Power Member
RankRankRank
Total Posts:  2322
Joined  2011-11-16

So if I understand this correctly…

If someone edits the metadata file in the support folder, then ‘reimports’ the metadata.. it will adjust the metadata in the CMS and any future reimporting of the metadata will follow the imported version, the original metadata having been wiped from the CMS, untill ofc the support file gets overwritten by an update to the item from DAZ. Furthermore, if one were to back up the edited metadata file, merge it with any revised one from an update from DAZ, then place the updated file in the support folder, this updated version would also maintain the structure the user wished in the CMS rather then constantly blowing out the data with some default data that didn’t follow the end users personal preferences. Is this essentially correct?

All of this is ofc with the provision that DAZ does not support modifying the metadata files and anyone doing so would be doing it at their own risk.

(A train and a car both have their purposes, but a train stops at places I don’t care about, and doesn’t go everywhere I do.)

 Signature 

Just because I may have a strong opinion doesn’t make it any more (or less) correct than any other, just that I feel passionately a particular way at that moment.

Profile
 
 
Posted: 22 November 2012 01:58 PM   [ Ignore ]   [ # 5 ]
Administrator
Avatar
RankRankRankRank
Total Posts:  14445
Joined  2003-10-09

That should be the case, though I’ve had very limited success in getting it to work.

 Signature 

DAZ Studio Frequently Asked Questions

Index of free DAZ Studio scripts and plugins list

Profile
 
 
Posted: 22 November 2012 02:53 PM   [ Ignore ]   [ # 6 ]
Power Member
RankRankRank
Total Posts:  2322
Joined  2011-11-16

Hmm, well I had hopes of being able to possibly create an interface around modifying/customizing the files.. but if manually editing isn’t functional, automating it would be a non starter.

 Signature 

Just because I may have a strong opinion doesn’t make it any more (or less) correct than any other, just that I feel passionately a particular way at that moment.

Profile
 
 
Posted: 22 November 2012 10:55 PM   [ Ignore ]   [ # 7 ]
Member
Rank
Total Posts:  49
Joined  2004-12-22

Thanks cridgit for the tutorials and forum links.  Lost of great info there.  I guess from what I am reading, that those of us who have restructured our runtimes are a bit out of luck at this time for using supplied metadata?  Right now, I can re-import metadata, but while I can see it in the list for import, the info doesn’t seem to actually install.  Nothing shows up.  I assume this is because it can’t find the files where it expects them to be for association.  I had hoped I could install them anyway and edit the paths to link to where I located the files, but I guess not.  I really like the idea of using metadata, but not sure I want to re-install the huge amount of content I have in order to use it.

Profile
 
 
Posted: 23 November 2012 04:27 AM   [ Ignore ]   [ # 8 ]
Active Member
RankRank
Total Posts:  901
Joined  2009-12-07
Wildlyfe - 22 November 2012 10:55 PM

Thanks cridgit for the tutorials and forum links.  Lost of great info there.  I guess from what I am reading, that those of us who have restructured our runtimes are a bit out of luck at this time for using supplied metadata?  Right now, I can re-import metadata, but while I can see it in the list for import, the info doesn’t seem to actually install.  Nothing shows up.  I assume this is because it can’t find the files where it expects them to be for association.  I had hoped I could install them anyway and edit the paths to link to where I located the files, but I guess not.  I really like the idea of using metadata, but not sure I want to re-install the huge amount of content I have in order to use it.


You’re welcome.

If you look inside one of the metadata DSX files, you’ll see the assets have relative path references (relative to My Library) e.g. runtime/poses/pose1.pz2). That means the content directory doesn’t matter. But if you’ve renamed files or moved them within the My Library folder, those relative paths won’t work.

As far as I know you can edit base paths in Studio, but I haven’t tried doing this to get metadata to work.

There is a batch install DOS script you can use if you’re on Windows - let me know if you need the link. I predict it’ll only be a matter of time before you take the plunge and reinstall wink  A number of people I know of have already done this. Its just easier that way.

 Signature 

PoseMaster for DAZ Studio and PoseMaster People Pack now sold at Rendo.

Profile
 
 
Posted: 23 November 2012 06:28 AM   [ Ignore ]   [ # 9 ]
Power Member
RankRankRank
Total Posts:  2322
Joined  2011-11-16

Actually, I rename all of my items also so that they have natural names rather then bg_btn_smzt or whatever unintelligible thing they get named at times. You know, the name one sees in the interface. I’m guessing this also breaks metadata?

 Signature 

Just because I may have a strong opinion doesn’t make it any more (or less) correct than any other, just that I feel passionately a particular way at that moment.

Profile
 
 
Posted: 23 November 2012 07:51 AM   [ Ignore ]   [ # 10 ]
Active Member
RankRank
Total Posts:  901
Joined  2009-12-07
Gedd - 23 November 2012 06:28 AM

Actually, I rename all of my items also so that they have natural names rather then bg_btn_smzt or whatever unintelligible thing they get named at times. You know, the name one sees in the interface. I’m guessing this also breaks metadata?


It should break it if you do that BEFORE installing the metadata, but if you do it AFTER in Studio (right-click rename) I would expect Studio is smart enough to know to update the metadata references as well. Haven’t tried though, so you might want to give it a shot.

 Signature 

PoseMaster for DAZ Studio and PoseMaster People Pack now sold at Rendo.

Profile
 
 
Posted: 23 November 2012 08:35 AM   [ Ignore ]   [ # 11 ]
Power Member
RankRankRank
Total Posts:  2322
Joined  2011-11-16

Ok, there is no way that would work. Renaming 1000’s (10’s of 1000’s even) of files one at a time by right clicking etc… is ... not workable to put it mildly. I batch rename the files many at a time, and we can’t do that within the DS interface. This points out exactly what the problem is when a program requires everything to go through it’s interface to maintain integrity. And before anyone says it, media players etc.. all allow external applications to modify metadata and they update their internal database to match. It appears to me what is really needed is a way to modify the support files externally and have the DS program reimport the metadata to sync it internally, and to be able to save out changes made internally to the support files. This is the way better modern programs work.

 Signature 

Just because I may have a strong opinion doesn’t make it any more (or less) correct than any other, just that I feel passionately a particular way at that moment.

Profile
 
 
Posted: 23 November 2012 10:23 AM   [ Ignore ]   [ # 12 ]
Active Member
RankRank
Total Posts:  901
Joined  2009-12-07
Gedd - 23 November 2012 08:35 AM

Ok, there is no way that would work. Renaming 1000’s (10’s of 1000’s even) of files one at a time by right clicking etc… is ... not workable to put it mildly. I batch rename the files many at a time, and we can’t do that within the DS interface. This points out exactly what the problem is when a program requires everything to go through it’s interface to maintain integrity. And before anyone says it, media players etc.. all allow external applications to modify metadata and they update their internal database to match. It appears to me what is really needed is a way to modify the support files externally and have the DS program reimport the metadata to sync it internally, and to be able to save out changes made internally to the support files. This is the way better modern programs work.


Nice concept in theory grin

Practically though, how do you propose any program (DAZ Studio, Poser, Maya, iTunes, Media Player, Powerpoint, Picasa etc.) identifies an asset (a file) if it has been renamed and/or moved BEFORE metadata has been added? I would very much like to here your proposed solution, because with my 30 years of computing experience (yeah since 1983 baby) all I know is the filename/path is the ONLY way to uniquely identify an asset WITHOUT metadata (some kind of supplementary signature added to the file).

The media players etc. you are referring to have the following characteristics:
1) they are bundled with the operating system which enables the operating system to provide media-editing features to provide for a seamless experience (I’m referring to both Windows and Mac).
2) they identify the file using either the filename or its metadata in the first place, without which they simply don’t know what song/album it is.

Personally, I simply don’t understand why you feel compelled to rename the files and move them around when you have higher-level tools at your disposal for organizing things. However, that’s your right and choice, but please provide a feasible proposal and perhaps it could be done. I really can’t think of how this is possible without using either filename/path OR metadata.

 Signature 

PoseMaster for DAZ Studio and PoseMaster People Pack now sold at Rendo.

Profile
 
 
Posted: 23 November 2012 01:36 PM   [ Ignore ]   [ # 13 ]
Power Member
RankRankRank
Total Posts:  2322
Joined  2011-11-16

First of all, neither Mediamonkey, XBMC, nor Media Companion come with any OS. They handle metadata very well as does many programs. The ones that come bundled with the OS, MS Media Player and ITunes *do not* handle metadata properly as they *only* keep the metadata within an internal indexing system. This means the metadata doesn’t get properly shared to other applications that look to the (music etc…) They are an example of how not to handle metadata. Metadata can be handled by placing it in the header of the file, if the file structure is defined in such a way as to accommodate that (this is the preferred way when available) or in a ‘sidecare’ file. This sidecar file can be xml, proprietary, etc… This in effect is what DAZ is doing with the support files. So far so good. They are actually following best practices up to this point. Where they fall away from that is when they don’t handle the sidecar files in a way that sets a standard that other programs can access and modify these sidecar files, and further, provide a way in the program (DAZ Studio) to syncronize these sidecar files with the database (CMS.)

Edit: technically they do provide a way to syncronize the database as one can ‘re-import’ and this should do exactly that. However, this doesn’t seem to be working properly.

Second Edit: There are many programs that handle metadata using the methods I have discribed and this *is* considered current best practice. I simply pointed out the three above as they are readily accessible for looking at. Mediamonkey and Media Companion probably being best two of the three for simply understanding functionality.

 Signature 

Just because I may have a strong opinion doesn’t make it any more (or less) correct than any other, just that I feel passionately a particular way at that moment.

Profile
 
 
Posted: 23 November 2012 07:04 PM   [ Ignore ]   [ # 14 ]
Active Member
RankRank
Total Posts:  901
Joined  2009-12-07

We’ve got 2 issues here. Firstly, does DAZ Studio handle metadata correctly and allow external modification of metadata. Secondly, should we be allowed to rename and reorganize our files and folders as we see fit and still expect our metadata to work inside DAZ Studio.

Let me tackle issue #1 first. Your statement that DAZ Studio does not support this is simply not correct. DAZ Studio handles external modification of DSX metadata perfectly. See the screenshots below for an example. The point is clear - anybody or any program can modify the DSX files and after reimporting the metadata DAZ Studio absorbs those changes.

You might ask why its necessary to reimport the metadata rather than read it “live” from the DSX files, or why not automatically import updated metadata when DAZ Studio starts up. The former case won’t provide the performance of a database, and the latter will slow down startup times for ALL users.

Second issue: “why can’t I rename my files and folders as I see fit outside of DAZ Studio”? The problem here is that when DAZ Studio first identifies a new asset, the only information it has about what that asset is, is its file name and path. Without that information THERE IS NO WAY IN GODS KINGDOM to know what the asset is. So if you insist on changing the files and paths DAZ Studio won’t be able to connect the asset’s metadata to it, and you’re on your own. If you leave the file names and paths intact and import the metadata, DAZ Studio knows what the asset is and attaches the metadata to it. If you THEN change the file name and path (through DAZ Studio), it should keep track for you (I haven’t tested this). Suggesting the program is deficient because it doesn’t allow you to arbitrarily change file names and paths outside of its control, doesn’t make sense.

Hopefully I have now more clearly stated my position and shared enough information with you so that you can see the point. I’m not diagreeing with your right to choose the workflow that suits you. I’m explaining why there are limitations as a result of your choice.

Image Attachments
Picture1.jpgPicture2.jpgPicture3.jpgPicture4.jpgPicture5.jpg
 Signature 

PoseMaster for DAZ Studio and PoseMaster People Pack now sold at Rendo.

Profile
 
 
Posted: 23 November 2012 11:09 PM   [ Ignore ]   [ # 15 ]
Power Member
RankRankRank
Total Posts:  2322
Joined  2011-11-16

If any program tried to read the data *live* it would cause it to ‘break’ too often (file locking issues etc.. when multiple apps tried to access the same file.)

Actually, I apologize. The issue that I believe is tripping you up is actually a good point and one I didn’t want to get into necessarily because I didn’t want to get that deep into it. I’ll be brief because I don’t want to bore everyone. Basically the added complexity DAZ has is that unlike a video or music file, a 3d object is multiple objects, and therefore more complicated to deal with. The issue with paths and filenames have to be either avoided being changed or done so in a way that is robust enough to handle it properly and efficiently. Going into it further as to how to deal with that is too boring for most so I’ll spare everyone from that. The one thing I will point out is that separating ‘tags’ from files and paths, ie… keeping them to separate and well defined areas of the sidecar file, is a step towards making this feasible.

Understand, I haven’t played with trying to modify the files. I was actually responding to Richard’s post saying he hadn’t had much luck manually modifying the files. Since I trust in his competence, I figured there are probably gotchas involved. That was the impetus of all of this on my part really. I know that all of this is in flux to some extent as DAZ formalizes some of this so I’m not in a rush to try delving too deeply into this atm.

That last part, the fact that import doesn’t zero out old metadata, nor provide a simple interface where all of the metadata is visible in a single screen for a given object/set of objects… that’s an example of a gotcha.

 Signature 

Just because I may have a strong opinion doesn’t make it any more (or less) correct than any other, just that I feel passionately a particular way at that moment.

Profile
 
 
   
1 of 2
1