anaximanes_2000 said:

crosswind said:

I think "re-import metadata" wrote no required Category data into "Categories" table, so of course there were no "linked data" in the table of "Category_Content" ... That was why you saw Lost and Found.

Not sure if you have tried re-installing IM package that was generated by CPA via DIM... Normally you should keep those IM package files of 3rd party product, just in case...

To your first point, I am totally ok with having it show up as "Lost and Found" inside of the thing.  The thing I am not understanding, to use Pumpkina as an example is why these 3rd party things work (which were installed after being packaged by CPA) and then don't work on the new computer.  That's literally all I need to know so I can fix the issue.  Like installing Pumpkina a 4th time, she now works.  Her little Runtime thing is the same, everything is the same.  So why does she work again now, but the 3 previous installs from 3 previous incarnations of Windows show up in Smart Content as empty.  If I can figure out why something works the first time, then maybe one of you that understands all this, can give me a way to make sure it ALWAYS works so I don't have to bust my @(# every time I move computers or reformat.  There must be a way to do it, without ever having to literally go by a TB hard drive just to keep every single "CPA" created Zip file so I can reinstall everything every time (and thus keep getting multiple dupilcates of it in Smart Content, i.e. 4 copies of Pumpkina in Smart Content now.)  To your second point, tthis is exactly what I want to avoid! lol

The only way to re-produce the issue so as to find out the culprit is to have CPA and simulate your process... but unfortunately I don't have it. Just wait and see if the author can give you some clue...

crosswind said:

I think "re-import metadata" wrote no required Category data into "Categories" table, so of course there were no "linked data" in the table of "Category_Content" ... That was why you saw Lost and Found.

Not sure if you have tried re-installing IM package that was generated by CPA via DIM... Normally you should keep those IM package files of 3rd party product, just in case...

To your first point, I am totally ok with having it show up as "Lost and Found" inside of the thing.  The thing I am not understanding, to use Pumpkina as an example is why these 3rd party things work (which were installed after being packaged by CPA) and then don't work on the new computer.  That's literally all I need to know so I can fix the issue.  Like installing Pumpkina a 4th time, she now works.  Her little Runtime thing is the same, everything is the same.  So why does she work again now, but the 3 previous installs from 3 previous incarnations of Windows show up in Smart Content as empty.  If I can figure out why something works the first time, then maybe one of you that understands all this, can give me a way to make sure it ALWAYS works so I don't have to bust my @(# every time I move computers or reformat.  There must be a way to do it, without ever having to literally go by a TB hard drive just to keep every single "CPA" created Zip file so I can reinstall everything every time (and thus keep getting multiple dupilcates of it in Smart Content, i.e. 4 copies of Pumpkina in Smart Content now.)  To your second point, tthis is exactly what I want to avoid! lol

You can @DoctorJellybean who's the author of CPA, or PM him. He can explain how CPA works.

Just for your info. - : There're 4 related key tables in cms database that essentially make Smart Content work: Product, Categories, Content and Category_Content.
- "Product" table is the parent which stores primary product data.
- "Categories" table stores all categories (parent + children categories...).
- "Content" table stores all line items of each asset file of a product.
- "Category_Content" table links assigned category in Categories table to each item in Content table.

I show you an example for your better understanding, as attached screenshots.

So, the prerequisites to show items in Smart Content with correctly-assigned categories is that all metadata are imported into the above tables with no failure. In your case, as per my exp., I think "re-import metadata" wrote no required Category data into "Categories" table, so of course there were no "linked data" in the table of "Category_Content" ... That was why you saw Lost and Found.

Not sure if you have tried re-installing IM package that was generated by CPA via DIM... Normally you should keep those IM package files of 3rd party product, just in case...

Anyway, all culprits of "missing and/or wrong metadata" can be found out as long as you know the "structure of metadata" and have a proper tool...

NorthOf45 said:

Why the older ones with decent metadata do not is still an unknown...

NOoooooo.... Well hopefully someone can help me.  Thanks again for having tried!  But yeah I have her in here three times, because I have had to remake her in CPA 3 times, one for every previous reformat and now will have to be a fourth unless someone can explain this CPA installed stuff and how to fix it.

Well, Pumkina metadata looks better, but why it does not display in Smart Content is a mystery, again. The new ones seem to work properly, so DIM is creating the metadata files as it should and adding the Products and their assets where they should go. Why the older ones with decent metadata do not is still an unknown...

@NorthOf45 - Well, I appreciate it, but I still do not get it.  That Runtime/Support file and over 1 million other files is backed up for me on 3 hard drives.  I backed everything up again before the reformat, which means the Ajuzam files are all precisely identical to what they originally were when they were showing up in Smart Content as "Lost and Found".  Like I've said, I got all first party stuff with the reimport metadata, and so you say thoise CPA creations must never have had the right metadata to begin with (apparently CPA does not function like that).  So, I started thinking, "Ok, North said Ajuzam is Poser or whatever, so I'll look at a Daz figure."  So I have Pumpkina for G8F.  As with all the 3rd party stuff she was working fine and of course now I have her thumnail in Smart Content, but it's all empty when I go into it.  So, I did what you said, right?  I went to DIM, I used the Runtime path, and opened her thing, and she has all the assets.  Screenshot here.  SO maybe that will help you figure out what is actually wrong, because she does not work either like the rest of them.  I DID check Evilyna Outfit while I was at it and yes, it shows assets as well, but unlike Pumpkina, I can see the outfit files fine in Smart COntent.  So, what does this mean?

@crosswind - I copied what you said for the next time I have to wipe this computer clean.  Thank you

wasn't there a program released not too long ago that allowed installs of these types of products and also set up the metadata? I agree though that this is something we, as customers, should not have to do. 

1. When DIM installs a package with no metadata file (Runtime/Support .dsx file), it creates one with the minimum required information to create the Product entry in the database. It also defines all the assets included in that Product and includes them in the Product, which you can see in Smart Content. DIM also creates a manifest in the Manifest Archive folder that contains all the information about the Product, who installed it, where it was installed (Library root folder), some internal tracking information (GUID, installer date, etc.) and the full path (relative to the Library root folder) of every file installed.

Your new and old installations seem to have all the manifests intact. You mentioned that Richard Haseltine told you to include them when copying the old installation. But that is not the data needed by Smart Content.

There is a set of metadata files (.dsx) in the Runtime/Support folder that should contain all the information that DIM created when installing the third-party files. The first-party files have metadata files included in the package, and they get installed at the same time. Look at your DIM-created Evilyna metadata file in the Runtime/Support folder and compare it to the Ajuzarn metadata file that you showed me earlier. The Evylina file will have something that looks like this:

<ContentDBInstall VERSION="1.0">
 <Products>
  <Product VALUE="Product name from CPA details page ">
   <StoreID VALUE="DAZ 3D by default"/>
   <GlobalID VALUE="long string of alphanumeric characters from CPA details page"/>
   <ProductToken VALUE="some number"/>
   <Artists>
    <Artist VALUE="maybe? Not part of CPA Info"/>
   </Artists>
   <Assets>
    <Asset VALUE="some path to an asset, like Runtime/Libraries/Character/Ajuzarn.cr2 or People/Genesis 9/Clothing/... etc.">
     ... More metadata minutiae
    </Asset>
     ... More assets
   </Assets>
   <SupportAssets VALUE="eg., /Runtime/Support/ DAZ_3D_4939938107523997-1_Ajuzam_DR_Test1.dsx">
    <SupportAsset VALUE="..."/>
     ... More Support Assets
   </SupportAssets>
    .... other stuff
  </Product>
 </Products>
</ContentDBInstall>

Your old Ajuzarn metadata is, most importantly, missing the <Assets>...</Assets> block of data, so it cannot define and add those files to the Product, therefore it remains empty. You absolutely must have that information to successfully create the Product with all of its assets (CPA cannot do that for you). Why it is missing is a mystery, but it could have been because of an old version of DIM that did work properly. Restoring bad data just gives bad results.

2. If your metadata files have all the information they should have, and you copy the Runtime/Support folder (along with everything else, and the manifest archive) to the new installation, you can re-import the metadata knowing that ALL the products will be available, and visible in Smart Content. This is good data, good results.

I hope this answers your questions. Good metadata is the key.

anaximanes_2000 said:

Dar, North, I might not be communicating well, I am sorry.  Ok, so I tested Evilyna on another computer about an hour ago. I installed Daz, and used install manage ONLY to install Content Package Assist.  Then, I did CPA and install the 3rd part outfit.  I opened that Daz and it worked perfectly.  I did a backup of the Daz3d folder and My Daz 3d Library things, which was all the basic stuff Daz comes with and that 3rd party outfit.  Ok, so then I reformatted that computer about 30 mins ago.  Reinstalled Daz, and Evilyna outfit is broken.  It is just like all my 3rd party stuff here.  The CPA thing did it again.  Now, this is thing I am either asking wrong or not understanding and please hold my hand through this...

Right now, on my real computer, as you saw in the screenshot, the 3rd party outfit that I installed is working fine in Smart Content (albeit the files are in lost and found).  So, why do all these CPA created 3rd party installs work just fine.  We know they ALWAYS break once you copy and paste your entire Daz library onto a new computer, but like right now, at this moment, the outfit works.  I get that listerally the 800 or so 3rd party stuff I have already installed on my computer multiple times now (since i always have to redownload them all, put them all through CPA, reinstall them all with DIM, and then lose Smart Content access to them again the next time I migrate Daz) and will not be able to apparently avoid having to do all that work yet again, but to avoid this fate ever happening to me again, with me being as direct and succunct as possible, can you answer 1. and 2., please?

1. Why is the outfit's files all showing up in Smart Content right now, what file does it have and where is this file that previous 3rd stuff is missing after my backup?

2. How do I replicate what is happening right now with that outfit having these working 3rd party CPA created and installed files going forward forever when backing up my massive amount of third party models?

Naturally and most likely, you lost the needed metadata...even if you've "re-imported everything"... Well, if you can attach the DSX file (in which metadata is stored...) of the product down below (under the folder: your Daz Library\runtime\support), I might find out the culprit and tell you why...

anaximanes_2000 said:

I go through this everytime... EVERY TIME.  So, here on Daz I bought a product several years ago called Content Package Assist, and somehow I always do this.  But it's always 2 or 3 years in between.  So, Daz was really busted (shocker, I know) and so I had to reinstall windows entirely.  No problem, backed up all my files in the "Public Documents" folder, which was the "My DAZ 3D Library" and "Daz3D" folders.  Easy.  Now, Content Package Assist of course had worked, and over time everything from third party sites was in there, working in Smart Content, all was right with the world.

 

Get the new computer, move everything back over.  All my 1st Party stuff is BAM right back in Smart Content after reimporting all metadata...

 

All 3rd party things show up in Smart Content too... at least the thumbnails do.  ALL of them are missing all their files that were there before the move.

 

Now, once again, I am lost.  If I right click and say "Show in Content Library" it takes me right to it.  I can click it and click, "Edit metadata" for example, and lo and behold, there are all the files.  So, Daz knows they are there.  So, what is the automated (not a man having to do hundreds of third party files) way to import all the files back into their existing smart content thing?  It's 2024 and Daz's estimated annual revenue is currently $32.5M per year.  So, obviously they would have a simple and easy way to do this and I will be grateful if you can let me know the simple process.

I don't have CPA while I always install 3rd-party with Content Wizard. However, as a heavy user of Smart Content, I strongly recommend the quickest and safest way for migrating Daz3D database from your old PC to new PC, as below:

- DO NOT export user-data as well as re-import metadata...etc.,  you may never get correct result especially for the 3rd-party products. Instead, directly migrate cms database !
- The cms database locates in the cluster folder : %appdata%\DAZ 3D\cms in your old PC. After you get everything done in your new PC with Daz3D product directry setup... copy the cluster folder from your old PC to the same location in your new PC. It'll simply take 1 ~ 2 minutes.

However, I'm not sure if you still have the cms database in your old PC or if you're able to recover it...but if you do the migration next time, pls go with this way.

You can have my Metadata since I have no use for it, LOL

As I said before, CPA does not create, modify or otherwise manipulate metadata. It packages what it is given into a DIM-compatible zip file. The metadata that Daz Studio uses is either supplied by the vendor (like first-party products from DAZ 3D), or generated automatically by DIM when installing the package if there is none included (like your recent Evilyna product). Your installation has the older generated metadata files, but they appear to be lacking important information. Why it is missing is a mystery.

If you re-create the third-party packages, a proper metadata file will be created by DIM when you install it, and can be re-imported in the future. CPA works properly and does what it should. Just be sure you are using the proper folder structure for the contents, otherwise the installation will be messed up. It does not do anything with metadata other than include it for installation if it is provided, like any other file. That metadata is what drives the Smart Content.

As to why they "break" every time, the metadata files used when re-importing metadata appear to be missing important information, so the Products are not properly filled with their assets.

There is a way to recreate the metadata files without having to re-process everything. It might be just as much work, but only need be done once, not every tme you need to reformat or change machines.

The files are in their proper place in the folders, and that is all that is really needed to recreate the Product contents. Copy the Product name, as shown in the Product pane, into the clipboard for future use. You can add existing files to a Product (one or more assets at a time or entire folders at once) from the Content Library view. The Product itself already exists, although empty for now.

Go to a relevant file or folder for that Product, and right-click on an asset (or multiple assets in view) or folder for that Product. Select "Create a Product from..." and in the dialog, paste the EXACT name of the Product that already exists. Accept, and it should prompt that the Product already exists, and if you want to add the files to the existing Product. Yes, you do, so accept it. Repeat for all assets for that Product. When complete, that Product should show all included assets in the Smart Content view.

This new information will be considered USER DATA that you can later export from Content DB Maintenance (any time, and regularly as backup) and re-import when needed. The thing is, you have to import all of it at the same time, which might not always be desired, especially if not up to date.

You could also export the metadata from the Product view to create the Product metadata file, like one that would be created when installing, and would now be considered VENDOR DATA. It could also be re-imported later by selecting one or more Products from DIM or The Content DB Maintenance -> Re-Import Metadata utility.

To use the Ajuzarn as an example, you know the Product name is "Azujarn DR Test1". Copy that string for later (to avoid having to type it in every time) and go to the Figures/Creatures folder in Poser Formats. Right-click on "Azujarn DR.cr2" and select "Create Product from...". In the dialog , paste your Product name "Azujarn DR Test1" and accept. When asked if you want to add to the existing Product, accept.

Next, go to "Materials/Creatures" and right-click on the "AzujarnDR" folder, right-click and select "Create Product from...". Paste in the same EXACT Product name as before and Accept. When asked if you want to add to the existing Product, accept.

Do the same for the "Pose/Creatures/AzujarnDR" folder. That should be all you need to at least get all the user-facing files to show in Smart Content (might need to refresh the view). One last step. In the Product view, right-click on the Product thumbnail and select Edit Metadata. You will see a button in the bottom left corner that says Export. This will create a fresh copy of the metadata file (the name and path is shown higher up at the Support label) in the Runtime/Support folder. This new metadata file will now contain the necessary asset definitions that populate the Product. (The metadata can do so much more, like Content Type, Compatibility, Default Categories and such, but you don't seem to be interested in all that, and would be a lot more work to set up.)

Rinse and repeat for every Product that is installed but has no assets appearing in Smart Content. It might be faster than downloading everything, processing in CPA and re-installing, again.

I hope I didn't overwhelm you with all that, but it only needs to be done once.

Soooooo, I do have at least a modicum of understanding for the metadata of the migrated Rendo vendors not being complete, so that most of the items end up in Lost&Found. At least I see them there, am aware that I have bought them, and can eventually correct the metadata. But now I've just bought a bunch of Lully dresses, and NONE of them have ANY metadata at all. They don't show up in Smart Content in any way, shape or form. Now, that is really really not okay. I rely on smart content for genesis clothes. If I'd known that, I could have set up the installers better on my own, via Content Package. There's got to be SOME effort put into this from DAZ's side at least. How can you sell us products that are not set up to your own rules and standards at all?!

Please fix this asap. This is literally YOUR job, DAZ. The PAs do almost everything else. So please, get to it.

Thanks.

(Yes, am also putting in a ticket. But maybe others want to chime in as well, or can be made aware of the dimensions of the problem if they are not yet.)

NorthOf45 said:

open the Content DB Maintenance utility and select "Export User Data". This will write one or more USserData_x.dsx files into the Runtime/Support folder. Copy them to the new machine into the same folder, and run the Re-Import Metadata utility, selecting only the UserData files They should appear in the listing in a different grouping from the DIM-installed products.

No it was this computer, I reformatted Windows because of THIS DAZ ISSUE.  Anyway, I will just have to manually install all my 3rd Party files a firth time... but at least I added this to my notes for the next time I have to do a reformat.  Thanks for all your help!  Sorry it's a total failure.  I hate Daz, but earn a living with it.  As far as that one I did, the Outfit to prove to you that it shows all the things to me (at least until the next reformat/new computer), is this what gave you the idea of this UserData thing, or is that a seperate thing we can look into to see why CPA installed stuff in Smart Content will always work until a reformat?

Another idea came to me. You said that everything was working on the old computer. That implies that you had recovered metadata for the third-party products. The one we looked at had no data for the assets, so they get left in limbo. If you had previously recovered the metadata, it would have had to be created somehow, and was in the database. Since it was not in the metadata file, it was as UserData, which has any info that was added by the user. It is not imported with a re-import metadata operation unless it was exported first.

If you still have access to the old computer, open the Content DB Maintenance utility and select "Export User Data". This will write one or more USserData_x.dsx files into the Runtime/Support folder. Copy them to the new machine into the same folder, and run the Re-Import Metadata utility, selecting only the UserData files They should appear in the listing in a different grouping from the DIM-installed products.

crosswind said:

Content Type makes these logos appear. If you correctly save the preset files, Content Type(s) will be automatically and correctly assigned and stored in cms database.

However, if you change the folder names or move the files to other folders, etc. you'll lose the logos...  You can assign Content Types to DUF files at anytime by: in Content Library, right click on the folder, for instance, "Characters" folder, click "Edit Metadata..." in the popup menu, then assign a value to the field of "Type".

Thank you very much for this to solve the problem

Content Type makes these logos appear. If you correctly save the preset files, Content Type(s) will be automatically and correctly assigned and stored in cms database.

However, if you change the folder names or move the files to other folders, etc. you'll lose the logos...  You can assign Content Types to DUF files at anytime by: in Content Library, right click on the folder, for instance, "Characters" folder, click "Edit Metadata..." in the popup menu, then assign a value to the field of "Type".

Okay. Ajuzarn has not been assigned to a Product, and is in limbo. I get this when an asset in a product is not included for some reason, usually left out of the metadata file, or has the wrong filename.

Okay, edit the metadata file in the Runtime/Support folder (DAZ_3D_4939938107523997-1_Ajuzam_DR_Test1.dsx). (Now you can use the DIM Show Installed Files list to get to it quickly). It is just a text file in XML format, but is human readable. Search for Ajuzarn DR.cr2. If it is there, it should be in the "Ajuzarn DR Test1" product.

Re-importing the metadata usually sets it straight. I do not know why it is not working. All I can suggest at this point is to re-install it and check (after a refresh in the folder) if you see a Product entry in that pane. Once you do, it will surely show up in Smart Content.

Sorry for the confusion. LOCAL USER is in DAZ Studio -> Content Library -> Products -> L

Files as blue hyperlinks is good, files are where they should be, which means the manifests are correct. Yes, you are close...

Now just to find the Product contents. I vaguely remember having problems many years ago with misplaced references, and I do believe it was because of LOCAL USER (I think an asset can only belong to one product at a time, even if used by more than one product). If you delete the references to Ajuzam in LOCAL USER and Re-Import Metadata, they should show up in the proper Product.