supressing the duplicate formulas found dialog

Hi all,

I'm a huge fan of batch rendering, but lately I got the "duplicate formulas found" dialog again when loading up genesis 3. I have found no defects except for longer loading times and the dialog popping up.

Now this dialog box messes up my "automated" process of batch rendering, as is wants an "OK" every time it pops up (which is every time a new scene opens up to be rendered.).

Hence the question: can that dialog box be suppressed so it doesn't mess up batch rendering? (other than going through the log and finding out what causes these formulas and uninstalling these products)

Thanks a lot,

Me

«1

Comments

  • Richard HaseltineRichard Haseltine Posts: 108,114

    It is possible to run DS silently from a console, I believe (though it's not a feature I have looked at or used). However, it would probably better to solve the underlying issue - especially if this is affecting all figures using the same base. Load one, then go to Help>Troubleshooting>View Log, jump to the end, and search backwards for Duplicate to find the full text including the property name(s).

  • Ok... Looks like have to go through that list again then... I did so some time ago and cleaned it all up (had to uninstall a lot of stuff though). Now I installed some new stuff and the problem is there again...

    The dialog says: "make sure content is correctly installed". But what does that mean? Basically you cannot manually install, so how could an installation be done wrong and cause this?

    because I identified some morph packs that cause this error that I would like to keep instead of uninstalling...

    Thanks a lot,

    Me

  • Singular BluesSingular Blues Posts: 737
    edited July 2017

    Open Studio.

    Go Help > Troubshooting > View Log File.

    Your text editor will open with the Studio Log. Go to the end of file (Ctrl+End windows, I assume Command+End Mac. Those who worship at the House that Jobs built will hopefully correct me if I'n wrong). Invoke Search. Type "duplicate" set it search up or backwards. Click whatever needs clicking to run the search. This will take to the last instance of Duplicate error. This is not going to be super helpful. It will tell you the fragment URI that is duplicated and which file.

    If you know how to edit DSF files, you can stry seaching the DSF to fix the issue, but the most likely solution will be just remove it. use the file name to Search the morphs folder of your figure to locate the problem file.

    What is a duplicate formula? DzUri is a qobject framework in the script api that allows ERC links to be links. It's a kind of hyper link for internal Daz properties. A duplicate formula is one which connects a control property's URI to the same target property. So imagine that CtrlVictoria8.dsf (if there is such a file, I assume) has and id of  CtrlVictoria8, and that id is linked to FBMVictoria8. Now for some reason, there's another file called Victoria8.dsf but there's a line in the file indicating CtrlVictoria8 and linking to FBMVictoria8. since the same id targets the same output property, it's a dupe. So two different files with the same id URI could cause the issue. It's hard to make that happen but it can happen. More likely, ERC freeze was run on the same property twice. Say ERC freeze connects A property to B property at 50%. But the author realizes it should be 80%. The correct thing to do at this point is ERC Bake, the adjust the 50 to 80 and ERC freeze once more. If, instead, the author changes that 50 to 80 without doing the bake, and Freezes again, you get to different ERC links in the same file. on at .5 and one a .3. And daz will allow that until you reload the figure. Then it breaks. This is more difficult to fix, sinces it's not going to be clear which of the formlas is the right formula. It's helpful if you're the author, because then yu have at least some idea what you were trying to do. 

    This is part of why the log is not super helpful. It doesn't telly you whole URI of the target property, which is okay if the perperty is just a property, like a morph or controller, but an issue of the property is sub unite of a node, like [Bone] xRotate. All you get from the log is xRot, which could be any one of near 200 bones. Also, it only gives one file. if both dupes are in the same file, then you might be able to sort it, but if they aren't, you won't know where the other dupe is. Which could be a problem because the one found in the log might be the one you need, while the other is the problem. Daz can't tell which one is the right one. If it could it would problably not bother telling you about the error.

    Note, if you have a literal copy of a DSF, this will not cause the dupe error. Daz will load copy one of Property X, the overwrite it with copy two. Ony when you have Property X connected to Property Z 2 times in the same file, or Property X links to Z and, for whatever reason, Y contains a line linking X to Z will you get this. 

    Post edited by Singular Blues on
  • Given that I know which product causes this... Is it helpfull to contact the vendor? Or is it entirely possible this error is specific to my pc?

    Thanksa lot,

    Me

  • Richard HaseltineRichard Haseltine Posts: 108,114

    First try a full uninstall - making sure all files have really gone - then reinstall. If the issue oersists, bug reoprt it.

  • Singular BluesSingular Blues Posts: 737
    edited July 2017

    It would be helpful to know exactly which file(s) are throwing the dupe, as opposed to just the product, but the first step I take is to hunt downthe offending file anyway. You'd be surprised how easy it is to end up with weirdness in your morphs. Once I know all possible offenders are removed, (if I don't know for sure one is broken, I make a back up), I then remove and reinstall the suspect product. If it works, I add the back up files on at a time. If the issue returns, then I know the issue was the last file I added.

    I guess if you can be sure the issue is always caused by the one product, and not some other files, you have no choice but make bug report. But I find that you really have to do a whole lot of troublshooting to be sure it's not just you. That or ask someone else who has the same product if they have the problem. Personal opinion. I really wouldn't know what the best thing to do is. I tend to just fix the problem myself. I dislike waiting.

    Post edited by Singular Blues on
  • CybersoxCybersox Posts: 9,281

    The big problem with this is that it assumes that you have an easy way to uninstall the items in question.  I never had a problem until I upgraded from 4.6 to 4.78, and then, suddenly, I had literally hundreds of duplicate items for Genesis and G2F, a number that seems to escalate with each new upgrade.  That's just way too much work to try to fix, and many are custom morhs transferred via GenX that I'd have to recreate from scratch.  so Ijust put up with the lag, and keep hoping that someone will come out with a utility that will comb them out automatically. 

  • The big problem with this is that it assumes that you have an easy way to uninstall the items in question.  I never had a problem until I upgraded from 4.6 to 4.78, and then, suddenly, I had literally hundreds of duplicate items for Genesis and G2F, a number that seems to escalate with each new upgrade.  That's just way too much work to try to fix, and many are custom morhs transferred via GenX that I'd have to recreate from scratch.  so Ijust put up with the lag, and keep hoping that someone will come out with a utility that will comb them out automatically. 

    I couldn't agree with you more ! :) Maybe our dream has come true. Does anybody know if this product does the trick?:

    https://www.daz3d.com/content-checkup

    Thanks a lot,

    Me

  • TotteTotte Posts: 14,716

    The big problem with this is that it assumes that you have an easy way to uninstall the items in question.  I never had a problem until I upgraded from 4.6 to 4.78, and then, suddenly, I had literally hundreds of duplicate items for Genesis and G2F, a number that seems to escalate with each new upgrade.  That's just way too much work to try to fix, and many are custom morhs transferred via GenX that I'd have to recreate from scratch.  so Ijust put up with the lag, and keep hoping that someone will come out with a utility that will comb them out automatically. 

    I couldn't agree with you more ! :) Maybe our dream has come true. Does anybody know if this product does the trick?:

    https://www.daz3d.com/content-checkup

    Thanks a lot,

    Me

    Hello,

    sorry to disappoint you but no, it does not take care of that problem, which is a pretty intricate problem by itself which requires lots of editing inside the duf files.
    But it's a good idea for a script if I could figure out exactly what causes it. (But when that is figured out no one will ever cause it to happen again so it's a dead end script if it's done.

    Content Checkup handles missing texture maps, bad path, specially those errors experienced in DS with Poser items (where Poser do some things behind the back of the user, which sometimes works, sometimes screws things up even worse)

     

  • CybersoxCybersox Posts: 9,281
    Totte said:
     

    Hello,

    sorry to disappoint you but no, it does not take care of that problem, which is a pretty intricate problem by itself which requires lots of editing inside the duf files.
    But it's a good idea for a script if I could figure out exactly what causes it. (But when that is figured out no one will ever cause it to happen again so it's a dead end script if it's done.

     

     

    I wouldn't bet on that being such a dead end script.  I'm noticing that a lot more stuff seems to sneak through the DAZ Q.C. these days and takes forever to get fixed, if it ever does, and, as near as I can tell, a large percentage of the problems in my runtime either come from products that didn't come from DAZ to begin with (mostly from Renderosity and various freebie sites like ShareCG and Deviant Art, where the standards of QC are either lower than DAZ's or simply non-existent).  Which means that as long as those items are still out there, there will always be some need for a script like this.   

  • CathyMCathyM Posts: 261

    Hi Guys! I submitted a help ticket on July 14 because I was getting a duplicate formulas error when using M3D Clayton Shapes for Michael 7 everytime I loaded any form of G3M or characters based on G3M (including M7, Gianni 7, etc.).  I am using Windows 10 64 bit and Daz Studio latest released build.  I received an answer from Daz on the same day telling me that - and I quote: "This is a known issue and occurs when both "M3D Clayton Shapes for Michael 7" and "Growing Up for Genesis 3 Male(s)" are installed. I have sent this on to our QA Team, however, since this does not actually affect any functions in Daz Studio (it just causes the message to come up), it is not high priority. They advised that they would get to this when they can. I'll let you know when I hear back." 

    Personally I uninstalled the M3D Clayton Shapes for Michael 7 until this problem is fixed, since there is a delay when loading any of these models with the error there, and also it made me nervous - If I built a large scene with any of these characters included, what would happen if they were saved with the error, and then the error was fixed and I tried loading the scene again? Uninstalling removed the error.  Anyway just thought this info might be helpful, and if I hear anything further about a fix I'll let you know smiley  -Cathy

  • The thing is indeed, these errors also pop up after upgrading to a new version of DS eventhough in the previous version everything was fine.

    By the way, sometimes the log really does not give enough information as to which file is causing the issue... I've got a massive amunt of "missing property" errors on certain animation files for G3 eventhough to my knowledge I've got no animation files stored on G3 (and a windows search action confirmed this...) So indeed a script to make this go away is HIGH on the priority list or at least it should be for the DAZ team...

    Kind regards,

    Me

  • Robert FreiseRobert Freise Posts: 4,574

    I just recently started getting the duplicate formulas error everytime I load any G3 male figure

    When I looked in the log it gave me no info at all as to what was causing it I also saw an extremely long list of morph data being reported as unfound however all the morphs mentioned worked just fine and the data was located where Studio said it couldn't find it

  • I just recently started getting the duplicate formulas error everytime I load any G3 male figure

    When I looked in the log it gave me no info at all as to what was causing it I also saw an extremely long list of morph data being reported as unfound however all the morphs mentioned worked just fine and the data was located where Studio said it couldn't find it

    I feel your pain... I get (apart from the duplicate errors) the same kind of list that runs on and on and on... I cleaned it up a while ago, but the depressing thing is, I never got G3 to load up in under 29 seconds... While I'm running on an i7 10 core processor and 128 GB of RAM... yet when I load up G3 and have my performance manager open at the same time, the processor, the RAM memory and even the disk that is being read are never used for more than 8%, 10% and 50% respectively. Also When comparing the behaviour of the performance manager with the logged events in the log and when they are logged, it appears that while loading, DS is doing absolutely nothing (detectable) for the better part of 18 seconds in that loading time... Go figure. All of that was before this issue started appearing again... Now G3 will take like six whole minutes to load up... DAZ should fix this real soon. At least for content distributed through its own website, there should be no way it gets messed up like this by simply updating to a newer version of DS or installing some new stuff. In my opinion, if they ever move to Daz Studio 5, that increase in number can only be justified if this kind of hastle is permanently solved in that release :).

    Kind regards,

    Me

  • TotteTotte Posts: 14,716

    Looks like this is a possible script to develop then, I might a look at it, need to get all those "conflicting" figures first (a list of good conflicting items would be great). I know, I'm lazy (probably why I became a programmer int he first place), but a list of items known to conflict or causing this is good, so I can stufy the problem and figure out a way to fit it.

    "M3D Clayton Shapes for Michael 7" + "Growing Up for Genesis 3 Male(s)" is one combo, I will check if I have those.

    Why it happens after updates, might be that updated base figures (guess wild here, correct me if I'm wrong) might have updated morphs control target names.

     

     

     

     

  • Wow, OK... Do I send you my log file or something?

    Thanks a lot,

    Kind regards,

    Me.

  • TotteTotte Posts: 14,716

    You can send me the log to guru66 #and here we put the character that looks like a cinemon bun# icloud.com.
    I won't promise anything yet, but this is the kind of tricky things I like to program.

  • Totte said:

    Looks like this is a possible script to develop then, I might a look at it, need to get all those "conflicting" figures first (a list of good conflicting items would be great). I know, I'm lazy (probably why I became a programmer int he first place), but a list of items known to conflict or causing this is good, so I can stufy the problem and figure out a way to fit it.

    "M3D Clayton Shapes for Michael 7" + "Growing Up for Genesis 3 Male(s)" is one combo, I will check if I have those.

    Why it happens after updates, might be that updated base figures (guess wild here, correct me if I'm wrong) might have updated morphs control target names.

     

     

     

     

    If, by Base figures. you mean Genesis 3 Male, then no. That's not it.

    Morphs are self contained. While they can also contain links to each other, a change of morph name would only exist in the file for that morph. Changes to the Base figure wouldn't have any direct impact. Changes to the morphs that come with the base figure could, but those should then be a problem of the base figure, not of any morph pack.

    Duplicate formulas usually ends up beiing something like what I said above. The only way to find out what causes it at any given time is to play the crapshoot with the log file. And like I said before, the Log is only going to give you one file as the culprit. Which is fine when you have duplicates in a single file, but it seems even odds that the dupes can be from formulas in two separate files. The log won't ever tell you what that first file was, only the second. I suppose if there were 3 or more, it would log those too. The behavior is logical, to an extent. It appears the log is designed to capture the error type and the file it was processing when it got the error. If you have dupes in multiple files, it will process the first, and not get an error. Some time later it will then find the second, throw error and log it.

    This particular case with M7 pack and Growing Up is clearly a multiple files issue. Otherwise, it wouldn't be "a known issue and occurs when both "M3D Clayton Shapes for Michael 7" and "Growing Up for Genesis 3 Male(s)" are installed." It only happens when two separate products are installed. On the upside, the log will indicate which file it found second, meaning you can narrow down the product of the first. There's only two options. 

    I'm not going to presume to be a master of Daz script, but I'm not sure how one would go about automating a fix for this. A skim over the product file list for the two items doesn't show anything obvious (though it wouldn't), so I can't speak to why it is happening in this case, but I'd bet good money the issue is unique. 2 products from 2 vendors with a huge SKU delta suggests this wasn't a case of some item getting mixed up during the authoring phase.

  • vwranglervwrangler Posts: 4,972
    Totte said:

    Looks like this is a possible script to develop then, I might a look at it, need to get all those "conflicting" figures first (a list of good conflicting items would be great). I know, I'm lazy (probably why I became a programmer int he first place), but a list of items known to conflict or causing this is good, so I can stufy the problem and figure out a way to fit it.

    "M3D Clayton Shapes for Michael 7" + "Growing Up for Genesis 3 Male(s)" is one combo, I will check if I have those.

    Why it happens after updates, might be that updated base figures (guess wild here, correct me if I'm wrong) might have updated morphs control target names.

    For those two specific products, the issue was that Growing Up got updated into a conflict with M3D Clayton Shapes that didn't exist before the update. Zev0 and Male-M3dia are working on their products to get them updated out of that conflict. I've no idea where that sits right now; the history of that discussion is in this thread link here.

  • TotteTotte Posts: 14,716
    Totte said:

    Looks like this is a possible script to develop then, I might a look at it, need to get all those "conflicting" figures first (a list of good conflicting items would be great). I know, I'm lazy (probably why I became a programmer int he first place), but a list of items known to conflict or causing this is good, so I can stufy the problem and figure out a way to fit it.

    "M3D Clayton Shapes for Michael 7" + "Growing Up for Genesis 3 Male(s)" is one combo, I will check if I have those.

    Why it happens after updates, might be that updated base figures (guess wild here, correct me if I'm wrong) might have updated morphs control target names.

     

     

     

     

    If, by Base figures. you mean Genesis 3 Male, then no. That's not it.

    Good, then this line is harder to explain: " I never had a problem until I upgraded from 4.6 to 4.78, and then, suddenly, I had literally hundreds of duplicate items for Genesis and G2F, a number that seems to escalate with each new upgrade. "

     

  • Can I still send you my log file or does this mean the scripting is off the table?

    Thanks a lot,

    Me

  • TotteTotte Posts: 14,716

    Can I still send you my log file or does this mean the scripting is off the table?

    Thanks a lot,

    Me

    Send it, it's a good startingpoint of items clashing (which I need to I can have it broken, then try to fix it )

  • Singular BluesSingular Blues Posts: 737
    edited July 2017
    Totte said:
    Totte said:

    Looks like this is a possible script to develop then, I might a look at it, need to get all those "conflicting" figures first (a list of good conflicting items would be great). I know, I'm lazy (probably why I became a programmer int he first place), but a list of items known to conflict or causing this is good, so I can stufy the problem and figure out a way to fit it.

    "M3D Clayton Shapes for Michael 7" + "Growing Up for Genesis 3 Male(s)" is one combo, I will check if I have those.

    Why it happens after updates, might be that updated base figures (guess wild here, correct me if I'm wrong) might have updated morphs control target names.

     

     

     

     

    If, by Base figures. you mean Genesis 3 Male, then no. That's not it.

    Good, then this line is harder to explain: " I never had a problem until I upgraded from 4.6 to 4.78, and then, suddenly, I had literally hundreds of duplicate items for Genesis and G2F, a number that seems to escalate with each new upgrade. "

     

    Maybe, maybe not. 

    Updating versions of Studio might add a new base figure, but it shouldn't necessarily overwrite and existing one. But even if it did, it is still as I said.

    The data leading to dupes has to do with how ERC links work.

    Say Morph A is linked to B. One might assume that A would just have a line that says "If this morph's value changes, then change Morph B by that amount times this factor." But that would be wrong. What it actually says is "If Morph A's value changes, then change Morph B by that amount times this factor." This may seem semantic but computers don't make assumptions. they do exactly as instructed. Sorry if I'm dumbing this down too much, but it's really important to understand what happening with dupe formulas (which is distinct from dupe IDs. that's a different error).

    The critical difference above is if you transplant the line about morph A controlling B to Morph C, it will still be a valid ERC link.

    Mind you this is one thing that can happen to cause a dupe error. This is exactly whats going on in the case of the Clayton/Growing Up conflict. Zev0 made a patch and posted it in the thread. A respondent said it didn't fix the error. So I pulled it. I opened the offending file and there is exactly a case of "Morph C has a line saying Morph A controls Morph B" This is almost certainly where the conflict is coming from. It's actually 2 lines, but it seems no one is complaing about the other causing errors. Specifically, the file inside Growing Up links Clayton's pJCMClaytonBodyAbsUpperLowerBendDownHD3 to CTRLMD_N_XRotate_35 and CTRLMD_N_XRotate_40. CTRLMD_N_XRotate_40 is the problem target. Now, there may be a darned good reason why Growing Up is trying to modify Clayton's behavior, but this is still the conflict. Because somewhere in Clayton, the same line exists.

    Someone having a Studio upgrade cause this kind of thing is really, really weird. Because, again, it would have to be a case of something like the above, where some file was telling Studio to link some other file to some 3rd file, and if that happened, then it would have happened to everyone. The only thing I can think of that might cause this kind of behavior resulting from an upgrade is if they upgrade added a new content library or improperly applied an update. (Say the upgrade or update removed a file and added a new one to replace it. New file, new name, but same id doing similar jobs. This would throw the dupe error in in incorrect update because the old file would not be removed in that case, but I can't imagine this happening a lot.) Of course, that's possibly a limit of my imagination. I'd need to see the user's log file.

    Of course, if the user is considering sending it, you must cause the error to happen before sending the log. Studio's log rolls and it logs a lot of stuff. Just messing about with the program could cause the error to be rolled out of the log file. So, to send the log: cause the error, then, directly open/copy the log. No passing go or anything. Like nuking the site from orbit, it's the only way to be sure.

    Post edited by Singular Blues on
  • TotteTotte Posts: 14,716

    Update of the program alone is code and libs only. Updates to content is most probably required, unless there is some caching involved, where caching is based on the order of things being installed, so when you install A before B irt works, but when the cache is rebuilt (or new content is installed), B ends up before A, causing the problem to show. But I don't think there is any kind of morph caches.

    Otherwise I don't think a Studio Update could be the problem, most likely people have a bunch of updates and new content in DIM and then install all.

  • Singular BluesSingular Blues Posts: 737
    edited July 2017

    My thinking, also, but I don't know if the installation packs for Studio unpack the included content as well as the program during upgrades. I've never look that closely at it because upgrades have never cause me any issues.

    Post edited by Singular Blues on
  • TotteTotte Posts: 14,716

    Content are always in separate installers.

  • CathyMCathyM Posts: 261
    edited July 2017

    Thanks guys for looking into this and trying to explain what is causing these errors... I appreciate all the explanations here, though sadly I have to confess that mostly this just goes over my head. frown

    While it would be fantastic if there was a way to fix the issues and do away with these errors, (in my own opinion) I would really even be satisfied if they could just make the error stop showing up - if as Daz says, it isn't actually doing any harm, why can't it just be supressed, as gibrril_fa945a6cee was hoping for? 

    I've found another incident of this error shows up in Genesis 2 Females and every character for her when having Second Circle's Beth for G2F installed, although I am not sure what the other model is that is causing it...(Beth is from Rendo, so probably not a concern for Daz)

    While I don't do animations, I did try using The batch skin converter in order to update some of my V4 textures to .duf, and found that this error as was mentioned requires me to click on "ok" every single time the script runs, which was making me crazy, so I had to uninstall that character in order to proceed. I think any of the batch file processing scripts will have this result. 

    I agree with gibrril_fa945a6cee, this problem, even though it may not cause any harm, is not acceptible, and I would really like to be able to use the things I've had to uninstall in order to make it go away.  Just my opinion - Cathy

    Post edited by CathyM on
  • Singular BluesSingular Blues Posts: 737
    edited July 2017

    I'm sorry the explanations are too technical, but the technical is important to understand the point. Unfortunately that means you just have to take my word.

    There is no fix for duplicate formulas errors because there is no one thing that causes it. One might argue I'm wrong about there being no one thing, semantically, but from a computer's point of view, each cause of the issue is unique. The only solution is to remove the offending product until it gets fixed, if it ever gets fixed. (Unless you are savvy enough and lucky enough to figure out the cause of the issue). Bear in mind the exact cause of the Clayton/Growing Up conflict was only found because one of people who had the probalem knew exactly how to check for changes to their product library and happened to only have one thing change on Genesis 3 male. The Log error pointed at the product that hadn't changed.

    It's pure blind luck they were able find the one that had changed. There just isn't a way to write computer code that could fix the problem.  The best one might do is code that would find every file that might be part of the problem.

    Post edited by Singular Blues on
  • TotteTotte Posts: 14,716

    @Singular Blues: Yes, it's difficult to describe complicated technical stuff in a simple way without losing the important parts.

     

  • Indeed. Sometimes I think of clever metaphors. Sometimes other people do. For example, I got a good understanding for air combat BFM from a clever trick I found on the internet, and I explained the trick over the internet to someone else (I forgot where I found the trick, but I remember the trick) and they got it, too. 

    I guess this is one of the things that in order to understand what is up, you need a solid understanding of what is up. There's probably a path to teach the details, but I've never been a good teacher. Either I think of/find some trick that makes the point or I don't. I'm not dismissive about it. I do try. I remember when I didn't know how any of this worked, and I couldn't imagine how I'd ever find out. I know the feeling. I know what it's like to not know what the hell someone's on about. I learned bit by tiny bit in facing little problems that I couldn't buy a solution for. And by being wrong. A lot. Repeatedly. So I can't use my experience as a model even leaving aside that I'm a terrible teacher.

Sign In or Register to comment.