Digital Art Zone

 
     
Metadata GlobalID = random hex string?
Posted: 14 July 2012 04:24 PM   [ Ignore ]
Member
Avatar
Rank
Total Posts:  60
Joined  2008-03-27

I’m looking at developing a semi-automated metadata generator to bring the many thousands items of non-DAZ sourced content in my Runtime under content management. I’ve got most of it figured out, but the sticking point is the GlobalID. About the only documentation I can find is that it’s “the database-internal ID - and is automatically set”, which is fine if you just want to read or edit existing metadata, but not very helpful when you’re trying to create new metadata from scratch outside DAZ Studio.

Initially I thought the GlobalId was some form of hash value, however this thread on uninstalling a product from CMS implies the GlobalID isn’t necessarily tightly related to the object it identifies because it uses a technique of overwriting one product’s GlobalID with another’s in order to delete an unwanted entry from the CMS.

Secondly, in the Content DB Editor screen, there’s a Generate button next to the GlobalID (see first screen image). If, without changing anything in the product, you click Generate, you get a completely different GlobalId (see second screen image), and every time you click you get a different id. If the GlobalId was a hash generated from the product details, it should generate the same id each time (and the field should probably be non-editable).

Does anyone know if the GlobalId really is a random hexadecimal string, as it appears to be.

Thanks.

Image Attachments
Content_DB_Editor_1.gif
Content_DB_Editor_2.gif
Profile
 
 
Posted: 16 July 2012 02:27 AM   [ Ignore ]   [ # 1 ]
Administrator
Avatar
RankRank
Total Posts:  315
Joined  2003-10-09

The Global ID is not a digest of the product details, as those details can change. It is a Universally Unique Identifier as defined by QUuid.  Once a Global ID is set for a given product, it should not be changed or shared by another product. Changing the Global ID for a product, even if all the other information is exactly the same, doesn’t update that product’s Global ID… it defines an entirely new product with the exact same details, which of course defeats the purpose. Sharing the Global ID between products doesn’t “share” anything so much as it creates a conflict of the data being defined for the product.

Below is a script that does essentially the same thing that the Content DB Editor Generate button does, except this shows a MessageBox instead of inserting it into a LineEdit.

// Call the QUuid::createUuid().toString() wrapper on DzApp
var sUuid App.createUuid();

// Remove the leading curly brace
sUuid sUuid.replace( /^\{+/, "" );
 
// Remove the trailing curly brace
sUuid sUuid.replace( /\}+$/, "" );

// Show the user
MessageBox.informationsUuid"Universally Unique Identifier""&OK;" ); 


-Rob

Edit: Removed extraneous semi-colon from MessageBox button text.

 Signature 

- Documentation Center
- Install Manager
- DAZ Studio 4.x
- DSON Importer
- DSON File Format Specification

Profile
 
 
Posted: 16 July 2012 04:44 AM   [ Ignore ]   [ # 2 ]
Member
Avatar
Rank
Total Posts:  60
Joined  2008-03-27
rbtwhiz - 16 July 2012 02:27 AM

The Global ID is not a digest of the product details, as those details can change. It is a Universally Unique Identifier as defined by QUuid.  Once a Global ID is set for a given product, it should not be changed or shared by another product. Changing the Global ID for a product, even if all the other information is exactly the same, doesn’t update that product’s Global ID… it defines an entirely new product with the exact same details, which of course defeats the purpose. Sharing the Global ID between products doesn’t “share” anything so much as it creates a conflict of the data being defined for the product.

Thanks Rob, that points me in the right direction.

I think the Python equivalent to your script would be something like:

import uuid

print struuid.uuid1() )  #  generate a host id and timestamp uuid 


I think I’d rather do something more along the lines of:

import uuid

print struuid.uuid5uuid.NAMESPACE_URLproductURL ) )  #  generate a SHA-1 hash uuid 

where productURL holds the page URL of the product.

The advantage of using the SHA-1 hash of the product page URL is that the uuid would be stable—it wouldn’t matter how many times you hit the Generate button, you’d always get the same uuid for the same product, whereas Studio keeps generating different uuids every time you hit the Generate button.


Later, after some experimentation…

I think the equivalent to what Studio is doing is more likely:

import uuid

print struuid.uuid4() )  #  generate a random uuid 

as the Studio generated uuids are less stable than timestamp based uuids.

Profile
 
 
Posted: 16 July 2012 09:14 AM   [ Ignore ]   [ # 3 ]
Administrator
Avatar
RankRank
Total Posts:  315
Joined  2003-10-09

The product page URL is one of those details I mentioned that can change. Not just within the same domain, but change domains as well. It happens often. The page I linked to states that the value is random - QUuid::createUuid(). This is intentional. The idea being that the owner of a product (or someone acting on their behalf) would define the value once at creation and then register/publish that value - ideally through a centralized service, but at least through a centralized location on a given store.

-Rob

 Signature 

- Documentation Center
- Install Manager
- DAZ Studio 4.x
- DSON Importer
- DSON File Format Specification

Profile
 
 
Posted: 16 July 2012 12:40 PM   [ Ignore ]   [ # 4 ]
Power Member
Avatar
RankRankRank
Total Posts:  1829
Joined  2006-02-17
rbtwhiz - 16 July 2012 09:14 AM

The product page URL is one of those details I mentioned that can change. Not just within the same domain, but change domains as well. It happens often. The page I linked to states that the value is random - QUuid::createUuid(). This is intentional. The idea being that the owner of a product (or someone acting on their behalf) would define the value once at creation and then register/publish that value - ideally through a centralized service, but at least through a centralized location on a given store.

-Rob

I know the you know this Rob, but for those who don’t:

UUID’s are not guaranteed to be unique between systems.  Multiple entities generating UUID/GUID can lead to conflicts.  The ideal solution would be for DAZ to have a UUID request system where a UUID could be requested for freebies or products and avoid the possibility of conflicts.

Kendall

 Signature 

Any opinions expressed in this post are those of Kendall Sears and may, or may not, be more, or less, valid than any other opinion.

The contents of this post are intended for the DAZ forum only, do not re-post any portion to any other forum without his permission.

Profile
 
 
Posted: 16 July 2012 11:11 PM   [ Ignore ]   [ # 5 ]
Active Member
Avatar
RankRank
Total Posts:  520
Joined  2005-09-12
Kendall Sears - 16 July 2012 12:40 PM
rbtwhiz - 16 July 2012 09:14 AM

The product page URL is one of those details I mentioned that can change. Not just within the same domain, but change domains as well. It happens often. The page I linked to states that the value is random - QUuid::createUuid(). This is intentional. The idea being that the owner of a product (or someone acting on their behalf) would define the value once at creation and then register/publish that value - ideally through a centralized service, but at least through a centralized location on a given store.

-Rob

I know the you know this Rob, but for those who don’t:

UUID’s are not guaranteed to be unique between systems.  Multiple entities generating UUID/GUID can lead to conflicts.  The ideal solution would be for DAZ to have a UUID request system where a UUID could be requested for freebies or products and avoid the possibility of conflicts.

Kendall


I have to agree on this - one centralized UUID/GUID generator and registry. Or There Will Be Duplicates.

 Signature 

The only constant is change—and it’s a variable.

Profile
 
 
Posted: 17 July 2012 06:59 AM   [ Ignore ]   [ # 6 ]
Member
Avatar
Rank
Total Posts:  60
Joined  2008-03-27
rbtwhiz - 16 July 2012 09:14 AM

The product page URL is one of those details I mentioned that can change. Not just within the same domain, but change domains as well. It happens often. The page I linked to states that the value is random - QUuid::createUuid(). This is intentional. The idea being that the owner of a product (or someone acting on their behalf) would define the value once at creation and then register/publish that value - ideally through a centralized service, but at least through a centralized location on a given store.

IETF RFC 4122 provides for five different mechanisms for generating uuids:

•  MAC address (time based);
•  DCE Security;
•  MD5 hash (name based);
•  random; and
•  SHA-1 hash (name based).

If the page you linked to states that all uuids are random, this would be a limitation of the Qt uuid library.


A centralised service sounds wonderful in theory, but…

As the vendor for the only software product that uses this metadata, the central repository would need to be something DAZ provided, and I haven’t noticed anyone in Utah rushing to volunteer.

The second issue would be whether the central service could reasonably vet requests for products sold at other stores. Could someone reasonably be expected to know that Merlin’s Medieval Swords sold at CP was a different product to Merlin’s Medieval Swords at RMP (and is actually the same product known as Merlin’s Medieval Swords II at RMP). Do they simply say as long as the combination of store name and product name is unique, issue a uuid?

Another issue would be products where the creator is Poser only and has no interest in supporting DAZ Studio, is no longer in 3D art, or indeed in some cases sadly no longer alive—who creates the uuid on their behalf? The metadata for these products will be generated by end users who want to bring these products under the CMS, but non of it will be officially sanctioned by the content creator. How will the centralised authority handle requests of that nature?

The store that would gain most from running a centralised service is DAZ, and I don’t see the cost/benefit being favourable.


Conversely, each store running their own service isn’t going to work either. Unless Smith Micro decides to hook into the CMS in Poser, it would be counter productive for them to host a CMS related service. Equally, I don’t see small, Poser-oriented vendors wanting to do the work either — if they don’t even test in Studio, or include Studio materials, they’re not going to provide metadata. Again, it will be the end users who produce this metadata.


If uuids are generated randomly, and two users produce and post metadata for the same product, you’re going to have two sets of metadata for the same product with different uuids. If, on the other hand, the uuids are generated from some easily derived source, then both sets of user generated data should have the same uuid (since it’s exactly the same product, they should have the same uuid). The product URL isn’t by any means a perfect solution, and I’m wide open to suggestions (within the limits of the uuid generator library — I did think of faking a URL with artist name and product name, but neither is guaranteed stable between stores).

Profile
 
 
Posted: 17 July 2012 07:32 AM   [ Ignore ]   [ # 7 ]
Power Member
Avatar
RankRankRank
Total Posts:  1829
Joined  2006-02-17

In a centralized service, one would have to rely on the creator primarily to register the products separately.  The creator knows if the items are different or not.  In the case where the creator is no longer available (for whatever reason) then it would probably have to be a “first register” gets the UUID and subsequent register requests get the same return.


In the case where the creator is “Poser Only” then the item would probably need to remain out of the CMS, or it is possible to assign “impossible” UUIDs for local use (i.e.  000000-xxx-...) and these UUIDs wouldn’t propagate to other systems.  If in a situation where the CMS is asked to register a 000000 UUID (as opposed to creating one itself) it would replace the external UUID and generate a local 000000 UUID as a substitute.


As long as there is no charge to get a UUID, I can’t see that creators wouldn’t want to take advantage of the opportunity.  The biggest hurdlea are educating the Content Creator and making a process where creating the metadata isn’t a PITA in the first place.


Kendall

Aelfric - 17 July 2012 06:59 AM
rbtwhiz - 16 July 2012 09:14 AM

The product page URL is one of those details I mentioned that can change. Not just within the same domain, but change domains as well. It happens often. The page I linked to states that the value is random - QUuid::createUuid(). This is intentional. The idea being that the owner of a product (or someone acting on their behalf) would define the value once at creation and then register/publish that value - ideally through a centralized service, but at least through a centralized location on a given store.

IETF RFC 4122 provides for five different mechanisms for generating uuids:

•  MAC address (time based);
•  DCE Security;
•  MD5 hash (name based);
•  random; and
•  SHA-1 hash (name based).

If the page you linked to states that all uuids are random, this would be a limitation of the Qt uuid library.


A centralised service sounds wonderful in theory, but…

As the vendor for the only software product that uses this metadata, the central repository would need to be something DAZ provided, and I haven’t noticed anyone in Utah rushing to volunteer.

The second issue would be whether the central service could reasonably vet requests for products sold at other stores. Could someone reasonably be expected to know that Merlin’s Medieval Swords sold at CP was a different product to Merlin’s Medieval Swords at RMP (and is actually the same product known as Merlin’s Medieval Swords II at RMP). Do they simply say as long as the combination of store name and product name is unique, issue a uuid?

Another issue would be products where the creator is Poser only and has no interest in supporting DAZ Studio, is no longer in 3D art, or indeed in some cases sadly no longer alive—who creates the uuid on their behalf? The metadata for these products will be generated by end users who want to bring these products under the CMS, but non of it will be officially sanctioned by the content creator. How will the centralised authority handle requests of that nature?

The store that would gain most from running a centralised service is DAZ, and I don’t see the cost/benefit being favourable.


Conversely, each store running their own service isn’t going to work either. Unless Smith Micro decides to hook into the CMS in Poser, it would be counter productive for them to host a CMS related service. Equally, I don’t see small, Poser-oriented vendors wanting to do the work either — if they don’t even test in Studio, or include Studio materials, they’re not going to provide metadata. Again, it will be the end users who produce this metadata.


If uuids are generated randomly, and two users produce and post metadata for the same product, you’re going to have two sets of metadata for the same product with different uuids. If, on the other hand, the uuids are generated from some easily derived source, then both sets of user generated data should have the same uuid (since it’s exactly the same product, they should have the same uuid). The product URL isn’t by any means a perfect solution, and I’m wide open to suggestions (within the limits of the uuid generator library — I did think of faking a URL with artist name and product name, but neither is guaranteed stable between stores).

 Signature 

Any opinions expressed in this post are those of Kendall Sears and may, or may not, be more, or less, valid than any other opinion.

The contents of this post are intended for the DAZ forum only, do not re-post any portion to any other forum without his permission.

Profile
 
 
Posted: 17 July 2012 10:18 AM   [ Ignore ]   [ # 8 ]
Active Member
RankRank
Total Posts:  901
Joined  2009-12-07
Kendall Sears - 16 July 2012 12:40 PM

The ideal solution would be for DAZ to have a UUID request system where a UUID could be requested for freebies or products and avoid the possibility of conflicts.
Kendall


I like the idea, but please forgive me I can’t help but picture it with a smile on my face wink ...

1. Login. Get logged out.
2. Login again. See a different person’s name on the welcome page.
3. Fill out (entire) UUID request form and click submit.
4. Website hangs.
5. Restart browser.
6. Login again.
7. Can’t find UUID request form.
8. Google “DAZ UUID request form”.
9. Follow first link which turns out to be to old website so it doesn’t work.
10. Manually enter the correct URL and find form.
11. Fill out entire form again and click submit.
12. Website tells you the price has changed for an item you didn’t order. In fact you weren’t even shopping!
13. Submit a support ticket.
15. Wait.
16. Get kicked out of Platinum Club.
17. Buy V6.
18. Get V6 for free.
19. Get UUID.

That was 5 minutes out of my life, but boy it made me laff!

 Signature 

PoseMaster for DAZ Studio and PoseMaster People Pack now sold at Rendo.

Profile
 
 
Posted: 17 July 2012 11:02 AM   [ Ignore ]   [ # 9 ]
Power Member
Avatar
RankRankRank
Total Posts:  1829
Joined  2006-02-17
cridgit - 17 July 2012 10:18 AM
Kendall Sears - 16 July 2012 12:40 PM

The ideal solution would be for DAZ to have a UUID request system where a UUID could be requested for freebies or products and avoid the possibility of conflicts.
Kendall


I like the idea, but please forgive me I can’t help but picture it with a smile on my face wink ...

1. Login. Get logged out.
2. Login again. See a different person’s name on the welcome page.
3. Fill out (entire) UUID request form and click submit.
4. Website hangs.
5. Restart browser.
6. Login again.
7. Can’t find UUID request form.
8. Google “DAZ UUID request form”.
9. Follow first link which turns out to be to old website so it doesn’t work.
10. Manually enter the correct URL and find form.
11. Fill out entire form again and click submit.
12. Website tells you the price has changed for an item you didn’t order. In fact you weren’t even shopping!
13. Submit a support ticket.
15. Wait.
16. Get kicked out of Platinum Club.
17. Buy V6.
18. Get V6 for free.
19. Get UUID.

That was 5 minutes out of my life, but boy it made me laff!


Wow.  Just wow.


On a more serious note, I’d think that one would need two stages at most.


Stage 1:  Apply for a vendor number (assuming DAZ doesn’t assign a “freebie number”, or some such)
Return:  Number to use in Stage 2


Stage 2:  3 entry form—a) vendor number b) item type   c) Item name
Return: Assuming no conflict, UUID is issued.  If conflict, request necessary change.


Kendall

 Signature 

Any opinions expressed in this post are those of Kendall Sears and may, or may not, be more, or less, valid than any other opinion.

The contents of this post are intended for the DAZ forum only, do not re-post any portion to any other forum without his permission.

Profile
 
 
Posted: 18 July 2012 03:27 AM   [ Ignore ]   [ # 10 ]
Member
Avatar
Rank
Total Posts:  60
Joined  2008-03-27
cridgit - 17 July 2012 10:18 AM


I like the idea, but please forgive me I can’t help but picture it with a smile on my face wink ...

1. Login. Get logged out.
2. Login again. See a different person’s name on the welcome page.
3. Fill out (entire) UUID request form and click submit.
<snip>
17. Buy V6.
18. Get V6 for free.
19. Get UUID.

That was 5 minutes out of my life, but boy it made me laff!


I know that was just the Reader’s Digest condensed version, ’coz you left out quite a few steps like:

18.1 Wait a week while a support ticket is processed to take your free purchase order out of pending status.

I’ve had a rough day at work, so I appreciate the laugh.

Kendall Sears - 17 July 2012 07:32 AM

In a centralized service, one would have to rely on the creator primarily to register the products separately.  The creator knows if the items are different or not.  In the case where the creator is no longer available (for whatever reason) then it would probably have to be a “first register” gets the UUID and subsequent register requests get the same return.

In the case where the creator is “Poser Only” then the item would probably need to remain out of the CMS, or it is possible to assign “impossible” UUIDs for local use (i.e.  000000-xxx-...) and these UUIDs wouldn’t propagate to other systems.  If in a situation where the CMS is asked to register a 000000 UUID (as opposed to creating one itself) it would replace the external UUID and generate a local 000000 UUID as a substitute.

As long as there is no charge to get a UUID, I can’t see that creators wouldn’t want to take advantage of the opportunity.  The biggest hurdlea are educating the Content Creator and making a process where creating the metadata isn’t a PITA in the first place.


There’s no charge for making D|S materials, but there are still a lot of products being released to the market that only have Poser materials, despite a fair, and I suspect growing, level of resistance to such products from D|S users. Some content creators just aren’t interested in the D|S market.

Profile