Expiring Gallery Links - Major issue for how we use the forums and DAZ Deals

2»

Comments

  • kprkpr Posts: 285
    edited April 13

    It's because you're not meant to use a CDN URL as a link.

    You're meant to use the direct link to the primary URL (like, daz.com/gallery/.../.../ main.jpg) and then the CD-Network sorts out which node will serve a particular user the resource.

    In brief: The way we previously linked to gallery pages/images was not correct IF that was a CDN... I suspect it wasn't a CDN, just a sub-domain pointed at servers not on the Daz network <-- If that's so, that's why "it worked". The new Google thing is different...

    CDN's are replication systems that - for a fee - copy resources as they are requested and store them on the closest distributed-content-node to the user. The system caches that resource in that geographic location for a little while - so when other users in that geo-location request the primary resource, they instead get the "local copy" (until it expires, and then rinse and repeat). This takes bandwidth and resource workload off the primary site by displaying the copy when a user asks for the original. Here's a very informative page and helpful diagram: https://www.synopi.com/cdn-content-delivery-network/

    The Google CDN has very temporary Universal Resource Locators (good for a user session in average time-on-page surfing - last time I looked, Amazon's AWS worked the same way) - Because sites update images and pages often, the "local copy" is intended to be temporary. Hence CDNs are meant to be used for high-traffic resources (but many, many sites use them for all resources).

    AFAIK the website owner (Daz) can set the time-to-refresh on the CDN... It's currently 15 minutes... an unnecessarily short time span for gallery images as they are infrequently updated.

    But asking for "long refresh times" isn't what should happen... It'll still be "broke" if that gets done. 

    Instead: What there isn't on the Daz Gallery (and AFAIK, never has been), is what you usually see in such systems: "Direct Link".(either to the page and/or the image itself) possibly because Daz thinks the CDN won't function correctly and they'll get hit with the bandwidth and resource tax - If the CDN is correctly implemented, that shouldn't happen.

    To fix old links Daz would need to re-enable the sub-domain to point at whatever storeage-location the images are now held and/or maybe write some code that amended forum pages to a new URL (see @Elor one comment up)

    Post edited by kpr on
  • xyer0xyer0 Posts: 6,321

    Don't know if this has been mentioned yet, but the thumbnails of Daz3D products used on gallery posts are all swirly wheels now.

  • LeanaLeana Posts: 12,735

    xyer0 said:

    Don't know if this has been mentioned yet, but the thumbnails of Daz3D products used on gallery posts are all swirly wheels now.

    The whole store was just swirlies up to a few minutes ago for me. Seems to work again now.

  • csaacsaa Posts: 930

    WendyLuvsCatz said:

    I may be mistaken but it could be so Ai bots cannot scrape images for training

    WendyLuvsCatz,

    One clever CDN provider came up with a way to outsmart the AI scrapers. Below is the blub at Slashdot about this. Here is the link from Cloudflare itself about AI Labyrinth.

    Cheers!

     

    'Web infrastructure provider Cloudflare unveiled "AI Labyrinth" this week, a feature designed to thwart unauthorized AI data scraping by feeding bots realistic but irrelevant content instead of blocking them outright. The system lures crawlers into a "maze" of AI-generated pages containing neutral scientific information, deliberately wasting computing resources of those attempting to collect training data for language models without permission.

    '"When we detect unauthorized crawling, rather than blocking the request, we will link to a series of AI-generated pages that are convincing enough to entice a crawler to traverse them," Cloudflare explained. The company reports AI crawlers generate over 50 billion requests to their network daily, comprising nearly 1% of all web traffic they process. The feature is available to all Cloudflare customers, including those on free plans. This approach marks a shift from traditional protection methods, as Cloudflare claims blocking bots sometimes alerts operators they've been detected. The false links contain meta directives to prevent search engine indexing while remaining attractive to data-scraping bots.'

  • ElorElor Posts: 3,128

    I don't know if it's related, but I have pictures who lost a like a couple of days ago, got it back during the week-end alongside other pictures than won one and afaik, they all lost it again today.

  • Matt_CastleMatt_Castle Posts: 3,003

    This has been about two weeks now. Can we actually expect any resolution to this having broken many many thousands of links across the forums?

  • tsroemitsroemi Posts: 3,451

    csaa said:

    WendyLuvsCatz said:

    I may be mistaken but it could be so Ai bots cannot scrape images for training

    WendyLuvsCatz,

    One clever CDN provider came up with a way to outsmart the AI scrapers. Below is the blub at Slashdot about this. Here is the link from Cloudflare itself about AI Labyrinth.

    Cheers!

     

    'Web infrastructure provider Cloudflare unveiled "AI Labyrinth" this week, a feature designed to thwart unauthorized AI data scraping by feeding bots realistic but irrelevant content instead of blocking them outright. The system lures crawlers into a "maze" of AI-generated pages containing neutral scientific information, deliberately wasting computing resources of those attempting to collect training data for language models without permission.

    '"When we detect unauthorized crawling, rather than blocking the request, we will link to a series of AI-generated pages that are convincing enough to entice a crawler to traverse them," Cloudflare explained. The company reports AI crawlers generate over 50 billion requests to their network daily, comprising nearly 1% of all web traffic they process. The feature is available to all Cloudflare customers, including those on free plans. This approach marks a shift from traditional protection methods, as Cloudflare claims blocking bots sometimes alerts operators they've been detected. The false links contain meta directives to prevent search engine indexing while remaining attractive to data-scraping bots.'

    I know this post was a while ago, but I just saw it and, OMG, this is INSANE! The resources that need to be wasted just to keep the crawlers at bay! Like in so, SO many other areas, THIS is why we can't have anything nice ... - Thanks for putting up the explanation, I never knew it was so bad already. 

  • csaacsaa Posts: 930

    tsroemi said:

    csaa said:

    One clever CDN provider came up with a way to outsmart the AI scrapers. Below is the blub at Slashdot about this. Here is the link from Cloudflare itself about AI Labyrinth.

    I know this post was a while ago, but I just saw it and, OMG, this is INSANE! The resources that need to be wasted just to keep the crawlers at bay! Like in so, SO many other areas, THIS is why we can't have anything nice ... - Thanks for putting up the explanation, I never knew it was so bad already. 

    tsroemi,

    Fascinating, right? It reminds me of the "Spy vs Spy" comic strips in those old Mad Magazine.  He's a section lifted from that Cloudflare announcement explaining their design logic:

    The false links contain appropriate meta directives to prevent search engine indexing while remaining attractive to data-scraping bots.

    "No real human would go four links deep into a maze of AI-generated nonsense," Cloudflare explains. "Any visitor that does is very likely to be a bot, so this gives us a brand-new tool to identify and fingerprint bad bots."

    This identification feeds into a machine-learning feedback loop—data gathered from AI Labyrinth is used to continuously enhance bot detection across Cloudflare's network, improving customer protection over time.

    It certainly adds depth to the so-called "agentic AI" systems being rolled out. Gatekeeper AI interacting with crawler AI to trap the later in a labyrinth -- something to think about as those tech stocks reach eye-watering valuations. indecision

    Cheers!

  • Matt_CastleMatt_Castle Posts: 3,003
    edited May 12

    We're more than a month on now.

    At this point, can we please just have a straight answer? Whether that's that Daz does have the intention of trying to fix the issue, or that they've concluded that it's not feasible/worth their time/permissible within the tenets of the cult of the Dark Despoiler (delete as applicable) and that they plan to leave the forum as a hollow husk with thousands of broken image embeds.

    Post edited by Matt_Castle on
Sign In or Register to comment.