UberEnvironment2 - Can Anybody Explain The Result Of This Simple Test? [UE2 problems/fixes]

12357

Comments

  • HoroHoro Posts: 10,117
    edited December 1969

    I start asking myself whether we are not chasing ghosts here (not you Spooky). I've been doing a bit of reading books on the subject lately and discovered that programs like Maya, 3ds Max, XSL and Cinema 4D, among others, seem to have IBL implemented like UE2 has. They use a diffuse convolved HDRI for the environment light, a moderately high res HDRI as reflection map and a hig-res LDRI to show the environment.

    Now this is what omnifreaker suggests on his website for UE2. Use a specular convolved HDRI with a low Phong exponent for the light. Such an HDRI cannot produce strong direction light and this is exactly what it must not do because the sampling alg used gets confused. This method gives nice GI light.

    A few other programs like LightGen or Bryce use a different approach to extract the lights: they create a rig of point light sources, usually 256. These distribute the lights according to their intensities on a sphere and they give true directional light - at the expense of banded shadows, even with 4096 lights as Bryce can do. There are methods to (motion) blur them by rotating the light rig. Another method is to wiggle the dome (or sphere) with the light rig. Bryce uses this method with some success. For TA (true ambience, Bryce's GI method), these point light sources can be converted to area lights, the direction effect gets somewhat blurred but still works.

    If what I've read is true and I understand it correctly, UE2 works like many IBL implementations by using a diffuse or specular convolved completely blurred HDRI for the ambient light and a LDRI backdrop. As for the reflection map, I'm not yet certain DS/UE2 can actually use an HDRI. Finally, if there is a key light in the scene like a bright window, the sun or spot lights, these have to be added by conventional light source(s). It is obvious that trying to align a backdrop with an HDRI that is not meant to be hi res and well focused is vain, as are any direction tests.

    The only real fix UE2 needs is in-depth documentation how this feature is supposed to work.

  • Takeo.KenseiTakeo.Kensei Posts: 1,303
    edited December 1969

    I don't think we're chasing ghost even if the moment is almost the good time to do it. Omnifreaker's Light is made to get Diffuse lightning and occlusion but it doesn't mean that you can't have something better. There are different GI light method. I've read about the one you describe for Bryce. It is also doable for 3Delight even if it's an old tech.
    However if we're gonna chase some ghost, who are we gonna call?

  • HoroHoro Posts: 10,117
    edited December 1969

    I like your way of thinking. Give DS another GI light method. I just think I fell into the trap of thinking UE2 must work the way I have known from Bryce due to my ignorance that there are also other methods.

  • SzarkSzark Posts: 10,634
    edited December 1969

    In terms of finding out if UE2 was misaligned Horo you weren't chasing ghosts.

    Have you guys seen this product http://www.daz3d.com/new-releases/real-light-hdr-gels-bulbs ? Where is states and I quote

    DAZ Studio's new support for .HDR format means that these HDRI files are much smaller in size than the previously needed .TIFFs!
  • HoroHoro Posts: 10,117
    edited October 2013

    I doubt that UE2 is actually misaligned. It was never meant to use a high resolution HDRI with a prominent light source. Though it can be made to work and give acceptable results. Here comes the misalignment in.

    The HDR format stores the data as RGBE, which means that for R, G and B only the mantissa is stored and for all three colours a common exponent - the one of the three colours that is highest. So, 3 single floats with 32-bit each can be saved in 4 bytes or 32 bits. Float TIFFs can't do that and they do need 96 bits. HDRs are indeed smaller in size than TIFFs, but only on the HD, not in memory.

    EDIT to add: HDR can also be uncompressed, compressed with RLE (run length encoding) and the version 2 stores the colour and exponent planes separately and puts the RLE compressor over the data. This is much more efficient then the version 1 but needs more time to extract to recombine the RGB pixel.

    Post edited by Horo on
  • SzarkSzark Posts: 10,634
    edited December 1969

    Well all the tests shown in this thread and other, plus the fix all lead to the HDRI/Tiff is misalighen as in unpside down Horo. I am not talking about the sphere either

  • HoroHoro Posts: 10,117
    edited December 1969

    Yeah, I know, I've lost a lot of time with this and the longer I experimented the more confused I got. I wonder whether it wouldn't be easier to start anew with an IBL implementation for DS than trying to bug-fix UE2 (just keep it as it is as a legacy option for compatibility reasons).

  • SzarkSzark Posts: 10,634
    edited December 1969

    That was my reason for posting a link to that product. If DS can use HDRI's natively like is shows on the 3Delight website then I would be the first to buy it.

  • millighostmillighost Posts: 261
    edited December 1969

    Horo said:
    I doubt that UE2 is actually misaligned. It was never meant to use a high resolution HDRI with a prominent light source. Though it can be made to work and give acceptable results. Here comes the misalignment in.

    The HDR format stores the data as RGBE, which means that for R, G and B only the mantissa is stored and for all three colours a common exponent - the one of the three colours that is highest. So, 3 single floats with 32-bit each can be saved in 4 bytes or 32 bits. Float TIFFs can't do that and they do need 96 bits. HDRs are indeed smaller in size than TIFFs, but only on the HD, not in memory.

    Actually, RGBE stores only an 8-bit mantissa with an 8-bit exponent. That gives a very low precision (only 8 bit), but a very high dynamic range (1:10^77), which practically nobody ever uses. If you want to save space on disk it is usually better to use TIF's logluv encoding, which also uses 32 bit, but stores the color information in CIE coordinates, so there is less color-banding if R,G, and B would require very different exponents. You can use tdlmake to create it: tdlmake -logluv -float input.hdr output.tif. Particularly useful with DS/3delight, because it does support environment maps and mipmaps, so no conversion is needed when using it with DS/3delight.

    EDIT to add: HDR can also be uncompressed, compressed with RLE (run length encoding) and the version 2 stores the colour and exponent planes separately and puts the RLE compressor over the data. This is much more efficient then the version 1 but needs more time to extract to recombine the RGB pixel.

  • HoroHoro Posts: 10,117
    edited December 1969

    Yes, Radiance HDR is not very famous for colour precision with only 16 million colours per EV (exposure value) and who needs a dynamic range of 253 EV?, it is the most widely used format, though. OpenEXR has a much higher colour fidelity with 1 billion colours per EV and the dynamic range is sufficient with 32 EV (48-bit, compression ZIP, PIZ, RLE). LovLuv gets 1 million colours (24 bit) and 33 million (32 bit) per EV and a dynamic range of 16 EV (24 bit) or 126 EV (32 bit) and compresses RLE. TIFF-float and PFM need the full 96 bit but get 4.7 x 10^21 colours per EV and there are 253 EVs. There are several programs that can convert between some formats. The free Picturenaut comes to mind, FDRTools (no LogLuv), Fhotoroom (no LogLuv), LuminanceHDR (qtpfsgui).

  • 3dcheapskate3dcheapskate Posts: 2,689
    edited November 2013

    Re: The Semi-Mythical 'Ketthrove's Fix"...

    With reference to post #117 - I've had a message that DAZ_Spooky's apparently still trying to find the link to this elusive 'ketthrove's fix'. Does one of the other contributers to this thread have the link - it was apparently mentioned in a bug report, or a post about a bug report? If so then please can you post it here. After all, i's not much use knowing that...

    " Ketthrove’s fix has been approved by our Dev that wrote Shader mixer as the best way to handle this until we get this fixed."

    ...when there's no way to find out what the darn thing actually is! ;-)

    Post edited by 3dcheapskate on
  • SzarkSzark Posts: 10,634
    edited December 1969

    do you ever get the feeling that we are flogging a dead Daz 2 horse . :)

    Will somone please be our superhero and make us a new HDRI formatted environmnent light.....pwease..

  • barbultbarbult Posts: 23,147
    edited December 1969

    Szark said:
    do you ever get the feeling that we are flogging a dead Daz 2 horse . :)

    Will somone please be our superhero and make us a new HDRI formatted environmnent light.....pwease..


    Sounds to me like a job for the DT-AoA team. They already create lights and HDRI environments.
  • SzarkSzark Posts: 10,634
    edited December 1969

    I do hope so barbult. or omnifreaker

  • 3dcheapskate3dcheapskate Posts: 2,689
    edited December 1969

    Szark said:
    do you ever get the feeling that we are flogging a dead Daz 2 horse . :)..

    Why does 'basil fawlty thrashing car' spring to mind? (Type the phrase into youtube if you're unfamiliar with it...)

  • SzarkSzark Posts: 10,634
    edited December 1969

    I am a fan of Monty Python which followed John to Fawtly Towers

  • millighostmillighost Posts: 261
    edited December 1969

    Szark said:
    do you ever get the feeling that we are flogging a dead Daz 2 horse . :)

    Will somone please be our superhero and make us a new HDRI formatted environmnent light.....pwease..


    But what is a HDRI formatted environment? Horo seems to want a light that works like Bryce's, for which it should be determinable how they work, but apart from that? There are environment maps usable in luxrender which differ from those to be used in the shader mixer nodes. Both are different from those to be used in renderman. Environment maps you find in the internet, come at least in two variations, depending on the way how they are created, i.e. are they created using a mirror probe, or by using a panoramic camera? If you look at e.g. the "HDR ProSets Yosemite" product from the store here, you might notice that the creator probably also had not a right idea of how to deal with it, because the light is simply rotated in seemingly random directions, so that the sun more or less matches the sun in the background image. The only thing that comes anywhere near a documentation for the mapping is Pixar's Renderman documentation, which also applies to 3delight (because 3delight tries to be renderman-compliant). This latter one is the one that the Uberenv2 uses. Might not be the best way to do it, but at least i know what it does.
  • SzarkSzark Posts: 10,634
    edited December 1969

    well since you put it like that anything thing that does the job well and not something that gets broken and left for years to be fixed.

    My prefernece would be one like Lux uses or a hybrid with a backgrond OFF swtich but still using the HDRi as a light source like UE2.

  • HoroHoro Posts: 10,117
    edited December 1969

    As I noted above, there is more than one way how IBL can work. Yes, I do prefer to have one single HDRI that creates proper directional light and can be used as backdrop at the same time, preferably with a tone-mapper integrated. I could make an indoor giving correct light by flipping, mirroring, put it upside down rotating it and what not. See image below, above spherical, below the tortured image that works. This method also worked for an outdoor with the sun as prominent light source. I consider this success just a lucky accident.

    Using a diffuse convolved HDRI as light source, a moderately high resolution HDRI as reflection map on a sphere primitive and a high resolution LDRI as backdrop on yet another sphere primitive works best. If there is a prominent light source, supply it with a spot light. Many well known 3D applications use this method. There are even free programs that create just that from an HDRI for several different application, automating the process (SmartIBL or sIBL, available from the Picturenaut website). If you don't need the backdrop, just skip that sphere.

    I do agree that the Yosemite set millighost refers to has not got it completely right but it is not that far off. It is, by the way, a beautiful set that also works for Carrara and I made it to work for Bryce - money good invested. It just doesn't work properly for UE2.

    Everybody going really for IBL using UE2 will get stuck by the simple fact that there is no documentation that tells the user how this feature is supposed to work. And I boldly claim that not one person at DAZ 3D does really know how UE2 works. Before any bug-fix or work-around is attempted, understanding how it is supposed to work is mandatory.

    Just because I got yet another method that appears to work correctly doesn't mean I got it right and that I know how UE2 works. It may as well be yet another lucky accident. It's all tinkering and guesswork.

    UeUeyz.jpg
    512 x 514 - 39K
  • SzarkSzark Posts: 10,634
    edited December 1969

    Has anyone looked at UE2 in 4.6.1.33. Everything looks as it should but I haven't done any test like you guys have.

  • 3dcheapskate3dcheapskate Posts: 2,689
    edited November 2013

    Latest I've got is 4.6.1.17, which has the same UE2 problem as earlier versions (to be specific - I only did a single front/top/left render and got exactly the same result as post 33 and I got exactly the same error as the first image in that post).

    P.S. Given up on getting a link to ketthrove's fix... oh well, I should have known better...

    Post edited by 3dcheapskate on
  • Takeo.KenseiTakeo.Kensei Posts: 1,303
    edited December 1969

    I didn't check either but I've seen an environment sphere in the new separate light pack. Thought it was eventually fixed but my request had no new entry so may be it is not done yet

  • SzarkSzark Posts: 10,634
    edited December 1969

    Thank guys.

    Takeo the environment sphere has been there for over a year now of I am not mistaken. ;)

  • SzarkSzark Posts: 10,634
    edited December 1969

    It would be nice to get an official update on this..... please..with cherries on top. :)

  • mjc1016mjc1016 Posts: 15,001
    edited January 2014

    A couple of things that haven't been brought up...

    DS uses Y-axis as up/down. 99.9% of the rest of the world uses Z-axis. Including 3Delight...draw your own conclusions...

    Other IBL solutions for DS do exist, already...buried in either Shader Mixer or Shader Builder. But all will probably suffer similar offset problems.

    Also, if you don't need tiling/bump/displacement, etc you don't need UV mapping...the 'visible' sphere's alignment issue can be easily solved by using an unmapped sphere. I don't think the actual sphere that contains the 'light' is mapped...in fact, it may not even have any geometry at all.

    Since millighost mentioned Luxrender's IBL...here's the Luxrender Wiki page on it...

    http://www.luxrender.net/wiki/Environment_map

    Pay close attention to the axis information on the latlong map (about halfway down the page)...

    Post edited by mjc1016 on
  • SzarkSzark Posts: 10,634
    edited December 1969

    mjc thanks for that I will have a read/ I do know, well heard it from DT that the Shader Mixer can now do native HDRI's so maybe it is time to delve in there and see what is what.

  • kitakoredazkitakoredaz Posts: 3,526
    edited December 1969

    I bought today HDR yosemite kits,
    http://www.daz3d.com/hdr-prosets-yosemite-pack-one

    and it seems add distant light with UE2, then it had been adjusted rotaiton too.
    so that I believed , if it has been correctly cast AO shadow. but no no no.
    the shadow almost caused by Distant light, and when I render the image,
    it show shadow from image hot point light.

    but if I remove the setted distant light, and test other free AO shadow catcher,
    actuall shadow caused by image hot point are perfectly different direction.

    then I test with white plane. distant light are needed to make right direction shadow ,
    but actuall IBL effect are ignored.

    though the HD image is really beautiful,
    then I can apply , the set one click easy,, but I disapointted much.

    I hope, if vendor noticed the problem, leave it off. (without add distant light opositte direction)
    as if it seems pretend , the IBL set work coreclty,,.:roll:

    Ithink,,, it is somehitng fake.

    keepshadow2.png
    734 x 950 - 489K
    fakedistantlight.JPG
    898 x 619 - 100K
  • 3dcheapskate3dcheapskate Posts: 2,689
    edited January 2015

    I haven't looked at this thread for over a year - I think my will to live failed after post #141 ! LOL But now I'm back again, hand-cranking a rusty copy of DS4.6.1.17 to take her out for a test run - soooo...

    Has UE2 been fixed yet ?

    (And if not, I wonder how many people are happily using UE2, oblivious to the problem ? But then I guess that this is one of those problems that you only really notice if you dig quite deep, eh? )


    mjc1016: I wish somebody had pointed out that Luxrender wiki page way back when ! I think the "DS Y-axis is up" along with those three mapping images could have saved a lot of brain-strain ! :D

    Post edited by 3dcheapskate on
  • HoroHoro Posts: 10,117
    edited December 1969

    Has UE2 been fixed yet ?

    Not to my knowledge. I had a contact with omnifreaker quite a while ago and it transpired that he sold this product to DAZ 3D and they are technically responsible to fix it. Additionally, I was told that 3Delight was not updated for years and there is no motivation to update the shaders until 3Delight is updated and RenderMan should be used instead.
  • ParrisParris Posts: 392

    Ok, I am very late to the party, sorry. Is anyone still looking at this thread? I didn't know about this inquiry until recently or I would have contributed what I know sooner ... which might be pretty much all the answers, I think. And some of your guesses are close or on the money. Good sleuthing!

    First, the illusive Ketthrove fix: I'm sorry but this isn't a remedy that would help you fix the problem yourself because we users don't have access to the light shader in UE2, so we can't make corrections to how it deals with coordinate space, but here it is nonetheless in shader mixer bricks.

    How do I know this is the "Ketthrove Fix"? Because I was the original poster of the thread that it came from. It was about shader mixer and how environment mapped reflections get messed up, but the same principle of wrong coordinate space applies to UE2. I popped the question, Age of Armour demonstrated the issue with a test (see below, note the position of the white car) and a possible work around, and Ketthrove graciously shared the final solution. 

    Age of Armour's Case Study

    ketthrove_fix.jpg
    1512 x 945 - 204K
    3Spheres1.jpg
    1280 x 720 - 85K
Sign In or Register to comment.