Create normal map from mesh with reduced-resolution mesh subtracted?

YofielYofiel Posts: 204
edited December 1969 in Carrara Discussion

HI folks, I'm still pretty new at this, so maybe I don't say it all correctly. I made a grayscale hieght map from NASA topographic data and imported it into Carrara as a terrain. But the dataset is so huge it brought my computer to its knees.So instead, I first pulled up PhotosShop and made a normal map from the high-res satellite grayscale data.. Nvidia's PhotoShop plugin for making normal maps went into lala land, so I installed xnormal, which worked a little clumsily, but it worked.

Then I exported the terrain from Carrara as an .obj file and mported it back in as a vertex object. I opened the vertex mesh in the Carrara editor. I decimated it and untrangulated it very carefully, as each mouse click took at least 10 mminutes. Finally I got the mesh down to a reasonable size, so now I have a lo-res mesh from the same data, and it's really smooth:)

So then I imported the normal map I'd made in Photoshop from the high res data and put it on the low-res mesh as a bump map. Now the low-res mesh has high-res detail. BUT it''s overemphasized and jagged. What I need to do is 'subtract' the low-res terrain from the high-res terrain and make a normal map from 'height difference.' This must be something other folks do for other data, and itg should be easier with a terrain mesh, which at least has a flat bottom and exactly the same bottom corners.

Is ther a way to subtract one mesh from another and get the difference data in Carrrara?

Comments

  • ManStanManStan Posts: 0
    edited December 1969

    Did you select "normal" rather then bump when you import the normal? They don't interpose. A bump map's lowest point is 0, all bump is added to that. A normal map has a mid point of 0 with low and high being +-. None the less there is a slider for adjusting how intense the bump is.

  • 3DAGE3DAGE Posts: 3,311
    edited December 1969

    HI emeyer :)

    So then I imported the normal map I’d made in Photoshop from the high res data and put it on the low-res mesh as a bump map. Now the low-res mesh has high-res detail. BUT it’‘s overemphasized and jagged.

    A Normal map, or Bump map,. creates a lighting effect on the surface, to make it look rough, but it doesn't alter the geometry of the model.

    What I need to do is ‘subtract’ the low-res terrain from the high-res terrain and make a normal map from ‘height difference.’

    No,. that may not do what you imagine. since the geometry isn't being changed by the normal map.

    Pictures help a lot,. they show us, what you're seeing, and doing, and help explain something which can sometimes be hard to explain.

    If the Hi-res Normal map , made in photoshop, is over emphasised, then you can reduce the "bump" effect setting in the shader,.or reduce the brightness of the normal map in the shader, either method will reduce the perceived bump effect

    Traditionally,. you would have a low polygon "base" object, then you'd add several levels of subdivision smoothing, which allow you to create finer surface details, to export the "High-res" normal map. but because subdivision smoothing can be "virtually" added, this allows you to go remove the sub-division smoothing and return to the original low polygon base model.

    Is ther a way to subtract one mesh from another and get the difference data in Carrrara?

    Yes,. most 3D programs (which handle modelling) have a "boolean" function, where one object can be added to, or subtracted from another, or the intersecting shape between both can be left,. or subtracted.

    It can leave you with a mess of untidy vertices and edges which may need some care.

    but, Boolean is used to cut one solid object from another,. like a sphere from a cube,.
    What you're thinking of doing would be cutting one deformed "Plane" from another deformed "Plane".

    Plane's don't have any depth ..unlike a cube,. so boolean "cutting" a sphere from a plane, with give you a plane with a hole.
    it's not easy to perform a boolean cut using two deformed planes, and get what you imagined the results would be..

    I think adjusting the normal map levels and perhaps the scene lighting wil give you the effect you're looking for.

    Hope it helps :)

  • YofielYofiel Posts: 204
    edited February 2013

    Well maybe I can explain a little more what puzzles me.. Just looking at a cross section of the mesh: suppose there is a small, flat island, ten feet above sea level. So the cross section of the low-res mesh has pixel points at 0ft, 10ft, 0ft.

    The low-res mesh was made from the high-res mesh. which has pixels representing altitudes 0ft, 5ft, 10ft, 5ft, 0ft. When I make the normal map from the high-res mesh, the normal map contains tangential data representing heights of 0ft, 5ft, 10ft, 5ft, 0ft. Typically, this would be a applied to a flat surface, such as a plane, to create an apparent height of 0ft, 5ft, 10ft, 5ft, 0ft on the flat surface at render time.

    But I want to put the normal map on the low-res mesh rather than a flat surface, so I can replicate trees, draw rivers and roads, and so on, on the low-res mesh surface. But when I put the normal map on the low-res mesh, I get apparent heights of 0ft, 5ft, ***20ft***, 5ft, 0ft, right?

    Capture.PNG
    467 x 338 - 10K
    Post edited by Yofiel on
  • Design AcrobatDesign Acrobat Posts: 459
    edited February 2013

    I noticed that the OP said he untriangulated and decimated the mesh.

    A normal map or a bump map can't put in large displaced detail. To do displace a mesh needs displacement map. What I mean is if the mesh has very deep valleys and high mountains, they will not be displaced with a normal or a bump map.

    See the sphere on the right with displaced geometry. Notice the low poly sphere on the left with a normal map - there is no displacement.


    What I'm saying if you start with a displacement mesh with a map, then untriangulate and decimate it, you will not get the results you expect.

    low_and_high.jpg
    605 x 412 - 47K
    Post edited by Design Acrobat on
  • YofielYofiel Posts: 204
    edited February 2013

    Well yes, but in this case the mesh is terrain sized. I'd want to be inside that valley looking up at the mountains. In the low res mesh, I just removed coplanar surfaces and halved the number of points. I need the low-res mesh so, in the valleys one can look up and see the mountains. Hence I'm trying to make a bump map (well actually a normal map to better represent data at tangential angles) that adds more detail to the low-res mesh from the original high-res mesh.

    Post edited by Yofiel on
  • YofielYofiel Posts: 204
    edited February 2013

    ...then I'm overlaying a tile from the original high-res mesh where I'm zooming in from orbit...this shows what I'm doing....

    When I add the normal map, the lower big map appears the same height as the little more detailed map, but the height appearance has to be exact, or it won't look right. And that's where I'm having problems )

    meshexample.jpg
    1024 x 698 - 169K
    Post edited by Yofiel on
  • Design AcrobatDesign Acrobat Posts: 459
    edited February 2013

    emeyer said:
    Well yes, but in this case the mesh is terrain sized. I'd want to be inside that valley looking up at the mountains. In the low res mesh, I just removed coplanar surfaces and halved the number of points. I need the low-res mesh so, in the valleys one can look up and see the mountains. Hence I'm trying to make a bump map (well actually a normal map to better represent data at tangential angles) that adds more detail to the low-res mesh from the original high-res mesh.

    Actually Bryce can import terrain maps from geo servers as I recall.

    But, one can similate the same thing by converting the bump map in PShop to (black and white under the Layers function) and then export as a tiff map.

    You can get results like what is seen below.
    1. Tiff map
    2. Vertex Map (after smoothing)
    Then, to make areas where you want trees, grasses, place buildings, you use PShop to make layers where the grasses, trees etc should be, make the other layers invisible, then use the surface replicator to insert the maps for trees, grass ets in the surface replicator.

    If this is not your intentions, then I apologize, but this saves a lot of time and is more accurate if you start out with the terrain editor and a good map.

    terrain_vertex_modeler.jpg
    640 x 373 - 79K
    terrain_map_small.jpg
    640 x 402 - 58K
    Post edited by Design Acrobat on
  • YofielYofiel Posts: 204
    edited December 1969

    Yes, that's exactly what I'm trying to do, and I find the terrain editor in Carrara better than in Bryce, maybe it's just because I've become more familiar with its UI. But if I draw all the data at high res, it brings my computer to its knees, because as well as the terrains, I have a model of ancient Athens imported from Google Earth (due to the amazing Collada format), with V4 flirting with Michael. So altogether it's just too much data, and that's why I'm trying to merge meshes of different resolutions.

    meshexample2.jpg
    960 x 547 - 192K
  • Design AcrobatDesign Acrobat Posts: 459
    edited February 2013

    emeyer said:
    Yes, that's exactly what I'm trying to do, and I find the terrain editor in Carrara better than in Bryce, maybe it's just because I've become more familiar with its UI. But if I draw all the data at high res, it brings my computer to its knees, because as well as the terrains, I have a model of ancient Athens imported from Google Earth (due to the amazing Collada format), with V4 flirting with Michael. So altogether it's just too much data, and that's why I'm trying to merge meshes of different resolutions.

    Yeah, that's a lot of data for a regular pc to handle.

    If you are comfortable with the terrain editor in Carrara, then I suggest make representational maps rather than digitally accurate maps. That is, get the look and feel of the map, but then make overlay maps with the color, placement of trees, grass and objects with maps in photoshop, paint it to accuracy (not precision) and the make other layers invisible, export in a jpg or psd for carrara's use.

    This way, you can precisely place your buildings and other objects using maps for the surface replicator. Each type of object would probably require its own map for place. One for buildings, one for trees and one for flora, then adjust. It sounds like quite a project.

    The infinite Skills series available in the store (paid tutorial) gives a very good explanation if this this process of making your own scenes with accuracy (not precision).

    For precision, you're gonna need a super computer with dedicated software like Google Earth has. heh

    or use Google Sketchup which is another purchase.

    But, it can be done in Carrara with some effort and you are satisfied with accuracy, rather than precision.

    Post edited by Design Acrobat on
  • YofielYofiel Posts: 204
    edited December 1969

    Also I could add, I expect the mip mapping in 8.5 will help with moire, which does appear in texture maps of this size. But with the dataset size I'm not playing in the beta yet. At the moment I'm just trying to get accurate reprentation of height. Am I mistaken in thinking the normal map is exaggerating peaks, because I am drawing it on a low-res mesh of the same height data?

  • Design AcrobatDesign Acrobat Posts: 459
    edited December 1969

    emeyer said:
    Also I could add, I expect the mip mapping in 8.5 will help with moire, which does appear in texture maps of this size. But with the dataset size I'm not playing in the beta yet. At the moment I'm just trying to get accurate reprentation of height. Am I mistaken in thinking the normal map is exaggerating peaks, because I am drawing it on a low-res mesh of the same height data?

    I've never used normal maps for precision. You can get accuracy, but not precision.

    The only method I've used is make a map from a TIFF, importing that into the terrain editor.
    Caution: Once smoothing is applied to get rid of the jaggies (for purposes of anti-aliasing) you will get 'not so precise' details.

    The texture artists here probably can address the moire problem. Of course, the higher resolution map you make, the better the render, but it will also bog down your system. One can do 4K x 4K maps in Carrara without too much difficulty.

  • YofielYofiel Posts: 204
    edited December 1969

    If you are comfortable with the terrain editor in Carrara, then I suggest make representational maps rather than digitally accurate maps. That is, get the look and feel of the map, but then make overlay maps with the color, placement of trees, grass and objects with maps in photoshop, paint it to accuracy (not precision) and the make other layers invisible, export in a jpg or psd for carrara's use.

    Aha. Thank you yes, that is a way around the problem, but in this case it doesn't work fo rthe opening scene, as I am animating a zoom and flying in, like in Vista Pro flight control....What I plan to do is invisibly morph V4 and M4 from humans into giant Gods in the sky, and zoom from them in their God morph in the sky to human-scaled figures on the ground.

    What did occur to me is another workaround: there is some way of substituing low-res for high-res data via LOD controls depending on distance, so perhaps I could use the original high res meshes without decimation, then at low res, any loss in precision due to normal maps would not matter, and I substitute in the higher res data and use 2d backdrops for the ground scenes. But I'm not sure how to set it up...have you tried that?

  • Design AcrobatDesign Acrobat Posts: 459
    edited February 2013

    emeyer said:
    If you are comfortable with the terrain editor in Carrara, then I suggest make representational maps rather than digitally accurate maps. That is, get the look and feel of the map, but then make overlay maps with the color, placement of trees, grass and objects with maps in photoshop, paint it to accuracy (not precision) and the make other layers invisible, export in a jpg or psd for carrara's use.

    Aha. Thank you yes, that is a way around the problem, but in this case it doesn't work fo rthe opening scene, as I am animating a zoom and flying in, like in Vista Pro flight control....What I plan to do is invisibly morph V4 and M4 from humans into giant Gods in the sky, and zoom from them in their God morph in the sky to human-scaled figures on the ground.

    What did occur to me is another workaround: there is some way of substituing low-res for high-res data via LOD controls depending on distance, so perhaps I could use the original high res meshes without decimation, then at low res, any loss in precision due to normal maps would not matter, and I substitute in the higher res data and use 2d backdrops for the ground scenes. But I'm not sure how to set it up...have you tried that?

    In the animation and working aspect of your project use target helpers as proxies for your meshes. Make your meshes invisible and use the target helpers centered on the pivot points of the objects. This will help with the 'fly arounds'.

    Post edited by Design Acrobat on
  • YofielYofiel Posts: 204
    edited February 2013

    Thank you. That could keep me going a bit longer before I get stuck again. Also I'm thinking Inagoni's SWAP could help. I could assemble in low resolution and then swap in the high-res map for render, it works, it's just REALLY SLOW!

    ...One person suggested I should use Max instead. If I were going for another app for this, I guess it would be Vue Infinite. But actually, I don't think the software makes much difference with datasets this size. More memory and a power graphics could help, but I'm reluctant to upgrade my Nvidia 470, it's been so reliable unlike most other dozen graphics cards over the decades, I really hesitate to upgrade.

    Post edited by Yofiel on
  • Design AcrobatDesign Acrobat Posts: 459
    edited December 1969

    emeyer said:
    Thank you. That could keep me going a bit longer before I get stuck again. Also I'm thinking Inagoni's SWAP could help. I could assemble in low resolution and then swap in the high-res map for render, it works, it's just REALLY SLOW!

    ...One person suggested I should use Max instead. If I were going for another app for this, I guess it would be Vue Infinite. But actually, I don't think the software makes much difference with datasets this size. More memory and a power graphics could help, but I'm reluctant to upgrade my Nvidia 470, it's been so reliable unlike most other dozen graphics cards over the decades, I really hesitate to upgrade.

    Vue Infinite would be better for fly arounds imo. But go with what you have I say.

  • YofielYofiel Posts: 204
    edited December 1969

    I think I kind of decided to stay on Carrara for at least a year. The web states an Evga 670 2gb would be twice as fast for slightly less power consumption than what I have, a Evga 470 with 1gb. Currently the price on that one is about $420 including tax, and it doesn't appear it's giong to depreciate significantly for quiote a while, although I could be wrong, I think TSMC's 28nm process is likely to remain very expensive.

    But I'm not convinced doubling the speed or adding more graphics memory would actually make that much difference.

  • Design AcrobatDesign Acrobat Posts: 459
    edited December 1969

    emeyer said:
    I think I kind of decided to stay on Carrara for at least a year. The web states an Evga 670 2gb would be twice as fast for slightly less power consumption than what I have, a Evga 470 with 1gb. Currently the price on that one is about $420 including tax, and it doesn't appear it's giong to depreciate significantly for quiote a while, although I could be wrong, I think TSMC's 28nm process is likely to remain very expensive.

    But I'm not convinced doubling the speed or adding more graphics memory would actually make that much difference.

    Only CPUs will help you in Carrara in total performance not GPUs.

    But, GPUs will help when visualizing your work product in Open GL as the processing power in the GPU can keep up.

  • YofielYofiel Posts: 204
    edited December 1969

    True. Sometimes the GPU doesn't make that much difference for OpenGL either, although they're alot better than they used to be...

    [long puase while I reinstall DAZ content in Studio with the new install manager that just appeared.... .... .... .....another improvement :}

  • 3DAGE3DAGE Posts: 3,311
    edited December 1969

    HI Emeyer :)

    If you're tying to do a fly in from orbit to the surface of a planet,. and you're using a single scene, or a single mesh object,.... forget it.
    you won't possibly have the resolution in a single scene at ground level which you want.. or too much resolution to make working practical.

    Go and look at how those Fly-in's from space (using 2D satellite images) are actually created,.

    It's a series of images,. each one focused on a different size of the same overall area,. and so a greater perceived level of detail,. each one is "faded" into the last image in post production. ...they also use layers of Cloud or fog to obscure the transitions.

    You could create the images you need for the effect within carrara, by rendering a series of 2D images, and using post production, (or the same process, applied within Carrara) to create the fly-in sequence in a video editor.
    This may save you a lot of hassle.

    Also,. you mention that the terrain seems distorted when you add the Normal map.
    The Normal, or Bump,. cannot change the physical geometry,. they are used to create a Shadow and Lighting effect.

    Only a "Displacement" map will change the original geometry.

    BUT... if you're using a terrain,. then there are two settings for the mesh which will effect the rendered surface height.
    the "Preview Quality" settings, and the "Final Quality" settings.

    Make sure that both of those settings are the same in the terrain editor,. then there should be no visible change in the height of the terrain in the Scene preview, and in the final rendered image.

    Your normal map will give you the illusion of finer detail on a low poly mesh, as will displacement and bump,. but none of those things will allow you to travel from ground level to space, and observe the ground details diminishing as you move away, or increasing as you get closer. going the opposite way.

    for example, you start out in space,. looking at continents,. then as you zoom in, you'll see more detail, mountain ranges, lakes, etc,.. but it's only at a few thousand feet from the ground, that you begin to see details such as a forest, roads, individual buildings etc... but in a 3D program, you would be calculating the lighting on each leaf on each tree, all the way from orbit.
    so, it makes sense not to have all of those details in a single scene.

    but rather, to create a set of scenes which you can use to render out a few images from each scene,. with each image a closer view of your subject,. and each Scene,. focused on a smaller "scene area", with higher detail in each scene.

    so instead of using the entire (high res) image map on a single terrain ,. you'd create a detailed terrain from a smaller section of your map,

    Hope that makes sense :)

  • DartanbeckDartanbeck Posts: 21,219
    edited December 1969

    emeyer said:

    What did occur to me is another workaround: there is some way of substituing low-res for high-res data via LOD controls depending on distance, so perhaps I could use the original high res meshes without decimation, then at low res, any loss in precision due to normal maps would not matter, and I substitute in the higher res data and use 2d backdrops for the ground scenes. But I'm not sure how to set it up...have you tried that?
    Carrara will render a high res figure really speedily if it's far away. I think it performs a bit of LOD without any special setup. Pretty sure.
Sign In or Register to comment.