Loading...
> >

Daz3D + Unity Live Capture issue with Blend Shape

OrbbOrbb Posts: 9
edited March 2024 in Unity Discussion

Hi there!

I'm using Unity Live Capture for the first time and I'm new to DAZ (v4.22 Pro).

I have imported into Unity my 3D character from DAZ using the DazToUnity Bridge (2023 v1.1). Everything works fine except with Unity Live Capture.

The head of my character is perfectly reactive to the live capture, but that's all. Eyes, mouth, facial muscles, nothing works and I think it's because of the error just bellow...

On the first screenshot, I got this error I can't solve. "The rig's SkinnedMeshRenderer components have not been assigned meshes with blend shapes, so blend shape mapping cannot be used."


Screenshot 1

For your information, my character looks like this (ignore the eyes):


Screenshots 2 & 3

And it looks like I've got everything I need to animate his face:


Screenshot 4

Thank you in advance for your help!

screenshot1.png
491 x 551 - 48K
screenshot2.png
1062 x 906 - 745K
screenshot3.png
531 x 435 - 202K
screenshot4.png
436 x 1274 - 66K
Post edited by Orbb on

Comments

  • TriCounterTriCounter Posts: 575
    edited March 2024

    Just popping in here and yes you will have an issue as the Genesis 9 setup has separated meshes for eyes. mouth etc..

    You would either need to use a tool to merge the geometry once you are in Unity.  The second option is a script to align all the blendshapes in the eyes, mouth.etc...

    If you are just playing then it wouldn't take too long to write a script (I used to have one when playing with the Tafi products, not sure where it is or how performant it was).

    If serious about using Daz/Unity I would recommend that you should merge the geometry using a tool like maybe "Mesh Baker" there is a free version which you could check out on the Unity Asset store.

    I can't say I've tested Mesh Baker (even though I own the full version) but will try and give it test for you as it's a topic that interests me but I'm short of time at moment :)

     

    Post edited by TriCounter on
  • OrbbOrbb Posts: 9

    Hi dortybassa!

    Thank you for the reply!

    I was wondering if I could send my .obj character in Blender, rig his face there and then import the fbx in Unity. Could this work?

    I will take a look at Mesh Baker! Thank you again!

  • TriCounterTriCounter Posts: 575

    I have no experience with Blender but you should be able to do it using that fine.

    I've just noticed you are using the rig and not the blendshapes.  You need to export the FACS blend shapes.

    1) When you send to Unity make sure you select "Advance Settings" and "Show FBX Dialog"

    2) when the FBX dialog pops up, click "Include Morphs" and click on the "Export Rules"

    3) add 2 rules "Genesis9.facs.ctrl" "Export"   and  "Genesis9.facs.bs" "export"

    You will then have the belanshapes causing your error but now you will have the blendshapes attached to each mesh (tear, eyelashes, eyes, mouth) and either need to sync them or merge the meshes.

    I just ran a test using Mesh Baker and it merges the meshes fine. 

    There is a problem that it still duplicates the blendshapes and would require some small edits to the scripts (so would require the paid version as that has the source code).

  • TriCounterTriCounter Posts: 575

    Just popping back to say my tests with Mesh Baker have worked really well  The Eyebrow, Eye, Mouth, Tear, Eyelash and Main Shape can all be merged into a single mesh with one copy of the FACS blendshape list in a few seconds. 

    There is also of course the way to do it without mesh baker (using a script to align the blenshapes on all 6 meshes)

    I'll run through both processes in a video if you would like?

  • OrbbOrbb Posts: 9

    Thanks again for your help dortybassa!

    I will try your solution and keep you updated!

    Yes, I would really appreciate a video! Thank you for your help!

  • TriCounterTriCounter Posts: 575

    When you have the Facs blendshapes on your mesh, the Unity Arkit sets up very easily, it automatically assigns the correct blendshapes to the face mapper component :)

    I was surprised how easily I got the G9 talking, with Mesh Baker it's a breeze.

    For seperated meshes, the face moves but wthout the eyelashes, mouth etc..it looks silly ... I've created an event script to add to each of the meshes to receive the updated blend shape value.  Now I just have to figure out when and where is the best time to push an event for them to hear..

    Off to bed now, Should get a quick tutorial up tomorrow, before I forget what I;m doing lol ;)

     

     

  • OrbbOrbb Posts: 9

    I've made a video to show you exactly what I do. I've done exactly what you said and I still can't see the "blend shapes" menu...  Maybe I made some mistakes that you'll see? 

  • TriCounterTriCounter Posts: 575

    It's my instructions being wrong!

    3) add 2 rules "Genesis9.facs_ctrl" "Export"   and  "Genesis9.facs_bs" "export"

     

    It's an underscore, sould be Genesis9.facs_ctrl and Genesis9.facs_bs

  • OrbbOrbb Posts: 9
    edited March 2024

    It's working! Thank you! Here is a short video: cheeky

    I can't wait to see your video on Mesh Baker!

    Also, any idea why the eyes of my character are white? I can't do anything in Unity to get them right!

    Post edited by Orbb on
  • TriCounterTriCounter Posts: 575
    edited March 2024

    Yes, it seems to be hardware dependant (don't get it on my main PC with Nvidia graphics but get it on my Intel laptop).

    Go to any material that uses the Daz Transparent shader (card eyebrows, left eye moisture, right eye moisture, Tear)

    Scroll all the way down for each to  "Surface Inputs" and find "Alpha Clip Threshold" which will be set to "0".  For some reason you have to slightly increase this value (I've got mine set to "0.02")

    Post edited by TriCounter on
  • OrbbOrbb Posts: 9
    edited March 2024

    dortybassa, thank you so much for your help! Seriously, without you, I would have spent so much time finding the awnsers! My wife went to work and she took the phone (we use one phone) so I can't try the live capture right now but here is a screenshot!



    I can't wait to put this voice over it:

    Post edited by Orbb on
  • TriCounterTriCounter Posts: 575
    edited March 2024

    How are you getting on?   The little script to keep blendshapes in all the meshes works well for me but Mesh Baker is problematic and doesn't want to play.

    Live Capture doesn't like what it spits out :(

    Maybe blender?  I tried to combine the meshes in Daz but it makes a mess.

    If the script is useful to you there is a link below.. it's not really prety but

    http://www.tricounter.com/BlendController.cs

    OK Found my new favourite Unity Asset, easily combine meshes, blendshapes, remove shapes etc. (Skinn Pro sells on Unity Asset Store but cheaper on Itch)

    https://cwmanley.itch.io/skinn-vertex-mapper

    You may well be able to get it to work with Mesh Baker but I'm now wondering why I never owned Skinn Pro before

     

     

    Post edited by TriCounter on
  • OrbbOrbb Posts: 9

    Hi dortybassa!

    I didn't need to use Mesh Backer or even a script to achieve what I had in mind. Here's a first attempt: 

    What do you think?

  • TriCounterTriCounter Posts: 575

    It looks great :)

    What are you using to combine the meshes (Mouth/Eye/Eyelash etc...)?

    Or do you just add all the meshes as renderers on the Face Mapper component?  Never thought about doing that until now lol

  • OrbbOrbb Posts: 9

    Here is my settings on the Face Mapper:


     

  • TriCounterTriCounter Posts: 575

    I'm so stupid it never crossed my mind to add all the meshes to the Face Mapper :)

  • OrbbOrbb Posts: 9

    Take a look, tell me what do you think: 

  • TriCounterTriCounter Posts: 575

    Really cool :)

  • ArtiniArtini Posts: 9,905

    Wow, Orbb, looks great.

     

  • OrbbOrbb Posts: 9

    Thank you guys and thank you again for your time dortybassa wink

Sign In or Register to comment.