Facial animation tests with Josh Crockett's Intelligencia and Nenana

Comments

• mindsong Posts: 1,693

  December 2018

  wow. neat.

  Interesting to me, is at the start of both, I liked the general look, but felt that uncanny valley thing in *spite* of the obvious quality of the capture, processing, and rendering.

  But... as I watched it, my resistance to that 'feeling' went away, and by the end, the characters were both far more 'real' to me than when they started - not because the production changed, but because my willingness to 'get used it' did.

  That the quality of this is so high and it still has that edge screams to me how absolutely sensitive we people are to the facial cues that comprise a huge amount of our communication. I wonder if you kicked up the 'exageration' sliders to get a more toon-like effect, if it would work differently, and if, opposite real humans (or real aliens...), it would still work, or if it would detract/distract. I'm thinking a review of that same choice as used in 'men in black' might be interesting study for me.

  Thanks for the post - I *love* this stuff, and am humbled and inspired by your progress.

  I've been experimenting with Brekel v1/v2/Kinect and the fclone demo for facial work, as well as manually tweaking the mimic-pro interpretations of audio, and still haven't gotten my realistic stuff to go beyond that valley, but can get some toony stuff that's maybe useable.

  cheers, and keep us posted!

  --ms
• Bryan Steagall Posts: 233

  December 2018

  Thank you Mindsong!  I really appreciate your comments.

  This is one of the difficulties of doing facial animation, and one of the reasons why I like to work with either toon characters or ones such as Josh's that are far enough away from a human face that you can minimize that uncanny valley feel. (which is why movies like Avatar were so successful)  We are very attuned and sensitive to facial cues, and ever since I've started doing facial mocap, I feel I'm even more observant (staring hours on end at facial performances.. ha!)  Even with the highest levels of attention to detail, and extremely sophisticated modeling and rigging, such as with projects like Saya (seen as the first example in Dynamixyz's reel http://www.dynamixyz.com/our-work/ ) we are still not quite there.. (Saya is probably one of the best I've seen)

  Performer allows the operator/animator to make all kinds of choices in regards to the resulting animation, but the end result in large part depends on the sophistication of the model and rig.  I'm not a modeler or rigger, so I have to depend on characters such as what I get here, while although they are very good, they lack a lot of what I would like to have that would make the animations even better.  So you end up making the best with what you have available (not saying this is the best I can do, but close enough).   To give you an idea of what I'm taling about, Squadron 42's facial animation is probably one of the best I've seen for games, and their rigs are composed of hundreds of blendshapes and bones (as with Saya's) https://www.youtube.com/watch?v=G1eLecTsTSw  With Daz characters, I leverage both the blendshapes and bones to get what you saw here, but I'm still limited on what I can do

  I'll be posting a side-by-side video soon of the facial performance of the actress and the retargeting, and you can tell me what you think..

  Thanks!
• mindsong Posts: 1,693

  December 2018

  Bryan Steagall said:

  Thank you Mindsong!  I really appreciate your comments.

  I certainly hope they were taken as being constructive, as I am very impressed. That said, I've also become very (too?) discerning for my own standards, and assume your postings are intended to draw out real thoughts, rather than simple applause. Publishing a work with the 'blind' hope that something is good, when it's really not that good is my recurring nightmare, and I assume it is for many others who want to produce to a standard worth watching. I always feel presumptuous when I judge the works of others- work that is far better than anything I've done - but don't mean it that way, as my judgement and standards naturally outrun my abilities as a matter of course - even when I approach my own standards, they tend to move up and away again! :^) A curse and a wonderful never-ending challenge, but I am 'getting there - slowly...

  Bryan Steagall said:

  This is one of the difficulties of doing facial animation, and one of the reasons why I like to work with either toon characters or ones such as Josh's that are far enough away from a human face that you can minimize that uncanny valley feel. (which is why movies like Avatar were so successful)  We are very attuned and sensitive to facial cues, and ever since I've started doing facial mocap, I feel I'm even more observant (staring hours on end at facial performances.. ha!)  Even with the highest levels of attention to detail, and extremely sophisticated modeling and rigging, such as with projects like Saya (seen as the first example in Dynamixyz's reel http://www.dynamixyz.com/our-work/ ) we are still not quite there.. (Saya is probably one of the best I've seen)

  ditto on watching people with a *very* different eye nowadays. I truly appreciate the animation masters, who have a brilliant genius for reducing facial emotions to their essence (e.g. the incredibles - wow)..., but most of that is still hand-keyed. The facial mocap stuff I've messed with is arguably 'accurate', but still incomplete to my eye, so I think our own art, in the 'realm' we're working in, is to identify and efficiently fill those gaps with the most critical 'essense' until the tech (we can afford) catches up.

  Bryan Steagall said:

  Performer allows the operator/animator to make all kinds of choices in regards to the resulting animation, but the end result in large part depends on the sophistication of the model and rig.  I'm not a modeler or rigger, so I have to depend on characters such as what I get here, while although they are very good, they lack a lot of what I would like to have that would make the animations even better.  So you end up making the best with what you have available (not saying this is the best I can do, but close enough).   To give you an idea of what I'm taling about, Squadron 42's facial animation is probably one of the best I've seen for games, and their rigs are composed of hundreds of blendshapes and bones (as with Saya's) https://www.youtube.com/watch?v=G1eLecTsTSw  With Daz characters, I leverage both the blendshapes and bones to get what you saw here, but I'm still limited on what I can do

  watching that video, it's amazing to see the level of resource investment going into this business, so I look at what you've produced and am blown away with what you've done. Joe Pingleton is pushing that bar the same way in what I would call our local 'consumer level' realm. I suppose that's part of the appeal/puzzle for me: can I actually pull this off and even compare with the enterprise players. Looking at the improvements in still-frame renders, I believe so. Maybe someday make a living doing so... Not ready to quit my day job, heh.

  Bryan Steagall said:

  I'll be posting a side-by-side video soon of the facial performance of the actress and the retargeting, and you can tell me what you think..

  look forward to it! thanks for the links/thoughts.

  --ms

  Bryan Steagall said:

  Thanks!