How to use Mimic Pro for Carrara effectively Tutorials
Sci Fi Funk
Posts: 1,197
I've spent a while now learning NLA, Mimic for Carrara (long time user of the stand alone version).
At the time of making the first video I was only 2 days into it. We are now 10 days into it and I need to stop here, explain what I've learned and hand over the (new?) technique for others to take further.
PART 1 - http://www.youtube.com/watch?v=JGifaQfWGjY
Using Mimic Pro for Carrara. Setting up for an easier life. Basic tweaks. Phonemes.
PART 2 - http://www.youtube.com/watch?v=70Jx9yrRZKw
A Detailed editing session, just using MIMIC Pro.
PART 3 - http://www.youtube.com/watch?v=EpkTXREwTWk
NLA Sequencer Intro.
PART 4 - http://www.youtube.com/watch?v=CY-MvEjyzFo
NLA Sequencer Editing blocks idea
PART 5 - http://www.youtube.com/watch?v=QVQkJFFl02Y
ADSR NLA blocks with Mimic Pro.
I'd love it if people want to take this further. I was thinking of offering the blocks as a $1.99 Car File on DAZ. You can set this up yourselves, but it's very dull and takes a long time. See what you think when I've completed the series and let's take it from there.
Daz 3D is part of 
Connect
DAZ Productions, Inc.
224 S 200 W, Suite #250
Salt Lake City, UT 84101
Licensing Agreement | Terms of Service | Privacy Policy | EULA
© 2024 Daz Productions Inc. All Rights Reserved.
Comments
Hi, I have some experience using this software and can suggest the following:
Sometimes, despite your best efforts; these digital puppets WILL NOT BLINK. Not sure if the model is corrupted, tried everything; not sure why but sometimes it just happens.
I used to get excited about doing alot of expressions but over time found that they sometimes warp the morph uncomfortably. I find it best to do or start with an expression and then 0...mimic ---expression... otherwise gets odd.
As far as selling aniblocks there are some expressions in the aniblock package like blink and breathe. They work ok or you can simply keyframe them on a blank model and save as NLA.
Good luck with your voyage !
Thanks AnAlias. Yes starting from 0 or making sure a previous movement doesn't (unintentionally) bleed into what you want to express now is very important!
Very good start, SciFiFunk. The CMU dictionary was a revelation to me. I was just looking for such a thing. Can't wait for further installments.
Thanks Argus. I recorded part 2 tonight and I'll upload tomorrow.
Fantastic Tutorial,
I have been trying similar techniques in DAZ studio and found a free online converter ( http://www.truespel.com/en/ ) that converts the text to simple phonetic notation for American English. I find it gives better results that straight text.
Your method seems much easier than what I have tried. I used Mimic Pro Standalone and Genesis Figure for all my tests. The tricks I found were to alter the .DMC file to exaggerate the head and neck movements and to watch the lip sync and quickly remove any small phenomes that seem out of place. I import the Mimic output to DAZ Studio and then add aniblocks for body movement, extra blinks, expressions and brows. I have had mixed results as it does require lots of hand editing and personal judgement. You can see my experiments at http://www.youtube.com/watch?v=ijbzHLozYk8&list=PLnSggzklmsJHDlHLZhw0_L7Dju-Yt-HcP
Mimic Pro for Carrara looks like the better method for a pure Carrara approach.
Thanks for the information.
Yeah thanks Joe - you experience is invaluable.
btw - I cant reply to you on youtube for some reason. It's not that I'm ignoring you!
This is similar to what I am going to cover, but if you see the series though you'll see a number of techniques to improve things, at the cost of animating time/effort.
Tomorrow I talk about NLA animation, it's taken me over a week to understand what Is going on, but now hopefully as I share my (new) knowledge I can join the NLA party and we can build on our experiences.
I can already see that if you become an expert in NLA, long term the animating time reduces with reuse (looping + reversing + speed changes). At some point I'd like to turn my DAZ animation data into Carrara NLA clips - then I can re-do episodes 1-4 in Carrara. Ep 1 and 2 are looking especially out of date now.
Very cool, as always, SciFi Funk!
That happens to me sometimes too... Aggravating!I am not sure why you couldn't reply.
NLA's seem like they are the real power of Carrara. Aniblocks seem to be the key to lip sync in DAZ studio, but NLA's take the idea to the next level.
Another thing I forgot to mention about my experience with lip sync is that eye movement is very important. It makes the difference between movement and the appearance of thinking. I have been pointing the eyes at a target and moving it in an aniblock and it add so much more life. Hopefully this helps.
Check out Chris Jones eye experiment at http://www.youtube.com/watch?v=TAZIvyAJfeM. Amazing realism, and very inspiring.
Not trying to hijack the thread, but that A3 walk cycle animation I did a while ago uses an NLA clip for the walk cycle. Initially both arms were like the left arm. Kind of moving in a general, stiff way at the sides. I did this intentionally in case I wanted to layer clips specifically for arm movements. Interestingly, using a target helper and IK tracking over-road the NLA clip and allowed me to retain the shoulder roll from the clip and keep the hand on her hip.
http://youtu.be/nkWBRuJlQhA
And if you're interested in the clip, it's for A3. I tested on a V3, but due to scaling I think, it didn't work correctly. You don't even want to see what it did to poor Hiro! Also, this will not keep the hand on the hip. The arms will stiffly swing at the sides. It's a walk in place clip, but I was able to use a motion path and get minimal feet slide.
https://dl.dropboxusercontent.com/u/7370483/Sexy-Walk-Clip.cbr
Part 2.
http://www.youtube.com/watch?v=70Jx9yrRZKw
This time I cover a typical editing session in Mimic pro IF you understand how it actually works AND you have a decent knowledge of what each viseme actually represents.
I demonstrate how and when to subsititute mimics default visemes vs your own choices.
Very good tutorial. I learned a lot. But I was wondering: have you tried to blend the phonemes together? Like my first pic shows, there is a feature that permits to blend phonemes. The second pic shows that you can choose how long you want to blend the phonemes, up to the point where they don't interact whit each other at all.
No I haven't !
That's excellent info Argus. Many thanks! I somewhat takes the wind out of my sails with what I am about to suggest, but I still think my idea is more flexible, although this narrows the gap considerably.
Next part out tomorrow.
Part 3 is an intro to NLA as a prerequisite for Mimic with NLA which is the next video in the series.
NOTE. Bonus content how to set up a half decent lighting rig for test animations in just a few seconds.
http://www.youtube.com/watch?v=EpkTXREwTWk
Fantastic Tutorial. I love that you are working in NLA's in Carrara as its much easier than what I have been trying.
You hit the same problems I was facing using Mimic Pro Standalone. The differences between mimic output and the phoneme's in the figures. That is why I found that modifying the .DMC file is the key to cutting down on the hand editing of the output. The trick seems to be matching up the output phoneme to the figures phoneme. This became very apparent when I tried to create a .DMC for Millenium Sub-Dragon. Since there are very few phonemes in Millenium Sub-Dragon figure. I tried an example using this custom DMC on a long poem with no editing and it worked ok, not great. You can see my test at this link. http://www.youtube.com/watch?v=A-zSRwFPvX4
This led me to start thinking like a puppeteer. The Muppets are a great example of Lip Sync because they aren't built with any of phonemes but they convey life and convincing lip sync. There trick seems to be simply the timing of open and closed mouths and body movement. I tried a quick version with DAZ Studios puppeteer and found that it is a quick way to get the feel of the audio lip sync. The immediacy of this method is very cool but I find the performance nature very different from the normal way I work. More practice is needed and there is something valuable to be learned along this path.
The placement of the phonemes is also very important. In Mimic Pro standalone I found that they need to be as close as possible to each other so that they don't pop between phonemes. But with NLA's that may be different.
The placement of the silences also plays an important role as it is very noticeable. Nothing more distracting than having the wrong mouth on a pause. That's why a editing pass on the silences is the quickest way to start the process.
Sorry to ramble, but its nice to talk to someone with the same problems. Thanks again for the great tutorials.
joeping - ramble away my friend - as you say it's good to talk with someone else whos been through the joy and pain of this, and I respect your editing of the dmc files.
I would love to see a video of your editing here so that perhaps I could attempt something similar (I appreciate the genesis file - I just can't use it as I'm sticking with M4s for some time).
Once we get to part 5 of this series we'll be in a position to talk with my idea on the table.
I have to be honest though - it's been intense, and I'm taking a short break to go back to music before applying this all to my episode 9.
Thanks Sci Fi Funk,
The DMC editing is pretty boring and time consuming, but I think it will be worth it in the long run. The main thing was the realization that Mimic outputs a set of phonemes and those need to be mapped to the figure. I was trying it on Genesis a product specifically not supported with the stand alone version. I had heard that it did a good job on M4 & v4. The stand alone version is so out of date it may be set up for M3 & V3. I think they don't even sell it anymore.
Sometimes a break is what is needed to get some perspective. I got to the point where I needed a break and have spent a few months away from the subject. Your experiments have motivated me about how some of my problems could be improved. Now I just have to justify purchasing Mimic Pro for Carrara. =)
I recommend, see the video below (there is one more to come). Look at the integration (read less chance of importing / editing going wrong).
If you are patient it's often on offer.
Part 4
THE NEW SYSTEM.
I believe this is an important breakthrough. I've applied synthesizer envelope theory to 3d animation blocks. Behold the additional editing capability!
http://www.youtube.com/watch?v=CY-MvEjyzFo
I've been told that only the stand-alone version of Mimic can edit DMC files, not the Mimic for Carrara plugin. But each time I try to load a Genesis CR2, Mimic says it can't find the figure. Is there a way to make a CR2 of Genesis that can be accepted by Mimic so that I can edit DNC files?
I've been told that only the stand-alone version of Mimic can edit DMC files, not the Mimic for Carrara plugin. But each time I try to load a Genesis CR2, Mimic says it can't find the figure. Is there a way to make a CR2 of Genesis that can be accepted by Mimic so that I can edit DNC files?
Can I ask what is your downside for just "doing it" in Carrara without the DMC?
Very interesting tutorial.
argus1000: I used the technique this tutorial to export a Genesis Figure to Mimic Pro Stand alone. http://www.youtube.com/watch?v=XDYEvSb7jQs
I didn't know the Carrara Mimic Plug in comes with a Genesis .DMC. It probably works better than my old version. =)
I'm not sure I understand the question. I work in Carrara with the Mimic plugin for Carrara and the standard DMC file that is provided. But I heard that if you modify or edit this DMC file, you can get better results. I though a DMC file was essential. I'm a bit confused because I'm still learning Mimic.
Two thoughts on this.
1. I used to have similar problems with DAZ figures and Carrara. I would load a Gen4 figure in, apply morphs and clothing. At some point after working with the scene, several things would stop working - the ability to "blink" was always one of them. This was around the introduction of C8. I am just getting back into Carrara after a while and so far I have NOT seen this problem with C8.5. It may have been fixed.
2. When using NLAs it is easy to get tripped up. Honestly, I am still learning the ins and outs of it. However, if the NLA is somehow controlling the "blink" (keeping eyes open) they you cannot override it. You have to make sure the NLA is not controlling that body part.
Thanks, joeping. The updated DMC file for Genesis might not come with the app, but can be found here:
http://www.daz3d.com/forums/discussion/28504/
Agreed. Sometimes what we perceive as a bug is in fact NLA controlling what we asked it to control. lol.
However I have found that there are bugs as described in my later videos. e.g. not showing the result of your editing, while you are editing only after the clip is saved again. V annoying as you are forced to guess or restart the session.
I feel there are a lot of things that don't "update" now. There is no way to update a shader in the main preview, and reference shaders also will not update unless you go to each instance and re-select the reference shader...
Do we have a bug tracker?
I must admit I haven't raised one for this. I'm using my network rendering doesn't work for IES lights as a test run before I add any more.
I'm not sure I understand the question. I work in Carrara with the Mimic plugin for Carrara and the standard DMC file that is provided. But I heard that if you modify or edit this DMC file, you can get better results. I though a DMC file was essential. I'm a bit confused because I'm still learning Mimic.
I have the old Mimic Pro that writes the DMCs, but I don't use it.... I am not aware of any helpful controls that allow you to create nuances, other than just editing the poses. It's been a while since I used it - and I may be confusing it with CrazyTalk (which *does* have a lot of nuance controls).
I set up the associations in Carrara (for mouths generally it is like 5-7 poses, which I re-use on similar "visemes". I actually like the "soft" look of this better than the DMCs which can look a little ar•tic•ul•at•ed - somewhat distracting - especially the letter L which suddenly makes the tongue appear but you don't see it otherwise.
imho, people's faces just don't move much when they talk... most people just talk slackjawed and expressionless unless it is public speaking.
I have been saving a Master Scene of my character with all her Mimic setup and call it "mimic ready". For me it hasn't gotten more complicated than that. (I try to render figures and background separate, it's faster)
But I just tested whether you can export the character with all the Mimic assignments (to get it into another Carrara scene). It is pretty easy. You have to drag all your poses and clips into a track (just one), then export your figure via the Browser / Scenes or Browser / Objects (use a group to save clothing, etc w the figure).... When you drag the figure into your new scene the NLA track is still there and all your Mimic assignments are attached to their poses.... In otherwords, all the poses export along with the figure by riding along in the NLA track. If you forget a pose it will not be saved with the figure and that assignment under Mimic will be blank.
You can't just "drag all the poses" onto the NLA track, they have to be dragged one-by-one, so that's a bit of work if you have a lot of poses...
For what it's worth, here is a discussion of "dynamic visemes" on something called DISNEY RESEARCH. Since Carrara uses NLA for the mouth shapes, you could explore a little of using clips instead of poses for each viseme - that's something Carrara Mimic can do that Mimic Pro can't, although I'm not sure Mimic's analyzer would be be as "dynamic" as Disney's - they seem to be inventing a new way of looking at lipsync.
http://www.disneyresearch.com/project/dynamic-visemes/
WOW Thanks Holly, as always you provide great information.
The results of the Disney technique looks amazing and are very inspiring.