Loading...
> >

Cararra and Mimic Pro Experiment and Question

JoepingletonJoepingleton Posts: 746
edited December 1969 in Carrara Discussion

View the animation on YOUTUBE.COM by clicking here.

This is my first experiment with Mimic Pro Standalone and Carrara 8.5 Pro.

I processed an audio file from a previous experiment in the stand alone version of Mimic Pro using my edited Genesis DMC file. I couldn't figure out how to import the PZ2 output from Mimic into Carrara. So I imported it into DAZ Studio and then imported the DAZ file to Cararra. Then I applied some quick brow movement in a NLA clip.

Is there a way to import a PZ2 file directly into Carrara, so that I can eliminate the DAZ Studio step?

«1

Comments

  • evilproducerevilproducer Posts: 9,040
    edited December 1969

    Could you use File-->Import?

    Another Possibility may be to create a folder for it in the Pose directory of one of your runtimes that is in your Content browser?

  • JoepingletonJoepingleton Posts: 746
    edited December 1969

    Thanks for the information.

    Unfortunately File --> Import didn't work, that is why I felt the need to post the question.

    Importing the PZ2 to DAZ and saving it in a content directory might work. I was looking to see if maybe I was missing something for a native Carrara technique. Maybe I need to break down and buy the Mimic Pro for Carrara plug in. =)

  • evilproducerevilproducer Posts: 9,040
    edited December 1969

    joeping said:
    Thanks for the information.

    Unfortunately File --> Import didn't work, that is why I felt the need to post the question.

    Importing the PZ2 to DAZ and saving it in a content directory might work. I was looking to see if maybe I was missing something for a native Carrara technique. Maybe I need to break down and buy the Mimic Pro for Carrara plug in. =)

    I think you mis-understand. I used to save various aniblocks in D|S by baking them to D|S keyframes and using the PFE script. I just manually created a folder under Poses in the runtime of my choosing, then saved the resulting file to it. Before I figured out how they worked I wasn't as disciplined. I just created a folder in my documents and saved them there. I couldn't get at them, so when I figured out where they should go, I simply moved them to the location I had created in Runtime-->Library-->Poses-->My folder name. As long as that runtime is referenced by the Content Browser (not Smart Content) they should show up the next time you launch Carrara.

    As an added bonus, I select my character in the instances palette and double click the icon in the browser and Carrara applies it to the figure. I've DLed plenty of free Poser animated pose files and still poses and have never needed D|S to access the .pz2.

  • evilproducerevilproducer Posts: 9,040
    edited December 1969

    Make sure to name the file specifically so you'll know what it is as Carrara or Mimic will probably give it a generic thumbnail.

  • JoepingletonJoepingleton Posts: 746
    edited December 1969

    Thanks Evil,
    I didn't understand what you said correctly. Your explanation of the method worked out awesome and eliminated the need to go into DAZ Studio.

  • evilproducerevilproducer Posts: 9,040
    edited December 1969

    Sometimes my shorthand makes sense to me and nobody else. ;-)

  • JoepingletonJoepingleton Posts: 746
    edited February 2014

    Thanks for sharing you knowledge Evil Producer. It helped me greatly with today's experiment.

    View the revised animation on YOUTUBE.COM by clicking here.

    This is my first experiment with Mimic Pro Standalone and Brekel Pro Face Kinect Motion Capture Lip Synchronization combined in Carrara 8.5 Pro on a Genesis Figure.

    I captured myself saying 13 sentences from the TMIT database using Brekel Pro Face Kinect Motion Capture. Then I processed the audio file in the stand alone version of Mimic Pro using my edited Genesis DMC file. In Carrara 8.5 Pro I imported the resulting two PZ2 files into NLA clips and applied them in additive mode to a genesis figure.

    Not perfect by any means, but I am pleased with this as a base for further editing.

    What do you think?

    timit01.jpg
    633 x 570 - 109K
    Post edited by Joepingleton on
  • DartanbeckDartanbeck Posts: 21,210
    edited December 1969

    Yeah, I like it.
    I agree that some tweaking for a bit more natural transitions into and out of "w" shapes might really sell the effect nicely as a great base to start from, and then alter a few things between different characters for individuality, etc., but that's a great response by the software and your efforts, I think. Great job!

  • JoepingletonJoepingleton Posts: 746
    edited February 2014

    Thanks Dartanbeck,

    View the revised animation on YOUTUBE.COM by clicking here.
    This is a revision to yesterdays experiment with Mimic Pro Standalone Lip Synchronization with in Carrara 8.5 Pro on a Genesis Figure. The main difference is I didn't use the motion capture from my last experiment. That eliminated most of the jerkiness of the movement.

    I used the audio from yesterday of myself reading 13 sentences from the TMIT database. Then I processed the audio file in the stand alone version of Mimic Pro using my edited Genesis DMC file and eliminating any lip motions that popped. In Carrara 8.5 Pro I imported the resulting PZ2 file into NLA clips. I created 3 Brow NLA clips (Up, Down, Squeeze) and 4 Eye NLA clips (Right, Left, Up, Down). These were placed in additive tracks over the lip sync NLA track.

    I may have removed too many mouth movements in order to get a smoother lip sync result. What do you think?

    Post edited by Joepingleton on
  • DartanbeckDartanbeck Posts: 21,210
    edited December 1969

    I like it. There are a couple of spots that I might tweak, if this was a finished part for production, but I like the 'less' approach. I feel that the animators for Star Wars - The Clone Wars 3d animations did an amazing job of this. Like, for the "r" sound, they used more tongue motion than lip pursing, which really made their synch look natural.

    Yeah, I like what you've got going here. It seems that you spoke out the pronunciations in the sentences on purpose as part of your test. Perhaps not. But when you see someone speaking fluidly, like they're talking about something that just rolls out, you'll notice that their mouths don't phoneme to each sound - but is more a blur. This is why I just cannot understand how people can learn to read lips!

    I can tell you've already noticed this though, because you're working it out really nicely... Bravo!

  • JoepingletonJoepingleton Posts: 746
    edited December 1969

    Thanks for the tips,

    Yes I spoke the spoke out the pronunciations in the sentences on purpose as part of the experiment.

    Holly wetcircuit posted a great link in scififunk's mimic forum http://www.disneyresearch.com/project/dynamic-visemes/. It changes the way to look at lip sync.

    This also turned me on to the TIMIT Database, which contains a bunch of sentences used to test hearing aids and voice recognition systems. These seemed like a good base to test out lip sync techniques and results. Plus they are kind of tongue twisted for a non performer like myself. =)

    I used the sentences found here http://data.cstr.ed.ac.uk/mocha/mocha-timit.txt

  • argus1000argus1000 Posts: 701
    edited December 1969

    joeping said:

    This is a revision to yesterdays experiment with Mimic Pro Standalone Lip Synchronization with in Carrara 8.5 Pro on a Genesis Figure. The main difference is I didn't use the motion capture from my last experiment. That eliminated most of the jerkiness of the movement.

    I like it very much. I like the head and eye movements best. Mouth movements, which are probably the hardest to capture, IMHO, need some work though. But the experiment is very promising, even illuminating.

  • wetcircuitwetcircuit Posts: 0
    edited December 1969

    joeping said:
    This also turned me on to the TIMIT Database, which contains a bunch of sentences used to test hearing aids and voice recognition systems. These seemed like a good base to test out lip sync techniques and results. Plus they are kind of tongue twisted for a non performer like myself. =)

    I used the sentences found here http://data.cstr.ed.ac.uk/mocha/mocha-timit.txt

    Oh! Those are cool... Hahaha. I love weird sentences that make you think there should be more story... Like an oddly specific Magic 8 Ball.

    "Carl lives in a lively home." "Most young rabbits rise early every morning." LOL

    They also remind me of those memory mnemonics, like "Richard Of York Gave Battle In Vain" to remember ROYGBIV

  • JoepingletonJoepingleton Posts: 746
    edited March 2014

    View the animation on YOUTUBE.COM by clicking here.

    This is my first attempt with the Carrara Mimic plug In. The audio is from the 11 Second Club March Contest ( http://www.11secondclub.com/competitions/current ). This time I tried to remove the "swimmy" motion of the previous experiments. Any feedback would be greatly appreciated before I submit it to the Contest.

    March01.jpg
    633 x 570 - 172K
    Post edited by Joepingleton on
  • JoepingletonJoepingleton Posts: 746
    edited March 2014

    View the animation on YOUTUBE.COM by clicking here.

    This is a speed test of the Mimic Plug in for Carrara Pro 8.5
    I found the audio file, imported it into Carrara via the Mimic plug in. Added an aniblock for the body movement. Added a NLA track with emotions. Pointed the eyes at a target and moved it around. It took an hour from idea to final render.

    I should have bought the Mimic Plug in for Carrara long ago. It's so much easier being able to edit everything in one place.
    Thanks for the help.

    silentium_amoris_by_oscar_wilde_by_joepingleton-d79ms5z.jpg
    633 x 570 - 153K
    Post edited by Joepingleton on
  • DartanbeckDartanbeck Posts: 21,210
    edited December 1969

    holly wetcircuit said:
    "Richard Of York Gave Battle In Vain" to remember ROYGBIV
    Restistors? What, did you go to electronic engineering school?
  • JoepingletonJoepingleton Posts: 746
    edited December 1969

    View the animation on YOUTUBE.COM by clicking here.

    This is another speed test of the Mimic Plug in for Carrara Pro 8.5
    I found the audio file, imported it into Carrara via the Mimic plug in on a Genesis Figure. Added a NLA track with brow movement NLA clips. Pointed the eyes at a target.

    Thanks for looking.

    the_broken_heart_by_john_donne_animation_by_joepingleton-d79zc5f.jpg
    633 x 570 - 201K
  • JoepingletonJoepingleton Posts: 746
    edited March 2014

    Has anyone tried Mimic with a language other than English. There doesn't seem to be any problems with different accents. Sorry if this sounds stupid, Are there different Phonemes?

    I was reading about chereme (the basic unit of signed communication) and was thinking that Mimic could generate a DMC file to generate hand poses for sign language. A chereme, as the basic unit of signed communication, is functionally and psychologically equivalent to the phonemes of oral languages, and has been replaced by that term in the academic literature. Cherology, as the study of cheremes in language, is thus equivalent to phonology.

    Post edited by Joepingleton on
  • argus1000argus1000 Posts: 701
    edited December 1969

    joeping said:
    View the animation on YOUTUBE.COM by clicking here.This is another speed test of the Mimic Plug in for Carrara Pro 8.5
    I found the audio file, imported it into Carrara via the Mimic plug in on a Genesis Figure. Added a NLA track with brow movement NLA clips. Pointed the eyes at a target.Thanks for looking.

    Very good test. Do you edit the phonemes or is this Mimic straight out of the box? Do you use a text file?

  • JoepingletonJoepingleton Posts: 746
    edited December 1969

    Thanks, I used Mimic out of the box and a text file.

  • wetcircuitwetcircuit Posts: 0
    edited December 1969

    joeping said:
    Has anyone tried Mimic with a language other than English. There doesn't seem to be any problems with different accents. Sorry if this sounds stupid, Are there different Phonemes?

    I was reading about chereme (the basic unit of signed communication) and was thinking that Mimic could generate a DMC file to generate hand poses for sign language. A chereme, as the basic unit of signed communication, is functionally and psychologically equivalent to the phonemes of oral languages, and has been replaced by that term in the academic literature. Cherology, as the study of cheremes in language, is thus equivalent to phonology.

    I don't know this specific subject, but it seems like you could fool Mimic into assembling NLA animated poses from a "phonetic" script... Should be easy to test with some short hand gestures -- but to be honest it would probably be better to just assemble directly on the NLA track by hand (once you have all the hand poses)... I can imagine all sorts of issues trying to get Mimic to do a workaround. At that point you could just do it directly without Mimic....

  • DartanbeckDartanbeck Posts: 21,210
    edited December 1969

    That looks fantastic.

  • JoepingletonJoepingleton Posts: 746
    edited March 2014

    Thanks for the feedback. A NLA approach might be more accurate for sign language since it is normally word based.
    Shouldn't any language have about the same mouth positions if they are converted to Phonemes?

    View the animation on YOUTUBE.COM by clicking here.
    This is another speed test of the Mimic Plug in for Carrara Pro 8.5
    I found the audio file, imported it into Carrara via the Mimic plug in on a Genesis Figure. Added a NLA track with brow movement NLA clips. Pointed the eyes at a target.

    Since I don't speak Spanish I can't tell if it looks correct. Any Spanish speakers have an opinion?

    Thanks for looking.

    SONETOWATTEAU01.jpg
    633 x 570 - 221K
    Post edited by Joepingleton on
  • argus1000argus1000 Posts: 701
    edited December 1969

    joeping said:
    Thanks, I used Mimic out of the box and a text file.

    Your test is pretty good. As for myself, I have a problem with Mimic Pro for Carrara. If you watch my video at:
    http://youtu.be/rc0jfxMSnzw
    you will see that my character has rubbery lips, as if the phonemes come too fast for her to speak. Watch her lips closely. Is there a solution to that? And, BTW, do you edit your DMC files?

  • DartanbeckDartanbeck Posts: 21,210
    edited December 1969

    Rubber lips?
    Perhaps it's part of the "W" phoneme that's bothering you. Some phonemes might get a bit too literal with how a sound should look, because the shape was made for that sound alone, not mixed in. For an automated lip sync run, I think Mimic did a great job. Now you can go in and tweak away to get it more to match her persona.

    But, like mentioned in the manual for Mimic, you'll often get the best lip syncing if you do create custom NLAs for each phoneme - or just the ones that aren't right for the character. Like to stop from forming too much of a doughnut when Owing, we could use less pucker and a touch of bottom lip down. Just food for thought.

  • DartanbeckDartanbeck Posts: 21,210
    edited December 1969

    joeping said:
    Thanks for the feedback. A NLA approach might be more accurate for sign language since it is normally word based.
    Shouldn't any language have about the same mouth positions if they are converted to Phonemes?

    View the animation on YOUTUBE.COM by clicking here.
    This is another speed test of the Mimic Plug in for Carrara Pro 8.5
    I found the audio file, imported it into Carrara via the Mimic plug in on a Genesis Figure. Added a NLA track with brow movement NLA clips. Pointed the eyes at a target.

    Since I don't speak Spanish I can't tell if it looks correct. Any Spanish speakers have an opinion?

    Thanks for looking.

    NLAs are also how we can tweak out our own phonemes from the DMC for each individual.
  • JoepingletonJoepingleton Posts: 746
    edited March 2014

    Thanks for the great tips.

    I didn't edit the DMC file for my last experiments. You may want to delete some of the phonemes NLA tracks that are too quick to get rid of some of the pop. Also offsetting the mouth from the audio by 4 frames so that you see the mouth before hearing the sound helps. Moving the silence NLA clips closer to the first words so you get a cleaner start on the phrase also helps.

    Post edited by Joepingleton on
  • DartanbeckDartanbeck Posts: 21,210
    edited December 1969

    joeping said:
    Thanks for the great tips.

    I didn't edit the DMC file for my last experiments. You may want to delete some of the phonemes NLA tracks that are tooquick to get rid of some of the pop. Also offsetting the mouth from the audio by 4 frames so that you see the mouth before hearing the sound helps. Moving the silence NLA clips closer to the first wordsso you get a cleaner start on the phrase also helps.

    No...
    Thank YOU for the tips! ;)
  • argus1000argus1000 Posts: 701
    edited December 1969

    joeping said:
    Thanks for the great tips.
    I didn't edit the DMC file for my last experiments. You may want to delete some of the phonemes NLA tracks that are too quick to get rid of some of the pop. Also offsetting the mouth from the audio by 4 frames so that you see the mouth before hearing the sound helps. Moving the silence NLA clips closer to the first words so you get a cleaner start on the phrase also helps.

    Thanks for the valuable tips.
  • JoepingletonJoepingleton Posts: 746
    edited March 2014

    View the animation on YOUTUBE.COM by clicking here.
    Thanks for the tips and feedback.

    This is another speed test of the Mimic Plug in for Carrara Pro 8.5
    I found the audio file, imported it into Carrara via the Mimic plug in on a Genesis Figure. Added a NLA track with brow movement NLA clips. Pointed the eyes at a target. Then I added some kinetic typography in After Effects.

    I edited the MIMIC NLA clips placement in the timeline. Some I removed and I added some to try to start seeing what looked better. I mostly played with the silences. I liked this experiments lip sync more than the default Mimic output.

    Hopefully the Typography distracts from the flaws. What do you think?

    fireAndIce01.jpg
    633 x 749 - 443K
    Post edited by Joepingleton on
Sign In or Register to comment.