Hello DazGods,

This is a mini-tutorial which suggests a method for turning a photo of a real person in to a 3D character in a video.  I'm sure this isn't the only way to do this so feel free to suggest alternative methods, I'll be all ears.

The method I'll describe requires 3 softwares.

1) DAZ (free)
2) CrazyTalk ($30)
3) Hitfilm Express (free)

If you've never used any of these apps, the learning curve for this tutorial can be very ballpark estimated as follows.  Your mileage may vary considerably depending upon your skill level.

1) DAZ = a week for very basic animation.
2) CrazyTalk = a week to learn enough to do this.
3) Hitfilm Express = a month at least if starting from scratch.

This post won't provide detailed instructions for every step, but is instead a general overview offered to inspire your own investigations.   Here's what I do...

=============

1) Animate a character in DAZ.  The only limitation here is that your character should be squarely facing the camera throughout the animation.

You can render your animation in iRay, or do what lazy people like me do, simply screen record the Viewport in DAZ.  :-)  However you export, the end product should be a video.

=============

2) Import a headshot in to CrazyTalk, animate the face, and add a voice track if desired.

If you aren't familiar with CrazyTalk these tutorials can provide an introduction:

https://www.youtube.com/playlist?list=PLNV5zSFadPdlt8f3Oht6MnAjYwoKTZG-A

Quick summary, if you are comfortable in DAZ you will have no problem with CrazyTalk.  Depending on your skill level I'd guess it would take one day to a week to learn CrazyTalk sufficiently to accomplish what is needed for this tutorial.

I'm using 2D heads exported out of CrazyTalk, which is why I suggested that your DAZ character should always be facing the camera.  If you're more skilled than I am you may be able to overcome this limitation.

=============

3) Import your DAZ video and CrazyTalk head video in to Hitfilm, or most any other video editor which can work with layers.

- Put the CrazyTalk video on a layer above your DAZ video.

- Mask the CrazyTalk video to remove any unwanted areas.

- Track the eyes in the DAZ video to create a motion track that your CrazyTalk video can follow.  Thus, the real person's face will pan and zoom appropriately in proportion to your DAZ animation.

Those of you who are already comfortable with some video editor will probably find this a relatively simple task. If you aren't already familiar with some video editor this won't be a trivial job.  

=============

Again, this is just one way to attach a real person's face to your DAZ animation.  There are most likely other and probably better ways to do this.  If you know such better ways, please share!

My skill level is that I'm very familiar with CrazyTalk, am comfortable with basic operations of the free version of Hitfilm, and have only been using DAZ a few days.  I'm getting some fairly decent end products of real person => DAZ animation.  Hardly award winning, but fun enough to keep me learning.

Interesting that the face even has gender. Okay, next to your old lady is a man wearing a turban and has a beard. He is looking downward at the sand and water. I should use Reallusion CrazyTalk to make all the faces talk. Ve have vays of making zem talk ;)

Thanks Wendy - Emotiguy was one of my first thoughts as well - maybe I'll pursue that a bit further.  have used CrazyTalk on occasion but would like to easily integrate this with some ofhter 3d props.

crazytalk was my first thought

otherwise emotiguy serves me well with a bit of vertex room fiddling and shading

I had a very similar problem a couple of months ago - I needed to create some talking animations for an education project at work.  Having used Mimic 3 many years ago, I tried Mimic Live and wasn't overly happy with the results, which is why I took a look at the Reallusion programs.

Crazytalk 8: If you just want talking head style characters with a reasonable degree of realism then this is the cheapest solution.  It will do text to speech (but the Windows voices it uses are limited), or it will take a recorded file.  The results aren't perfect, but it is of vidoe game cutscene quality and more importantly it renders very quickly.  You are, however, limited to the characters that come with the program or with the add-on packs such as the Realistic Human 100 pack, which can become expensive.  If you are not too worried about a large variety of characters, and especially if you are short of time then this is a good solution.
iClone 7: There is a brief explanation of the workflow involved for using Daz characters in iClone 7 here: https://www.reallusion.com/iclone/daz3d/

There is an iClone 7 bundle that includes 3D Exchange, Character Creator 2 (soon to be upgrades to v3) and the Daz Genesis Extension for $299 (though if you have an earlier version of iClone you might get a lower price when you log in).  Link to the bundle here: https://www.reallusion.com/store/product.html?l=1&p=ic
iClone 7 alone gives you a lot more options for talking heads than Crazytalk just from within the program.  Setting up the Daz Genesis 8 or 3 export/import is fiddly the first time because you have to copy phonemes to Daz Studio and add them to the figure before exporting.  Then you take the figure into 3D Exchange where you can work on the shaders and from there you can send it to iClone 7. 
Once you have the Daz figure in iClone 7, you can use text to speech, record "live" or use a recorded speech clip.  I need to revisit and refine my project for next summer, so beforfe then I plan to ask some friends to record dialogue for me for the different characters.  You can also add scenes or HDRI backgrounds.
I haven't had time to master the program, so only used it for talking heads, but there are a lot of animation options for the figures that you can buy and add in.

For me, where iClone 7 beats Daz Studio is in the speed of animation and rendering.  Of course the quality is not equal to an iray rendered still in DS, but it is more than sufficient for most animation projects.
Regarding the quality though, I think those of us that do this is a hobby or a profession forget how amazing it can seem to other people.  I created around 10 animations introducing tasks in a project, and completed them over a weekend.  The kids that worked through the project thought that the animations were amazing.

One last thing - I wasn't certain of exactly what I wanted and was on a tight deadline, so I bought Crazytalk 8, and then later bought the iClone 7 / 3D Exchange / Daz Extension.  There are demos available for you to try out the products and I would recommend trying them before shelling out a lot of cash.

I'm sure other people will post their experiences with iClone, but if you want any more info then let me know.

 

Thanks for your feedback Reaper since I am looking to leverege my huge daz model collection then iclone seems more suited to that than crazytalk. I did actually download crazytalk8 but so far have been less than impressed.

I had a very similar problem a couple of months ago - I needed to create some talking animations for an education project at work.  Having used Mimic 3 many years ago, I tried Mimic Live and wasn't overly happy with the results, which is why I took a look at the Reallusion programs.

Crazytalk 8: If you just want talking head style characters with a reasonable degree of realism then this is the cheapest solution.  It will do text to speech (but the Windows voices it uses are limited), or it will take a recorded file.  The results aren't perfect, but it is of vidoe game cutscene quality and more importantly it renders very quickly.  You are, however, limited to the characters that come with the program or with the add-on packs such as the Realistic Human 100 pack, which can become expensive.  If you are not too worried about a large variety of characters, and especially if you are short of time then this is a good solution.
iClone 7: There is a brief explanation of the workflow involved for using Daz characters in iClone 7 here: https://www.reallusion.com/iclone/daz3d/

There is an iClone 7 bundle that includes 3D Exchange, Character Creator 2 (soon to be upgrades to v3) and the Daz Genesis Extension for $299 (though if you have an earlier version of iClone you might get a lower price when you log in).  Link to the bundle here: https://www.reallusion.com/store/product.html?l=1&p=ic
iClone 7 alone gives you a lot more options for talking heads than Crazytalk just from within the program.  Setting up the Daz Genesis 8 or 3 export/import is fiddly the first time because you have to copy phonemes to Daz Studio and add them to the figure before exporting.  Then you take the figure into 3D Exchange where you can work on the shaders and from there you can send it to iClone 7. 
Once you have the Daz figure in iClone 7, you can use text to speech, record "live" or use a recorded speech clip.  I need to revisit and refine my project for next summer, so beforfe then I plan to ask some friends to record dialogue for me for the different characters.  You can also add scenes or HDRI backgrounds.
I haven't had time to master the program, so only used it for talking heads, but there are a lot of animation options for the figures that you can buy and add in.

For me, where iClone 7 beats Daz Studio is in the speed of animation and rendering.  Of course the quality is not equal to an iray rendered still in DS, but it is more than sufficient for most animation projects.
Regarding the quality though, I think those of us that do this is a hobby or a profession forget how amazing it can seem to other people.  I created around 10 animations introducing tasks in a project, and completed them over a weekend.  The kids that worked through the project thought that the animations were amazing.

One last thing - I wasn't certain of exactly what I wanted and was on a tight deadline, so I bought Crazytalk 8, and then later bought the iClone 7 / 3D Exchange / Daz Extension.  There are demos available for you to try out the products and I would recommend trying them before shelling out a lot of cash.

I'm sure other people will post their experiences with iClone, but if you want any more info then let me know.

I also have iClone/Crazytalk pipeline products and there is a face 3D generator in that system where you can use charicatures to generate genesis head morphs. The intent is not to make celebrity mimics. 

I am also planning to do realistic medically related image textures for genesis characters and develop 3D anatomy.  

The two will never meet as long as artists keep trying to turn regular renders into comic art.

What everyone is missing is that comics aren't drawn with the same proportions and consistent scale like any 3D model.

The closest approximation is the camera that exxagerates the scale of the closest objects, but since this is done PER OBJECT, in drawing, that alone won't get.

You would have make your scene and then scale/morph actual limbs and body parts to create an artistic emphasis on action points.

-----------

Then when you combine all the short cuts that comics take, for the sake of time and the medium's limitations- you always end up with something that doesn't look like a comic. Editing a render so it looks 'almost drawn' is a baby step towards the far goal.

yes.

What I keep finding is many of the styles or simple frames that I'm trying to emulate (literally reproduce in DS), is that they are notably 'unreal' in exactly the way you describe. Usually drawn to look like they *should* look, rather than like things really do! - The shadows will never work that way, they're wrong in the drawing!, The body can't do that! It's broken! heh.

My tools are too good.

Using the ever-more accurate 3D tools to render exaggeration and stylization is proving to be an art in itself... Morphed characters like Ollie and Edie, and George, with toonstyled lines and cel shaders helps, but comics are clearly about cleverly indicating dynamic ideas on static pages, and the decades of tricks and iconic elements that 'say' certain things by convention, are tough to duplicate in a medium that's been refined for accurate realism. Tools like CrazyTalk (reallusion) are bridging this gap in the animation side of things, and mechanisms like Motion Studio are bending the presentation norms, but it's still a free-for-all as far as 3D novel/comic 'norms'.

The good news is those who are going down this path have a whole new world of imagery and styles that can still be established as baselines for the future, rather than having to match a certain existing style (e.g. a specific manga flavor). We're the first ones down this path (in a 50 year timeframe), and we don't have to match a style. But we do have to invent one that folks will like. And apparently that's much harder to do when the style giants that precede us in the comic work (from archie to batman) aren't there for us to lean on so much in this brave new world of imagery and communication.

In a world of chalk, we've been given acrylics and told go paint a ceiling! I think this should be fun!

--ms

Anyway back on topic
I really wish Daz would ,at the very least
 unvault the venerable stand alone mimic 3 pro
even as 32 bit only app it still is quite useful
for creating ,editing& refining lipsynch.
perhaps even give it away with one of those M4/V4 "clearances

Personally I find the existing DAZ Mimic implementation to be adequate, if not elegant. I accept that DAZ is not highly focused on animation. I have always liked Reallusion's handling of lipsync, ever since I discovered Crazytalk years ago, but I can't accept their never-ending "bump you up" way of doing business.

After I did this, I figured this lipsync was close enough, and other things became more important to me.

Well, the solar eclipse will be coming to town in a couple of hours, and I've got it on the brain now.

Thanks a lot...

Daz Studio renders + Crazytalk are a good mix. 

Morphs are great in Daz Studio to have a lot of characters for this kind of work. Then Crazytalk allows me to add effects on the voices too.

I have added another video.

6R

...made with Daz Studio and various characters (2D renders) + Crazytalk. 

These are simple videos to learn some things very quickly here : http://cartoon6r.free.fr

6R

CrazyTalk is a lot of fun

It was the first graphics program I ever used back in 2008 on my Vista based Dell Laptop.

On photos but yes goes well on  art drawings and 3D

Here is a short video I made quickly for testing a mix between Daz Studio (2D Render) and Crazytalk8 (make the character speak).

Dont' forget, it's only a 2D picture made with Daz Studio !!! Crazy, no ???  :-)

https://www.youtube.com/watch?v=Hie-_ps7wxs&feature=youtu.be

6R

you might be interested in Crazytalk Animator by Reallusion

We are not allowed to give offsite links but you can Google it

You can literally photograph your models then using the 2D rigging in the program animate them 

it has a 3D like depth effect though actually 2D vector graphics.

The price slows me down every time. Also, I can't find if they are still using the Indigo render engine in 7.

Hm, i found only this in the FAQ:

Will the current iClone 6 plug-ins and services, be compatible with iClone 7?

The current iClone 6's plug-ins and services will be compatible with iClone 7 (official launch version). You will simply need to update some of them.

Below is the list:
- Character Creator 2
- 3DXchange 7
- CrazyTalk Pipeline 8
- popVideo 3
- Perception Neuron Mocap Plug-in
- Kinect Mocap Plug-in for XBox One
- Avatar Toolkit Plug-in
- Substance Designer 5
- Indigo Render Plug-in (limited to iClone 6 features)

But what is limited to iClone 6 features :)

I have so much content I've never used. Must be more efficient and give in less to temptation. ;)

That's just crazy talk. :-)

 

No, CrazyTalk isn't a DAZ Original last I checked.

His I have tinkered with DAZ Studio for many years and also use Crazytalk 8.1 for creating custom 3d heads from photos.  Is there a way to take these heads and use them on Genesis characters or any other characters in DAZ?  I think it can be done with Character Creator which is given for free with iClone, but I hate to have to buy another program just to do this exporting.  Are there any other alternatives?

Thanks!

I have to say, we have some talented and productive people here.  It's taking me some time to realize this, since this site doesn't exactly toot its horn about animation.  I'm not sure I understand the reason for this.  It does me a lot of good to see what other people are doing.  I see posts moaning about DAZ's failings as an animation tool.  Seeing the work of others helps me put these posts in perspective.

Oh well I don't know enough of the biz to consider myself more than an amateur. =^-^= But thank you. I have a Blue yeti mic and Audacity, so you can say I'm on the cheap just like many folks here.

If you have villain roles... I love to be villains. I like to rant and shake my fists and chuckle and scheme and plot. However, I don't want to be type cast so whatever you wanna give me is fine.

I use lipsync in DAZ 32 bit... I wish they'd put it in the 64 bit version. But they truly have been dropping the ball on a lot with animation.

Villains, eh?  I'll bet you are an excellent villain.  I found some of the posts in this thread to be a bit unsettling, but there's only so much I can do in a vacuum.  I won't give up my ideas about collaboration.

Me, it's time I actually create my first piece of 3D work.  Everybody has something to show but me.  At the moment I have a table full of new hardware to install that will hopefully make rendering a bearable experience, if it doesn't send me to the poorhouse first.

I learned about timelines and keyframes from Flash. Tweening made a believer out of me. Then I discovered lipsync in Crazytalk and I was wowed. Then I decided to go 3D.  I can't resist it, but I'm finding making the switch more difficult than I expected.

I have certainly used rendered heads as a basis for my characters in crazytalk iclone etc so yes it is quite doable.

I actually found Genesis 3 works quite well in iClone, am wondering if your FBX export has something ticked or unticked that you are losing the morphs, it was discussed recently on the Reallusion forum.

http://forum.reallusion.com/305353/Morphed-DAZ-Characters-into-3DXChange-Pipeline

HD morphs do not export of course.

 

 

That's interesting.  I read the thread.  They refer to "the tutorial."  I wonder which tutorial they mean.  Do you happen to know?

I configured the DAZ Morph Export rules according to a recent RL video that deals with importing G3 characters.  That's where the "head." statement comes from.  He was specific about putting a dot after the word.

I have certainly used rendered heads as a basis for my characters in crazytalk iclone etc so yes it is quite doable.

I actually found Genesis 3 works quite well in iClone, am wondering if your FBX export has something ticked or unticked that you are losing the morphs, it was discussed recently on the Reallusion forum.

http://forum.reallusion.com/305353/Morphed-DAZ-Characters-into-3DXChange-Pipeline

HD morphs do not export of course.