Animate models within live action
laynemoore
Posts: 0
I don't know if this is the correct forum, if not please forgive. I posed this issue in adobe's after effects forum and a couple of guys acted like I came over to kick sand in their vaginas. Made me feel really stupid. You all are much more civil.
Here's what I'm wondering is possible: I want to animate a hovering 3D robot within a live action scene. I found some instruction on opening the model in Photoshop Extended and saving a format that allows me to import as 3D file into AE. I would assume that I could make the live action a 3D layer and move this model around in 3D space within the live action.
My question, though, may or may not involve some degree of motion tracking. It's revolutionized the way I storyboard my projects. I love the concept of putting an object into the scene and locking it down. But this is different. I want to put that object into the scene but animate movement to interact with the shot. One suggested I don't use AE at all for this and do everything in my 3D software (which is Carrara 8 Pro). But I'm not seeing how that is possible. I thought I would be going the other direction and doing the animation within AE with keyframes or something.
I know I'm over-simplifying the work involved but, surely I'm not the only person who wants to do this type of shot. How would you approach it?
If it won't work, I'm bummed because part of my movie has a monster who is stalking characters in live action sequences.
Daz 3D is part of 
Connect
DAZ Productions, Inc.
224 S 200 W, Suite #250
Salt Lake City, UT 84101
Licensing Agreement | Terms of Service | Privacy Policy | EULA
© 2024 Daz Productions Inc. All Rights Reserved.
Comments
I don't use AE, so I can't help with the work flow for that. I had always assumed that the 3D space in AE utilized 2D elements like billboards, does it import 3D geometry as well? If not, then unless it's a static figure and there's no change in perspective, you'll need to animate it.
I have a few questions regarding your project and scene in general.
What motion tracking software do you use? I know there's a couple that can export cameras to Carrara. Syntheyes (I think that's it's name) comes to mind.
How familiar are you with Carrara and 3D apps. in general?
What Is the robot supposed to be animated interacting with?
Can you give a better description of the scene and the elements involved?
This is just an example I had done as a test. I don't have motion tracking software, so the camera is locked down. The blowing grass and the ground use a Shadow Catcher, which is invisible to the renderer except for shadows and also acts as a mask for any 3D geometry behind it.
I used my live action background plate as a reference in Carrara for setting up my lighting and framing the shot. I then rendered the animation with an alpha channel and composited the background plate and the CG elements in Final Cut Pro.
http://www.youtube.com/watch?v=_8puXTRXt7Y&feature=share&list=UU6wB1FKPN4DWpuoVsQY2o8Q
I use the motion tracker in AE CS6.
I'm a newbie but fast learner.
The robot's movements are hovering and turning. I'd like to animate some lights on him too. In cockpit scenes I use the physical model because he's behind the seats (like C3PO's seat). I need to animate the CG model when he's with a group of people walking. I want him to look like he's hovering along with them. The shots are either horizontal or facing the camera. If I can get the hang of it, there are gun fight sequences where he hovers through the action.
My movie is Space Victories. It's feature length. It's an ambitious project that will take a lot of time for me and my son to do post. Original orchestral soundtrack composed by myself and my talented son (we use a Roland Fantom X8 to record the scores). With green screen, AE, Strata Foto, Photoshop Extended and Carrara 8 Pro, we're going to pull it off. The movie is part of a 5 book series I'm writing but filming the first movie starting with book IV (tip of the hat to Lucas). This robot is in quite a few scenes and I'd really love to get it right.
I'll be posting quite a few questions as we get more into it, especially when it's time to create the monster. Ugh! I may end up buying one instead. Anyone like to barter for original scores?
-------------------------------------------------------------------------------------
How awesome is that? Can we import live action footage into Carrara?
I do believe Dr. Evil is on the right track.
I'd go with those who suggest you do it in Carrara. Yes, you can import your live action footage into Carrara and use it as a Background (or is it Backdrop, I always get those two confused...) and animate your robot or whatever interacting with the backdrop footage. Just go into Scene/Effects/Backdrop and it will allow you to import a "map" (image or video) into the scene to use as a fixed, full screen backdrop for all of your rendered frames.
Now, hopefully you were a good boy and planned for this when you made the live action footage. Which means you used a tripod, and kept track of the placement of the camera relative to the acting figures. And also the light levels and sources. If you did, then you can set up your Carrara scene to match that. Because that's the biggest challenge, making sure the coordinate systems of your action footage and your Carrara scene are in sync.
And as Evil suggested, you can use Shadow Catchers to place shadows from the CG objects in your renders so it's like they really existed in the live action footage and cast shadows on the ground, etc.
To use a still picture, movie or image sequence as a backdrop, I recommend setting the render output to the same aspect ratio as the movie being used in the Background. Next, go to the View menu and select, Show Production Frame. You should see a wireframe rectangle representing the frame appear. Once that is done, look at the top right side of the Assembly Room window and find the circle icon with the up arrow in the middle. Click it, and another window will open. This is the Interactive Renderer. Select the checkbox that says Show Backdrop. Click Okay. The next thing to do is select Scene in the Scene Hierarchy. In the tab at the top of the screen that says Effects, click the triange next to Backdrop and then choose Map from the pull down menu. A dialogue will open allowing you to navigate to your file.
If you use a motion tracker software for syncing camera movements, you may have the option of exporting the camera to Carrara, then you should be able to sync the live action footage you shot with the robot because the camera movements are based on the motion tracking data. If you don't see Carrara listed, you may still be able to import the data from within Carrara. I would start a thread here specifically for motion tracker software and Carrara compatibility. There are people here that do use it. Again, there's a couple different programs I've read about, but the only one I can remember is Syntheyes.
I know Carrara Pro is supposed to have a AE exporter to export Carrara's camera, but I've never used it, and I don't know how up to date it is.
That is AMAZING - I didn't know we could do this in Carrara. That opens some possibilities for me. So using the video reference to animate the character, then exporting it back into AE, I can then do rotoscoping if necessary?
You all are helpful - thank you.
You can do a lot in Carrara. When I composite amodel into a video or photo, I could do most of the work in Carrara, but I still prefer to render the subject with an alpha and composite in a program that specializes in compositing. I feel that I have more control that way.
Hi laynemoore :)
Carrara has a special shader called "Shadow catcher", which can be applied to a floor or other objects, and will catch the shadows from your 3D model,. this can be used to composite the shadows back into your live footage and add to the illusion of your 3d model being in the real scene.
If you're also new to using After effects,...
You should check out the free tutorial videos on www.videocopilot.com
they cover almost all aspects of using After effects to composite and create effects with live footage or animation.
They often have some free give-away's, like stock FX footage too.
Hope it helps :)
Hello all,
Carrara has some great tools for supporting matchmoving, Syntheyes is what i use although i hav'nt had the chance
to play for quite a while. Here's a few examples i have done using Syntheyes + Carrara + Hitfilm:
http://www.youtube.com/user/pscamm/videos?flow=grid&view=0
"Statues in my Garden" was the latest one, "CG + live Actor Composite" was the previous one where the main object is to
demonstrate eyeline interaction between the live actor and the CG charactor, and of course my first very old attempt at all
this which is "CG Monster in my Garden".
If Syntheyes is used then the Output file auto loads up your live footage into Carrara's backdrop ready to use. A couple of
really important things though. Always set a 'Lock' contraint to your animated Camera immediately on opening the file or
you may accidently move the camera's position at some point along the timeline and you wont get it back. Also Carrara
only allows certain codecs to run in the live backdrop, USA formats seem to be more readily supported than the UK
equivelents like NTSC v PAL etc. Also if you are using Full HD footage then produce a cut down version of it at around
SD resolution and load that up into the backdrop in place of the HD origonal, this will massively inprove interactive render
times while your working.
I would definatly suggest using the 8.5 Beta for this kind of workflow as i managed to get Pierre to sort out quite a few bugs
and issues on the matchmoving support side of things. One really important one for example is that the 'Directors' Camera
View also showed the Backdrop video too. He added a checkbox in the interactive render settings box called 'Show Backdrop
- Director Camera' If you uncheck this then your directors camera view is clear and usable for navigation while animating. Another
little addition was to add checkboxes in the properties tray called 'Show Backdrop'. I found myself adding new cameras to the
scene positioned in certain places to aid the animation process and this check box disables the video in these camera's again
so your able to concentrate on animating and your video is only there in your actual render camers view.......Great feature.
Good Luck
Paul
Blender now has some serious matchmoving tools now, too. There's also the Pixel Farm's PFMatch.
I just (a few hours ago) finished a great tutorial on Blender's new tools called "Track Match Blend" that walks you through all the steps for many kinds of matching. Once you have a tracked camera in Blender, you could export that to FBX for import to Carrara.
Blender is open source, and free. (Though that awesome tutorial is about $30 on DVD.) PFMatch or Syntheyes start at about $400.
Once you have the matched camera moves, Carrara is very capable of doing the compositing work during the render, though I have run into minor issues on occasion. The main thing you will use the the "Shadow Catcher" object option.
A few things to remember when you shoot the live action:
* Make sure the video is really clean. Good lighting and a short shutter speed will help the process.
* The video should not have any zoom movement
* After the action shot, use the same camera, in the same settings, at the same zoom, to take reference shots of the whole scene from different angles. These will come in handy later, and if you don't have it, it's hard to re-do.
* record as much as you can about the lights in the scene. You will need this to set up the 3D lights in Carrara. (or any tool.)
* record dimensions of things in the scene, so that the matchmove can be scaled correctly.
This is just a still sample I did a few years ago, using the Daz Lion: (Not video, and not matchmoved, but using the shadow catcher.)
Alright, personally I'd put the breaks on all this bringing video into your 3D program stuff. Honestly, AE is always going to be the better compositor.
Shadow catching, yeah -- that's something you might want to snag from your 3D program (though I don't see the need unless you're doing some really, really high level stuff and you're literally, like, making models of all the furniture and matching all of the movements of all the people with CGI models, etc).
So okay, probably too many questions. As far as casting a shadow goes, that's easy as nothing. You copy your footage, black it out, flip and skew and opacity accordingly. Easy peasy quick shadow.
As for "interacting" with things in the video image. I hope you don't mean literally because of course that's impossible. If you have CS6 (I don't), I think their auto rotoscoping might help you out by creating some quick masks around your moving objects (people) that your ship/robot can then dip and dive around.
Also, no you cannot bring a 3D model into AE and animate it in (like, puppeteer it) AE. Not sure if you asked that. And unless there's something above my cerebral pay grade (there certainly could be), I do not know of any way that you can bring an ANIMATED 3D figure into AE's "true" 3D space. Like AE has sorta fake 3D space where it's like any animation program -- just flat layers at varying distances from each other. And then it has like "true" 3D, but that is only for OBJECTS. Like, stationary objects. You can then animate those stationary objects, but only as OBJ files, ya know? Like you're not going to be moving arms and stuff. Just XYZ axis, rotation, etc. Hope that makes sense.
So what I *think* you want to do (which is what I do) is you want to render out your 3D animation as a PNG sequence with transparent background. Then toss on your masking (if necessary), drop your quick and dirty shadow, add effects, color correct and -- voila! That's the most common sense, quick and dirty way to do it. I don't think I render anything out of Carrara ever that isn't a PNG with transparent background.
You CAN match move if you absolutely need to. But personally, if you're doing a (indie) VFX shot, I would plan on sitting that camera on a tripod, film extra wide (e.g. more frame then you need), and then add movement/shake in post. Match moving in a non-professional program (which AE isn't quite professional, though it's amazing) is never going to be perfect unless you frame by frame by frame by frame that mother. And even then there's no guarantee. And who the hell would ever frame by frame anything that extreme at 24fps? You can do a *ton* with post camera movements. I've seen guys do entire pans despite the camera being fixed on a tripod. They just blew up the image, camera shake and bam.
Anyway, that'd be my advice. Stick to AE for all of your video compositing. PNG transparent background.
Good luck.
I agree about not using Carrara to actually composite the image. That's what AE, AVID or FCP, etc. is for. When I bring video in, it is for the specific purpose of matching lighting. I disagree with you about using the shadow catcher. I've used the method you mentioned and I've had some good luck and some not so good luck. To me the shadow catcher has been the most consistent. Additionally it can act as a mask in your scene as well. Plus, it's pretty quick and doesn't slow down my renders.