Is there AI that can analyze an image pose and apply it to DAZ char?

Is there AI that can analyze an image pose and apply it to DAZ char?

Comments

  • eeyuneeyun Posts: 46

    It's doable - Clip Studio Paint has advertised this functionality for its own 3D mannequins for a decade(?)

    BUT - follow the money - DAZ sells content, including pose packs. A tool such as you describe would allow someone to recreate a pose from the promo image in the Daz marketplace.

  • lkopop908lkopop908 Posts: 87

    Ive asked the same thing and gotten pretty much the same response^. Good news tho, Blender will have that feature and Daz will either make it part of the subscription model or be left in the dust.(I am currently salty about Alpha being the only way to render on 500xx and it crashing....alot! and have been dipping into Blender)

    Ive been dabbling with doing the pose myself and using Qwen inpainting to put clothes on or such, cuts render time down I can tell the Ai to make say a dress blow in the wind or be held up or anything, without having to mess with Dforce. Been saving a ton of time doing that since you know...no Ai poses anytime soon. 

     

  • WendyLuvsCatzWendyLuvsCatz Posts: 40,422

    the argument given makes as much sense as nobody can manually pose a DAZ figure using a thumbnail on a promo image as a reference 

    selling a visually copied pose would on the otherhand be a derivative work

  • Richard HaseltineRichard Haseltine Posts: 108,979

    The "argument" is an assertion by someone with no access to Daz' decision making - this is why we usually pull speculative comments, they are taken as facts when they are nothing of the kind.

  • eeyuneeyun Posts: 46
    edited January 5

    :-) If manually copying poses was that easy, nobody would buy them - people want the shortcuts provided by premade poses, characters , sets etc.

    Daz has the "add to cart" button to get you that shortcut, I'm just saying they're unlikely to provide (or support) a "copy pose from promo image" button. 

    Post edited by eeyun on
  • lkopop908lkopop908 Posts: 87
    Personally the reason I want Ai is I have bought to many poses from Daz and even more from 3P sites. Heck would be nice if we could get a Ai to find the pose I want in my poses. Things are not properly labeled making it a royal pain to sift thru, like clothes from 3P (-_-), tho I have been working on it since it being cluttered wasnt helping. Just have to remember not to basic dump new content and actually place it in its proper folders. This seems as close to Ai we're gonna get for a while. https://www.daz3d.com/mocap-live
  • JamesJames Posts: 1,178

    I want to use AI to analyze my own photos and then apply it to genesis, without needing expensive suit like Rokoko. 
     

  • WendyLuvsCatzWendyLuvsCatz Posts: 40,422
    edited January 13

    OpenPose is the model needed to create such a script or plugin for those keen to try

    https://github.com/CMU-Perceptual-Computing-Lab/openpose

    there are licensing conditions to take into account 

    noncommercial so there's that

    I shudder to think what the commercial license costs but obviously exists as various video to FBX animation sites use it

    I have used a few myself and imported the results into DAZ studio

    Post edited by WendyLuvsCatz on
  • I don't think I've ever seen a pose that didn't require at least some manual tweaking to make it perfect. 

    Getting from an image to a pose file. A workflow sequence that I've used in the past was to copy the pose from the image by simply standing, sitting or whatever in the pose yourself, putting your own body into the desired pose, and then using an ipad or a phone to mocap the pose with depth sensing apps and save as BVH or FBX. I haven't used this method a lot but I remember doing some work with it a little while back for a video. Mobile devices are getting very good these days capturing body shapes and movements and saving them in useful formats. 

    Here's a link to a YouTube video I made using my own physical body movement to "capture" the desired sequence of poses:     https://youtu.be/ELdoIqow4Q4

  • Seven193Seven193 Posts: 1,146

    Does Daz Studio know how to take any humanoid bvh skeleton and fit it to a Daz Figure?  If so, then I would say, yes, it can be done. If AI can export your pose to bvh, then you're one step away from getting it into Daz Studio.

     

  • JamesJames Posts: 1,178

    I would be great if I can just take a pic, and the character automatically pose the way I pose, without needing export import etc. 

  • WendyLuvsCatzWendyLuvsCatz Posts: 40,422

    James said:

    I would be great if I can just take a pic, and the character automatically pose the way I pose, without needing export import etc. 

    well that you actually can do using ai and Openpose, Facebook's Meta Ai or Google's Gemini will do it for you with the 2 pictures and a prompt too

  • JamesJames Posts: 1,178

    WendyLuvsCatz said:

    James said:

    I would be great if I can just take a pic, and the character automatically pose the way I pose, without needing export import etc. 

    well that you actually can do using ai and Openpose, Facebook's Meta Ai or Google's Gemini will do it for you with the 2 pictures and a prompt too

    I don't get it. 
    What I have in mind is the character in DAZ3 posing as I pose. 

  • WendyLuvsCatzWendyLuvsCatz Posts: 40,422

    all require import and conversion, even https://www.rokoko.com/products/vision

    if you are not against ai, changing the pose on a rendered image using a reference is faster

    often free for limited goes, too many options to list but most ai websites do it

  • JamesJames Posts: 1,178

    I see, a rendered image. 

    I was thinking to change the pose on daz3d.
    It would be handy, for example like:
    The scene: A burger fast food restaurat. 
    I want to fill it with crowds. 
    Just imagine how long will I have to pose each person.  To make the scene. 
    If I have the tool like AI posing, it would cut time cost very much. 

     

  • StratDragonStratDragon Posts: 3,278

    I have Gemini pro free as a college student, it was unable to produce a usable pose file after it sounded like it was understanding the syntax of the uncompressed files and the results from multiple angles repeatedly. The time I wasted I could have built a library of poses with stone tools. if you asked it to make a pose for someone who had a building fall on them then it's useful, but you would have to ask for a pose of someone touching their nose to get it.

  • I think the main issue with AI-enabled posing, in the sense that it takes a DAZ model and applies a pose from a reference image, is that there isn't an easy mapping from the open posing standards (and even some of the commercial ones) to the DAZ model structure (e.g. Genesis 8, 9, etc). It's not only that the names are different, but there are just more keypoints, rotation standards, etc. that need to be factored in.  

  • TugpsxTugpsx Posts: 818
    edited January 20

    sidcarton1587 said:

    I think the main issue with AI-enabled posing, in the sense that it takes a DAZ model and applies a pose from a reference image, is that there isn't an easy mapping from the open posing standards (and even some of the commercial ones) to the DAZ model structure (e.g. Genesis 8, 9, etc). It's not only that the names are different, but there are just more keypoints, rotation standards, etc. that need to be factored in.  

    There is a difference with the Daz skeleton vs a lot of others and a re-mapping option is required to get the perfect pose. In ComfyUI, Openpose presents an opportunity to acquire a pose from an image (I also have a screen crab module that can capture poses from anything on your screen) and generate an image which can then be used to output a FBX / BVH file. With the correct re-mapped skeleton the BVH is usable in Daz.
    Check your ComfyUI modules for the required nodes.

     

    Post edited by Tugpsx on
  • eeyuneeyun Posts: 46

    Not sure if you're talking about the OpenPose ControlNet - but (coming from Daz Studio) I found it interesting that the skeleton it uses is entirely 2D.

  • Seven193Seven193 Posts: 1,146

    sidcarton1587 said:

    I think the main issue with AI-enabled posing, in the sense that it takes a DAZ model and applies a pose from a reference image, is that there isn't an easy mapping from the open posing standards (and even some of the commercial ones) to the DAZ model structure (e.g. Genesis 8, 9, etc). It's not only that the names are different, but there are just more keypoints, rotation standards, etc. that need to be factored in.  

    They wouldn't remap the skeleton, they would need to refit it. Select a pair of fitting points between the two skeletons that match up, leg-to-leg, arm-to-arm, etc...then run a fitting algorithm to stretch the Daz skeleton onto the OpenPose skeleton.  This is what they do to wrap a humanoid mesh onto a Daz figure mesh. If they can do it with vertices, then you can do it with joints too.

     

  • It's possible to get some output you could then use for Daz, using ComfyUI or something.  In almost all cases however, it's just easier to do a pose manually, isn't it?

    Anyway, here's Grok's opinion, if you want it.  It's doable.

  • WendyLuvsCatzWendyLuvsCatz Posts: 40,422

    just use the Rokoko one I llinked

Sign In or Register to comment.