Anihead looks great what about an anibody?

takezo_3001

I love the idea of mocap from 2D video, and this is certainly the best step into that direction! Imagine, if 3D mocap info from 2D video (Not just the head) were made available for Daz Studio/Genesis, still waiting for this to become a thing!

SasquatchIsCool

There is a company that does AI mocap for like 5 bucks a month.  :)

takezo_3001

shawnmarms said:

There is a company that does AI mocap for like 5 bucks a month.  :)

Which company? Plus, that's much different than having the freedom of me/us using any video we choose (Not just PG material) for animating genesis characters... 

Leonides02

I believe he means this one:

https://getrad.co/#product

But the output is pretty lackluster.

kwannie

Takezo there is a system to extract motion from 2D. I first learned about it on Neko Sentai but there was not much discussion there.

https://forum.neko-sentai.com/showthread.php?t=14031&page=94#

I shareed the infon with Mcasual hoping he could add his genius to using it in DAZ and pdr0 here in the forums actually istalled it and got it working a bit. The newest info I had is in the link below.

https://github.com/facebookresearch/Densepose

Let us know if you try it out and can make sense out of it.

takezo_3001

kwannie said:

Takezo there is a system to extract motion from 2D. I first learned about it on Neko Sentai but there was not much discussion there.

https://forum.neko-sentai.com/showthread.php?t=14031&page=94#

I shareed the infon with Mcasual hoping he could add his genius to using it in DAZ and pdr0 here in the forums actually istalled it and got it working a bit. The newest info I had is in the link below.

https://github.com/facebookresearch/Densepose

Let us know if you try it out and can make sense out of it.

YES, This is precisely what I was talking about! Thanks for the info, I'll give it a go, there's a ton of programs/info that I never knew existed!

EDIT: You need linux for the incredibly over-complicated install process with this program, I'll simply wait until this tech matures enough for a simple executable for the program... (Or until MCasual comes up with a script!) though the forum still has links to other solutions that I may explore; I knew that the MMD crowd would be close to the solution!

wiz

Anahead looks to be a pretty simple app based off a machine learning (aka "deep learning") library called DLIB which has a couple of nicely trained head pose estimators. There have been a lot of refinements, but it all starts here, in an article called Head Pose Estimation using OpenCV and DLIB.

It needs a few other things before it's viable, like distributing the movement over the top 3 neck joints instead of using just the Atlas (which is kinda flawed in Genesis). A Mac port would be nice, too. Not everyone has access to a real computer.

AFAIK there's no "whole body" equivlent of DLIB. OpenNI crashed and burnt years ago.

takezo_3001

 

wiz said:

Anahead looks to be a pretty simple app based off a machine learning (aka "deep learning") library called DLIB which has a couple of nicely trained head pose estimators. There have been a lot of refinements, but it all starts here, in an article called Head Pose Estimation using OpenCV and DLIB.

It needs a few other things before it's viable, like distributing the movement over the top 3 neck joints instead of using just the Atlas (which is kinda flawed in Genesis). A Mac port would be nice, too. Not everyone has access to a real computer.

AFAIK there's no "whole body" equivlent of DLIB. OpenNI crashed and burnt years ago.

Yeah, I would hope there would be a one for the entire figure, we're close to that tech; there's nothing commercially available yet so it's a while off before we can use it for Daz characters!

 

wiz

takezo_3001 said:

 

wiz said:

Anahead looks to be a pretty simple app based off a machine learning (aka "deep learning") library called DLIB which has a couple of nicely trained head pose estimators. There have been a lot of refinements, but it all starts here, in an article called Head Pose Estimation using OpenCV and DLIB.

It needs a few other things before it's viable, like distributing the movement over the top 3 neck joints instead of using just the Atlas (which is kinda flawed in Genesis). A Mac port would be nice, too. Not everyone has access to a real computer.

AFAIK there's no "whole body" equivlent of DLIB. OpenNI crashed and burnt years ago.

Yeah, I would hope there would be a one for the entire figure, we're close to that tech;

Not really. A face is a pretty well defined thing, lots of landmarks, and mostly uncovered, and the head is basically one solid object. It's 6DOF (3 rotation axes, 3 translation axes). As you add degrees of freedom, things break down. Look at how much trouble Anihead has with the neck, or the fact that it can't do the mouth and eyes. And the neck can be solved in just one or two 3DOF joints. A basic skeleton like the one used for Kinect 2 has 23 joints, each 3DOF. It does add a neck joint around C4 and a joint around C7-T1 to the basic C1 "atlas" joint, which makes for much more natural movement. The hands and feet are woefully inadequate, and the entire spine is done with one joint around L1.

takezo_3001 said:

there's nothing commercially available yet so it's a while off before we can use it for Daz characters!

I could do it with a K2 or K3. Wouldn't even consider tryng with a single camera. That would involve refining guesses based on joint movement over time. It would take forever to lock.

takezo_3001

 

wiz said:

takezo_3001 said:

 

wiz said:

Anahead looks to be a pretty simple app based off a machine learning (aka "deep learning") library called DLIB which has a couple of nicely trained head pose estimators. There have been a lot of refinements, but it all starts here, in an article called Head Pose Estimation using OpenCV and DLIB.

It needs a few other things before it's viable, like distributing the movement over the top 3 neck joints instead of using just the Atlas (which is kinda flawed in Genesis). A Mac port would be nice, too. Not everyone has access to a real computer.

AFAIK there's no "whole body" equivlent of DLIB. OpenNI crashed and burnt years ago.

Yeah, I would hope there would be a one for the entire figure, we're close to that tech;

Not really. A face is a pretty well defined thing, lots of landmarks, and mostly uncovered, and the head is basically one solid object. It's 6DOF (3 rotation axes, 3 translation axes). As you add degrees of freedom, things break down. Look at how much trouble Anihead has with the neck, or the fact that it can't do the mouth and eyes. And the neck can be solved in just one or two 3DOF joints. A basic skeleton like the one used for Kinect 2 has 23 joints, each 3DOF. It does add a neck joint around C4 and a joint around C7-T1 to the basic C1 "atlas" joint, which makes for much more natural movement. The hands and feet are woefully inadequate, and the entire spine is done with one joint around L1.

takezo_3001 said:

there's nothing commercially available yet so it's a while off before we can use it for Daz characters!

I could do it with a K2 or K3. Wouldn't even consider tryng with a single camera. That would involve refining guesses based on joint movement over time. It would take forever to lock.

What I am waiting for is mocap info derived from 2D video that's already being worked on but yeah, we're still far from that being a stand alone program (and not a service) for genesis characters!

wiz

takezo_3001 said:

 

wiz said:

I could do it with a K2 or K3. Wouldn't even consider tryng with a single camera. That would involve refining guesses based on joint movement over time. It would take forever to lock.

What I am waiting for is mocap info derived from 2D video that's already being worked on but yeah, we're still far from that being a stand alone program (and not a service) for genesis characters!

Ugh. GPU farms "in the cloud", Cherrypicked cases, unusably oversimplified skeletal model, enormous lock time.

You will have some wait before that hits a point where it can realistically work a skeleton as complex as current DAZ models. If you can't get an AR/VR/MR/XR company or a game company to throw big $ at it, it's going to be a concept model forever.

 

takezo_3001

wiz said:

takezo_3001 said:

 

wiz said:

I could do it with a K2 or K3. Wouldn't even consider tryng with a single camera. That would involve refining guesses based on joint movement over time. It would take forever to lock.

What I am waiting for is mocap info derived from 2D video that's already being worked on but yeah, we're still far from that being a stand alone program (and not a service) for genesis characters!

Ugh. GPU farms "in the cloud", Cherrypicked cases, unusably oversimplified skeletal model, enormous lock time.

You will have some wait before that hits a point where it can realistically work a skeleton as complex as current DAZ models. If you can't get an AR/VR/MR/XR company or a game company to throw big $ at it, it's going to be a concept model forever.

Yep, it will indeed be a useless concept model, as this is when I originally saw the tech visualized all the way back in 2010... no one wants to rely on some website to upload/calculate their own creations, it's like having to rely on 3rd party canned mainstream-accepted general purpose animations forever, unless you have access to an actual MoCap studio! Looks like I'll be waiting for another 10 years then!

wolf359

Mocap suits are becoming more affordable however you still need specialized software to parse the data like Autodesk MOBU etc........... I doubt you will be seeing any free solutions ( in Daz studio), for this type of thing anytime soon