mcjKinectKit for Kinect V1 and in June 2018 Kinect V2 - script and exe for DS 3, 4

mCasual

 kinectV2 version iwas released June 15th 2018

https://sites.google.com/site/mcasualsdazscripts9/mcjkinectv2kit

 

https://sites.google.com/site/mcasualsdazscripts4/mcjkinectkit001
direct link to the 001v4 package
https://sites.google.com/site/mcasualsdazscripts4/mcjkinectkit001/mcjKinectKitSecretPreBetaVersion_v4.zip?attredirects=0&d=1

there will be a PC-exe application, the "server"

which will be launched by a Daz Studio script, the "client"

then the server will transmit a constant stream of skeleton poses

which the client will apply to a ( low-poly ) figure

So, as youfrantically gesticulate in front of your Kinect camera

your figure will frantically gesticulate back at you

and all this will be recordable

----
similar to what i did for the wii-mote and for the gamepads

Tugpsx

Sweet, looking forward to it's release.

mCasual

video

https://www.youtube.com/watch?v=VNwkHcRpja4

first tests the stickman posed was derived from joint positions

in this one it's derived from joint absolute angles (quaternions)

so we're getting closer to seeing Amy (Aiko3) start moving !

mCasual

some progress

i'll possibly include mechanisms that remove weird limb rotations

------

maybe the kinect server should talk to the kinect client ( a daz script ) in Daz Script language

ajd_at_072df9a27d

It is fantastic
:-)

mCasual

some walk-around mocap

throttlekitty

Way cool, thanks for sharing!

I don't have a kinect, maybe I can find a cheap one somewhere, I've always wanted to try out some of the interesting things I see people doing with it.

SimonWM

This is awesome as it is and I almost feel bad asking this question but will there be a way to connect more than one Kinect to counteract occlusion?

mCasual

SimonWM said:

This is awesome as it is and I almost feel bad asking this question but will there be a way to connect more than one Kinect to counteract occlusion?

well not in the near future

for complicated things like that you'll have to get ipisoft or breckel pro

i was supposed to wait until spring/summer before playing with the kinect
but i succumbed to temptation

so the plan is the roll out a usable kinect-mocap system for DS 1, 2, 3, 4 PC-Win7

then resume work on mcjBuilder, then the Bullet Physics engine, then the cage Deformer, then 3d painting, then ... projects i announced but didnt get to in 2010-2014 :)

--------
current issue, the kinect driver doesn't compensate the skeleton poses for the current kinect sensor tilt
(unless my kinect's accelerometer is toast? )

jaebea

Casual, you are such a blessing to this community!!

SimonWM

Casual said:

SimonWM said:

This is awesome as it is and I almost feel bad asking this question but will there be a way to connect more than one Kinect to counteract occlusion?

well not in the near future

for complicated things like that you'll have to get ipisoft or breckel pro

i was supposed to wait until spring/summer before playing with the kinect
but i succumbed to temptation

so the plan is the roll out a usable kinect-mocap system for DS 1, 2, 3, 4 PC-Win7

then resume work on mcjBuilder, then the Bullet Physics engine, then the cage Deformer, then 3d painting, then ... projects i announced but didnt get to in 2010-2014 :)

--------
current issue, the kinect driver doesn't compensate the skeleton poses for the current kinect sensor tilt
(unless my kinect's accelerometer is toast? )

All Hail Casual!!! Looking forward to this project and ALL of the rest! And a big Happy New Year to you, Sir!

mCasual

i didn't mention it in previous renders but the kinect drivers give me the figure poses in the kinect's "space"

so if the kinect is tilted the animation ( and implied ground plane ) is tilted

but that's now fixed

and i'll even bring the animation's ground level to daz studio's Y=0 level

mCasual

the pose transfer from the stickman to Amy/Aiko is not correct yet
so it will be better next demonstration

but even then, i did have to fix the animation usin mcjCycleFilter

( free script that i use a lot to clean up animations and make loops loop smoothly )

the hand and feet animations will probably always require "manual" cleanup

the rest of the body, if you're carefull will me almost immediately usable

the manual will include a section about using my free animation tools which help fix animation issues

Tugpsx

Nice progress, thanks for sharing.

ghastlycomic

This is pretty dang cool.

mCasual

improved the transfer from the stickman's pose to Amy/Aiko
the arms

there's 2 phases
1 getting the limbs to point the same way
2 getting the limbs to be twisted the same way

for the collar bones, i'll probably introduce an adjustable angle offset

note that if you're familiar with mcjParent
you could animate a target that follows a hand position
then have mcjAutoLimb2014 re-build the arm animation

same thing with the feet/legs

and using mcjCycle filter you could smoothen the target trajectories

------

improved the transfer from the stickman's pose to Amy/Aiko
the legs and feet

now i have to factor Aiko3's joint orientations
because they are rarely perfectly aligned with the World's X, Y or Z axis

this may allow us to handle figures like TheGirl

mCasual

here you can see Aiko looks extremely way better with the collarbones with an Z Rotation offset of 20 degrees

mCasual

with the collarbone rotation offset = good

with the feet glued in place using mcjAutoLimb2014 ... not so bad but not so good

will i tackle the "re-targeting" issue?

maybe ... later ... the priority is to get you something to work with if you have a kinect

( though you must know there's breckel and ipisoft and a few others ( nuicapture? actinia? ) )

mCasual

i tried having the Aiko3 figure move in real-time in Daz Studio during the motion-capture
but it seemed to reduce the capture rate too much

so for the capture shown below

i start a Daz Script named mcjKinectClient
and through it, i launch the mcjKinectServer, which is, may i remind you, a PC-Windows application

the server displays a stick-person which moves as fast as me ( very close to 30fps )

and transmits the poses to the mcjKinectClient, which records them in Aiko3's timeline

but without changing the currently displayed frame, so Aiko doesn't move during motion-capture

then when you stop the capture

you can repair the animation problems and render it

-----

but i still havethings to fix ( feet ) before the release

SixDs

Soooo...., I'm picturing you standing up in front of your Kinect camera doing Amy's funky dance for the benefit of illustration? :)

mCasual

SixDs said:

Soooo...., I'm picturing you standing up in front of your Kinect camera doing Amy's funky dance for the benefit of illustration? :)

actually that's the real-life Aiko3 which was motion captured at the beach

Exclusive Video Footage Proving it all :
https://www.youtube.com/watch?v=TFn286BeIQ0

mCasual

another Video Demonstration

here i used the very low-polygon-count SecondLife poser-figure

and did motion capture with real time animation in daz studio

if you have a fast PC, you'll probably be able to use more complex figures

or, do the motion capture with just the green stickman as your feedback

https://www.youtube.com/watch?v=3oBtAeEW-2s

note the lag between the Kinect Server window and the Daz Studio viewport
it's mainly because the Kinect server ( for now ) accumulates a few
poses before sending them to the daz script client )

mCasual

and the result of the motion capture session, rendered by daz studio
after a few corrections done using mcjKeepOrient and mcjRepeatAction-drop-to-floor

https://www.youtube.com/watch?v=In_mIIWdDls&feature=youtu.be

ghastlycomic

This is looking really cool. Damn you got the programming skills for sure.

I'm going to have to get the USB adaptor/power supply for my kinect.

mCasual

ghastlycomic said:

This is looking really cool. Damn you got the programming skills for sure.

I'm going to have to get the USB adaptor/power supply for my kinect.

if you google for

amazon kinect adapter

that gets you the type i use, it was about $8

i read that some of the old kinects don't work on the PC ,

but the ones that were sold with the "kinect adventures" game pack ( what i bought) do work

mCasual

still working on it

one of the issues is the weight of the animation data

another is ( possibly ) being able to record the kinect's audio during motion-capture

16KHz audio i think ... that must not be a big burden for the CPU(s)

and i want by record/stop buttons .... ( someday maybe speech-recognition-based )

3dOutlaw

Casual said:

ghastlycomic said:

This is looking really cool. Damn you got the programming skills for sure.

I'm going to have to get the USB adaptor/power supply for my kinect.

if you google for

amazon kinect adapter

that gets you the type i use, it was about $8

i read that some of the old kinects don't work on the PC ,

but the ones that were sold with the "kinect adventures" game pack ( what i bought) do work

Is it the EastVita one you mention? This looks cool!!!

mCasual

3doutlaw said:

Casual said:

ghastlycomic said:

This is looking really cool. Damn you got the programming skills for sure.

I'm going to have to get the USB adaptor/power supply for my kinect.

if you google for

amazon kinect adapter

that gets you the type i use, it was about $8

i read that some of the old kinects don't work on the PC ,

but the ones that were sold with the "kinect adventures" game pack ( what i bought) do work

Is it the EastVita one you mention? This looks cool!!!

mine is exactly like that
http://www.amazon.ca/Xbox-Kinect-Adapter-Power-Supply/dp/B004IXRXGY

the other brands, as long as they look like that and have a similar price should work

Toyen

Wow! What a great idea for motion capture at home! : )

Are you planning to make this compatible with DS only?

Makes me wish I had a kinect : )

mCasual

Toyen said:

Wow! What a great idea for motion capture at home! : )

Are you planning to make this compatible with DS only?

Makes me wish I had a kinect : )

eventually i may have a version that outputs BVH or Poser-Animations, so that could be used in Poser/Carrara and others

note that there's free versions of Breckel, and (maybe?) IPI-Soft which let you do motion capture with the kinect and save them to disk as BVH, which is a format that many 3d-software can use

a few years ago i did this motion capture with my kinect and breckel -- i did have to use many of my other free scripts to correct problems with the raw animation from Breckel

https://www.youtube.com/watch?v=wYEv7gtg68w

also, be aware that importing BVH animations ( including the 1000s of free CMU BVHs ) in Daz Studio is often problematic - i found that importing the BVHs and applying them to the free SecondLife avatar, usually works, whereas importing the BVH and applying it to Aiko/Victoria or Genesis often fails

after successfully importing the BVH and applying it to the SecondLife figure, you can save it as a Daz Studio Pose-Preset-animation, then apply that to the Aiko3/Victoria3/4 or Genesis figure

But the Breckel and IpiSoft forums probably discuss those issues at length, and you can probably now, do your motion capture, and bam apply it to Genesis - newer versions of Daz Studio like 4.7 also have improved BVH handling

mCasual

I'm sure Darrin Stephens would approve of my marketing skills.

the start/stop and led buttons work, i have to implement the delayed-start

and possibly i'll add a control for the kinect tilt-motor
( one of the quirks of the kinect is that we are only allowed to activate that motor a few times and let it rest between activations )

and i guess the control of the smoothing will also be part of the server