Digital Art Zone

 
   
1 of 2
1
mcjHIDKit - live puppetry in daz studio using a microphone and a gamepad (WIN-PC)
Posted: 14 October 2012 07:15 AM   [ Ignore ]
Power Member
Avatar
RankRankRank
Total Posts:  1998
Joined  2006-01-19

────────────────────────────────────────────

download and manual——-> https://sites.google.com/site/mcasualsdazscripts2/mcjhidkit

────────────────────────────────────────────
History
────────────────────────────────────────────
Feb 7 2013 10:00 pm Handles higher data rates like 30 Hz
Feb 6 2013 3:00 am   initial release, DS1, DS2 versions non-operational
Feb 6 2013 3:20 am   version for DS1 DS2 fixed
Feb 6 2013 4pm de-selection of target nodes while the script is running - practical for Live Shows and screen-captures
────────────────────────────────────────────

you will be able to control a character using a gamepad ( ... a gamepad with an HID driver )

and any sound present at the microphone ( or soundcard in general ) will be usable to drive a morph channel like “mouthOpen”

all this can be done live and can be recorded on Daz studio’s timeline

in the emotiguy example shown below, i recorded movements generated using the microphone,
in 3 passes, one for the mouth-open morph, one for the wink morph and one for the smile morph

but i could’ve done it in 1 pass or live, using the mic + 2 joystick buttons

the system should work with most modern (HID) gamepads like my logitech usb-rumblepad

if will work only on a windows PC since it relies on an exe program called the “server”. On Studio’s side the ‘client’ is a daz script which should be usable on DS 1, 2, 3, 4, 4.5

Image Attachments
complexanim.gifrumble.pngoyoyo.gifcliser.png
Profile
 
 
Posted: 14 October 2012 03:38 PM   [ Ignore ]   [ # 1 ]
Member
Rank
Total Posts:  121
Joined  2004-10-23

This is another very interesting project. I like where you are going in the development of interactive animation for Daz Studio.
I wonder if the use of an Xbox Kinect would serve as an input device. Whereas the information generated by the onscreen figure could be segmented and used to drive the character movement.

I know it would take a decent resolution for facial movement, but for starters, body movement such as raising an arm, bending hands, lift a shoulder or even fingers may be within the relm of the project as it gets refined.

Again thanks for sharing your projects with us and we look forward to the results of your quest to enhance Daz Studio and more.

Profile
 
 
Posted: 15 October 2012 06:53 AM   [ Ignore ]   [ # 2 ]
Power Member
Avatar
RankRankRank
Total Posts:  1998
Joined  2006-01-19
Tugpsx - 14 October 2012 03:38 PM

This is another very interesting project. I like where you are going in the development of interactive animation for Daz Studio.
I wonder if the use of an Xbox Kinect would serve as an input device. Whereas the information generated by the onscreen figure could be segmented and used to drive the character movement.

I know it would take a decent resolution for facial movement, but for starters, body movement such as raising an arm, bending hands, lift a shoulder or even fingers may be within the relm of the project as it gets refined.

Again thanks for sharing your projects with us and we look forward to the results of your quest to enhance Daz Studio and more.

eventually i may try something with the kinect, there’s OpenKinect OpenNI and brekel which i gather have all that’s needed to get skeletal poses from a kinect connected to a PC . Last year James Walsh http://vimeo.com/19570277 had a working kinect plugin for daz studio - there may be posts about it elsewhere in the forum. Longer ago he was able to implement GLSL shaders like chrome, in Studio’s 3dviewport

a few weeks ago i started making tests with OpenCV (computer vision) , i bought an HD webcam, and made a baseball cap with 2 colored markers on it, and wrote a program that reports the position of the markers and forwards them to daz studio, 10 or 30 times per second. i’ll eventually get back to that project. With more complex processing i could also detect and follow eyes. with 3 markers i could get head position and orientation.

there’s a system that exists with 3 IR beacons on a baseball cap + a webcam, and can be used for head tracking, it’s compatible with some PC Games.

with my WiiKit it wouldnt take much work to so things like this http://www.youtube.com/watch?v=Jd3-eiid-Uw

Profile
 
 
Posted: 15 October 2012 07:09 AM   [ Ignore ]   [ # 3 ]
Member
Rank
Total Posts:  121
Joined  2004-10-23

Awesome. Keep up the great work we all look forward to your projects.
.

Profile
 
 
Posted: 16 October 2012 10:39 AM   [ Ignore ]   [ # 4 ]
Power Member
Avatar
RankRankRank
Total Posts:  1998
Joined  2006-01-19

the buttons will be able, to work as momentary switches,

meaning, you press a button, the animated object jumps to its full-on position

or

they will function as dampened switches,

meaning, you press the button, the animated object ramps up until it reaches the full-on position, 

stays there as long as you press the button,

and when you release the button, it ramps down until it reaches its full-off position

—-

the finite state machine ( http://en.wikipedia.org/wiki/Finite-state_machine )
for the dampened buttons

Image Attachments
motorized_button.png
Profile
 
 
Posted: 16 October 2012 06:02 PM   [ Ignore ]   [ # 5 ]
Power Member
Avatar
RankRankRank
Total Posts:  1998
Joined  2006-01-19

here you can see an animation done live using 2 gamepad buttons
they are “dampened”, with an “attack” and “release” time of half a second

 

Image Attachments
dampen.gif
Profile
 
 
Posted: 16 October 2012 10:29 PM   [ Ignore ]   [ # 6 ]
Member
Rank
Total Posts:  121
Joined  2004-10-23

Wow you work fast. Great idea.

Profile
 
 
Posted: 17 October 2012 01:18 PM   [ Ignore ]   [ # 7 ]
Power Member
Avatar
RankRankRank
Total Posts:  1998
Joined  2006-01-19

adding push-on-push-off outputs for each button

i guess there must be other types i need to add, like dampened push-on-push-off

trying to think where the complete list of button types is posted , i would think, on a site for PLCs

Image Attachments
popo.jpg
Profile
 
 
Posted: 19 October 2012 03:58 PM   [ Ignore ]   [ # 8 ]
Power Member
Avatar
RankRankRank
Total Posts:  1998
Joined  2006-01-19

in red you can see the undampened output of Button #1 of the game pad was used to animate the YTranslate parameter of a Null node

and in red you can see the dampened output of the same button #1 was used to animate the YTranslate of a another null node

then mcjTransport was used to make the paths of those 2 nulls visible ( amd setInterpolation script was user to linearize the speed of the nulls )


in the second picture, you can see in green the unfiltered output for the gamepad button #1

and in red the push-on-push-off output for the gamepad button #1

Image Attachments
damppp.jpgkeen.jpg
Profile
 
 
Posted: 21 October 2012 12:50 PM   [ Ignore ]   [ # 9 ]
Power Member
Avatar
RankRankRank
Total Posts:  1998
Joined  2006-01-19

the dampened push-on-push-off buttons

press and release, the output of the switch will ramp from 0 to 1
then stays at the 1 level
press again the output of the switch will ramp from 1 to 0
and stay at the 0 level

etc ...

Image Attachments
dampenedpopo.png
Profile
 
 
Posted: 18 December 2012 02:21 PM   [ Ignore ]   [ # 10 ]
New Member
Total Posts:  6
Joined  2012-07-17

awesome purchased key rig and daz had the nerve 2 tell me 2 downgrade 2 daz 3.x in order 2 use this….make this work and i’ll be the first to pay $20 bucks 4 it…u already a legend…me thinks this will make u a god!!

Profile
 
 
Posted: 18 December 2012 02:23 PM   [ Ignore ]   [ # 11 ]
New Member
Total Posts:  6
Joined  2012-07-17

honestly i think the best use of the game-pad is controlling the camera…just play mw3…

Profile
 
 
Posted: 18 December 2012 02:46 PM   [ Ignore ]   [ # 12 ]
Power Member
Avatar
RankRankRank
Total Posts:  1998
Joined  2006-01-19
yungexec - 18 December 2012 02:21 PM

awesome purchased key rig and daz had the nerve 2 tell me 2 downgrade 2 daz 3.x in order 2 use this….make this work and i’ll be the first to pay $20 bucks 4 it…u already a legend…me thinks this will make u a god!!

actually i’m almost certain this will work on all versions of Daz Studio

it’s the same system as the wii-kit,
( https://sites.google.com/site/mcasualsdazscripts/mcjwiikit-for-daz-studio-3-pc-version-zero-beta-not-stable-yet )

on Daz Studio’s side of things, , there’s a script, which is compatible with Daz Studio 1, 2, 3, 4 ( not sure if i tested it on DS4.0 or 4.5)
it launches a program (for windows only) that constantly watches what the gamepad is doing

good thing you reminded me i was forgetting this project !

so i’ll finish the dress for Aiko3 and get back to it before xmas

Profile
 
 
Posted: 26 January 2013 10:36 AM   [ Ignore ]   [ # 13 ]
Power Member
Avatar
RankRankRank
Total Posts:  1998
Joined  2006-01-19

been working on it a bit today

i linked the buttons of the gamepad (logitech dual action rumble) to the YTranslate of the button objects

the 2 analog joysticks ( 4 potentiometers) were linked to the XTranslate and ZTranslate coordinates of 2 button objects

here’s the content of the links table

link joy.btn.1.val button_1 Y Translate -50 0
link joy
.btn.10.val button_10 Y Translate -50 0
link joy
.btn.11.val button_11 Y Translate -50 0
link joy
.btn.12.val button_12 Y Translate -50 0
link joy
.btn.2.val button_2 Y Translate -50 0
link joy
.btn.3.val button_3 Y Translate -50 0
link joy
.btn.4.val button_4 Y Translate -50 0
link joy
.btn.5.val button_5 Y Translate -50 0
link joy
.btn.6.val button_6 Y Translate -50 0
link joy
.btn.7.val button_7 Y Translate -50 0
link joy
.btn.8.val button_8 Y Translate -50 0
link joy
.btn.9.val button_9 Y Translate -50 0
link joy
.or.1 button_13 Z Translate 1 -178
link joy
.or.2 button_13 X Translate 1 306
link joy
.or.3 button_14 Z Translate 1 -178
link joy
.or.4 button_14 X Translate 1 128 

 

Image Attachments
nubuttonz.gifblinka.gif
Profile
 
 
Posted: 03 February 2013 09:52 PM   [ Ignore ]   [ # 14 ]
Power Member
Avatar
RankRankRank
Total Posts:  1998
Joined  2006-01-19

~ news update ~~ news update ~~ news update ~~ news update ~~ news update ~

just a quick news bulletin to say, i worked on the gamepad server side of things

and now have a steady stream of data coming out of it

the release web page will be here :

https://sites.google.com/site/mcasualsdazscripts2/mcjhidkit

i’ll possibly tie all loose ends tomorrow and release it !

the image below shows the 4 ouputs for button 1 of my gamepad

raw value, dampened, push-on-oush-off-dampened and push-on-push-off

the status of the audio and the gamepad were sent from the server to the client in daz studio 10 times per second.

the final version will probably include the ability to increase the sampling rate to 30 times per second

Image Attachments
workin.jpgmcjClientDS1DS2.ds_.pngmcjClient.dsa_.png
Profile
 
 
Posted: 05 February 2013 05:40 PM   [ Ignore ]   [ # 15 ]
Power Member
Avatar
RankRankRank
Total Posts:  1998
Joined  2006-01-19

the latest version of the HIDServer will display the states

you will also have the opportunity to change the report rate, the ramp up and ramp down times for the “dampened” buttons

in the third image you can see the report rate was 10 Hertz, the ramp up and ramp down times were 5000 ms

Image Attachments
id.jpghid2.jpgdpopop.gif
Profile
 
 
   
1 of 2
1