Loading...
> > >

DAZ Director Plugin (WIP)

2»

Comments

  • sidcarton1587sidcarton1587 Posts: 116

    Tugpsx said:

    sidcarton1587 said:

    Tugpsx said:

    I see you rolled off USD support into its own project. USD Load and USD Export for Daz studio, very nice.

    I started adding to the script server but then it occurred to me that I don't want to stuff everything into it, in the interest of keeping it lean. So instead added a mechanism for other DAZ plugins to register an endpoint with the script server so that they too can be exposed via REST endpoints (and the Python package). The USD support is basically an example plugin that works that way. Now comes the hard part, actually make USD import/export work. :-)

    So true, this link forms the basis for exporting importing  in Studio . https://github.com/daz3d. It has the source for the various requirements but i'm sure you already have access to this information. 

    Yep. I've been distracted by RL and other projects -- so many projects. Hopefully I can get back to this in a couple of weeks. 

     

     

     

  • sidcarton1587sidcarton1587 Posts: 116

    Hi gang, 

     Here is the next update of the Script Server, v2.5.0. Release v2.5.0 — Interaction Posing, IK Solver, and Body Measurement Fixes · bluemoonfoundry/daz-script-server


     Updates include:

    •  Full interaction posing system for placing two figures together in physically plausible poses and aligning individual limbs to world-space targets using an iterative IK solver. This is a very experimental build, but it looks like it's at least moving in the right direction. Preset recipe builders generate a ready-to-apply InteractionRecipe from a pair of figure labels: Available presets: build_sit_recipe, build_touch_recipe, build_kiss_recipe, build_fight_recipe. All presets produce an InteractionRecipe that serialises to JSON and can be stored and replayed.
    •  IK limb alignment moves a hand or foot end-effector to a world-space target using a Jacobian-based iterative solver running inside a single batched HTTP call sequence:
    • Body Measurement Accuracy Fixes (docs/examples/body_measurements.py)
    • Torso loop selection — Slices now score each cross-section loop by centroid distance from the body centerline and pick the most-centered loop, filtering out arm cross-sections that caused ~70 cm overestimates on A-pose figures.
    • Robust outlier rejection — the heuristic fallback for figures without a gender keyword in their scene label (e.g. "MadisonG9") now discards slices above 1.35× the median before selecting the bust peak. --figure-type flag — pass G9F, G9M, G8F, G8M, G8.1F, or G8.1M to force a calibration entry when auto-detection cannot determine figure gender from the scene label.
    • A bunch of new examples with full documentation 

    • Extended Test Suite

      • All tests reorganised under tests/ package (test_dazpy.py, test_api.py, test_dazpy_integration.py, test_performance.py, harness_integration.py).
      • tests/README.md added — documents every test class and individual test across all five modules.
      • Integration harness (tests/harness_integration.py) covers the full interaction posing pipeline end-to-end; --auto flag for CI/smoke use.

     

  • TugpsxTugpsx Posts: 901

    sidcarton1587 said:

    Hi gang, 

     Here is the next update of the Script Server, v2.5.0. Release v2.5.0 — Interaction Posing, IK Solver, and Body Measurement Fixes · bluemoonfoundry/daz-script-server


     Updates include:

    •  Full interaction posing system for placing two figures together in physically plausible poses and aligning individual limbs to world-space targets using an iterative IK solver. This is a very experimental build, but it looks like it's at least moving in the right direction. Preset recipe builders generate a ready-to-apply InteractionRecipe from a pair of figure labels: Available presets: build_sit_recipe, build_touch_recipe, build_kiss_recipe, build_fight_recipe. All presets produce an InteractionRecipe that serialises to JSON and can be stored and replayed.
    •  IK limb alignment moves a hand or foot end-effector to a world-space target using a Jacobian-based iterative solver running inside a single batched HTTP call sequence:
    • Body Measurement Accuracy Fixes (docs/examples/body_measurements.py)
    • Torso loop selection — Slices now score each cross-section loop by centroid distance from the body centerline and pick the most-centered loop, filtering out arm cross-sections that caused ~70 cm overestimates on A-pose figures.
    • Robust outlier rejection — the heuristic fallback for figures without a gender keyword in their scene label (e.g. "MadisonG9") now discards slices above 1.35× the median before selecting the bust peak. --figure-type flag — pass G9F, G9M, G8F, G8M, G8.1F, or G8.1M to force a calibration entry when auto-detection cannot determine figure gender from the scene label.
    • A bunch of new examples with full documentation 

    • Extended Test Suite

      • All tests reorganised under tests/ package (test_dazpy.py, test_api.py, test_dazpy_integration.py, test_performance.py, harness_integration.py).
      • tests/README.md added — documents every test class and individual test across all five modules.
      • Integration harness (tests/harness_integration.py) covers the full interaction posing pipeline end-to-end; --auto flag for CI/smoke use.

     

    Sweet and with DS6 SDK released we can complie this to work in DS6. Thanks for the added features. Looking forward tot he updates. 

  • sidcarton1587sidcarton1587 Posts: 116

    Tugpsx said:

    sidcarton1587 said:

    Hi gang, 

     Here is the next update of the Script Server, v2.5.0. Release v2.5.0 — Interaction Posing, IK Solver, and Body Measurement Fixes · bluemoonfoundry/daz-script-server


     Updates include:

    •  Full interaction posing system for placing two figures together in physically plausible poses and aligning individual limbs to world-space targets using an iterative IK solver. This is a very experimental build, but it looks like it's at least moving in the right direction. Preset recipe builders generate a ready-to-apply InteractionRecipe from a pair of figure labels: Available presets: build_sit_recipe, build_touch_recipe, build_kiss_recipe, build_fight_recipe. All presets produce an InteractionRecipe that serialises to JSON and can be stored and replayed.
    •  IK limb alignment moves a hand or foot end-effector to a world-space target using a Jacobian-based iterative solver running inside a single batched HTTP call sequence:
    • Body Measurement Accuracy Fixes (docs/examples/body_measurements.py)
    • Torso loop selection — Slices now score each cross-section loop by centroid distance from the body centerline and pick the most-centered loop, filtering out arm cross-sections that caused ~70 cm overestimates on A-pose figures.
    • Robust outlier rejection — the heuristic fallback for figures without a gender keyword in their scene label (e.g. "MadisonG9") now discards slices above 1.35× the median before selecting the bust peak. --figure-type flag — pass G9F, G9M, G8F, G8M, G8.1F, or G8.1M to force a calibration entry when auto-detection cannot determine figure gender from the scene label.
    • A bunch of new examples with full documentation 

    • Extended Test Suite

      • All tests reorganised under tests/ package (test_dazpy.py, test_api.py, test_dazpy_integration.py, test_performance.py, harness_integration.py).
      • tests/README.md added — documents every test class and individual test across all five modules.
      • Integration harness (tests/harness_integration.py) covers the full interaction posing pipeline end-to-end; --auto flag for CI/smoke use.

     

    Sweet and with DS6 SDK released we can complie this to work in DS6. Thanks for the added features. Looking forward tot he updates. 

    Ugh. I'm still having a heck of time installing DS6 (keeps getting stuck in Retry) and the SDK doesn't show up in my IM even after following all the steps people have outlined. Very frustrating. 

    But as soon I get over those hurdles for sure I'll include DS6 versions. 

     

     

  • sidcarton1587sidcarton1587 Posts: 116

    FYI, I've added automatic nightly build of the repo so people who want to check out the bleeding edge stuff can do so prior to the formal releases. 

    See Releases · bluemoonfoundry/daz-script-server

    The releases going forward now include the DLL for Windows as well as dylibs for Mac OS Intel and Apple Silicon. 

     

  • sidcarton1587sidcarton1587 Posts: 116
    edited June 25

    Hi gang, 

     We are now building test versions of the Script Server plugin for DS 4 and DS6, for Windows, MacOS Intel, and MacOS Apple Silicon. These artifacts are available in the Nightly build for present, while we continue testing them. They will be rolled into the next version as part of the standard release. Remember that if you want to use a newer version of the plugin you must remove any old DLL or dylib file first!

    See the Artifacts section of the Nightly release here: https://github.com/bluemoonfoundry/daz-script-server/releases

    Post edited by sidcarton1587 on
  • TugpsxTugpsx Posts: 901

    sidcarton1587 said:

    Hi gang, 

     We are now building test versions of the Script Server plugin for DS 4 and DS6, for Windows, MacOS Intel, and MacOS Apple Silicon. These artifacts are available in the Nightly build for present, while we continue testing them. They will be rolled into the next version as part of the standard release. Remember that if you want to use a newer version of the plugin you must remove any old DLL or dylib file first!

    See the Artifacts section of the Nightly release here: https://github.com/bluemoonfoundry/daz-script-server/releases

    Works great in DS6. Thanks for the new samples. Your project is moving along nicely thanks for sharing. 

  • sidcarton1587sidcarton1587 Posts: 116
    edited June 29

    Hi gang, 

     So it's been a lot of work over the last couple weeks that I want to share. 

    The Script Server version 2.6.0 is released for Studio 4 and 6, for Windows, MacOS-Intel, and MacOS Apple Silicon. See here Release v2.6.0 · bluemoonfoundry/daz-script-server
    The Studio MCP Server 0.4.0 is released as well, and it works with Script Server 2.6.0+. See here bluemoonfoundry/daz-mcp-server: MCP server wrapping the DazScriptServer HTTP plugin for DAZ Studio
    The README contains a substantial information about the MCP Server and how to to use it. 

    One thing I thought might be useful is an example of a conversation I had with Claude that used the MCP server to drive DAZ Studio. Here the is the transcript of the conversation: 

       https://claude.ai/share/5fca5cd1-3861-42d9-b407-b2732bf08f1e

    As we can see from the transcript, the end-to-end workflow seems to work but there are obviously places where the MCP Server doesn't get things right or doesn't understand. Which is okay, but I'm using this to essentially teach Claude when it makes mistakes or doesn't understanding things. The file SKILL_DAZ_STUDIO.md captures some of those lessons learned, which can then be given to Claude, or any other LLM, to make sure that going forward it doesn't make the same mistakes (fingers crossed). As I continue to make improvements, some of these lessons learned will get baked into the server code itself. 

    So try it out, let me know what works and what doesn't and what you'd really like to see feature-wise. If you are a newbie to LLMs and MCP Servers and this stuff doesn't make sense but sounds cool, send me a DM and I'd be happy to guide you through the process of getting started so you can play with this.

     

    [Edit: MCP .4 requires ScriptServer 2.6.0+]

    Post edited by sidcarton1587 on
  • TugpsxTugpsx Posts: 901

    @sidcarton1587 please check your PM for the script

  • sidcarton1587sidcarton1587 Posts: 116
    edited July 17

    Hi gang, 

     For those who are tracking I'm going to be publishing ScriptServer 2.7.0 and the updated MCP Server 0.5.0 in the next few days. A lot of improvements, bug fixes, and some interesting integration examples. 

     As I am putting finishing touches on this release I am looking forward to improvements for Version 3.x and one of the things that I'm thinking about is adding a Python server, either as part of Script Server or a seperate plugin, and wanted to get some feedback on whether this would be something that could be useful. The general idea is that you can "call out" from a DAZ script in a standard way Python scripts that execute and send back results to the DAZ script. In its simplest form, this would be similar to the Script IDE pane, except that you could write Python code instead of DAZ script code. In more advanced operations, this would allow you to send data from DAZ Studio to Python which could then do computations, or communicate to an external service from there (e.g. Blender or some other tool), and return results or information back to the DAZ Script (or use Script Server to do automations in Studio based on the results).

     I'm not entirely sure what the final architecture will look like, so all of this is subject to change. Here are some potential showcase concepts of what could be possible (full disclosure: they were edit and formatted by Gemini). In some cases, you can have the Python server work with the Script Server in a loop, so you don't even need to write the script response yourself. Anyway, still mulling through concepts. Welcome discussion. 

     

    Showcase 1: The "Instant Voice Actor" (Audio-to-LipSync Animation)

    Creating realistic mouth and facial animations to match a voice-acting track is traditionally a tedious, frame-by-frame process.

    • The Concept: Drag and drop an audio file (.wav or .mp3) of a voice line. The plugin automatically generates natural lip-sync and facial expression animations on the active figure's timeline.

    • The Tech Stack:

      • Whisper (by OpenAI) or Wav2Vec2 (by Meta) to analyze the audio file, perform phoneme alignment, and detect speech timing.

      • A Python mapping dictionary that translates phonemes (like "ah", "ee", "oh", "th") directly to DAZ's native viseme and mouth-shape morphs.

    • Why it Showcases the Power: It takes a highly complex task (audio processing and phoneme extraction), wraps it in a simple drag-and-drop UI, and outputs frame-by-frame timeline keyframes in DAZ Studio.

    Showcase 2: Text-to-PBR Material Generator (Stable Diffusion)

    Creating custom textures (like rusted sci-fi armor, worn leather, or detailed skin) usually requires opening external painting software like Substance Painter or Photoshop.

    • The Concept: Select a surface on a DAZ object, type a text prompt (e.g., "highly detailed medieval plate armor, scratched steel, brass engraving"), and hit generate. Seamless, tileable textures are applied directly to the surface in real-time.

    • The Tech Stack:

      • A lightweight, local Stable Diffusion instance (or a call to an online API) to generate seamlessly tileable texture maps.

      • OpenCV to generate corresponding normal, roughness, and metallic maps from the diffuse output.

      • dazpy to dynamically construct a new Iray PBR material, assign the textures to the correct channels, and apply it to the selected geometry.

    • Why it Showcases the Power: It shows how DAZ can be integrated with generative AI workflows, allowing artists to texture scenes dynamically using natural language.

    Showcase 3 : Dynamic Weather & Lighting Sync (API Integration)

    Setting up realistic environment lighting to match a specific mood, time, or location is a trial-and-error process.

    • The Concept: Type in a real-world city (e.g., "Tokyo, Japan") or select a preset mood. The plugin syncs DAZ’s Iray environment dome, sun position, tone mapping, and fog parameters to match the actual, real-time weather and time of day in that location.

    • The Tech Stack:

      • The requests library in Python to query real-world weather and solar position APIs (like OpenWeatherMap and SunCalc).

      • Python converts the solar elevation and azimuth angles into DAZ's Iray infinite-distant light directions.

      • If the API reports "Cloudy," Python adjusts the environment map intensity and adds volumetric fog to simulate overcast lighting.

    • Why it Showcases the Power: It demonstrates how easily Python can bridge DAZ Studio with the wider internet, enabling real-world, data-driven scene generation.

    Post edited by sidcarton1587 on
  • TugpsxTugpsx Posts: 901

    That would be very useful to have this type of expansion in Studio. Looking forward to the updates. 

  • sidcarton1587sidcarton1587 Posts: 116
    edited July 19

    Hi all, 

      I went ahead and publish version 2.7.2 of the Daz Script Server

     See here: Release v2.7.2 · bluemoonfoundry/daz-script-server

     The dazpy Python package is now published on PyPi, so you can `pip install dazpy` to install it seperately to support writing your own script code. 

     Here is the dazpy API documentation: dazpy — DAZ Studio Python SDK — dazpy 2.7.2 documentation

     And here is the HTTP API documentation: DazScript Server — HTTP API Reference

    As always, the best place to comment on and request new features and bug fixes is on GitHub:

      Issues · bluemoonfoundry/daz-script-server

      bluemoonfoundry/daz-script-server · Discussions · GitHub

     

     

     

     

     

    Post edited by sidcarton1587 on
Sign In or Register to comment.