DAZ Director Plugin (WIP)

sidcarton1587

So as I mentioned in a different post, I've started tinkering with integrating DAZ Studio with AI-assist to define the art of the possible. What I want to be clear about up-front is that this is not about AI image generation, and is not related to the DAZ AI Studio product released by DAZ. 

What I am talking about is using AI to help Studio users accelerate their workflows where they need the accelerating. The AI in this case becomes your "gopher" (agent in AI parlance), to get you past tedious tasks so you can focus on what's most interesting to you. That being said, what's tedious to you might be the most enjoyable part to someone else. So this tool would not be intended to replace the existing Studio interface, but support it. 

Anyway, let's get down to brass tacks. What are we talking about exactly?
 

Here's the "marketing" literature response:

DAZ Director connects an AI — specifically Claude but we can also use other models or local and public models— directly to DAZ Studio. Not as a helper that writes scripts you then have to paste and run yourself. Not as a chatbot that gives you instructions to follow manually. As a director. You describe what you want. It happens.

Here's what that looks like in practice.

You open a conversation pane (just another tab pane just like any other in DAZ Studio) and start talking about a scene. You want a portrait setup — a female character, dramatic and elegant, in a minimal interior. The AI talks through the possibilities with you: what kind of lighting would suit the mood, what camera angles would work, what the scene needs. Then, when you're ready, it starts building.

It searches your DAZ asset library — your actual installed content — finds a suitable character and environment, loads them into the scene, and positions everything. Then you say: "Create a cinematic close-up camera for her face, slightly low angle." The camera appears, framed exactly as described. You say: "Add a three-point lighting setup — soft portrait lights, warm key, cool fill." The lights are created, positioned, and configured. You say: "Now give me a camera animation that starts far back in the darkness, rushes in from behind her, and sweeps around to show her face." The keyframes are set. Press play.

None of that required opening a menu.

But here's what's equally important: at any point in that process, you can put down the conversation and pick up the mouse. Drag a light to a slightly different position. Tweak a morph slider by hand. Adjust the camera framing by eye. DAZ Director doesn't take over your workspace — it works alongside it. The dials and sliders you already know are still there, still yours, still often the fastest way to get exactly what you want.

So right now the basics can be demonstrated. The plugin can take a command do some pretty interesting things -- those things the marketing literature mentions above like finding content, adding it to the scene, moving cameras around, creating lights, all using natural language commands -- but it's not foolproof and it has many limitations that need to be worked. 

What I am hoping to get out of this discussion are answers about how people use DAZ and how something like this could help them use it better, more enjoyably. 

Some questions that immediately come to mind include:

• Is something like this really useful for DAZ Studio users? I'm doing this a proof of concept, but the concept keeps expanding. 
• What kinds of natural language command would you love to see? Full scene descriptions? Help posing characters? Expressions? Animations? What parts of your DAZ workflow do you find the most tedious?
• Keeping in mind that this doesn't come for free. Even if the plugin was low-cost or free, there is still the matter of paying for the AI charges. It would involve purchasing API credits from someone like Anthropic (Claude) or OpenAI (ChatGPT) or Google (Gemini) or downloading and using your own models -- with the understanding that that is more technically involved and works better when you have local resources to handle it (like a GPU with more 12GB+ VRAM for example). Is that tradeoff or cost something that you would consider?

Anyway, I'm in the process of putting together a demo video of the basic capabilities, and a more detailed post about the technical details, in other forums.  But I thought I'd put this out here to foster discussion. 

Tugpsx

Interesting, I have had a few projects on the back burner including a Pose Executor that uses phrases to pose characters in a scene even for animation.

sidcarton1587

Tugpsx said:

Interesting, I have had a few projects on the back burner including a Pose Executor that uses phrases to pose characters in a scene even for animation.

So did this incorporate an LLM or straight up NLP or something else? Honestly I've left posing as a special case that requires very specific attention. One of the hardest things to get right.