Remixing your art with AI

1323334353638»

Comments

  • WendyLuvsCatzWendyLuvsCatz Posts: 41,525
    edited August 11

    I have never rendered motion blur in DAZ Studio either...

    I doubt many modern cinematographers do either, nowadays everything is added in production 

    easier to edit in aberrations than remove them

    only homemovies have motion blur and photographers using film and a longer exposure 

    postwork is a thing

    Post edited by WendyLuvsCatz on
  • 3DIO3DIO Posts: 639
    edited August 11

    Again, you're talking about postworking, but what I'm getting at is how to access settings directly so that it can be done properly, and to a professional standard.

    By postworking output in an attempt to conform it to a specific standard, you're basically putting lipstick on a pig.  Much better is to have the  A.I. render-out the video using the correct settings right from the start.  I hope to be experimenting with this stuff myself soon enough, but basically that's what I really want and need it to do.

    Anyway, no biggie, I just though someone might fancy the challenge of testing it.

     

    Post edited by 3DIO on
  • @3DIO nope no luck yet. My ai got distracted. Anyone recognize this background? https://www.deviantart.com/barefootuptomysoul/art/EXT-AI-TOWN-NIGHT-1301923825
  • donniekeidicdonniekeidic Posts: 138

    Barefoot Upto My Soul said:

    @3DIO nope no luck yet. My ai got distracted. Anyone recognize this background? https://www.deviantart.com/barefootuptomysoul/art/EXT-AI-TOWN-NIGHT-1301923825

    Chinatown by Stonemason, oldie but a goodie

  • Chinatown by Stonemason, oldie but a goodie

    You are right! But my ai render changed the Chinese writing from dry clean only to hand wash.
  • edited August 23
    Artini said:

    Continuing my adventure with the Cat...

    That cat looks so serious LOL Onions and garlic are toxic poison for cats.
    Post edited by Barefoot Upto My Soul on
  • TugpsxTugpsx Posts: 921

    3DIO said:

    Again, that's a corrective measure and it's never the same as shooing (or in this case rendering) a scene at the correct shutter speed to start with.

    Bear in mind that in order to do what they do, A.I. models must have an internal grasp of Frame Rate, Shutter Speed, ISO. F-stop etc, so the question is how do you get at those parameters in order to render out something professional enough to either composite or match real footage?

    Since I never see these options, I'm hoping that it would undertand such instructions.  I've seen a video where an A.I. model understood lens type (which is promising), but that's not nearly enough for producing output suited to professional video work, or for producing something you can throw into DaVinci to use in an optimal, specifically intended way.

     

    Have you looked at some of the ComfyUI workflows available within ComfyUI interface?

    Some of the workflows actually have camera controls. 
    There is also a workflow from Avalon on YouTube (I bought his AI package, great set with seeds, workflow, prompt engineering, starter images, audio tracks for folley, music , speech, action etc). 

    Some interesting tricks is stating the camera model and type being used, depending on the AI model you use and it's awareness of real world cameras, you can prompt in or out motion blur. Set camera back information and more, not just lens. 
    There are a few real world camera assets here in the Daz store that can make a difference in your quest. Lots of experimenting with different camera settings and you might be able to get what you're looking for. 

  • 3DIO3DIO Posts: 639
    edited August 23

    @BarefootUptoMySoul
    On the plus side, it means that Evil Onion is safe around Richard!

    @Tugpsx
    Thanks for the heads-up on that, and I have to say that's the most promising thing I've heard so far in that respect.  The moment you mentioned being able to set "Camera Back" as well as lens properties, a huge smile crept across my face!

    Absolutely, I'm planning to use ComfyUI.  In fact I'm learning A.I. faster than I otherwise would be by watching videos about ComfyUI.  I've never seen the sort of tool you describe, but man, I'm damn glad to hear that one exists since I think the whole system would be very crude and unprofessional without it.

    Do you know the name of the ComfyUI tool that allows you to set that sort of stuff?
    Also, is there a term that should be used to search for that specific type of node?

     

    Post edited by 3DIO on
  • TugpsxTugpsx Posts: 921
    edited August 24

    3DIO said:

    @BarefootUptoMySoul
    On the plus side, it means that Evil Onion is safe around Richard!

    @Tugpsx
    Thanks for the heads-up on that, and I have to say that's the most promising thing I've heard so far in that respect.  The moment you mentioned being able to set "Camera Back" as well as lens properties, a huge smile crept across my face!

    Absolutely, I'm planning to use ComfyUI.  In fact I'm learning A.I. faster than I otherwise would be by watching videos about ComfyUI.  I've never seen the sort of tool you describe, but man, I'm damn glad to hear that one exists since I think the whole system would be very crude and unprofessional without it.

    Do you know the name of the ComfyUI tool that allows you to set that sort of stuff?
    Also, is there a term that should be used to search for that specific type of node?

     

    Check out Comfyui-prompt-camera-assistant, it should get you going. 

     

    PromptCameraAssistant

    Inputs

    • Mode: Still / Video
    • Camera
    • Lens type
    • Film / Sensor look
    • Aperture (f-stop)
    • ISO
    • Shutter speed
    • Bokeh style
    • Focus style
    • Motion blur preset
    Post edited by Tugpsx on
  • ArtiniArtini Posts: 10,903

    Barefoot Upto My Soul said:

    Artini said:

    Continuing my adventure with the Cat...

    That cat looks so serious LOL Onions and garlic are toxic poison for cats.

    Thanks, I did not know about that.

     

  • WendyLuvsCatzWendyLuvsCatz Posts: 41,525

    well, before I used the internet, I didn't know,

    Mum's Eddy finished off all our bolognaise, stews etc full of garlic and onions and he lived to 18 when I finally had him euthanased a year after mum's passing.

    he was very fragile by then and had kidney disease but didn't get it until 16 at least

  • 3DIO3DIO Posts: 639

    Tugpsx said:

    Check out Comfyui-prompt-camera-assistant, it should get you going. 

     

    PromptCameraAssistant

    Inputs

    • Mode: Still / Video
    • Camera
    • Lens type
    • Film / Sensor look
    • Aperture (f-stop)
    • ISO
    • Shutter speed
    • Bokeh style
    • Focus style
    • Motion blur preset

    Cheers mate, but still no luck here.  Can't find a thing about it, but then search for me has been going rapidly downhill for some time now and is pretty much useless.

    All I can find are videos about prompts, and nothing whatseover about nodes for controlling shutter speed or ISO.  If you know of a specific video that demonstrates "PromptCameraAssistant" in use, please post it so that I can check it out, cause I cannot find a thing about it.

     

  • TugpsxTugpsx Posts: 921

    3DIO said:

    Tugpsx said:

    Check out Comfyui-prompt-camera-assistant, it should get you going. 

     

    PromptCameraAssistant

    Inputs

    • Mode: Still / Video
    • Camera
    • Lens type
    • Film / Sensor look
    • Aperture (f-stop)
    • ISO
    • Shutter speed
    • Bokeh style
    • Focus style
    • Motion blur preset

    Cheers mate, but still no luck here.  Can't find a thing about it, but then search for me has been going rapidly downhill for some time now and is pretty much useless.

    All I can find are videos about prompts, and nothing whatseover about nodes for controlling shutter speed or ISO.  If you know of a specific video that demonstrates "PromptCameraAssistant" in use, please post it so that I can check it out, cause I cannot find a thing about it.

     

    Prompt Camera Assistant is availabel using Comfy Manager node installer. Search for "Camera"

      

  • 3DIO3DIO Posts: 639
    edited August 24

    Ah right, you mean to search from ComfyUI.  Can't at the moment since I had to uninstall it some months back because I broke the install through messing with it.  Still, you would think that something as important as that would have videos on YouTube about it.

    Apparently not, which is very strange!

     

    Post edited by 3DIO on
  • TugpsxTugpsx Posts: 921

    3DIO said:

    Ah right, you mean to search from ComfyUI.  Can't at the moment since I had to uninstall it some months back because I broke the install through messing with it.  Still, you would think that something as important as that would have videos on YouTube about it.

    Apparently not, which is very strange!

     

    If you are using the portable, there is a folder called update  that has tools to quickly reset comfy to a working version. Try the update stable to get you back to a stable version quickly.
    ComfyUI-Prompt-Camera-Assistant 

  • LindseyLindsey Posts: 2,017

    A year ago, I dabbled in using Stable Diffusion to generate AI images.  I tried a couple of WebUI's with Stable Diffusion: Automatic 1111 and Comfy Desktop.  There was a bit of a learning curve and was getting all sorts of weird images because didn't take the time to learn prompting techniques.  I've been using Krita (Krita.org) as a Photoshop replacement and found there was Stable Diffusion plugin and taps into a ComfyUI Server.  It integrates AI functionality into Krita.  It comes with a selection of checkpoints (models) you can install. I add free checkpoints and loras from Civitai.com.  All of this is free you can run on your own PC.  Go to Krita.org for the paint program then install the Krita AI Plugin (search for its installer).  It all very easy to download and install.  When you install the Krita AI plugin you're giving the option of creating a ComfyUI server that is run locally and controlled within Krita, or you can connect to an existing ComfyUI server that is externally managed.  The AI plugin can modify your DAZ renders by controlling the strength of applying a checkpoint.  One can even select parts of a DAZ image and render in-fills/modifications or render a 100% AI image.  

    I'm suprised how you can enter a very generalized idea of an image and perform a sequence of renders.  I found the prompt shown on the Krita screenshot on the internet. I've attached three pictures. One of the Krita screen with the AI "docker", and two images I picked out from a set of consecutive renders from the prompt.  Rendered the images at 3840 x 2160 to make desktop backgrounds. Many checkpoints for realistic renders look best in quite smaller resolutions due to the images they are trained on.  I used a digital image checkpoint that was installed with the Krita AI plugin that seem to render ok in a large size renders.

    Tip: To display the AI Plugin in Krita if you don't see it, go to Settings>Dockers> and enable the AI Image Generation check box. On the AI docker, select the gear box upper right, then select Connection to setup a local ComfyUI Server controlled within Krita or connect to existing ComfyUI server (didn't try that).

    Krita.jpg
    3857 x 2177 - 1M
    Tea_House 1.jpg
    3840 x 2160 - 822K
    Tea_House 2.jpg
    3840 x 2160 - 879K
  • 3DIO3DIO Posts: 639
    edited August 25

    @Tugpsx
    Thanks for the link, and wow, that is what I'm talking about!

    What that plugin does is 100% essential for anyone wanting to use this AI stuff to a professional standard.  The bloke who made that node deserves a medal for what he did, since he's the only one I've seen so far who truly understands the need for these parameters!  It's absolutely mind-blowing to think that this stuff (which is ground zero essential knowledge for professional video work) is not the first thing these so-called "Experts" on YouTube talk about when "Advising" people how to achieve professional video!  It honestly makes me wonder what they must be teaching in professional video courses these days!

    Then again, I've come to expect nothing more than click-bait on YouTube, and that's all 99% of these opportunists seem to exist for;  basically wasting everyone's time so that some click-baiter can make a few pence on a video.

    Trying my best not to rant here, but it's (very) telling of how bad things have actually gotten, while at the same time, being totally laughable.  But anyway, thanks again, and I'm not kidding that if such a node did not exist, then A.I. would be nothing more than a passing curiosity to me.

    @Lindsey
    Assuming you mean the "Krita-AI-Diffusion" plugin, I do agree it's great plugin!  It's one I tried to get running some months back, but sadly couldn't on my AMD card.  But the videos I saw of it were very good, and it gives a fantastic workflow being inegrated into Krita like that, even tappping into the power of ComfyUI to drive it.

     

    Post edited by 3DIO on
  • cosmosmcosmosm Posts: 100

    WendyLuvsCatz said:

    well, before I used the internet, I didn't know,

    Mum's Eddy finished off all our bolognaise, stews etc full of garlic and onions and he lived to 18 when I finally had him euthanased a year after mum's passing.

    Might be one of those things that breaks down when cooked.

    idk.   I know that my Taffy seemed to treat the raw ones as toxic waste, I felt so sorry for her the day my friend and I were cutting vegetables and one of us dropped a bit of onion (I think it was a scrap of dry peel) and she fled.   laugh

  • 3DIO3DIO Posts: 639
    edited September 3

    Possibly of interest to the AI crowd is Yamaha's new Omnivocal.  It uses AI, but is based on Yamaha's very own AI synthesis technology that can be edited real-time!

    It's currently in beta but is available in Cubase, and since Cubase is owned by Yamaha, they've made it so that it runs only in Cubase.  Quite basic at the moment, but even so, you already get to automate the individual parameters.  And as suggested in the first video, since this is just the beta, you can ony imagine what's coming, likely in Cubase Pro 16 when it releases with a bit of luck!

    The good thing about it being locked into Cubase, is it signals they thoroughly intend to put a lot of effort into it, no doubt use it to sell more copies of Cubase.  It's pretty good even in beta, so I'm majorly looking forward to what they add to it in the next release.  I'm picking-up Cubase Pro 16 when it releases anway, so Omnivocal is really a very nice bonus!

    I was going to download the demo, but didn't bother since I might as well wait for 16 to arrive first, something I really look forward to!
     

     

    Post edited by 3DIO on
  • ArtiniArtini Posts: 10,903

    Thanks for the heads up.

    These are pretty old videos (8-9 months old). The one below is more recent.

     

  • csaacsaa Posts: 1,074

    @Lindsey
    Assuming you mean the "Krita-AI-Diffusion" plugin, I do agree it's great plugin!  It's one I tried to get running some months back, but sadly couldn't on my AMD card.  But the videos I saw of it were very good, and it gives a fantastic workflow being inegrated into Krita like that, even tappping into the power of ComfyUI to drive it.

     @Lindsey3DIO:

    Thanks for the Krita-AI-Diffusion endorsement. I saw the plugin in Krita but didn't have much time for it. I too have  an AMD GPU and the idea of running ComfyUI server locally intrigues me. I'll give this a shot; if not, then I'll try the external server. 

    Cheers!

     

  • 3DIO3DIO Posts: 639
    edited September 4

    @Artini
    Perhaps there's some confusion.  Omnivocal is a vocal synthesizer plugin that is only available within the 'Elements', 'Artist' and 'Pro' versions of Cubase.

    The video you posted is not about Omnivocal, it's about getting started with Cubase (which will still come in handy for beginners, mind you).  But to be clear, the 'AI' and 'LE' vesions of Cubase do not come with the Omnivocal beta (as far as I'm aware).  I can imagine that people new to Cubase might think the 'AI' version has something to do with A.I, but it's got nothing to do with A.I.

    The Omnivocal plugin uses A.I. technology, but it's Yamaha's own A.I. technology that can be edited real-time.  This means it runs locally and there are no issues using the vocals on commercial records (Cubase is a professional DAW (Digital Audio Workstation)).  Omnivocal first appeared with the launch of the current version (15), which is why you won't see much about it until it comes out of beta and Yamaha start promoting it through Cubase (Yamaha own Steinberg and Cubase (and Cubase is packed with the same technology used on Yamaha hardware)).

    The official page for Omnivocal  is this very basic one  for now, but one of the most important aspects of Omnivocal, is this part :

    Yamaha Said :
    "A growing lineup of voice banks is coming soon. All voice banks can be used freely, including for commercial music,
    without requiring any permission or credit from Yamaha or Steinberg."


    I'm assuming the business model is to keep Omnivocal a Cubase-only plugin in order to attract more people to Cubase, and to fund development by charging for voicebanks in much the same way that Daz charge for content.  Personally I'll be going for a husky, smoky-sounding female voicebank.  Someone who would sound equally good at Jazz and Rock music.  The demonstration voicebanks it comes with are suited more to pop music I think.

    But do you see the real genius in doing it Yamaha's way?  Remember, the problem we face with A.I. is that there's no reliable way to prove which artist was used in the generation of the A.I.output, and so the real artist doesn't get paid.  With Omnivocal, though, you can bet your life that eventually, they will make available the voicebanks of famous singers.  Every one of them will be painstakingly converted into Yamaha's Omnivocal format by their engineers.  This is much better for the artist, since the artist gets a cut of the sale of every voicebank sold of their voice  :-)

    Way to go, Yamaha, I have always loved thee!


    @csaa
    Well I hope you get it working, you certainly seem clued-up in that respect.  But in the end, personally I gave up even trying to get my AMD card working, especially when I realised it's not just the fact that it's AMD holding me back, but also the fact that the card is too old and the required a driver type that is not available.  If you have a more modern AMD card, you might get it to work, but I would not even waste my time if you have anything like the sort of card I have.  I managed to install Krita, I managed to intall the plugin, and even managed to install ComfyUI.  Further, ComfyUI actually ran without generating an error, but the progress simply used to stay static when it got to the final generation output node (and the Krita-AI-Diffusion plugin depends on ComfyUI being installed and able to run in order to work).

    Ironic really cause my AMD CPU was the best decision I ever made in PC components, whereas my AMD GPU was (by far) the worst.  I will never buy an AMD GPU again, never.  It will have held me back for a whopping eight years by the time I've replaced the damn thing!

     

    Post edited by 3DIO on
  • ArtiniArtini Posts: 10,903

    Thanks for the further explanation. I were used Cubase in the past, but it was too advanced for me.

     

  • 3DIO3DIO Posts: 639
    edited September 7

    @Artini
    The video you posted shows just how easy it is to use at its core, so don't let it intimidate you.  Whenever I install a new version of Cubase, I always take the time to declutter the interface and toolbars, cause once that's done, it all feels much more comfortable, clean and easy (see attached image of how clean mine looks).

    Believe it or not, I can access every function from this simple interface, with the Edit controls appearing whenever I right click instead of cluttering-up the interface.

    That aside, I had to donwload the demo to take a look at Omnivocal, couldn't resist!  Absolutely amazing, and what's extra neat about it is that by adjusting the gap inbetween notes, you effect how the performance is delivered.  If, for example, you get it to sing a long note and then leave a gap before singing another note, it uses the gap to take a breath as a human would need to.  The trick to getting it sounding amazing is to use lots of automation lanes to control the parameters over time.

    Amazing, I'm shocked at how versatile even the default voice banks are.  At one point I had the male vocal sounding similar to Jon Anderson, who I know from the Jon and Vangelis albums.  To an extent, I even recreated the harmonising from their song "Genevieve" which sounded pretty damn good considering I didn't put that much detail and effort in to it!

    I also found some functionality not yet implemented, like a vocal mode that allows you to select between Soprano, Soprano 2, Alto, Alto 2, Tenor, Tenor 2, Bass, Bass 2.  And another unimplemented function for Articulation.  I'm guessing this stulff (along with phonetic and pronounciation editing) will come when it's out of beta, and seriously, I cannot wait, it's gonna be absolute bloody perfection!    At last, we have fully controllable, customisable, realistic singers inside our DAW!

    Clickable Video of the real Genevieve and its Harmonies:

     

    Custom Cubase Pro Interface.jpg
    1920 x 1040 - 353K
    Post edited by 3DIO on
  • ArtiniArtini Posts: 10,903

    Good for you.

    Have you heard about Melody Assistant?

    https://www.myriad-online.com/en/products/melody.htm

    That one is very nice. I have used it long time ago and were very happy with it.

     

     

  • ArtiniArtini Posts: 10,903

    Harmony Assistant is even better:

    https://www.myriad-online.com/en/products/harmony.htm

     

  • 3DIO3DIO Posts: 639
    edited September 7

    I could never use software like that since I never learnt to read musical notation.  I prefer to play live into the Cubase piano roll editor, ideally from a weighted 88-key keyboard, and correct any mistakes by dragging the note around with a mouse.  For harmony, I do all that by ear and find no hardship in it.  I'm one of those who never took the time to develop fancy playing skills, but nevertheless, can still do whatever I want, purely by ear.

    The only thing I have difficulty with is 'complex' chords, but Cubase has a chord construction (and chord sequencing) system which helps me out massively.

     

    Post edited by 3DIO on
  • ArtiniArtini Posts: 10,903

    Interesting, That explains well different preferences of music programs and why there are so many of them.

    Actually I have already purchased a music program from Unity Asset store,

    which generates music from prompts and also lyrics/text or one can choose random generation.

    All of these is done locally on the Nvidia RTX card with at least 12 GB of VRAM.

    The demo of it shows more than half an hour of music already created with it.

    My main interest is in creating videos for YouTube and I enjoy a lot creating graphics by myself,

    so having possibility of easy creation of music automatically fit me best.

     

  • WendyLuvsCatzWendyLuvsCatz Posts: 41,525

    I have been using Melody Assistant for years but I do read music and cannot work with piano roll interfaces 

    I should really upgrade to Harmony assistant one day but now is not a good time, need better hardware etc too, none of its happening 

Sign In or Register to comment.