Loading...
> >

Interpolation between images using AI

omvendtomvendt Posts: 200
in The Commons

Dear experts,

I am not a fan of AI generated images, but I was wondering if it would be possible to use it in ways that preserve the artistic intent of our work. For example,

A. Using AI to interpolate between two scenes. E.g. 1) Victoria standing with her back to a chair, 2) Victoria sitting in the chair. The chair cushion bends under her weight.  
B. Using AI to simplify posing: E.g. Victoria reaches out to Daniel, who takes her hand. 

In the first case, it would be great to get some intermediate images generated. Maybe enough for a short video. 
In the second case, it would be very helpful to have their fingers interlace automatically, since it is really tedious to manually do that.

Two questions:
a) Is A or B possible today? If so, how?
b) do you agree that these usecases represent a very different use of AI compared to describing the scene and then receiving the image or video? Personally, I think the artistic intent is preserved because everything in the scene would be selected and positioned by the (in my case, wannabee) artist.

Have a great Sunday.

Comments

  • WendyLuvsCatzWendyLuvsCatz Posts: 41,652
    edited September 27

    https://www.daz3d.com/forums/discussion/591121/remixing-your-art-with-ai#latest

    a fair few of us already do this among other things

    https://www.youtube.com/watch?v=AkqlF7lOsCw&list=PLdCZ-Zh9DvBA3pqDOWbbYAgn1MYtWtz3I&pp=8AUB

    Post edited by WendyLuvsCatz on
  • LuckydraftLuckydraft Posts: 39

    Both A a B are very possible with local generation. Either Krea2 and inpainting, or Qwen Image 2.1 which is a very recently released edit model - even in this case I'd do inpainting so the rest of the image doesn't degrade in quality over time. You will probably need to reroll a few times depending on how consistent you want the end result to be. You will need a good GPU for decent results without having quality degrade too much, 16GB VRAM or higher. There's the always option of using cloud GPUs for the occassional work though.

    In the first case, it would be great to get some intermediate images generated. Maybe enough for a short video. 

    How far you can get here really again depends on how much you're willing to tolerate inconsistencies. You can definitely get decent intermediate images, but they may not be as consistent or sequential as you'd want, or you'll have to spend a lot of time re-rolling to get things to match. Again I'd inpaint here to preserve general image quality. I haven't played around too much but MiniMax H3 (free, but you still need a good GPU) or Seedance (best close weights model as far as I know) might be able to get you there easier. 

    If you don't mind inconsistencies and don't care about image quality degradation over time, I'd say most of the recent AI models can do what you ask very easily. With AI the catch is always on those two.

  • omvendtomvendt Posts: 200

    Luckydraft said:

    Both A a B are very possible with local generation. Either Krea2 and inpainting, or Qwen Image 2.1 which is a very recently released edit model - even in this case I'd do inpainting so the rest of the image doesn't degrade in quality over time. You will probably need to reroll a few times depending on how consistent you want the end result to be. You will need a good GPU for decent results without having quality degrade too much, 16GB VRAM or higher. There's the always option of using cloud GPUs for the occassional work though.

    In the first case, it would be great to get some intermediate images generated. Maybe enough for a short video. 

    How far you can get here really again depends on how much you're willing to tolerate inconsistencies. You can definitely get decent intermediate images, but they may not be as consistent or sequential as you'd want, or you'll have to spend a lot of time re-rolling to get things to match. Again I'd inpaint here to preserve general image quality. I haven't played around too much but MiniMax H3 (free, but you still need a good GPU) or Seedance (best close weights model as far as I know) might be able to get you there easier. 

    If you don't mind inconsistencies and don't care about image quality degradation over time, I'd say most of the recent AI models can do what you ask very easily. With AI the catch is always on those two.

    Inpainting, does that mean that you carve out a "protected" part of the image that then remains constant?
    I have a RTX-5090, 64 GB machine coming next Friday (I hope - long story) so I should be able to run some models locally. Look forward to seeing what I can learn. 

  • LuckydraftLuckydraft Posts: 39

    Basically yes, with inpainting you can define a mask of the area the AI will modify. There's multiple inpaint methods and I'm sure as many tools to do so. The one I found most successful is that behind the scenes your mask gets cropped from the original image and resized to a resolution that works well for the AI (e.g. 1024x1024), and then the AI "updates" that crop, then once the AI has generated the new image, the process downsamples the new pixels back into the original resolution (lets say, 512x512), and then blends them back into the original image based on your mask. I picked a square resolution for the example but there's some other aspect ratios that you can work with.

    With a 5090 you should be able to run the very best local image models at max quality for sure.

  • MasterstrokeMasterstroke Posts: 2,510

    One useful idea for AI would be to only have to render 4? frames per second for animations instead of 24, 25 or 30 frames, while the AI is taking care of the tweeners.

Sign In or Register to comment.