9

I want a GAN to train on "before and after" images of dental surgeries; so that it can generate "after" pictures for fresh patients.

Input images are like these: https://img.webmd.com/dtmcms/live/webmd/consumer_assets/site_images/article_thumbnails/slideshows/cosmetic_dentistry_slideshow/650x350_cosmetic_dentistry_slideshow.jpg

Now being a web developer, I am getting into AI.

I have read a hell lot on Basic ML, AI, GANs, VAE, StyleGANs, etc.

Made small projects on Tensorflow, etc.

I have my basics and theoretical knowledge in place. But I dont know how to get started practically on the above idea I have.

What should be my approach?

Following are my questions:

How many "before and after" images will I need?

What architecture should I use? i.e. StyleGAN, VAE?

As I will need my custom model; training a fresh model would need a good GPU. What platform should I go for? Colab, Floyhub, or something else

One approach that comes to my mind is that - Process the image, do the color segmentation or something.

Add a static tooth image to area where tooth is missing. Then feed this image to a GAN, so that it can polish that image to look real.

Please guide me in the right direction. Sorry if any of my question is nonsense.

Lakshay Dulani
  • 265
  • 1
  • 2
  • 6

1 Answers1

5

It's a very specific problem and there's no right or wrong solution. I'll just write what I'd do in your position and hope that it is useful.

How many "before and after" images will I need?

You'll need a lot of images to consistently get good results, in the range of tens or preferably hundreds of thousands.

What architecture should I use? i.e. StyleGAN, VAE?

StyleGAN seems to be the most popular option for these sorts of tasks these days, so I'd stick with this.

As I will need my custom model; training a fresh model would need a good GPU. What platform should I go for? Colab, Floyhub, or something else

The platform you'll choose depends on whether or not you're willing to pay. Colab, Floydhub and Kaggle are the only free options with GPU afaik, but also offer paid versions (1, 2). All have their pros and cons. For example colab, which I'm more familiar with, offers only 12-hour sessions (after that it disconnects you and you have to reconnect).

Approach

Regarding the process you said, that's an interesting approach. I would have gone with something more basic like a segmentation mask, but I think your idea is better.

Djib2011
  • 8,068
  • 5
  • 28
  • 39