Discriminator missing for model finetuning

[slimsushi]

Hi,

I plan to finetune the pretrained generator model on a new personalized voice in german and I also plan to add noisy audio recordings to make it more robust in bad recording situations. Basically like you mentioned in your paper with: "[...] our model could be fine-tuned on a dataset comprised of only a single input speaker converted to a target voice in order to create a personalized voice conversion model."

Now my Question is: Do I also need the pretrained discriminator model to finetune the generator model? In your provided KoeAI/llvc_models on Huggingface is only the pretrained generator safed and I cannot find the corresponding discriminator model. Also in you train.py script the training only continues from the provided checkpoint if both the generator model and discriminator model are given. Is it still possible to train with a new discriminator initialized from scratch? Or is it possible that you provide me with the pretrained generator and discriminator model?

Thanks for your help

[ksadov]

I've found that initializing the discriminator from scratch works actually works better than using a pretrained discriminator when it comes to fine-tuning a model. I thought that the provided train script allowed for this but apparently not-- I'll see if I can get around to fixing this by the end of the week, or feel free to open a PR if you get it working for yourself.

[Yaodada12]

I've found that initializing the discriminator from scratch works actually works better than using a pretrained discriminator when it comes to fine-tuning a model. I thought that the provided train script allowed for this but apparently not-- I'll see if I can get around to fixing this by the end of the week, or feel free to open a PR if you get it working for yourself.

According to the paper, I loaded your G_5000.pth and used 3090 single card to train 39 epcoh on the train-360 data, but my loss is getting bigger and bigger. What is the reason? What is the normal level total loss（loss_disc+loss_gen_all） when training to 39 epochs?

[slimsushi]

According to the paper, I loaded your G_5000.pth and used 3090 single card to train 39 epcoh on the train-360 data, but my loss is getting bigger and bigger. What is the reason? What is the normal level total loss（loss_disc+loss_gen_all） when training to 39 epochs?

When you use the Checkpoint G_500000.pt provided by the author, then you use the already pretrained model on that already used dataset, Therefore it should overfit and the loss will increase, because it has already seen that data. When using the checkpoint try to use new unseen data to finetune it on your specific task.

[Yaodada12]

According to the paper, I loaded your G_5000.pth and used 3090 single card to train 39 epcoh on the train-360 data, but my loss is getting bigger and bigger. What is the reason? What is the normal level total loss（loss_disc+loss_gen_all） when training to 39 epochs?

When you use the Checkpoint G_500000.pt provided by the author, then you use the already pretrained model on that already used dataset, Therefore it should overfit and the loss will increase, because it has already seen that data. When using the checkpoint try to use new unseen data to finetune it on your specific task.

Sorry for misleading you. I am using the train-360 dataset that has been converted by other speaker.