Experiments

DreamBooth Training - Tokyo Revengers

2024
FINISHED - PASSEDyesCreative / LoRA

Fine-tuned a diffusion model on Tokyo Revengers character images to generate similar-styled characters.

DreamBoothDiffusion ModelsPython

What I Built

Fine-tuned a diffusion model on character images from Tokyo Revengers - specifically Mikey - to generate characters in the same visual style.

The DreamBooth Exercise

DreamBooth is about subject fidelity: you're training the model to know what a specific subject looks like, then generating that subject in new contexts. The tradeoffs are:

  • Too little training data - the model doesn't capture the subject accurately
  • Too much training data - the model overfits and loses the ability to generalize (the subject appears in every prompt whether you want it or not)
  • Regularization images - necessary to prevent the model from associating the trigger token with all images in a class, not just the specific subject

The experiment was an exercise in understanding these tradeoffs hands-on, using an anime character as the subject because the art style is distinctive enough to verify fidelity visually.

DreamBooth Training - Tokyo Revengers - image 1
DreamBooth Training - Tokyo Revengers - image 2
DreamBooth Training - Tokyo Revengers - image 3
DreamBooth Training - Tokyo Revengers - image 4
DreamBooth Training - Tokyo Revengers - image 5

Next experiment

Lip Sync - Sonic

Lip Sync - Sonic

FINISHED - PASSEDVideo Gen