| 1 | # Training |
| 2 | |
| 3 | Check your FFmpeg installation: |
| 4 | ```bash |
| 5 | ffmpeg -version |
| 6 | ``` |
| 7 | If not found, install it first (or skip assuming you know of other backends available). |
| 8 | |
| 9 | ## Prepare Dataset |
| 10 | |
| 11 | Example data processing scripts, and you may tailor your own one along with a Dataset class in `src/f5_tts/model/dataset.py`. |
| 12 | |
| 13 | ### 1. Some specific Datasets preparing scripts |
| 14 | Download corresponding dataset first, and fill in the path in scripts. |
| 15 | |
| 16 | ```bash |
| 17 | # Prepare the Emilia dataset |
| 18 | python src/f5_tts/train/datasets/prepare_emilia.py |
| 19 | |
| 20 | # Prepare the Wenetspeech4TTS dataset |
| 21 | python src/f5_tts/train/datasets/prepare_wenetspeech4tts.py |
| 22 | |
| 23 | # Prepare the LibriTTS dataset |
| 24 | python src/f5_tts/train/datasets/prepare_libritts.py |
| 25 | |
| 26 | # Prepare the LJSpeech dataset |
| 27 | python src/f5_tts/train/datasets/prepare_ljspeech.py |
| 28 | ``` |
| 29 | |
| 30 | ### 2. Create custom dataset with CSV |
| 31 | Prepare a CSV with two columns using a required header: `audio_file|text`. Audio paths must be absolute. |
| 32 | Use guidance see [#57 here](https://github.com/SWivid/F5-TTS/discussions/57#discussioncomment-10959029). |
| 33 | |
| 34 | ```bash |
| 35 | python src/f5_tts/train/datasets/prepare_csv_wavs.py /path/to/metadata.csv /path/to/output |
| 36 | ``` |
| 37 | |
| 38 | ## Training & Finetuning |
| 39 | |
| 40 | Once your datasets are prepared, you can start the training process. |
| 41 | |
| 42 | ### 1. Training script used for pretrained model |
| 43 | |
| 44 | ```bash |
| 45 | # setup accelerate config, e.g. use multi-gpu ddp, fp16 |
| 46 | # will be to: ~/.cache/huggingface/accelerate/default_config.yaml |
| 47 | accelerate config |
| 48 | |
| 49 | # .yaml files are under src/f5_tts/configs directory |
| 50 | accelerate launch src/f5_tts/train/train.py --config-name F5TTS_v1_Base.yaml |
| 51 | |
| 52 | # possible to overwrite accelerate and hydra config |
| 53 | accelerate launch --mixed_precision=fp16 src/f5_tts/train/train.py --config-name F5TTS_v1_Base.yaml ++datasets.batch_size_per_gpu=19200 |
| 54 | ``` |
| 55 | |
| 56 | ### 2. Finetuning practice |
| 57 | Discussion board for Finetuning [#57](https://github.com/SWivid/F5-TTS/discussions/57). |
| 58 | |
| 59 | Gradio UI training/finetuning with `src/f5_tts/train/finetune_gradio.py` see [#143](https://github.com/SWivid/F5-TTS/discussions/143). |
| 60 | |
| 61 | If want to finetune with a variant version e.g. *F5TTS_v1_Base_no_zero_init*, manually download pretrained checkpoint from model weight repository and fill in the path correspondingly on web interface. |
| 62 | |
| 63 | If use tensorboard as logger, install it first with `pip install tensorboard`. |
| 64 | |
| 65 | <ins>The `use_ema = True` might be harmful for early-stage finetuned checkpoints</ins> (which goes just few updates, thus ema weights still dominated by pretrained ones), try turn it off with finetune gradio option or `load_model(..., use_ema=False)`, see if offer better results. |
| 66 | |
| 67 | ### 3. W&B Logging |
| 68 | |
| 69 | The `wandb/` dir will be created under path you run training/finetuning scripts. |
| 70 | |
| 71 | By default, the training script does NOT use logging (assuming you didn't manually log in using `wandb login`). |
| 72 | |
| 73 | To turn on wandb logging, you can either: |
| 74 | |
| 75 | 1. Manually login with `wandb login`: Learn more [here](https://docs.wandb.ai/ref/cli/wandb-login) |
| 76 | 2. Automatically login programmatically by setting an environment variable: Get an API KEY at https://wandb.ai/authorize and set the environment variable as follows: |
| 77 | |
| 78 | On Mac & Linux: |
| 79 | |
| 80 | ``` |
| 81 | export WANDB_API_KEY=<YOUR WANDB API KEY> |
| 82 | ``` |
| 83 | |
| 84 | On Windows: |
| 85 | |
| 86 | ``` |
| 87 | set WANDB_API_KEY=<YOUR WANDB API KEY> |
| 88 | ``` |
| 89 | Moreover, if you couldn't access W&B and want to log metrics offline, you can set the environment variable as follows: |
| 90 | |
| 91 | ``` |
| 92 | export WANDB_MODE=offline |
| 93 | ``` |
| 94 |