强大的少样本语音转换与语音合成Web用户界面。

蓝梦实 ea83174fe0 Rewrite I18n Scan (#1294) 1 day ago
Docker 4c78f66431 明确Dockerfile中所使用的asr模型的版本,使得和cmd-asr.py中保持一致 5 months ago
GPT_SoVITS 368045c94e 尝试修复混合模式下短单词发音不清晰的问题 (#1286) 3 days ago
docs d65f0ff8b4 Update README (#1291) 2 days ago
i18n 2cf8e79aba Update zh_CN.json 3 months ago
tools ea83174fe0 Rewrite I18n Scan (#1294) 1 day ago
.dockerignore b0b039ad21 Docker镜像构建脚本对于镜像的Tag增加Git Commit的Hash值,便于知道镜像中应用版本 4 months ago
.gitignore 0b806dba37 一些小问题修复 (#1021) 2 months ago
Dockerfile 82a5672361 删除重复的 COPY 指令 (#1073) 2 months ago
GPT_SoVITS_Inference.ipynb 26e6fe6b15 Add inference-only 4 months ago
LICENSE 6e05506a85 Initial commit 6 months ago
README.md d65f0ff8b4 Update README (#1291) 2 days ago
api.py 582ba7d519 fix: 移除多余 my_utils.py (#1189) (#1251) 2 weeks ago
colab_webui.ipynb 9c4ba08ccb Fix the colab comma which make the colab notebook broken (#983) 2 months ago
config.py db40317d9c Update config.py 4 months ago
docker-compose.yaml c990387f7e 修改原有Dockerfile和文档中错误的端口号;Dockerfile中可以去掉 VOLUME 的声明,同时将端口暴露缩减成一行。 5 months ago
dockerbuild.sh b0b039ad21 Docker镜像构建脚本对于镜像的Tag增加Git Commit的Hash值,便于知道镜像中应用版本 4 months ago
go-webui.bat 14097eee3b Update go-webui.bat 5 months ago
go-webui.ps1 47e113a7b2 support powershell 5 months ago
gpt-sovits_kaggle.ipynb 0b806dba37 一些小问题修复 (#1021) 2 months ago
install.sh 7e533c6995 Update install.sh 5 months ago
requirements.txt d1d43661a9 修正numba与numpy匹配问题,在ubuntu24.04下测试通过 (#1236) 2 weeks ago
webui.py db50670598 Update webui.py 1 month ago

README.md

GPT-SoVITS-WebUI

A Powerful Few-shot Voice Conversion and Text-to-Speech WebUI.

[![madewithlove](https://img.shields.io/badge/made_with-%E2%9D%A4-red?style=for-the-badge&labelColor=orange)](https://github.com/RVC-Boss/GPT-SoVITS)
[![Open In Colab](https://img.shields.io/badge/Colab-F9AB00?style=for-the-badge&logo=googlecolab&color=525252)](https://colab.research.google.com/github/RVC-Boss/GPT-SoVITS/blob/main/colab_webui.ipynb) [![License](https://img.shields.io/badge/LICENSE-MIT-green.svg?style=for-the-badge)](https://github.com/RVC-Boss/GPT-SoVITS/blob/main/LICENSE) [![Huggingface](https://img.shields.io/badge/🤗%20-Models%20Repo-yellow.svg?style=for-the-badge)](https://huggingface.co/lj1995/GPT-SoVITS/tree/main) [![Discord](https://img.shields.io/discord/1198701940511617164?color=%23738ADB&label=Discord&style=for-the-badge)](https://discord.gg/dnrgs5GHfG) **English** | [**中文简体**](./docs/cn/README.md) | [**日本語**](./docs/ja/README.md) | [**한국어**](./docs/ko/README.md) | [**Türkçe**](./docs/tr/README.md)

Features:

  1. Zero-shot TTS: Input a 5-second vocal sample and experience instant text-to-speech conversion.

  2. Few-shot TTS: Fine-tune the model with just 1 minute of training data for improved voice similarity and realism.

  3. Cross-lingual Support: Inference in languages different from the training dataset, currently supporting English, Japanese, and Chinese.

  4. WebUI Tools: Integrated tools include voice accompaniment separation, automatic training set segmentation, Chinese ASR, and text labeling, assisting beginners in creating training datasets and GPT/SoVITS models.

Check out our demo video here!

Unseen speakers few-shot fine-tuning demo:

https://github.com/RVC-Boss/GPT-SoVITS/assets/129054828/05bee1fa-bdd8-4d85-9350-80c060ab47fb

User guide: 简体中文 | English

Installation

For users in the China region, you can click here to use AutoDL Cloud Docker to experience the full functionality online.

Tested Environments

  • Python 3.9, PyTorch 2.0.1, CUDA 11
  • Python 3.10.13, PyTorch 2.1.2, CUDA 12.3
  • Python 3.9, PyTorch 2.2.2, macOS 14.4.1 (Apple silicon)
  • Python 3.9, PyTorch 2.2.2, CPU devices

Note: numba==0.56.4 requires py<3.11

Windows

If you are a Windows user (tested with win>=10), you can download the integrated package and double-click on go-webui.bat to start GPT-SoVITS-WebUI.

Users in the China region can download the package by clicking the link and then selecting "Download a copy." (Log out if you encounter errors while downloading.)

Linux

conda create -n GPTSoVits python=3.9
conda activate GPTSoVits
bash install.sh

macOS

Note: The models trained with GPUs on Macs result in significantly lower quality compared to those trained on other devices, so we are temporarily using CPUs instead.

  1. Install Xcode command-line tools by running xcode-select --install.
  2. Install FFmpeg by running brew install ffmpeg.
  3. Install the program by running the following commands:
conda create -n GPTSoVits python=3.9
conda activate GPTSoVits

pip install -r requirements.txt

Install Manually

Install Dependences

pip install -r requirements.txt

Install FFmpeg

Conda Users
conda install ffmpeg
Ubuntu/Debian Users
sudo apt install ffmpeg
sudo apt install libsox-dev
conda install -c conda-forge 'ffmpeg<7'
Windows Users

Download and place ffmpeg.exe and ffprobe.exe in the GPT-SoVITS root.

Mac Users
brew install ffmpeg

Using Docker

docker-compose.yaml configuration

  1. Regarding image tags: Due to rapid updates in the codebase and the slow process of packaging and testing images, please check Docker Hub for the currently packaged latest images and select as per your situation, or alternatively, build locally using a Dockerfile according to your own needs.
  2. Environment Variables:
  • is_half: Controls half-precision/double-precision. This is typically the cause if the content under the directories 4-cnhubert/5-wav32k is not generated correctly during the "SSL extracting" step. Adjust to True or False based on your actual situation.
  1. Volumes Configuration,The application's root directory inside the container is set to /workspace. The default docker-compose.yaml lists some practical examples for uploading/downloading content.
  2. shm_size: The default available memory for Docker Desktop on Windows is too small, which can cause abnormal operations. Adjust according to your own situation.
  3. Under the deploy section, GPU-related settings should be adjusted cautiously according to your system and actual circumstances.

Running with docker compose

docker compose -f "docker-compose.yaml" up -d

Running with docker command

As above, modify the corresponding parameters based on your actual situation, then run the following command:

docker run --rm -it --gpus=all --env=is_half=False --volume=G:\GPT-SoVITS-DockerTest\output:/workspace/output --volume=G:\GPT-SoVITS-DockerTest\logs:/workspace/logs --volume=G:\GPT-SoVITS-DockerTest\SoVITS_weights:/workspace/SoVITS_weights --workdir=/workspace -p 9880:9880 -p 9871:9871 -p 9872:9872 -p 9873:9873 -p 9874:9874 --shm-size="16G" -d breakstring/gpt-sovits:xxxxx

Pretrained Models

Download pretrained models from GPT-SoVITS Models and place them in GPT_SoVITS/pretrained_models.

For UVR5 (Vocals/Accompaniment Separation & Reverberation Removal, additionally), download models from UVR5 Weights and place them in tools/uvr5/uvr5_weights.

Users in the China region can download these two models by entering the links below and clicking "Download a copy"(Log out if you encounter errors while downloading.)

For Chinese ASR (additionally), download models from Damo ASR Model, Damo VAD Model, and Damo Punc Model and place them in tools/asr/models.

For English or Japanese ASR (additionally), download models from Faster Whisper Large V3 and place them in tools/asr/models. Also, other models may have the similar effect with smaller disk footprint.

Users in the China region can download this model by entering the links below

Dataset Format

The TTS annotation .list file format:

vocal_path|speaker_name|language|text

Language dictionary:

  • 'zh': Chinese
  • 'ja': Japanese
  • 'en': English

Example:

D:\GPT-SoVITS\xxx/xxx.wav|xxx|en|I like playing Genshin.

Todo List

  • [ ] High Priority:

    • Localization in Japanese and English.
    • User guide.
    • Japanese and English dataset fine tune training.
  • [ ] Features:

    • Zero-shot voice conversion (5s) / few-shot voice conversion (1min).
    • TTS speaking speed control.
    • Enhanced TTS emotion control.
    • Experiment with changing SoVITS token inputs to probability distribution of vocabs.
    • Improve English and Japanese text frontend.
    • Develop tiny and larger-sized TTS models.
    • Colab scripts.
    • Try expand training dataset (2k hours -> 10k hours).
    • better sovits base model (enhanced audio quality)
    • model mix

(Additional) Method for running from the command line

Use the command line to open the WebUI for UVR5

python tools/uvr5/webui.py "<infer_device>" <is_half> <webui_port_uvr5>

If you can't open a browser, follow the format below for UVR processing,This is using mdxnet for audio processing

python mdxnet.py --model --input_root --output_vocal --output_ins --agg_level --format --device --is_half_precision 

This is how the audio segmentation of the dataset is done using the command line

python audio_slicer.py \
    --input_path "<path_to_original_audio_file_or_directory>" \
    --output_root "<directory_where_subdivided_audio_clips_will_be_saved>" \
    --threshold <volume_threshold> \
    --min_length <minimum_duration_of_each_subclip> \
    --min_interval <shortest_time_gap_between_adjacent_subclips> 
    --hop_size <step_size_for_computing_volume_curve>

This is how dataset ASR processing is done using the command line(Only Chinese)

python tools/asr/funasr_asr.py -i <input> -o <output>

ASR processing is performed through Faster_Whisper(ASR marking except Chinese)

(No progress bars, GPU performance may cause time delays)

python ./tools/asr/fasterwhisper_asr.py -i <input> -o <output> -l <language>

A custom list save path is enabled

Credits

Special thanks to the following projects and contributors:

Theoretical Research

Thanks to all contributors for their efforts