reorganize docs
This commit is contained in:
Xintao
42
docs/CONTRIBUTING.md
Normal file
42
docs/CONTRIBUTING.md
Normal file
@@ -0,0 +1,42 @@
|
||||
# Contributing to Real-ESRGAN
|
||||
|
||||
We like open-source and want to develop practical algorithms for general image restoration. However, individual strength is limited. So, any kinds of contributions are welcome, such as:
|
||||
|
||||
- New features
|
||||
- New models (your fine-tuned models)
|
||||
- Bug fixes
|
||||
- Typo fixes
|
||||
- Suggestions
|
||||
- Maintenance
|
||||
- Documents
|
||||
- *etc*
|
||||
|
||||
## Workflow
|
||||
|
||||
1. Fork and pull the latest Real-ESRGAN repository
|
||||
1. Checkout a new branch (do not use master branch for PRs)
|
||||
1. Commit your changes
|
||||
1. Create a PR
|
||||
|
||||
**Note**:
|
||||
|
||||
1. Please check the code style and linting
|
||||
1. The style configuration is specified in [setup.cfg](setup.cfg)
|
||||
1. If you use VSCode, the settings are configured in [.vscode/settings.json](.vscode/settings.json)
|
||||
1. Strongly recommend using `pre-commit hook`. It will check your code style and linting before your commit.
|
||||
1. In the root path of project folder, run `pre-commit install`
|
||||
1. The pre-commit configuration is listed in [.pre-commit-config.yaml](.pre-commit-config.yaml)
|
||||
1. Better to [open a discussion](https://github.com/xinntao/Real-ESRGAN/discussions) before large changes.
|
||||
1. Welcome to discuss :sunglasses:. I will try my best to join the discussion.
|
||||
|
||||
## TODO List
|
||||
|
||||
:zero: The most straightforward way of improving model performance is to fine-tune on some specific datasets.
|
||||
|
||||
Here are some TODOs:
|
||||
|
||||
- [ ] optimize for human faces
|
||||
- [ ] optimize for texts
|
||||
- [ ] support controllable restoration strength
|
||||
|
||||
:one: There are also [several issues](https://github.com/xinntao/Real-ESRGAN/issues) that require helpers to improve. If you can help, please let me know :smile:
|
||||
7
docs/FAQ.md
Normal file
7
docs/FAQ.md
Normal file
@@ -0,0 +1,7 @@
|
||||
# FAQ
|
||||
|
||||
1. **How to select models?**<br>
|
||||
A: Please refer to [docs/model_zoo.md](docs/model_zoo.md)
|
||||
|
||||
1. **Can `face_enhance` be used for anime images/animation videos?**<br>
|
||||
A: No, it can only be used for real faces. It is recommended not to use this option for anime images/animation videos to save GPU memory.
|
||||
271
docs/Training.md
Normal file
271
docs/Training.md
Normal file
@@ -0,0 +1,271 @@
|
||||
# :computer: How to Train/Finetune Real-ESRGAN
|
||||
|
||||
- [Train Real-ESRGAN](#train-real-esrgan)
|
||||
- [Overview](#overview)
|
||||
- [Dataset Preparation](#dataset-preparation)
|
||||
- [Train Real-ESRNet](#Train-Real-ESRNet)
|
||||
- [Train Real-ESRGAN](#Train-Real-ESRGAN)
|
||||
- [Finetune Real-ESRGAN on your own dataset](#Finetune-Real-ESRGAN-on-your-own-dataset)
|
||||
- [Generate degraded images on the fly](#Generate-degraded-images-on-the-fly)
|
||||
- [Use paired training data](#use-your-own-paired-data)
|
||||
|
||||
[English](Training.md) **|** [简体中文](Training_CN.md)
|
||||
|
||||
## Train Real-ESRGAN
|
||||
|
||||
### Overview
|
||||
|
||||
The training has been divided into two stages. These two stages have the same data synthesis process and training pipeline, except for the loss functions. Specifically,
|
||||
|
||||
1. We first train Real-ESRNet with L1 loss from the pre-trained model ESRGAN.
|
||||
1. We then use the trained Real-ESRNet model as an initialization of the generator, and train the Real-ESRGAN with a combination of L1 loss, perceptual loss and GAN loss.
|
||||
|
||||
### Dataset Preparation
|
||||
|
||||
We use DF2K (DIV2K and Flickr2K) + OST datasets for our training. Only HR images are required. <br>
|
||||
You can download from :
|
||||
|
||||
1. DIV2K: http://data.vision.ee.ethz.ch/cvl/DIV2K/DIV2K_train_HR.zip
|
||||
2. Flickr2K: https://cv.snu.ac.kr/research/EDSR/Flickr2K.tar
|
||||
3. OST: https://openmmlab.oss-cn-hangzhou.aliyuncs.com/datasets/OST_dataset.zip
|
||||
|
||||
Here are steps for data preparation.
|
||||
|
||||
#### Step 1: [Optional] Generate multi-scale images
|
||||
|
||||
For the DF2K dataset, we use a multi-scale strategy, *i.e.*, we downsample HR images to obtain several Ground-Truth images with different scales. <br>
|
||||
You can use the [scripts/generate_multiscale_DF2K.py](scripts/generate_multiscale_DF2K.py) script to generate multi-scale images. <br>
|
||||
Note that this step can be omitted if you just want to have a fast try.
|
||||
|
||||
```bash
|
||||
python scripts/generate_multiscale_DF2K.py --input datasets/DF2K/DF2K_HR --output datasets/DF2K/DF2K_multiscale
|
||||
```
|
||||
|
||||
#### Step 2: [Optional] Crop to sub-images
|
||||
|
||||
We then crop DF2K images into sub-images for faster IO and processing.<br>
|
||||
This step is optional if your IO is enough or your disk space is limited.
|
||||
|
||||
You can use the [scripts/extract_subimages.py](scripts/extract_subimages.py) script. Here is the example:
|
||||
|
||||
```bash
|
||||
python scripts/extract_subimages.py --input datasets/DF2K/DF2K_multiscale --output datasets/DF2K/DF2K_multiscale_sub --crop_size 400 --step 200
|
||||
```
|
||||
|
||||
#### Step 3: Prepare a txt for meta information
|
||||
|
||||
You need to prepare a txt file containing the image paths. The following are some examples in `meta_info_DF2Kmultiscale+OST_sub.txt` (As different users may have different sub-images partitions, this file is not suitable for your purpose and you need to prepare your own txt file):
|
||||
|
||||
```txt
|
||||
DF2K_HR_sub/000001_s001.png
|
||||
DF2K_HR_sub/000001_s002.png
|
||||
DF2K_HR_sub/000001_s003.png
|
||||
...
|
||||
```
|
||||
|
||||
You can use the [scripts/generate_meta_info.py](scripts/generate_meta_info.py) script to generate the txt file. <br>
|
||||
You can merge several folders into one meta_info txt. Here is the example:
|
||||
|
||||
```bash
|
||||
python scripts/generate_meta_info.py --input datasets/DF2K/DF2K_HR, datasets/DF2K/DF2K_multiscale --root datasets/DF2K, datasets/DF2K --meta_info datasets/DF2K/meta_info/meta_info_DF2Kmultiscale.txt
|
||||
```
|
||||
|
||||
### Train Real-ESRNet
|
||||
|
||||
1. Download pre-trained model [ESRGAN](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.1.1/ESRGAN_SRx4_DF2KOST_official-ff704c30.pth) into `experiments/pretrained_models`.
|
||||
```bash
|
||||
wget https://github.com/xinntao/Real-ESRGAN/releases/download/v0.1.1/ESRGAN_SRx4_DF2KOST_official-ff704c30.pth -P experiments/pretrained_models
|
||||
```
|
||||
1. Modify the content in the option file `options/train_realesrnet_x4plus.yml` accordingly:
|
||||
```yml
|
||||
train:
|
||||
name: DF2K+OST
|
||||
type: RealESRGANDataset
|
||||
dataroot_gt: datasets/DF2K # modify to the root path of your folder
|
||||
meta_info: realesrgan/meta_info/meta_info_DF2Kmultiscale+OST_sub.txt # modify to your own generate meta info txt
|
||||
io_backend:
|
||||
type: disk
|
||||
```
|
||||
1. If you want to perform validation during training, uncomment those lines and modify accordingly:
|
||||
```yml
|
||||
# Uncomment these for validation
|
||||
# val:
|
||||
# name: validation
|
||||
# type: PairedImageDataset
|
||||
# dataroot_gt: path_to_gt
|
||||
# dataroot_lq: path_to_lq
|
||||
# io_backend:
|
||||
# type: disk
|
||||
|
||||
...
|
||||
|
||||
# Uncomment these for validation
|
||||
# validation settings
|
||||
# val:
|
||||
# val_freq: !!float 5e3
|
||||
# save_img: True
|
||||
|
||||
# metrics:
|
||||
# psnr: # metric name, can be arbitrary
|
||||
# type: calculate_psnr
|
||||
# crop_border: 4
|
||||
# test_y_channel: false
|
||||
```
|
||||
1. Before the formal training, you may run in the `--debug` mode to see whether everything is OK. We use four GPUs for training:
|
||||
```bash
|
||||
CUDA_VISIBLE_DEVICES=0,1,2,3 \
|
||||
python -m torch.distributed.launch --nproc_per_node=4 --master_port=4321 realesrgan/train.py -opt options/train_realesrnet_x4plus.yml --launcher pytorch --debug
|
||||
```
|
||||
|
||||
Train with **a single GPU** in the *debug* mode:
|
||||
```bash
|
||||
python realesrgan/train.py -opt options/train_realesrnet_x4plus.yml --debug
|
||||
```
|
||||
1. The formal training. We use four GPUs for training. We use the `--auto_resume` argument to automatically resume the training if necessary.
|
||||
```bash
|
||||
CUDA_VISIBLE_DEVICES=0,1,2,3 \
|
||||
python -m torch.distributed.launch --nproc_per_node=4 --master_port=4321 realesrgan/train.py -opt options/train_realesrnet_x4plus.yml --launcher pytorch --auto_resume
|
||||
```
|
||||
|
||||
Train with **a single GPU**:
|
||||
```bash
|
||||
python realesrgan/train.py -opt options/train_realesrnet_x4plus.yml --auto_resume
|
||||
```
|
||||
|
||||
### Train Real-ESRGAN
|
||||
|
||||
1. After the training of Real-ESRNet, you now have the file `experiments/train_RealESRNetx4plus_1000k_B12G4_fromESRGAN/model/net_g_1000000.pth`. If you need to specify the pre-trained path to other files, modify the `pretrain_network_g` value in the option file `train_realesrgan_x4plus.yml`.
|
||||
1. Modify the option file `train_realesrgan_x4plus.yml` accordingly. Most modifications are similar to those listed above.
|
||||
1. Before the formal training, you may run in the `--debug` mode to see whether everything is OK. We use four GPUs for training:
|
||||
```bash
|
||||
CUDA_VISIBLE_DEVICES=0,1,2,3 \
|
||||
python -m torch.distributed.launch --nproc_per_node=4 --master_port=4321 realesrgan/train.py -opt options/train_realesrgan_x4plus.yml --launcher pytorch --debug
|
||||
```
|
||||
|
||||
Train with **a single GPU** in the *debug* mode:
|
||||
```bash
|
||||
python realesrgan/train.py -opt options/train_realesrgan_x4plus.yml --debug
|
||||
```
|
||||
1. The formal training. We use four GPUs for training. We use the `--auto_resume` argument to automatically resume the training if necessary.
|
||||
```bash
|
||||
CUDA_VISIBLE_DEVICES=0,1,2,3 \
|
||||
python -m torch.distributed.launch --nproc_per_node=4 --master_port=4321 realesrgan/train.py -opt options/train_realesrgan_x4plus.yml --launcher pytorch --auto_resume
|
||||
```
|
||||
|
||||
Train with **a single GPU**:
|
||||
```bash
|
||||
python realesrgan/train.py -opt options/train_realesrgan_x4plus.yml --auto_resume
|
||||
```
|
||||
|
||||
## Finetune Real-ESRGAN on your own dataset
|
||||
|
||||
You can finetune Real-ESRGAN on your own dataset. Typically, the fine-tuning process can be divided into two cases:
|
||||
|
||||
1. [Generate degraded images on the fly](#Generate-degraded-images-on-the-fly)
|
||||
1. [Use your own **paired** data](#Use-paired-training-data)
|
||||
|
||||
### Generate degraded images on the fly
|
||||
|
||||
Only high-resolution images are required. The low-quality images are generated with the degradation process described in Real-ESRGAN during trainig.
|
||||
|
||||
**1. Prepare dataset**
|
||||
|
||||
See [this section](#dataset-preparation) for more details.
|
||||
|
||||
**2. Download pre-trained models**
|
||||
|
||||
Download pre-trained models into `experiments/pretrained_models`.
|
||||
|
||||
- *RealESRGAN_x4plus.pth*:
|
||||
```bash
|
||||
wget https://github.com/xinntao/Real-ESRGAN/releases/download/v0.1.0/RealESRGAN_x4plus.pth -P experiments/pretrained_models
|
||||
```
|
||||
|
||||
- *RealESRGAN_x4plus_netD.pth*:
|
||||
```bash
|
||||
wget https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.2.3/RealESRGAN_x4plus_netD.pth -P experiments/pretrained_models
|
||||
```
|
||||
|
||||
**3. Finetune**
|
||||
|
||||
Modify [options/finetune_realesrgan_x4plus.yml](options/finetune_realesrgan_x4plus.yml) accordingly, especially the `datasets` part:
|
||||
|
||||
```yml
|
||||
train:
|
||||
name: DF2K+OST
|
||||
type: RealESRGANDataset
|
||||
dataroot_gt: datasets/DF2K # modify to the root path of your folder
|
||||
meta_info: realesrgan/meta_info/meta_info_DF2Kmultiscale+OST_sub.txt # modify to your own generate meta info txt
|
||||
io_backend:
|
||||
type: disk
|
||||
```
|
||||
|
||||
We use four GPUs for training. We use the `--auto_resume` argument to automatically resume the training if necessary.
|
||||
|
||||
```bash
|
||||
CUDA_VISIBLE_DEVICES=0,1,2,3 \
|
||||
python -m torch.distributed.launch --nproc_per_node=4 --master_port=4321 realesrgan/train.py -opt options/finetune_realesrgan_x4plus.yml --launcher pytorch --auto_resume
|
||||
```
|
||||
|
||||
Finetune with **a single GPU**:
|
||||
```bash
|
||||
python realesrgan/train.py -opt options/finetune_realesrgan_x4plus.yml --auto_resume
|
||||
```
|
||||
|
||||
### Use your own paired data
|
||||
|
||||
You can also finetune RealESRGAN with your own paired data. It is more similar to fine-tuning ESRGAN.
|
||||
|
||||
**1. Prepare dataset**
|
||||
|
||||
Assume that you already have two folders:
|
||||
|
||||
- **gt folder** (Ground-truth, high-resolution images): *datasets/DF2K/DIV2K_train_HR_sub*
|
||||
- **lq folder** (Low quality, low-resolution images): *datasets/DF2K/DIV2K_train_LR_bicubic_X4_sub*
|
||||
|
||||
Then, you can prepare the meta_info txt file using the script [scripts/generate_meta_info_pairdata.py](scripts/generate_meta_info_pairdata.py):
|
||||
|
||||
```bash
|
||||
python scripts/generate_meta_info_pairdata.py --input datasets/DF2K/DIV2K_train_HR_sub datasets/DF2K/DIV2K_train_LR_bicubic_X4_sub --meta_info datasets/DF2K/meta_info/meta_info_DIV2K_sub_pair.txt
|
||||
```
|
||||
|
||||
**2. Download pre-trained models**
|
||||
|
||||
Download pre-trained models into `experiments/pretrained_models`.
|
||||
|
||||
- *RealESRGAN_x4plus.pth*
|
||||
```bash
|
||||
wget https://github.com/xinntao/Real-ESRGAN/releases/download/v0.1.0/RealESRGAN_x4plus.pth -P experiments/pretrained_models
|
||||
```
|
||||
|
||||
- *RealESRGAN_x4plus_netD.pth*
|
||||
```bash
|
||||
wget https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.2.3/RealESRGAN_x4plus_netD.pth -P experiments/pretrained_models
|
||||
```
|
||||
|
||||
**3. Finetune**
|
||||
|
||||
Modify [options/finetune_realesrgan_x4plus_pairdata.yml](options/finetune_realesrgan_x4plus_pairdata.yml) accordingly, especially the `datasets` part:
|
||||
|
||||
```yml
|
||||
train:
|
||||
name: DIV2K
|
||||
type: RealESRGANPairedDataset
|
||||
dataroot_gt: datasets/DF2K # modify to the root path of your folder
|
||||
dataroot_lq: datasets/DF2K # modify to the root path of your folder
|
||||
meta_info: datasets/DF2K/meta_info/meta_info_DIV2K_sub_pair.txt # modify to your own generate meta info txt
|
||||
io_backend:
|
||||
type: disk
|
||||
```
|
||||
|
||||
We use four GPUs for training. We use the `--auto_resume` argument to automatically resume the training if necessary.
|
||||
|
||||
```bash
|
||||
CUDA_VISIBLE_DEVICES=0,1,2,3 \
|
||||
python -m torch.distributed.launch --nproc_per_node=4 --master_port=4321 realesrgan/train.py -opt options/finetune_realesrgan_x4plus_pairdata.yml --launcher pytorch --auto_resume
|
||||
```
|
||||
|
||||
Finetune with **a single GPU**:
|
||||
```bash
|
||||
python realesrgan/train.py -opt options/finetune_realesrgan_x4plus_pairdata.yml --auto_resume
|
||||
```
|
||||
271
docs/Training_CN.md
Normal file
271
docs/Training_CN.md
Normal file
@@ -0,0 +1,271 @@
|
||||
# :computer: 如何训练/微调 Real-ESRGAN
|
||||
|
||||
- [训练 Real-ESRGAN](#训练-real-esrgan)
|
||||
- [概述](#概述)
|
||||
- [准备数据集](#准备数据集)
|
||||
- [训练 Real-ESRNet 模型](#训练-real-esrnet-模型)
|
||||
- [训练 Real-ESRGAN 模型](#训练-real-esrgan-模型)
|
||||
- [用自己的数据集微调 Real-ESRGAN](#用自己的数据集微调-real-esrgan)
|
||||
- [动态生成降级图像](#动态生成降级图像)
|
||||
- [使用已配对的数据](#使用已配对的数据)
|
||||
|
||||
[English](Training.md) **|** [简体中文](Training_CN.md)
|
||||
|
||||
## 训练 Real-ESRGAN
|
||||
|
||||
### 概述
|
||||
|
||||
训练分为两个步骤。除了 loss 函数外，这两个步骤拥有相同数据合成以及训练的一条龙流程。具体点说：
|
||||
|
||||
1. 首先使用 L1 loss 训练 Real-ESRNet 模型，其中 L1 loss 来自预先训练的 ESRGAN 模型。
|
||||
|
||||
2. 然后我们将 Real-ESRNet 模型作为生成器初始化，结合L1 loss、感知 loss、GAN loss 三者的参数对 Real-ESRGAN 进行训练。
|
||||
|
||||
### 准备数据集
|
||||
|
||||
我们使用 DF2K ( DIV2K 和 Flickr2K ) + OST 数据集进行训练。只需要HR图像！<br>
|
||||
下面是网站链接:
|
||||
1. DIV2K: http://data.vision.ee.ethz.ch/cvl/DIV2K/DIV2K_train_HR.zip
|
||||
2. Flickr2K: https://cv.snu.ac.kr/research/EDSR/Flickr2K.tar
|
||||
3. OST: https://openmmlab.oss-cn-hangzhou.aliyuncs.com/datasets/OST_dataset.zip
|
||||
|
||||
以下是数据的准备步骤。
|
||||
|
||||
#### 第1步：【可选】生成多尺寸图片
|
||||
|
||||
针对 DF2K 数据集，我们使用多尺寸缩放策略，*换言之*，我们对 HR 图像进行下采样，就能获得多尺寸的标准参考（Ground-Truth）图像。 <br>
|
||||
您可以使用这个 [scripts/generate_multiscale_DF2K.py](scripts/generate_multiscale_DF2K.py) 脚本快速生成多尺寸的图像。<br>
|
||||
注意：如果您只想简单试试，那么可以跳过此步骤。
|
||||
|
||||
```bash
|
||||
python scripts/generate_multiscale_DF2K.py --input datasets/DF2K/DF2K_HR --output datasets/DF2K/DF2K_multiscale
|
||||
```
|
||||
|
||||
#### 第2步：【可选】裁切为子图像
|
||||
|
||||
我们可以将 DF2K 图像裁切为子图像，以加快 IO 和处理速度。<br>
|
||||
如果你的 IO 够好或储存空间有限，那么此步骤是可选的。<br>
|
||||
|
||||
您可以使用脚本 [scripts/extract_subimages.py](scripts/extract_subimages.py)。这是使用示例:
|
||||
|
||||
```bash
|
||||
python scripts/extract_subimages.py --input datasets/DF2K/DF2K_multiscale --output datasets/DF2K/DF2K_multiscale_sub --crop_size 400 --step 200
|
||||
```
|
||||
|
||||
#### 第3步：准备元信息 txt
|
||||
|
||||
您需要准备一个包含图像路径的 txt 文件。下面是 `meta_info_DF2Kmultiscale+OST_sub.txt` 中的部分展示（由于各个用户可能有截然不同的子图像划分，这个文件不适合你的需求，你得准备自己的 txt 文件)：
|
||||
|
||||
```txt
|
||||
DF2K_HR_sub/000001_s001.png
|
||||
DF2K_HR_sub/000001_s002.png
|
||||
DF2K_HR_sub/000001_s003.png
|
||||
...
|
||||
```
|
||||
|
||||
你可以使用该脚本 [scripts/generate_meta_info.py](scripts/generate_meta_info.py) 生成包含图像路径的 txt 文件。<br>
|
||||
你还可以合并多个文件夹的图像路径到一个元信息（meta_info）txt。这是使用示例:
|
||||
|
||||
```bash
|
||||
python scripts/generate_meta_info.py --input datasets/DF2K/DF2K_HR, datasets/DF2K/DF2K_multiscale --root datasets/DF2K, datasets/DF2K --meta_info datasets/DF2K/meta_info/meta_info_DF2Kmultiscale.txt
|
||||
```
|
||||
|
||||
### 训练 Real-ESRNet 模型
|
||||
|
||||
1. 下载预先训练的模型 [ESRGAN](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.1.1/ESRGAN_SRx4_DF2KOST_official-ff704c30.pth)，放到 `experiments/pretrained_models`目录下。
|
||||
```bash
|
||||
wget https://github.com/xinntao/Real-ESRGAN/releases/download/v0.1.1/ESRGAN_SRx4_DF2KOST_official-ff704c30.pth -P experiments/pretrained_models
|
||||
```
|
||||
2. 相应地修改选项文件 `options/train_realesrnet_x4plus.yml` 中的内容：
|
||||
```yml
|
||||
train:
|
||||
name: DF2K+OST
|
||||
type: RealESRGANDataset
|
||||
dataroot_gt: datasets/DF2K # 修改为你的数据集文件夹根目录
|
||||
meta_info: realesrgan/meta_info/meta_info_DF2Kmultiscale+OST_sub.txt # 修改为你自己生成的元信息txt
|
||||
io_backend:
|
||||
type: disk
|
||||
```
|
||||
3. 如果你想在训练过程中执行验证，就取消注释这些内容并进行相应的修改：
|
||||
```yml
|
||||
# 取消注释这些以进行验证
|
||||
# val:
|
||||
# name: validation
|
||||
# type: PairedImageDataset
|
||||
# dataroot_gt: path_to_gt
|
||||
# dataroot_lq: path_to_lq
|
||||
# io_backend:
|
||||
# type: disk
|
||||
|
||||
...
|
||||
|
||||
# 取消注释这些以进行验证
|
||||
# 验证设置
|
||||
# val:
|
||||
# val_freq: !!float 5e3
|
||||
# save_img: True
|
||||
|
||||
# metrics:
|
||||
# psnr: # 指标名称，可以是任意的
|
||||
# type: calculate_psnr
|
||||
# crop_border: 4
|
||||
# test_y_channel: false
|
||||
```
|
||||
4. 正式训练之前，你可以用 `--debug` 模式检查是否正常运行。我们用了4个GPU进行训练：
|
||||
```bash
|
||||
CUDA_VISIBLE_DEVICES=0,1,2,3 \
|
||||
python -m torch.distributed.launch --nproc_per_node=4 --master_port=4321 realesrgan/train.py -opt options/train_realesrnet_x4plus.yml --launcher pytorch --debug
|
||||
```
|
||||
|
||||
用 **1个GPU** 训练的 debug 模式示例:
|
||||
```bash
|
||||
python realesrgan/train.py -opt options/train_realesrnet_x4plus.yml --debug
|
||||
```
|
||||
5. 正式训练开始。我们用了4个GPU进行训练。还可以使用参数 `--auto_resume` 在必要时自动恢复训练。
|
||||
```bash
|
||||
CUDA_VISIBLE_DEVICES=0,1,2,3 \
|
||||
python -m torch.distributed.launch --nproc_per_node=4 --master_port=4321 realesrgan/train.py -opt options/train_realesrnet_x4plus.yml --launcher pytorch --auto_resume
|
||||
```
|
||||
|
||||
用 **1个GPU** 训练：
|
||||
```bash
|
||||
python realesrgan/train.py -opt options/train_realesrnet_x4plus.yml --auto_resume
|
||||
```
|
||||
|
||||
### 训练 Real-ESRGAN 模型
|
||||
|
||||
1. 训练 Real-ESRNet 模型后，您得到了这个 `experiments/train_RealESRNetx4plus_1000k_B12G4_fromESRGAN/model/net_g_1000000.pth` 文件。如果需要指定预训练路径到其他文件，请修改选项文件 `train_realesrgan_x4plus.yml` 中 `pretrain_network_g` 的值。
|
||||
1. 修改选项文件 `train_realesrgan_x4plus.yml` 的内容。大多数修改与上节提到的类似。
|
||||
1. 正式训练之前，你可以以 `--debug` 模式检查是否正常运行。我们使用了4个GPU进行训练：
|
||||
```bash
|
||||
CUDA_VISIBLE_DEVICES=0,1,2,3 \
|
||||
python -m torch.distributed.launch --nproc_per_node=4 --master_port=4321 realesrgan/train.py -opt options/train_realesrgan_x4plus.yml --launcher pytorch --debug
|
||||
```
|
||||
|
||||
用 **1个GPU** 训练的 debug 模式示例:
|
||||
```bash
|
||||
python realesrgan/train.py -opt options/train_realesrgan_x4plus.yml --debug
|
||||
```
|
||||
1. 正式训练开始。我们使用4个GPU进行训练。还可以使用参数 `--auto_resume` 在必要时自动恢复训练。
|
||||
```bash
|
||||
CUDA_VISIBLE_DEVICES=0,1,2,3 \
|
||||
python -m torch.distributed.launch --nproc_per_node=4 --master_port=4321 realesrgan/train.py -opt options/train_realesrgan_x4plus.yml --launcher pytorch --auto_resume
|
||||
```
|
||||
|
||||
用 **1个GPU** 训练：
|
||||
```bash
|
||||
python realesrgan/train.py -opt options/train_realesrgan_x4plus.yml --auto_resume
|
||||
```
|
||||
|
||||
## 用自己的数据集微调 Real-ESRGAN
|
||||
|
||||
你可以用自己的数据集微调 Real-ESRGAN。一般地，微调（Fine-Tune）程序可以分为两种类型:
|
||||
|
||||
1. [动态生成降级图像](#动态生成降级图像)
|
||||
2. [使用**已配对**的数据](#使用已配对的数据)
|
||||
|
||||
### 动态生成降级图像
|
||||
|
||||
只需要高分辨率图像。在训练过程中，使用 Real-ESRGAN 描述的降级模型生成低质量图像。
|
||||
|
||||
**1. 准备数据集**
|
||||
|
||||
完整信息请参见[本节](#准备数据集)。
|
||||
|
||||
**2. 下载预训练模型**
|
||||
|
||||
下载预先训练的模型到 `experiments/pretrained_models` 目录下。
|
||||
|
||||
- *RealESRGAN_x4plus.pth*:
|
||||
```bash
|
||||
wget https://github.com/xinntao/Real-ESRGAN/releases/download/v0.1.0/RealESRGAN_x4plus.pth -P experiments/pretrained_models
|
||||
```
|
||||
|
||||
- *RealESRGAN_x4plus_netD.pth*:
|
||||
```bash
|
||||
wget https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.2.3/RealESRGAN_x4plus_netD.pth -P experiments/pretrained_models
|
||||
```
|
||||
|
||||
**3. 微调**
|
||||
|
||||
修改选项文件 [options/finetune_realesrgan_x4plus.yml](options/finetune_realesrgan_x4plus.yml) ，特别是 `datasets` 部分：
|
||||
|
||||
```yml
|
||||
train:
|
||||
name: DF2K+OST
|
||||
type: RealESRGANDataset
|
||||
dataroot_gt: datasets/DF2K # 修改为你的数据集文件夹根目录
|
||||
meta_info: realesrgan/meta_info/meta_info_DF2Kmultiscale+OST_sub.txt # 修改为你自己生成的元信息txt
|
||||
io_backend:
|
||||
type: disk
|
||||
```
|
||||
|
||||
我们使用4个GPU进行训练。还可以使用参数 `--auto_resume` 在必要时自动恢复训练。
|
||||
|
||||
```bash
|
||||
CUDA_VISIBLE_DEVICES=0,1,2,3 \
|
||||
python -m torch.distributed.launch --nproc_per_node=4 --master_port=4321 realesrgan/train.py -opt options/finetune_realesrgan_x4plus.yml --launcher pytorch --auto_resume
|
||||
```
|
||||
|
||||
用 **1个GPU** 训练：
|
||||
```bash
|
||||
python realesrgan/train.py -opt options/finetune_realesrgan_x4plus.yml --auto_resume
|
||||
```
|
||||
|
||||
### 使用已配对的数据
|
||||
|
||||
你还可以用自己已经配对的数据微调 RealESRGAN。这个过程更类似于微调 ESRGAN。
|
||||
|
||||
**1. 准备数据集**
|
||||
|
||||
假设你已经有两个文件夹（folder）:
|
||||
|
||||
- **gt folder**（标准参考，高分辨率图像）：*datasets/DF2K/DIV2K_train_HR_sub*
|
||||
- **lq folder**（低质量，低分辨率图像）：*datasets/DF2K/DIV2K_train_LR_bicubic_X4_sub*
|
||||
|
||||
然后，您可以使用脚本 [scripts/generate_meta_info_pairdata.py](scripts/generate_meta_info_pairdata.py) 生成元信息（meta_info）txt 文件。
|
||||
|
||||
```bash
|
||||
python scripts/generate_meta_info_pairdata.py --input datasets/DF2K/DIV2K_train_HR_sub datasets/DF2K/DIV2K_train_LR_bicubic_X4_sub --meta_info datasets/DF2K/meta_info/meta_info_DIV2K_sub_pair.txt
|
||||
```
|
||||
|
||||
**2. 下载预训练模型**
|
||||
|
||||
下载预先训练的模型到 `experiments/pretrained_models` 目录下。
|
||||
|
||||
- *RealESRGAN_x4plus.pth*:
|
||||
```bash
|
||||
wget https://github.com/xinntao/Real-ESRGAN/releases/download/v0.1.0/RealESRGAN_x4plus.pth -P experiments/pretrained_models
|
||||
```
|
||||
|
||||
- *RealESRGAN_x4plus_netD.pth*:
|
||||
```bash
|
||||
wget https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.2.3/RealESRGAN_x4plus_netD.pth -P experiments/pretrained_models
|
||||
```
|
||||
|
||||
**3. 微调**
|
||||
|
||||
修改选项文件 [options/finetune_realesrgan_x4plus_pairdata.yml](options/finetune_realesrgan_x4plus_pairdata.yml) ，特别是 `datasets` 部分：
|
||||
|
||||
```yml
|
||||
train:
|
||||
name: DIV2K
|
||||
type: RealESRGANPairedDataset
|
||||
dataroot_gt: datasets/DF2K # 修改为你的 gt folder 文件夹根目录
|
||||
dataroot_lq: datasets/DF2K # 修改为你的 lq folder 文件夹根目录
|
||||
meta_info: datasets/DF2K/meta_info/meta_info_DIV2K_sub_pair.txt # 修改为你自己生成的元信息txt
|
||||
io_backend:
|
||||
type: disk
|
||||
```
|
||||
|
||||
我们使用4个GPU进行训练。还可以使用参数 `--auto_resume` 在必要时自动恢复训练。
|
||||
|
||||
```bash
|
||||
CUDA_VISIBLE_DEVICES=0,1,2,3 \
|
||||
python -m torch.distributed.launch --nproc_per_node=4 --master_port=4321 realesrgan/train.py -opt options/finetune_realesrgan_x4plus_pairdata.yml --launcher pytorch --auto_resume
|
||||
```
|
||||
|
||||
用 **1个GPU** 训练：
|
||||
```bash
|
||||
python realesrgan/train.py -opt options/finetune_realesrgan_x4plus_pairdata.yml --auto_resume
|
||||
```
|
||||
11
docs/feedback.md
Normal file
11
docs/feedback.md
Normal file
@@ -0,0 +1,11 @@
|
||||
# Feedback 反馈
|
||||
|
||||
## 动漫插画模型
|
||||
|
||||
1. 视频处理不了: 目前的模型，不是针对视频的，所以视频效果很很不好。我们在探究针对视频的模型了
|
||||
1. 景深虚化有问题: 现在的模型把一些景深 和 特意的虚化 都复原了，感觉不好。这个后面我们会考虑把这个信息结合进入。一个简单的做法是识别景深和虚化，然后作为条件告诉神经网络，哪些地方复原强一些，哪些地方复原要弱一些
|
||||
1. 不可以调节: 像 Waifu2X 可以调节。可以根据自己的喜好，做调整，但是 Real-ESRGAN-anime 并不可以。导致有些恢复效果过了
|
||||
1. 把原来的风格改变了: 不同的动漫插画都有自己的风格，现在的 Real-ESRGAN-anime 倾向于恢复成一种风格（这是受到训练数据集影响的）。风格是动漫很重要的一个要素，所以要尽可能保持
|
||||
1. 模型太大: 目前的模型处理太慢，能够更快。这个我们有相关的工作在探究，希望能够尽快有结果，并应用到 Real-ESRGAN 这一系列的模型上
|
||||
|
||||
Thanks for the [detailed and valuable feedbacks/suggestions](https://github.com/xinntao/Real-ESRGAN/issues/131) by [2ji3150](https://github.com/2ji3150).
|
||||
Reference in New Issue
Block a user