diff --git a/README.md b/README.md index 6104d7b..e919106 100644 --- a/README.md +++ b/README.md @@ -11,7 +11,7 @@ Real-ESRGAN aims at developing **Practical Algorithms for General Image Restoration**.
We extend the powerful ESRGAN to a practical restoration application (namely, Real-ESRGAN), which is trained with pure synthetic data. -:triangular_flag_on_post: The training codes have been released. A detailed guide will be provided later (on July 25th). Note that the codes have a lot of refactoring. So there may be some bugs/performance drops. Welcome to report issues and I wil also retrain the models. +:triangular_flag_on_post: The training codes have been released. A detailed guide will be provided later (on July 25th). ### :book: Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data @@ -54,6 +54,7 @@ You can download **Windows executable files** from https://github.com/xinntao/Re This executable file is **portable** and includes all the binaries and models required. No CUDA or PyTorch environment is needed.
You can simply run the following command: + ```bash ./realesrgan-ncnn-vulkan.exe -i input.jpg -o output.png ``` diff --git a/Training.md b/Training.md new file mode 100644 index 0000000..9e54f02 --- /dev/null +++ b/Training.md @@ -0,0 +1,97 @@ +# :computer: How to Train Real-ESRGAN + +The training codes have been released.
+Note that the codes have a lot of refactoring. So there may be some bugs/performance drops. Welcome to report issues and I will also retrain the models. + +## Overview + +The training has been divided into two stages. These two stages have the same data synthesis process and training pipeline, except for the loss functions. Specifically, + +1. We first train Real-ESRNet with L1 loss from the pre-trained model ESRGAN. +1. We then use the trained Real-ESRNet model as an initialization of the generator, and train the Real-ESRGAN with a combination ofL1 loss, perceptual loss and GAN loss. + +## Dataset Preparation + +We use DF2K (DIV2K and Flickr2K) + OST datasets for our training. Only HR images are required.
+You can download from : + +1. DIV2K: http://data.vision.ee.ethz.ch/cvl/DIV2K/DIV2K_train_HR.zip +2. Flickr2K: https://cv.snu.ac.kr/research/EDSR/Flickr2K.tar +3. OST: https://openmmlab.oss-cn-hangzhou.aliyuncs.com/datasets/OST_dataset.zip + +For the DF2K dataset, we use a multi-scale strategy, *i.e.*, we downsample HR images to obtain several Ground-Truth images with different scales. + +We then crop DF2K images into sub-images for faster IO and processing. + +You need to prepare a txt file containing the image paths. Examples in `meta_info_DF2Kmultiscale+OST_sub.txt` (As different users may have different sub-images partition, this file is not suitable for your purpose and you need to prepare your own txt file): + +```txt +DF2K_HR_sub/000001_s001.png +DF2K_HR_sub/000001_s002.png +DF2K_HR_sub/000001_s003.png +... +``` + +## Train Real-ESRNet + +1. Download pre-trained model [ESRGAN](https://drive.google.com/file/d/1b3_bWZTjNO3iL2js1yWkJfjZykcQgvzT/view?usp=sharing) into `experiments/pretrained_models`. +1. Modify the content in the option file `options/train_realesrnet_x4plus.yml` accordingly: + ```yml + train: + name: DF2K+OST + type: RealESRGANDataset + dataroot_gt: datasets/DF2K # modify to the root path of your folder + meta_info: data/meta_info/meta_info_DF2Kmultiscale+OST_sub.txt # modify to your own generate meta info + io_backend: + type: disk + ``` +1. If you want to perform validation during training, uncomment those lines and modify accordingly: + ```yml + # Uncomment these for validation + # val: + # name: validation + # type: PairedImageDataset + # dataroot_gt: path_to_gt + # dataroot_lq: path_to_lq + # io_backend: + # type: disk + + ... + + # Uncomment these for validation + # validation settings + # val: + # val_freq: !!float 5e3 + # save_img: True + + # metrics: + # psnr: # metric name, can be arbitrary + # type: calculate_psnr + # crop_border: 4 + # test_y_channel: false + ``` +1. Before the formal training, you may run in the `--debug` mode to see whether everything is OK. We use four GPUs for training: + ```bash + CUDA_VISIBLE_DEVICES=0,1,2,3 \ + python -m torch.distributed.launch --nproc_per_node=4 --master_port=4321 train.py -opt options/train_realesrnet_x4plus.yml --launcher pytorch --debug + ``` +1. The formal training. We use four GPUs for training. We pass `--auto_resume` to resume the training if necessary automatically. + ```bash + CUDA_VISIBLE_DEVICES=0,1,2,3 \ + python -m torch.distributed.launch --nproc_per_node=4 --master_port=4321 train.py -opt options/train_realesrnet_x4plus.yml --launcher pytorch --auto_resume + ``` + +## Train Real-ESRGAN + +1. After you train Real-ESRNet, you now have the file `experiments/train_RealESRNetx4plus_1000k_B12G4_fromESRGAN/model/net_g_1000000.pth`. If you need to specify the pre-trained path of other files. Modify the `pretrain_network_g` value in the option file `train_realesrgan_x4plus.yml`. +1. Modify the option file `train_realesrgan_x4plus.yml` accordingly. Most modifications are similar to those listed above. +1. Before the formal training, you may run in the `--debug` mode to see whether everything is OK. We use four GPUs for training: + ```bash + CUDA_VISIBLE_DEVICES=0,1,2,3 \ + python -m torch.distributed.launch --nproc_per_node=4 --master_port=4321 train.py -opt options/train_realesrgan_x4plus.yml --launcher pytorch --debug + ``` +1. The formal training. We use four GPUs for training. We pass `--auto_resume` to resume the training if necessary automatically. + ```bash + CUDA_VISIBLE_DEVICES=0,1,2,3 \ + python -m torch.distributed.launch --nproc_per_node=4 --master_port=4321 train.py -opt options/train_realesrgan_x4plus.yml --launcher pytorch --auto_resume + ``` diff --git a/experiments/pretrained_models/README.md b/experiments/pretrained_models/README.md new file mode 100644 index 0000000..d0cc4af --- /dev/null +++ b/experiments/pretrained_models/README.md @@ -0,0 +1 @@ +# Put downloaded pre-trained models here