diff --git a/README.md b/README.md index 703e916..7cc19e2 100644 --- a/README.md +++ b/README.md @@ -10,12 +10,10 @@ [English](README.md) **|** [简体中文](README_CN.md) +:fire: :fire: :fire: Add **small video models** for anime videos (**针对动漫视频的小模型**). Please see [anime video models](docs/anime_video_model.md). + 1. [Colab Demo](https://colab.research.google.com/drive/1k2Zod6kSHEvraybHl50Lys0LerhyTMCo?usp=sharing) for Real-ESRGAN google colab logo. -2. Portable [Windows](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.2.4/realesrgan-ncnn-vulkan-20210901-windows.zip) / [Linux](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.2.4/realesrgan-ncnn-vulkan-20210901-ubuntu.zip) / [MacOS](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.2.4/realesrgan-ncnn-vulkan-20210901-macos.zip) **executable files for Intel/AMD/Nvidia GPU**. You can find more information [here](#Portable-executable-files). The ncnn implementation is in [Real-ESRGAN-ncnn-vulkan](https://github.com/xinntao/Real-ESRGAN-ncnn-vulkan). - -Thanks for your interests and use:-) There are still many problems about the anime/illustration model, mainly including: 1. It cannot deal with videos; 2. It cannot be aware of depth/depth-of-field; 3. It is not adjustable; 4. May change the original style. Thanks for your valuable feedbacks/suggestions. All the feedbacks are updated in [feedback.md](feedback.md). Hopefully, a new model will be available soon. - -感谢大家的关注和使用:-) 关于动漫插画的模型，目前还有很多问题，主要有: 1. 视频处理不了; 2. 景深虚化有问题; 3. 不可调节, 效果过了; 4. 改变原来的风格。大家提供了很好的反馈。这些反馈会逐步更新在 [这个文档](feedback.md)。希望不久之后，有新模型可以使用. +2. Portable [Windows](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.3.0/realesrgan-ncnn-vulkan-20211212-windows.zip) / [Linux](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.3.0/realesrgan-ncnn-vulkan-20211212-ubuntu.zip) / [MacOS](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.3.0/realesrgan-ncnn-vulkan-20211212-macos.zip) **executable files for Intel/AMD/Nvidia GPU**. You can find more information [here](#Portable-executable-files). The ncnn implementation is in [Real-ESRGAN-ncnn-vulkan](https://github.com/xinntao/Real-ESRGAN-ncnn-vulkan). Real-ESRGAN aims at developing **Practical Algorithms for General Image Restoration**.
We extend the powerful ESRGAN to a practical restoration application (namely, Real-ESRGAN), which is trained with pure synthetic data. @@ -24,7 +22,10 @@ We extend the powerful ESRGAN to a practical restoration application (namely, Re :question: Frequently Asked Questions can be found in [FAQ.md](FAQ.md) (Well, it is still empty there =-=||). +:milky_way: Thanks for your valuable feedbacks/suggestions. All the feedbacks are updated in [feedback.md](feedback.md). + :triangular_flag_on_post: **Updates** +- :white_check_mark: Add small models for anime videos. More details are in [anime video models](docs/anime_video_model.md). - :white_check_mark: Add the ncnn implementation [Real-ESRGAN-ncnn-vulkan](https://github.com/xinntao/Real-ESRGAN-ncnn-vulkan). - :white_check_mark: Add [*RealESRGAN_x4plus_anime_6B.pth*](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.2.4/RealESRGAN_x4plus_anime_6B.pth), which is optimized for **anime** images with much smaller model size. More details and comparisons with [waifu2x](https://github.com/nihui/waifu2x-ncnn-vulkan) are in [**anime_model.md**](docs/anime_model.md) - :white_check_mark: Support finetuning on your own data or paired data (*i.e.*, finetuning ESRGAN). See [here](Training.md#Finetune-Real-ESRGAN-on-your-own-dataset) @@ -80,21 +81,23 @@ If you have some images that Real-ESRGAN could not well restored, please also op ### Portable executable files -You can download [Windows](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.1.2/realesrgan-ncnn-vulkan-20210801-windows.zip) / [Linux](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.1.2/realesrgan-ncnn-vulkan-20210801-ubuntu.zip) / [MacOS](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.1.2/realesrgan-ncnn-vulkan-20210801-macos.zip) **executable files for Intel/AMD/Nvidia GPU**. +You can download [Windows](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.3.0/realesrgan-ncnn-vulkan-20211212-windows.zip) / [Linux](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.3.0/realesrgan-ncnn-vulkan-20211212-ubuntu.zip) / [MacOS](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.3.0/realesrgan-ncnn-vulkan-20211212-macos.zip) **executable files for Intel/AMD/Nvidia GPU**. This executable file is **portable** and includes all the binaries and models required. No CUDA or PyTorch environment is needed.
You can simply run the following command (the Windows example, more information is in the README.md of each executable files): ```bash -./realesrgan-ncnn-vulkan.exe -i input.jpg -o output.png +./realesrgan-ncnn-vulkan.exe -i input.jpg -o output.png -n model_name ``` -We have provided three models: +We have provided five models: 1. realesrgan-x4plus (default) 2. realesrnet-x4plus 3. realesrgan-x4plus-anime (optimized for anime images, small model size) +4. RealESRGANv2-animevideo-xsx2 (anime video, X2) +5. RealESRGANv2-animevideo-xsx4 (anime video, X4) You can use the `-n` argument for other models, for example, `./realesrgan-ncnn-vulkan.exe -i input.jpg -o output.png -n realesrnet-x4plus` @@ -213,18 +216,7 @@ A common command: python inference_realesrgan.py -n RealESRGAN_x4plus -i infile ## :european_castle: Model Zoo -- [RealESRGAN_x4plus](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.1.0/RealESRGAN_x4plus.pth): X4 model for general images -- [RealESRGAN_x4plus_anime_6B](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.2.4/RealESRGAN_x4plus_anime_6B.pth): Optimized for anime images; 6 RRDB blocks (slightly smaller network) -- [RealESRGAN_x2plus](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.1/RealESRGAN_x2plus.pth): X2 model for general images -- [RealESRNet_x4plus](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.1.1/RealESRNet_x4plus.pth): X4 model with MSE loss (over-smooth effects) - -- [official ESRGAN_x4](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.1.1/ESRGAN_SRx4_DF2KOST_official-ff704c30.pth): official ESRGAN model (X4) - -The following models are **discriminators**, which are usually used for fine-tuning. - -- [RealESRGAN_x4plus_netD](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.2.3/RealESRGAN_x4plus_netD.pth) -- [RealESRGAN_x2plus_netD](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.2.3/RealESRGAN_x2plus_netD.pth) -- [RealESRGAN_x4plus_anime_6B_netD](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.2.4/RealESRGAN_x4plus_anime_6B_netD.pth) +Please see [docs/model_zoo.md](docs/model_zoo.md) ## :computer: Training and Finetuning on your own dataset diff --git a/README_CN.md b/README_CN.md index d4fcad6..5a87074 100644 --- a/README_CN.md +++ b/README_CN.md @@ -10,19 +10,22 @@ [English](README.md) **|** [简体中文](README_CN.md) -1. Real-ESRGAN的[Colab Demo](https://colab.research.google.com/drive/1k2Zod6kSHEvraybHl50Lys0LerhyTMCo?usp=sharing) google colab logo. -2. **支持Intel/AMD/Nvidia显卡**的绿色版exe文件： [Windows版](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.2.4/realesrgan-ncnn-vulkan-20210901-windows.zip) / [Linux版](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.2.4/realesrgan-ncnn-vulkan-20210901-ubuntu.zip) / [macOS版](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.2.4/realesrgan-ncnn-vulkan-20210901-macos.zip)，详情请移步[这里](#便携版（绿色版）可执行文件)。NCNN的实现在 [Real-ESRGAN-ncnn-vulkan](https://github.com/xinntao/Real-ESRGAN-ncnn-vulkan)。 +:fire: :fire: :fire: 添加了**针对动漫视频的小模型**, 更多信息在 [anime video models](docs/anime_video_model.md) 中. -感谢大家的关注和使用:-) 关于动漫插画的模型，目前还有很多问题，主要有: 1. 视频处理不了; 2. 景深虚化有问题; 3. 不可调节, 效果过了; 4. 改变原来的风格。大家提供了很好的反馈。这些反馈会逐步更新在 [这个文档](feedback.md)。希望不久之后，有新模型可以使用. +1. Real-ESRGAN的[Colab Demo](https://colab.research.google.com/drive/1k2Zod6kSHEvraybHl50Lys0LerhyTMCo?usp=sharing) google colab logo. +2. **支持Intel/AMD/Nvidia显卡**的绿色版exe文件： [Windows版](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.3.0/realesrgan-ncnn-vulkan-20211212-windows.zip) / [Linux版](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.3.0/realesrgan-ncnn-vulkan-20211212-ubuntu.zip) / [macOS版](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.3.0/realesrgan-ncnn-vulkan-20211212-macos.zip)，详情请移步[这里](#便携版（绿色版）可执行文件)。NCNN的实现在 [Real-ESRGAN-ncnn-vulkan](https://github.com/xinntao/Real-ESRGAN-ncnn-vulkan)。 Real-ESRGAN 的目标是开发出**实用的图像修复算法**。
我们在 ESRGAN 的基础上使用纯合成的数据来进行训练，以使其能被应用于实际的图片修复的场景（顾名思义：Real-ESRGAN）。 :art: Real-ESRGAN 需要，也很欢迎你的贡献，如新功能、模型、bug修复、建议、维护等等。详情可以查看[CONTRIBUTING.md](CONTRIBUTING.md)，所有的贡献者都会被列在[此处](README_CN.md#hugs-感谢)。 +:milky_way: 感谢大家提供了很好的反馈。这些反馈会逐步更新在 [这个文档](feedback.md)。 + :question: 常见的问题可以在[FAQ.md](FAQ.md)中找到答案。（好吧，现在还是空白的=-=||） :triangular_flag_on_post: **更新** +- :white_check_mark: 添加了针对动漫视频的小模型, 更多信息在 [anime video models](docs/anime_video_model.md) 中. - :white_check_mark: 添加了ncnn 实现：[Real-ESRGAN-ncnn-vulkan](https://github.com/xinntao/Real-ESRGAN-ncnn-vulkan). - :white_check_mark: 添加了 [*RealESRGAN_x4plus_anime_6B.pth*](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.2.4/RealESRGAN_x4plus_anime_6B.pth)，对二次元图片进行了优化，并减少了model的大小。详情 以及 与[waifu2x](https://github.com/nihui/waifu2x-ncnn-vulkan)的对比请查看[**anime_model.md**](docs/anime_model.md) - :white_check_mark: 支持用户在自己的数据上进行微调 (finetune)：[详情](Training.md#Finetune-Real-ESRGAN-on-your-own-dataset) @@ -76,21 +79,23 @@ Real-ESRGAN 将会被长期支持，我会在空闲的时间中持续维护更 ### 便携版（绿色版）可执行文件 -你可以下载**支持Intel/AMD/Nvidia显卡**的绿色版exe文件： [Windows版](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.2.4/realesrgan-ncnn-vulkan-20210901-windows.zip) / [Linux版](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.2.4/realesrgan-ncnn-vulkan-20210901-ubuntu.zip) / [macOS版](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.2.4/realesrgan-ncnn-vulkan-20210901-macos.zip)。 +你可以下载**支持Intel/AMD/Nvidia显卡**的绿色版exe文件： [Windows版](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.3.0/realesrgan-ncnn-vulkan-20211212-windows.zip) / [Linux版](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.3.0/realesrgan-ncnn-vulkan-20211212-ubuntu.zip) / [macOS版](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.3.0/realesrgan-ncnn-vulkan-20211212-macos.zip)。 绿色版指的是这些exe你可以直接运行（放U盘里拷走都没问题），因为里面已经有所需的文件和模型了。它不需要 CUDA 或者 PyTorch运行环境。
你可以通过下面这个命令来运行（Windows版本的例子，更多信息请查看对应版本的README.md）： ```bash -./realesrgan-ncnn-vulkan.exe -i 输入图像.jpg -o 输出图像.png +./realesrgan-ncnn-vulkan.exe -i 输入图像.jpg -o 输出图像.png -n 模型名字 ``` -我们提供了三种模型： +我们提供了五种模型： 1. realesrgan-x4plus（默认） 2. reaesrnet-x4plus 3. realesrgan-x4plus-anime（针对动漫插画图像优化，有更小的体积） +4. RealESRGANv2-animevideo-xsx2 (针对动漫视频, X2) +5. RealESRGANv2-animevideo-xsx4 (针对动漫视频, X4) 你可以通过`-n`参数来使用其他模型，例如`./realesrgan-ncnn-vulkan.exe -i 二次元图片.jpg -o 二刺螈图片.png -n realesrgan-x4plus-anime` @@ -208,18 +213,7 @@ A common command: python inference_realesrgan.py -n RealESRGAN_x4plus -i infile ## :european_castle: 模型库 -- [RealESRGAN_x4plus](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.1.0/RealESRGAN_x4plus.pth): X4 model for general images -- [RealESRGAN_x4plus_anime_6B](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.2.4/RealESRGAN_x4plus_anime_6B.pth): Optimized for anime images; 6 RRDB blocks (slightly smaller network) -- [RealESRGAN_x2plus](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.1/RealESRGAN_x2plus.pth): X2 model for general images -- [RealESRNet_x4plus](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.1.1/RealESRNet_x4plus.pth): X4 model with MSE loss (over-smooth effects) - -- [official ESRGAN_x4](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.1.1/ESRGAN_SRx4_DF2KOST_official-ff704c30.pth): official ESRGAN model (X4) - -下面是 **判别器** 模型, 他们经常被用来微调（fine-tune）模型. - -- [RealESRGAN_x4plus_netD](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.2.3/RealESRGAN_x4plus_netD.pth) -- [RealESRGAN_x2plus_netD](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.2.3/RealESRGAN_x2plus_netD.pth) -- [RealESRGAN_x4plus_anime_6B_netD](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.2.4/RealESRGAN_x4plus_anime_6B_netD.pth) +请参见 [docs/model_zoo.md](docs/model_zoo.md) ## :computer: 训练，在你的数据上微调（Fine-tune） diff --git a/docs/anime_model.md b/docs/anime_model.md index ffbd51a..45c1882 100644 --- a/docs/anime_model.md +++ b/docs/anime_model.md @@ -32,7 +32,7 @@ python inference_realesrgan.py -n RealESRGAN_x4plus_anime_6B -i inputs ### ncnn Executable File -Download the latest portable [Windows](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.2.4/realesrgan-ncnn-vulkan-20210901-windows.zip) / [Linux](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.2.4/realesrgan-ncnn-vulkan-20210901-ubuntu.zip) / [MacOS](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.2.4/realesrgan-ncnn-vulkan-20210901-macos.zip) **executable files for Intel/AMD/Nvidia GPU**. +Download the latest portable [Windows](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.3.0/realesrgan-ncnn-vulkan-20211212-windows.zip) / [Linux](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.3.0/realesrgan-ncnn-vulkan-20211212-ubuntu.zip) / [MacOS](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.3.0/realesrgan-ncnn-vulkan-20211212-macos.zip) **executable files for Intel/AMD/Nvidia GPU**. Taking the Windows as example, run: diff --git a/docs/anime_video_model.md b/docs/anime_video_model.md new file mode 100644 index 0000000..fd83267 --- /dev/null +++ b/docs/anime_video_model.md @@ -0,0 +1,121 @@ +# Anime Video Models + +:white_check_mark: We add small models that are optimized for anime videos :-) + +| Models | Scale | Description | +| ---------------------------------------------------------------------------------------------------------------------------------- | :---- | :----------------------------- | +| [RealESRGANv2-animevideo-xsx2](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.3.0/RealESRGANv2-animevideo-xsx2.pth) | X2 | Anime video model with XS size | +| [RealESRGANv2-animevideo-xsx4](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.3.0/RealESRGANv2-animevideo-xsx4.pth) | X4 | Anime video model with XS size | + +- [Anime Video Models](#anime-video-models) + - [How to Use](#how-to-use) + - [PyTorch Inference](#pytorch-inference) + - [ncnn Executable File](#ncnn-executable-file) + - [Step 1: Use ffmpeg to extract frames from video](#step-1-use-ffmpeg-to-extract-frames-from-video) + - [Step 2: Inference with Real-ESRGAN executable file](#step-2-inference-with-real-esrgan-executable-file) + - [Step 3: Merge the enhanced frames back into a video](#step-3-merge-the-enhanced-frames-back-into-a-video) + - [More Demos](#more-demos) + +--- + +The following are some demos (best view in the full screen mode). + +https://user-images.githubusercontent.com/17445847/145706977-98bc64a4-af27-481c-8abe-c475e15db7ff.MP4 + +https://user-images.githubusercontent.com/17445847/145707055-6a4b79cb-3d9d-477f-8610-c6be43797133.MP4 + +https://user-images.githubusercontent.com/17445847/145707046-8702a17c-a194-4471-8a53-a4cc44c9594c.MP4 + +## How to Use + +### PyTorch Inference + +```bash +# download model +wget https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.3.0/RealESRGANv2-animevideo-xsx2.pth -P experiments/pretrained_models +# inference +python inference_realesrgan_video.py -i inputs/video/onepiece_demo.mp4 -n RealESRGANv2-animevideo-xsx2 -s 2 -v -a --half --suffix outx2 +``` + +### ncnn Executable File + +#### Step 1: Use ffmpeg to extract frames from video + +```bash +ffmpeg -i onepiece_demo.mp4 -qscale:v 1 -qmin 1 -qmax 1 -vsync 0 tmp_frames/frame%08d.png +``` + +- Remember to create the folder `tmp_frames` ahead + +#### Step 2: Inference with Real-ESRGAN executable file + +1. Download the latest portable [Windows](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.3.0/realesrgan-ncnn-vulkan-20211212-windows.zip) / [Linux](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.3.0/realesrgan-ncnn-vulkan-20211212-ubuntu.zip) / [MacOS](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.3.0/realesrgan-ncnn-vulkan-20211212-macos.zip) **executable files for Intel/AMD/Nvidia GPU** + +1. Taking the Windows as example, run: + + ```bash + ./realesrgan-ncnn-vulkan.exe -i tmp_frames -o out_frames -n RealESRGANv2-animevideo-xsx2 -s 2 -f jpg + ``` + + - Remember to create the folder `out_frames` ahead + +#### Step 3: Merge the enhanced frames back into a video + +1. First obtain fps from input videos by + + ```bash + ffmpeg -i onepiece_demo.mp4 + ``` + + ```console + Usage: + -i input video path + ``` + + You will get the output similar to the following screenshot. + +

+ +

+ +2. Merge frames + + ```bash + ffmpeg -i out_frames/frame%08d.jpg -c:v libx264 -r 23.98 -pix_fmt yuv420p output.mp4 + ``` + + ```console + Usage: + -i input video path + -c:v video encoder (usually we use libx264) + -r fps, remember to modify it to meet your needs + -pix_fmt pixel format in video + ``` + + If you also want to copy audio from the input videos, run: + + ```bash + ffmpeg -i out_frames/frame%08d.jpg -i onepiece_demo.mp4 -map 0:v:0 -map 1:a:0 -c:a copy -c:v libx264 -r 23.98 -pix_fmt yuv420p output_w_audio.mp4 + ``` + + ```console + Usage: + -i input video path, here we use two input streams + -c:v video encoder (usually we use libx264) + -r fps, remember to modify it to meet your needs + -pix_fmt pixel format in video + ``` + +## More Demos + +- Input video for One Piece: + + https://user-images.githubusercontent.com/17445847/145706822-0e83d9c4-78ef-40ee-b2a4-d8b8c3692d17.mp4 + +- Out video for One Piece + + https://user-images.githubusercontent.com/17445847/145706827-384108c0-78f6-4aa7-9621-99d1aaf65682.mp4 + +**More comparisons** + +https://user-images.githubusercontent.com/17445847/145707458-04a5e9b9-2edd-4d1f-b400-380a72e5f5e6.MP4 diff --git a/docs/model_zoo.md b/docs/model_zoo.md new file mode 100644 index 0000000..1a30ebd --- /dev/null +++ b/docs/model_zoo.md @@ -0,0 +1,47 @@ +# :european_castle: Model Zoo + +- [:european_castle: Model Zoo](#european_castle-model-zoo) + - [For General Images](#for-general-images) + - [For Anime Images](#for-anime-images) + - [For Anime Videos](#for-anime-videos) + +--- + +## For General Images + +| Models | Scale | Description | +| ------------------------------------------------------------------------------------------------------------------------------- | :---- | :------------------------------------------- | +| [RealESRGAN_x4plus](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.1.0/RealESRGAN_x4plus.pth) | X4 | X4 model for general images | +| [RealESRGAN_x2plus](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.1/RealESRGAN_x2plus.pth) | X2 | X2 model for general images | +| [RealESRNet_x4plus](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.1.1/RealESRNet_x4plus.pth) | X4 | X4 model with MSE loss (over-smooth effects) | +| [official ESRGAN_x4](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.1.1/ESRGAN_SRx4_DF2KOST_official-ff704c30.pth) | X4 | official ESRGAN model | + +The following models are **discriminators**, which are usually used for fine-tuning. + +| Models | Corresponding model | +| ---------------------------------------------------------------------------------------------------------------------- | :------------------ | +| [RealESRGAN_x4plus_netD](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.2.3/RealESRGAN_x4plus_netD.pth) | RealESRGAN_x4plus | +| [RealESRGAN_x2plus_netD](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.2.3/RealESRGAN_x2plus_netD.pth) | RealESRGAN_x2plus | + +## For Anime Images + +| Models | Scale | Description | +| ------------------------------------------------------------------------------------------------------------------------------ | :---- | :---------------------------------------------------------- | +| [RealESRGAN_x4plus_anime_6B](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.2.4/RealESRGAN_x4plus_anime_6B.pth) | X4 | Optimized for anime images; 6 RRDB blocks (smaller network) | + +The following models are **discriminators**, which are usually used for fine-tuning. + +| Models | Corresponding model | +| ---------------------------------------------------------------------------------------------------------------------------------------- | :------------------------- | +| [RealESRGAN_x4plus_anime_6B_netD](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.2.4/RealESRGAN_x4plus_anime_6B_netD.pth) | RealESRGAN_x4plus_anime_6B | + +## For Anime Videos + +| Models | Scale | Description | +| ---------------------------------------------------------------------------------------------------------------------------------- | :---- | :----------------------------- | +| [RealESRGANv2-animevideo-xsx2](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.3.0/RealESRGANv2-animevideo-xsx2.pth) | X2 | Anime video model with XS size | +| [RealESRGANv2-animevideo-xsx4](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.3.0/RealESRGANv2-animevideo-xsx4.pth) | X4 | Anime video model with XS size | + +The following models are **discriminators**, which are usually used for fine-tuning. + +TODO diff --git a/inference_realesrgan_video.py b/inference_realesrgan_video.py index 353f143..c0786d5 100644 --- a/inference_realesrgan_video.py +++ b/inference_realesrgan_video.py @@ -39,6 +39,8 @@ def main(): parser.add_argument('-v', '--video', action='store_true', help='Output a video using ffmpeg') parser.add_argument('-a', '--audio', action='store_true', help='Keep audio') parser.add_argument('--fps', type=float, default=None, help='FPS of the output video') + parser.add_argument('--consumer', type=int, default=4, help='Number of IO consumers') + parser.add_argument( '--alpha_upsampler', type=str, @@ -133,8 +135,7 @@ def main(): reader.start() que = queue.Queue() - num_consumer = 4 - consumers = [IOConsumer(args, que, f'IO_{i}') for i in range(num_consumer)] + consumers = [IOConsumer(args, que, f'IO_{i}') for i in range(args.consumer)] for consumer in consumers: consumer.start() @@ -171,7 +172,7 @@ def main(): avg_fps = 1. / (timer.get_avg_time() + 1e-7) pbar.set_description(f'idx {idx}, fps {avg_fps:.2f}') - for _ in range(num_consumer): + for _ in range(args.consumer): que.put('quit') for consumer in consumers: consumer.join() diff --git a/inputs/video/onepiece_demo.mp4 b/inputs/video/onepiece_demo.mp4 new file mode 100644 index 0000000..29b4e52 Binary files /dev/null and b/inputs/video/onepiece_demo.mp4 differ