Comments (22)
I have successfully converted all my RTCW/ET textures with ESRGAN and my own automated RTCWHQ method, see https://github.com/Krischan74/RTCWHQ
from esrgan.
I've found a new code that preserves alpha channel.
Credit goes to RWTema: https://pastebin.com/sb5PCKxT
I did a few testing and it works BUT it takes very long to process even on my GTX1080.
Normally a picture from 320x240 takes one second. With that new code it takes more than one Minute on the same picture.
from esrgan.
I think it is possible. After reading images, only process the color channels with the ESRGAN model and directly copy (or maybe bicubic upsampling) the original alpha channel. After that, concatenate the color channels with the alpha channels, obtaining the final result.
from esrgan.
I am not familiar with images with alpha channels. I do not know whether I understand correctly.
- The simplest way may be to use the current model for RGB images and directly upsample the alpha channel. And finally combine RGB channels and Alpha Channels together.
However, there are some problems when the edges of the alpha channel mismatch the RGB channels, I found an example for game images here . - So another improved method is to also upsample the alpha channel, treating it as a gray image using the current model. I am not sure what the final effects looks like, may have a try.
- If having lots of training data with alpha channel images, we can fine-tune the current model with RGBA channels. (May also get the training pairs from other HD games, which will decrease the domain gap.)
If possible, could you @shogun1337 @Dilapidated provide some example images with RGBA channels? I may have a try with current models ;-)
from esrgan.
OpenCV seems to have support for reading an alpha channel, theres a cv2.IMREAD_COLOR flag that i've changed to cv2.IMREAD_UNCHANGED but sadly seems it nothing changed
from esrgan.
Having this supporting alpha channel on output would be awesome. Probably more of a BasicSR question but I was curious if using images with alpha channels would cause issues currently with training as well?
from esrgan.
@xinntao Sure, there are some texture samples with transparency from the "Return To Castle Wolfenstein" game
https://www.mediafire.com/file/8nu5gyy1asvq0mf/samples.rar/file
from esrgan.
@xinntao Thank you for looking into this deeper. I have included examples from two different games for variety/style purposes. Resident Evil 2 and New Super Mario Bros.
https://www.mediafire.com/file/f6twysc62h6kfpg/ALPHA_EXAMPLES.zip/file
from esrgan.
The results are also pixelated, maybe did he used the nearest neighbour interpolation?
from esrgan.
@Fraggoso Thank you for sharing this. It's good to see progress being made on retaining the alpha. Sadly I get the same pixelated artifacts as Shogun1337 and it does seem to take quite a bit longer. Hopefully @xinntao can further refine the code to fix these issues.
from esrgan.
@shogun1337 Can you post your image source so I can try it as well?
Like I said, it worked on my end.
from esrgan.
I have tried it with this script https://pastebin.com/xUdTHff9
It seems that the results are also not satisfying.
The results can be downloaded from here.
To control the noise and details, I also use the interpolation between RRDB_ESRGAN_x4.pth
and RRDB_PSNR_x4.pth
.
from esrgan.
@xinntao The results, at least for me, were expected to look like they do.
Your script isn't working on my end though. Did you change anything? I receive an error on line 35 img.shape
from esrgan.
@Fraggoso You can find this image in the pack i've posted ealier
(https://www.mediafire.com/file/8nu5gyy1asvq0mf/samples.rar/file)
@xinntao I think the results won't be better, even with extracting alpha channel from the original picture, bicubic upsizing in photoshop i'm getting very similar result:
(open this image on white background to see bicubic pixel artifacts with photoshop)
from esrgan.
@xinntao I receive an error if I run the script:
Traceback (most recent call last): File "test.py", line 35, in <module> H, W, C = img.shape
What am I missing?
from esrgan.
@Fraggoso
Hmm it works good with python 3.5 x64 (for windows), opencv 3.4.1, numpy 1.14.5+mkl, torch 0.4.1
Have you tried updating dependencies?
from esrgan.
Isn't it a way to simply ignore alpha channel and save the upscaled image without any alpha? That'd be good!
from esrgan.
Has there been any progress with this? Upscaling a game like Metroid Prime Trilogy will take forever without alpha channel handling. I can't even figure out how to programmatically sort the PNGs that are dumped by which ones have alpha and which ones don't, so I can process the ones with transparency with waifu2x-caffe, so I'd have to go through 10k+ images one by one and manually sort them.
from esrgan.
@Krischan74 incredible work, thank you very much, this can be used for every other games aswell
from esrgan.
Thanks. I've created this for my "Enemy Territory Special Edition" (ETSE) which is still under construction / testing. It won't appear on Github but perhaps on moddb as it is LARGE (10GB!). At least the cool map launcher ETlaunch will be in my repository soon.
Oh and many thanks to the guys from ESRGAN without ETSE wouldn't be possible. Before I found your cool project I upscaled thousands of textures MANUALLY in Photoshop with a special template I created. That looked already great but the ESRGAN results are even more stunning :-)
from esrgan.
Is it possible to edit the code so that you don't have to check if an image has alpha in them or not?
I'm not a coder, sadly or I could try to do it myself but I really think it should be an easy fix to tell in test.py IF the image has alpha than proceed with X if not than proceed with Z?
from esrgan.
If anyone else is interested in working with the alpha channel, you can look at my new repository, where I am creating a fork of the Real-ESRGAN with multi-channel support (essentially training with any number of channels, in particular 4-channel, with alpha channel support)
https://github.com/Robolightning/Multichannel-Real-ESRGAN
from esrgan.
Related Issues (20)
- Is there a way to do I/O with YUV instead of RGB?
- Inference ESRGAN with Multiple GPUs
- 有关参数的设置
- CUDA runs out of memory HOT 4
- _pickle.UnpicklingError: invalid load key, '<'. run in google colab HOT 4
- Can you put the complete training code of ESRGAN in this repo? BasicSR is too complex and not very friendly for beginners HOT 2
- 为什么我使用自己修改的rrdb之后训练进行插值会得到这个bug
- Not recognising my upscale model.
- When I use RDN as the generator for training, the details of the generated image will appear R, G or B color spots.
- Please stop changing the state keys HOT 1
- Do you have a x2 pre-trained model?
- When I trained Real-ESRNet, I encountered this problem. HOT 1
- Not an issue just a question about image extend
- fail on MacOS(VENTURA) with NDArray > 2**32
- Shortcut connection / Residual structure in unnecessary for RRDB
- Error: Found no NVIDIA driver on your system #3699
- What is differents between RealESRGAN_x4plus.pth and RealESRGAN_x4plusD.pth
- Why the result is not statisfactory as like the result in paper? HOT 1
- Can esrgan use the 4x-ultrasharp model? HOT 1
- Agrandar imagen
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from esrgan.