Comments (5)
The output values are indeed outside the range [0,1] and clamping the output doesn't remove the artifacts. I don't think that using sigmoid on the random image is a viable option because the loss function is meant to be used with the output of a neural network, which initially outputs random garbage before it learns to generate realistic images.
This loss function doesn't seem to discriminate against these strong artifacts, I was only able to get rid of them using value < 0.99997
(which is far higher than the score my denoising application can achieve), so I suspect that the network would never learn to get rid of these artifacts once they appear in the first few itterations.
Do you think not being able to detect these artifacts is an inherent issue with MS-SSIM or a bug?
from pytorch-msssim.
Would you mind providing a complete example of the issue you are facing?
I simplified the test script to accommodate your example and the output did not display the artifacts contained in your initial post. The simplified optimization script is as follows:
from pytorch_msssim import msssim
import torch
from torch import optim
import numpy as np
import torchvision
from PIL import Image
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
totensor = torchvision.transforms.ToTensor()
img1 = totensor(Image.open('kate.png'))
img1 = img1.reshape([1]+list(img1.shape))
img2 = torch.rand(img1.size())
img2 = torch.nn.functional.sigmoid(img2)
img1 = img1.to(device)
img2 = img2.to(device)
img1.requires_grad = False
img2.requires_grad = True
value = msssim(img1, img2)
optimizer = optim.Adam([img2], lr=0.01)
print("Initial MS-SSIM: %.5f" % (value.item()))
while value < 0.999:
optimizer.zero_grad()
msssim_out = -msssim(img1, img2)
value = -msssim_out.item()
print('Current MS-SSIM = %.5f' % value)
msssim_out.backward()
optimizer.step()
torchvision.utils.save_image(img2, 'out.png')
from pytorch-msssim.
I don't get the artifacts when I use the code you provided here, but they come back when I comment out the sigmoid function. It doesn't seem correct to use it because torch.rand returns uniformly distributed numbers between 0 and 1 which limits sigmoid to >=0.5
>>> img1.min(),img1.max()
(tensor(0.), tensor(0.6549))
>>> img2.min(),img2.max()
(tensor(1.2696e-05), tensor(1.0000))
>>> img2 = torch.nn.functional.sigmoid(img2)
>>> img2.min(),img2.max()
(tensor(0.5000), tensor(0.7311))
from pytorch-msssim.
While that would be true before optimization, the sigmoid application guarantees that the output of optimization is always in the interval [0,1], thus describing a valid image (without artifacts). The minimum value of 0.5 will decrease during optimization to reach the intended value. You could just shift the initial uniform sampling in order to begin the optimization with a [0,1] interval after the sigmoid if you prefer.
If you look at the minimum and maximum values during optimization for your version (without sigmoid) you should see that the minimum value drops below 0, creating the reported artifacts in the initial issue. This is an expected consequence of the optimization procedure used.
Let me know if this solved your concerns, or otherwise close the issue. Thank you for your feedback!
from pytorch-msssim.
Hello,
I apologize for the delay since my last answer. I cannot guarantee that this problem is an inherent MS-SSIM issue and not an implementation introduced error but I was not able to find or reproduce your issues. If your problem still persists, please feel free to re-open a new issue where we can actively discuss this. I will be closing this one due to age.
Regarding stability for the optimization, I will introduce a check in the value for the ssim and mc calculations in each scale iteration using a relu to ensure both values are always > 0, based on the implementation by https://github.com/VainF.
from pytorch-msssim.
Related Issues (20)
- nan problem HOT 2
- Output is out of range HOT 2
- Bug in your MSSSIM implementation HOT 3
- Is the threshold necessary? HOT 1
- Inquiry about standard deviations HOT 1
- ssim on mnist HOT 1
- Index map HOT 1
- Contrast Sensitivity (CS) for batch size > 1 HOT 1
- Typo in calculation? HOT 2
- Arbitrary NAN for very low MS-SSIM comparisons HOT 15
- Big in SSIM implementation, don't use this code for perceptual quality estimation HOT 3
- SSIM result is different from skimage.measure.compare_ssim HOT 1
- parameter misspelled in readme HOT 1
- Usage of this loss in the binary segmentation
- SSIM loss output range
- MS-SSIM Usage HOT 1
- Expected object of scalar type Double but got scalar type Float for argument #2 'weight' HOT 2
- Possible mistake in your implementation? HOT 3
- functools.lru_cache can be used for saving the window HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pytorch-msssim.