vlar-group/nvfi

NVFi in PyTorch (NeurIPS 2023)

nvfi's Issues

InDoorSeg cannot be learned.

The InDoorSeg training code is not incorrect, but the quality of generation in the extrapolation of the training viewpoint of the test data is relatively good, but the images of the unknown viewpoint (interpolation) tended to be black.
If you know of a solution, please let me know.

[Question] NVFi dataset making process.

I wonder how to make an NVFi dataset.

As I know, it is similar to the D-NeRF dataset structure. When I use ColMap to save videos as images, the time information is not
included in the JSON file. I wonder about how this was included in the JSON file.

I would appreciate it if you tell me how this process.

Bug in model evaluation—datasets with more than 1000 images have very low PSNRs. Easy fix.

The render_test_evaluation() function in train_nvfi() writes generated images to disk with files names r_some_number.png where the some_number indicates the position of the corresponding target in all_targets. However, the read_images_in_dir() function sorts these images by their filenames as strings and not numbers, and hence there is a mismatch between the ordering of generated examples and targets.

This shouldn't be impactful if there are fewer than 1000 images since the images have up to three zeros prepended to their index in the corresponding filename.

The fix is simple: in the read_images_in_dir() function in utils/metrics.py, one must replace

fnames.sort()

with

remove_non_numeric = lambda x: ''.join(filter(str.isdigit, x))
fnames = sorted(fnames, key=lambda x: int(remove_non_numeric(x)))

Missing libraries and argparse arguments

Hi,

I'm trying to run the code as instructed, but stumbled upon a few issues.

1. Some requirements are missing.
   in tensorf_model_utils.py there are the following imports which are not in the requirements:

import plyfile
import skimage.measure

A pip install of plyfile and scikit-image seems to fix it, so I guess it should be added to the requirements.txt.

2. in train_nvfi.py the argument config_args.keyframe is missing, causing an attribute error.
   I assume this would be a boolean flag, indicating whether the keyframe optimization in section 3.2 of your paper is used..?

Questions about the construction of get_vel_loss()

Hi! Thanks for your great job and finally it opened!
However, as I was reading the code of velocity part, I found that new random points and time were considered when calculating the velocity constrain. Is any reason to do so? Since usually I think the pts to render color should be the same with the pts calculating all the losses, and your code seems to be an exception.

Looking forward to your reply soon!