maluuba/geneva

Code to train and evaluate the GeNeVA-GAN model for the GeNeVA task proposed in our ICCV 2019 paper "Tell, Draw, and Repeat: Generating and modifying images based on continual linguistic instruction"

geneva's Issues

How to reproduce the best result?

I follow the instructions in README.md, but I can't get the best results as the paper said.
Here is what I haved done:

1. Follow the instructions from https://github.com/Maluuba/GeNeVA_datasets/, and generate the data file.
2. Download the pretrained object detector model from https://www.microsoft.com/en-us/research/project/generative-neural-visual-artist-geneva/
3. Use the default parameters and run python geneva/inference/train.py @example_args/iclevr-d-subtract.args

Here is my results in iclevr dataset, I trained this model with 2 NVIDIA TITAN Xp about 4 days:

It seems it's far from the performance in the origin paper.

How to change the specified text condition？

Excuse me，sir. I'm a student interested in this course. Sorry to bother you, I want to know how to use the specified text to generate the corresponding image? If I want to add a block diagram by pyqt, where should I add it? Wait for the good sound.

What is your pytorch version?

Hi I wonder which pytorch version you use? I ran into some weird warning issue. The one that bothers a lot is this warning message:
/opt/conda/conda-bld/pytorch_1573049304260/work/aten/src/ATen/native/cudnn/RNN.cpp:1268: UserWarning: RNN module weights are not part of single contiguous chunk of memory. This means they need to be compacted at every call, possibly greatly increasing memory usage. To compact weights again call flatten_parameters().

I think it is because you are applying DataParallel on the RNN, but I am not quite sure how to resolve it.
Thank you for taking a look at this issue.

pre-trained models

Do you have a pre-trained model? Where can I find it? Is that possible that we can just run one script to see your examples mentioned in your paper Fig. 5?

AttributeError: 'NoneType' object has no attribute 'seek'.

Hello! Thank you very much for your wonderful work!
When I reproduced the code, some problems occurred:
First of all, I can't seem to generate the corresponding h5 file on the CoDraw data set. Do you have a classified data connection?
Then, when I tested the model, some problems appeared:
During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "geneva/inference/test.py", line 64, in
tester = Tester(cfg, test_eval=True)
File "geneva/inference/test.py", line 31, in init
self.model.load(model_path, iteration)
File "/home/zhaoliuqing/GeNeVA/geneva/models/inference_models/recurrent_gan.py", line 111, in load
snapshot = _read_weights(pre_trained_path, iteration)
File "/home/zhaoliuqing/GeNeVA/geneva/models/inference_models/recurrent_gan.py", line 148, in _read_weights
snapshot = torch.load(pre_trained_path)
File "/home/zhaoliuqing/anaconda3/envs/geneva/lib/python3.6/site-packages/torch/serialization.py", line 358, in load
return _load(f, map_location, pickle_module)
File "/home/zhaoliuqing/anaconda3/envs/geneva/lib/python3.6/site-packages/torch/serialization.py", line 520, in _load
_check_seekable(f)
File "/home/zhaoliuqing/anaconda3/envs/geneva/lib/python3.6/site-packages/torch/serialization.py", line 179, in _check_seekable
raise_err_msg(["seek", "tell"], e)
File "/home/zhaoliuqing/anaconda3/envs/geneva/lib/python3.6/site-packages/torch/serialization.py", line 172, in raise_err_msg
raise type(e)(msg)
AttributeError: 'NoneType' object has no attribute 'seek'. You can only torch.load from a file that is seekable. Please pre-load the data into a buffer like io.BytesIO and try to load from it instead.
Have you ever encountered a similar problem? Thank you!

data preprocessing, Convert BGR and RGB

Hi,

This is a very minor issue.
(If I understand correctly, the background images are only used at the very first time step.)

Why not converting BGR to RGB in
https://github.com/Maluuba/GeNeVA/blob/master/geneva/data/codraw_dataset.py#L19 ,
as you have done in
https://github.com/Maluuba/GeNeVA/blob/master/geneva/data/codraw_dataset.py#L72 ?

Thanks!