jamt9000/dve

Descriptor Vector Exchange

dve's Issues

May I know how to sample the pixels which are used in the loss function?

Hi @jamt9000,
Thanks for the great work. I have one question about the implementation for pixel sampling. How to sample pixels from the original images, which will be used in the loss function? I am not quite understand these two lines of code in the implementation:

Could you please give some explanation about the function random_tps_weights()? What does 'weights1' represent and also the multiplication between 'self.F' and 'weights1'

Smaller image is used during landmark regression on MAFL?

DVE/data_loader/data_loaders.py

Line 178 in 2085144

data = data[:, self.crop:-self.crop, self.crop:-self.crop]

It seems that during unsupervised learning, the images are not cropped, while during landmark regression on MAFL (or other datasets), they are cropped by self.crop. Is that right? If yes, any reasons behind this?

Thanks for sharing the codes! It is really useful!

Missing test.py?

In section https://github.com/jamt9000/DVE#regressing-landmarks, a test.py command script is mentioned, but this file is missing from the repository.

Evaluation fails when training terminates before max epoch set in config

I've trained the SmallNet with 3D descriptors using the config file linked from the README.

For me, the training terminated already after 70 epochs instead of 100:

Running validation for epoch 71
validation epoch took 00h00m13s
    epoch          : 71
    loss           : 1.8158737950681234
    val_loss       : 1.765822774887085
Val performance didn't improve for 10 epochs. Training stops.

This resulted in the following error when the train.py script attempted to load the checkpoint for epoch 100 (value from config file) for the mini-evaluation:

Loading checkpoint: saved/models/celeba-smallnet-3d-dve-2019-08-08_17-54-21/2019-09-18_17-09-50/checkpoint-epoch100.pth ...
Traceback (most recent call last):
  File "train.py", line 241, in <module>
    main(config, args.resume)
  File "train.py", line 176, in main
    evaluation(config, logger=logger)
  File "/data/aschuh/source/dve/test_matching.py", line 171, in evaluation
    checkpoint = torch.load(ckpt_path)
  File "/data/aschuh/tools/pyenv/versions/dve/lib/python3.6/site-packages/torch/serialization.py", line 384, in load
    f = f.open('rb')
  File "/data/aschuh/tools/pyenv/versions/3.6.9/lib/python3.6/pathlib.py", line 1183, in open
    opener=self._opener)
  File "/data/aschuh/tools/pyenv/versions/3.6.9/lib/python3.6/pathlib.py", line 1037, in _opener
    return self._accessor.open(self, flags, mode)
  File "/data/aschuh/tools/pyenv/versions/3.6.9/lib/python3.6/pathlib.py", line 387, in wrapped
    return strfunc(str(pathobj), *args)
FileNotFoundError: [Errno 2] No such file or directory: 'saved/models/celeba-smallnet-3d-dve-2019-08-08_17-54-21/2019-09-18_17-09-50/check
point-epoch100.pth'

The BaseTrainer should probably store the last epoch (or change self.epochs) in this case (cf.

DVE/base/base_trainer.py

Line 139 in e6e0cdb

self.logger.info("Val performance didn\'t improve for {} epochs. "

) and that value be used in train.py at

DVE/train.py

Line 173 in e6e0cdb

epoch = config["trainer"]["epochs"]

instead of the value from the config file.

Error 404 when trying to download pretrained model files

http://www.robots.ox.ac.uk/~vgg/research/DVE/data/models/celeba-smallnet-3d-dve/2019-08-08_17-54-21/model_best.pth

"The requested URL /~vgg/research/DVE/data/models/celeba-smallnet-3d-dve/2019-08-08_17-54-21/model_best.pth was not found on this server."

Animal Datasets

Hello.
I'm very impressed with this work.

I found the experiment with human and animal face datasets on the paper.
I wonder if you have any plan to release preprocessed dataset like other human face dataset.
Thank you :)

AttributeError: 'CelebAPrunedAligned_MAFLVal' object has no attribute 'use_ims'

When trying to train a SmallNet on the celeba dataset using the configuration file http://www.robots.ox.ac.uk/~vgg/research/DVE/data/models/celeba-smallnet-3d-dve/2019-08-08_17-54-21/config.json, I get the following error.

Loss args OrderedDict([('normalize_vectors', False)])
Traceback (most recent call last):
  File "train.py", line 241, in <module>
    main(config, args.resume)
  File "train.py", line 169, in main
    trainer.train()
  File "/data/aschuh/source/dve/base/base_trainer.py", line 86, in train
    result = self._train_epoch(epoch)
  File "/data/aschuh/source/dve/trainer/trainer.py", line 191, in _train_epoch
    for batch_idx, batch in enumerate(self.data_loader):
  File "/data/aschuh/tools/pyenv/versions/dve/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 819, in __next__
    return self._process_data(data)
  File "/data/aschuh/tools/pyenv/versions/dve/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 846, in _process_data
    data.reraise()
  File "/data/aschuh/tools/pyenv/versions/dve/lib/python3.6/site-packages/torch/_utils.py", line 369, in reraise
    raise self.exc_type(msg)
AttributeError: Caught AttributeError in DataLoader worker process 0.
Original Traceback (most recent call last):
  File "/data/aschuh/tools/pyenv/versions/dve/lib/python3.6/site-packages/torch/utils/data/_utils/worker.py", line 178, in _worker_loop
    data = fetcher.fetch(index)
  File "/data/aschuh/tools/pyenv/versions/dve/lib/python3.6/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/data/aschuh/tools/pyenv/versions/dve/lib/python3.6/site-packages/torch/utils/data/_utils/fetch.py", line 44, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/data/aschuh/source/dve/data_loader/data_loaders.py", line 105, in __getitem__
    if (not self.use_ims and not self.use_keypoints):
AttributeError: 'CelebAPrunedAligned_MAFLVal' object has no attribute 'use_ims'

Question on keypoint computation

Sharing a question received by email:

Thanks for sharing your DVE codes. I found the new keypoints computed in your code is somehow counter-intuitive. Could you kindly explain: Why you construct a KD tree with the warped grid instead of the regular one?

This is a link to the questioned line in your code

DVE/utils/tps.py

Line 164 in 2085144

def warp_keypoints(self, keypoints, grid_unnormalized):