by Jeffrey Braga, Doris Chia-ching Lin
This project is an attempt to use TIRG function for image retrieval with a new furniture style dataset. The original code was published with the following paper:
Composing Text and Image for Image Retrieval - An Empirical Odyssey
Nam Vo, Lu Jiang, Chen Sun, Kevin Murphy, Li-Jia Li, Li Fei-Fei, James Hays
CVPR 2019.
github code source
The dataset was published with the paper below:
Learning Style Compatibility for Furniture
Divyansh Aggarwal, Elchin Valiyev, Fadime Sener, and Angela Yao
CVPR 2018.
dataset download
In order to implement TIRG for the furniture style dataset, we have created test_queries.txt
to map the index of source image and target image for image retrieval. Each image has captions which can be cross-referenced with testing queries. Here are our dataset stats:
Train Model
- FurnitureStyle: 133346 images
- 56520 unique captions
- Modifiable images 40076
Test Model
- FurnitureStyle: 28812 images
- 28812 test queries
- torchvision
- pytorch
- numpy
- tqdm
- tensorboardX
- Python 3.0.0 or above
- Need to have a
cuda
compatible graphic card
main.py
: driver script to run training/testingdatasets.py
: Dataset classes for loading images & generate training retrieval queriestext_model.py
: LSTM model to extract text featuresimg_text_composition_models.py
: various image text compostion models (described in the paper)torch_function.py
: contains soft triplet loss function and feature normalization functiontest_retrieval.py
: functions to perform retrieval test and compute recall performance
Download our generated test_queries.txt from here.
Make sure the dataset include these files:
<dataset_path>/splits/*.txt
<dataset_path>/houzz/<category>/<style>/*.jpeg
<dataset_path>/test_queries.txt`
note that the file name val_split
in /splits/
from origin dataset should be renamed to test_val_split
Run training & testing:
python main.py --dataset=furnitureStyle --dataset_path=./furniture-style \
--num_iters=160000 --model=concat --loss=batch_based_classification \
--learning_rate_decay_frequency=50000 --comment=furnitureS_concat
python3 main.py --dataset=furnitureStyle --dataset_path=./furniture-style \
--num_iters=160000 --model=tirg --loss=batch_based_classification \
--learning_rate_decay_frequency=50000 --comment=furnitureS_tirg