lingxiao-li/hae

ICCV 2023: The Euclidean Space is Evil: Hyperbolic Attribute Editing for Few-shot Image Generation

The Euclidean Space is Evil: Hyperbolic Attribute Editing for Few-shot Image Generation (ICCV 2023)

Few-shot image generation is a challenging task since it aims to generate diverse new images for an unseen category with only a few images. Existing methods suffer from the trade-off between the quality and diversity of generated images. To tackle this problem, we propose Hyperbolic Attribute Editing (HAE), a simple yet effective method. Unlike other methods that work in Euclidean space, HAE captures the hierarchy among images using data from seen categories in hyperbolic space. Given a well-trained HAE, images of unseen categories can be generated by moving the latent code of a given image toward any meaningful directions in the Poincaré disk with a fixing radius. Most importantly, the hyperbolic space allows us to control the semantic diversity of the generated images by setting different radii in the disk. Extensive experiments and visualizations demonstrate that HAE is capable of not only generating images with promising quality and diversity using limited data but achieving a highly controllable and interpretable editing process.

Description

Official implementation of HAE for few-shot image generation. Our code is modified from pSp.

Getting Started

Prerequisites

• Linux
• NVIDIA GPU + CUDA CuDNN (CPU may be possible with some modifications, but is not inherently supported)
• Python 3

Installation

• Clone this repo:

git clone https://github.com/lingxiao-li/HAE.git
cd HAE

• Dependencies:
  We recommend running this repository using Anaconda. All dependencies for defining the environment are provided in /HAE/hae/environment/hae_env.yaml.

Pre-trained Models

Then download the pre-trained models we provide from Google Drive

Then put the pre-trained models under /HAE/hae/pretrained_models

Training

Preparing your Data

Related datasets can be downloaded at: https://github.com/bcmi/Awesome-Few-Shot-Image-Generation

After downloading the pre-trained models and datasets, change the corresponding path at /Codes/hae/configs/paths_config.py

Train

Go to the path to HAE:

cd /PATH_TO/HAE/hae

Then you can train the model using:

python scripts/train.py \
--dataset_type=flowers_encode_eva \
--psp_checkpoint_path=/PATH_TO/psp_flowers.pt \
--exp_dir=OUTPUT_PATH \
--feature_size=512 \
--workers=8 \
--batch_size=8 \
--test_batch_size=8 \
--test_workers=8 \
--val_interval=80000 \
--save_interval=5000 \
--encoder_type=GradualStyleEncoder \
--start_from_latent_avg \
--lpips_lambda=1 \
--l2_lambda=1 \
--image_interval=1000 \
--hyperbolic_lambda=0.3 \
--reverse_lambda=1

Inference

Inference the images using:

python scripts/inference.py \
--exp_dir=OUTPUT_PATH \
--checkpoint_path=/PATH_TO/hae_flowers.pt \
--data_path=PATH_TO/flowers_eva/test \
--test_batch_size=4 \
--test_workers=4

Visualization

Please see the comments in the notebook:

Interpolation & Perturbation.ipynb and UMAP_Visualization.ipynb

Citation

If you use this code for your research, please cite our paper The Euclidean Space is Evil: Hyperbolic Attribute Editing for Few-shot Image Generation:

@InProceedings{Li_2023_HAE,
    author    = {Li, Lingxiao and Zhang, Yi and Wang, Shuhui},
    title     = {The Euclidean Space is Evil: Hyperbolic Attribute Editing for Few-shot Image Generation},
    booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
    month     = {October},
    year      = {2023},
    pages     = {22714-22724}
}

hae's Issues

Code error when using pre-trained model for inferrence

First, thanks for providing the source code for comparison!
However, when I run the inference.py with the pre-trained models and instructions you provided, the code does not run successfully. I need the generated images by HAE for comparative experiments, could you provide clearer instructions for the inference? If I reproduce your results successfully, I will cite your paper in my work.

VGGFace dataset

Hello, I am very interested in your research and want to execute your program. However, when using the VGG Face dataset, I don’t know how to obtain it. I saw it from the link you gave. But the file format is .npy or .zip. I don't know how to use it. If you have a format with ".jpg" , can you share it. Or tell me how I can get it. Thank you so much!

When is open source code？

When is open source code？