manycore-research/plankassembly

[ICCV 2023] PlankAssembly: Robust 3D Reconstruction from Three Orthographic Views with Learnt Shape Programs

PlankAssembly: Robust 3D Reconstruction from Three Orthographic Views with Learnt Shape Programs

Wentao Hu* · Jia Zheng* · Zixin Zhang* · Xiaojun Yuan · Jian Yin · Zihan Zhou

IEEE/CVF Conference on Computer Vision (ICCV), 2023

*These authors contributed equally to this work.

Note

This branch contains the implementation of PlankAssembly, which supports three types of inputs: (1) visible and hidden lines, (2) visible edges only, and (3) sidefaces. For raster images as inputs, please refer to the raster branch. For comparison with PolyGen, please refer to the polygen branch.

Setup

Our code has been tested with Python 3.8, PyTorch 1.10.0, CUDA 11.3, and PyTorch Lightning 1.7.6.

Installation

Clone the repository, then create and activate a plankassembly conda environment using the following commands.

# clone repository
git clone https://github.com/manycore-research/PlankAssembly.git
# create conda env
conda env create --file environment.yml
conda activate plankassembly

If you encounter any issue with provided conda environment, you may install dependencies manually using the following commands.

conda create -n plankassembly python=3.8
conda activate plankassembly
conda install pytorch==1.10.0 torchvision==0.11.0 torchaudio==0.10.0 cudatoolkit=11.3 -c pytorch -c conda-forge
pip install pytorch-lightning==1.7.7 torchmetrics==0.11.4 rich==12.5.1 'jsonargparse[signatures]'
pip install detectron2 -f https://dl.fbaipublicfiles.com/detectron2/wheels/cu113/torch1.10/detectron2-0.6%2Bcu113-cp38-cp38-linux_x86_64.whl
conda install -c conda-forge pythonocc-core=7.6.2
pip install numpy shapely svgwrite svgpathtools trimesh setuptools==59.5.0 html4vision

Dataset

The dataset can be found on Hugging Face Datasets. Please download the data first, then unzip the data in the project workspace.

The released dataset only contains 3D shape programs. To prepare the data for training and testing, please run the following commands.

We use PythonOCC to render three-view orthogonal engineering drawings and save them as SVG files.

# render complete inputs
python dataset/render_complete_svg.py
# render noisy inputs, please specify the noise ratio
python dataset/render_noisy_svg.py --data_type noise_05 --noise_ratio 0.05
# render visible inputs
python dataset/render_visible_svg.py

Then, pack the input line drawings and output shape programs into JSON files.

python dataset/prepare_info.py --data_path path/to/data/root

To visualize the 3D model, we could generate the ground-truth 3D meshes from shape.

python misc/build_gt_mesh.py --data_path path/to/data/root

Training

Use the following command to train a model from scratch:

# train a model with complete lines as inputs
python trainer_complete.py fit --config configs/train_complete.yaml

Testing

Use the following command to test with a pre-trained model:

# infer a model with complete lines as inputs
python trainer_complete.py test \
    --config configs/train_complete.yaml \
    --ckpt_path path/to/checkpoint.ckpt \
    --trainer.devices 1

Evaluation

To compute the evaluation metrics, please run the following command:

python evaluate.py --data_path path/to/data/dir --exp_path path/to/lightning_log/dir

Visualization

To visualize the results, we build 3D mesh models from predictions:

python misc/build_pred_mesh.py --exp_path path/to/lightning_log/dir

Then, we use HTML4Vision to generate HTML files for mesh visualization (please refer to here for more details):

python misc/build_html.py --exp_path path/to/lightning_log/dir

The 2D images presented in our paper are rendered using bpy-visualization-utils.

Model Checkpoints

The checkpoints can be found on Hugging Face Models. Or click the links below to download the checkpoint for the corresponding model type directly.

• Model trained on complete inputs: here
• Model trained on visible inputs only: here
• Model trained on sideface inputs: here

LICENSE

PlankAssembly is licensed under the AGPL-3.0 license. The code snippets in the third_party folder are available under Apache-2.0 License.

plankassembly's Issues

Input images

Hello,

I could run all steps with your data from the preparation until the visualization, and it looks very nice.

However, I still don't understand where to start with my own images which are raster images in jpg format. In your branch dedicated to raster images, I only see svg or json files but not the original images.

Could you please provide me a few precisions on this topic?

Thanks!

Best,
Guillaume

AssertionError

Dear，thank you very much for your work, but I found several issues when I reproduced your code.

Firstly, I run the fourth command (Python dataset/prepare_info. py -- root path/to/data/root), and there is no root parameter in the code.

Then I run the training command (python trainer_complete.py fit --config configs/train_complete.yaml), run 19 epochs and display “AssertionError: expecting key_padding_mask shape of (16, 1157), but got torch.Size([16, 1199])”.

How can I solve this problem？

Data format

Hello,

I have an additional question regarding the way of preparing the data for inference.

When I run the testing script using the configs/train.yaml, I get the following error:

File "/home/guist/PlankAssembly-atlas/plankassembly/data.py", line 246, in getitem
svgs = info['svgs']
KeyError: 'svgs'

I checked the json file and there is indeed no 'svgs' field so I was wondering how to modify the file so that it works. More generally, how should I prepare my custom images so that I can apply your model on them?

Thanks a lot for the answer.

Best regards,
Guillaume

Reconstructing 3d objects with curves

The current model performs fine with objects that have distinct edges or are composed of square and rectangular shapes. Can it be adapted to process data for images that contain curves, as shown in the SVG and 3D model below?

Additionally, can the bellow SVG be segmented into separate views—front, top, and side—to input into the model?

Unexpected and missing keys

Hello,

I am trying to run the testing script for raster images. I saved the pre-trained checkpoint in the models/ directory and the R-50.pth file in the weights/ directory. Then I typed the command line:

python trainer.py test --config configs/train.yaml --ckpt_path models/line_complete-checkpoint_999-precision=0.944-recall=0.934-f1=0.938.ckpt --trainer.devices 1

But I got the error below. Could you please tell me what went wrong?

Thank you!

Best regards,
Guillaume

raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for Trainer:
Missing key(s) in state_dict: "backbone2d.0.fpn_lateral2.weight", "backbone2d.0.fpn_lateral2.norm.weight", "backbone2d.0.fpn_lateral2.norm.bias", "backbone2d.0.fpn_lateral2.norm.running_mean", "backbone2d.0.fpn_lateral2.norm.running_var", "backbone2d.0.fpn_output2.weight", "backbone2d.0.fpn_output2.norm.weight", "backbone2d.0.fpn_output2.norm.bias", "backbone2d.0.fpn_output2.norm.running_mean", "backbone2d.0.fpn_output2.norm.running_var", "backbone2d.0.fpn_lateral3.weight", "backbone2d.0.fpn_lateral3.norm.weight", "backbone2d.0.fpn_lateral3.norm.bias", "backbone2d.0.fpn_lateral3.norm.running_mean", "backbone2d.0.fpn_lateral3.norm.running_var", "backbone2d.0.fpn_output3.weight", "backbone2d.0.fpn_output3.norm.weight", "backbone2d.0.fpn_output3.norm.bias", "backbone2d.0.fpn_output3.norm.running_mean", "backbone2d.0.fpn_output3.norm.running_var", "backbone2d.0.fpn_lateral4.weight", "backbone2d.0.fpn_lateral4.norm.weight", "backbone2d.0.fpn_lateral4.norm.bias", "backbone2d.0.fpn_lateral4.norm.running_mean", "backbone2d.0.fpn_lateral4.norm.running_var", "backbone2d.0.fpn_output4.weight", "backbone2d.0.fpn_output4.norm.weight", "backbone2d.0.fpn_output4.norm.bias", "backbone2d.0.fpn_output4.norm.running_mean", "backbone2d.0.fpn_output4.norm.running_var", "backbone2d.0.fpn_lateral5.weight", "backbone2d.0.fpn_lateral5.norm.weight", "backbone2d.0.fpn_lateral5.norm.bias", "backbone2d.0.fpn_lateral5.norm.running_mean", "backbone2d.0.fpn_lateral5.norm.running_var", "backbone2d.0.fpn_output5.weight", "backbone2d.0.fpn_output5.norm.weight", "backbone2d.0.fpn_output5.norm.bias", "backbone2d.0.fpn_output5.norm.running_mean", "backbone2d.0.fpn_output5.norm.running_var", "backbone2d.0.bottom_up.stem.conv1.weight", "backbone2d.0.bottom_up.stem.conv1.norm.weight", "backbone2d.0.bottom_up.stem.conv1.norm.bias", "backbone2d.0.bottom_up.stem.conv1.norm.running_mean", "backbone2d.0.bottom_up.stem.conv1.norm.running_var", "backbone2d.0.bottom_up.res2.0.shortcut.weight", "backbone2d.0.bottom_up.res2.0.shortcut.norm.weight", "backbone2d.0.bottom_up.res2.0.shortcut.norm.bias", "backbone2d.0.bottom_up.res2.0.shortcut.norm.running_mean", "backbone2d.0.bottom_up.res2.0.shortcut.norm.running_var", "backbone2d.0.bottom_up.res2.0.conv1.weight", "backbone2d.0.bottom_up.res2.0.conv1.norm.weight", "backbone2d.0.bottom_up.res2.0.conv1.norm.bias", "backbone2d.0.bottom_up.res2.0.conv1.norm.running_mean", "backbone2d.0.bottom_up.res2.0.conv1.norm.running_var", "backbone2d.0.bottom_up.res2.0.conv2.weight", "backbone2d.0.bottom_up.res2.0.conv2.norm.weight", "backbone2d.0.bottom_up.res2.0.conv2.norm.bias", "backbone2d.0.bottom_up.res2.0.conv2.norm.running_mean", "backbone2d.0.bottom_up.res2.0.conv2.norm.running_var", "backbone2d.0.bottom_up.res2.0.conv3.weight", "backbone2d.0.bottom_up.res2.0.conv3.norm.weight", "backbone2d.0.bottom_up.res2.0.conv3.norm.bias", "backbone2d.0.bottom_up.res2.0.conv3.norm.running_mean", "backbone2d.0.bottom_up.res2.0.conv3.norm.running_var", "backbone2d.0.bottom_up.res2.1.conv1.weight", "backbone2d.0.bottom_up.res2.1.conv1.norm.weight", "backbone2d.0.bottom_up.res2.1.conv1.norm.bias", "backbone2d.0.bottom_up.res2.1.conv1.norm.running_mean", "backbone2d.0.bottom_up.res2.1.conv1.norm.running_var", "backbone2d.0.bottom_up.res2.1.conv2.weight", "backbone2d.0.bottom_up.res2.1.conv2.norm.weight", "backbone2d.0.bottom_up.res2.1.conv2.norm.bias", "backbone2d.0.bottom_up.res2.1.conv2.norm.running_mean", "backbone2d.0.bottom_up.res2.1.conv2.norm.running_var", "backbone2d.0.bottom_up.res2.1.conv3.weight", "backbone2d.0.bottom_up.res2.1.conv3.norm.weight", "backbone2d.0.bottom_up.res2.1.conv3.norm.bias", "backbone2d.0.bottom_up.res2.1.conv3.norm.running_mean", "backbone2d.0.bottom_up.res2.1.conv3.norm.running_var", "backbone2d.0.bottom_up.res2.2.conv1.weight", "backbone2d.0.bottom_up.res2.2.conv1.norm.weight", "backbone2d.0.bottom_up.res2.2.conv1.norm.bias", "backbone2d.0.bottom_up.res2.2.conv1.norm.running_mean", "backbone2d.0.bottom_up.res2.2.conv1.norm.running_var", "backbone2d.0.bottom_up.res2.2.conv2.weight", "backbone2d.0.bottom_up.res2.2.conv2.norm.weight", "backbone2d.0.bottom_up.res2.2.conv2.norm.bias", "backbone2d.0.bottom_up.res2.2.conv2.norm.running_mean", "backbone2d.0.bottom_up.res2.2.conv2.norm.running_var", "backbone2d.0.bottom_up.res2.2.conv3.weight", "backbone2d.0.bottom_up.res2.2.conv3.norm.weight", "backbone2d.0.bottom_up.res2.2.conv3.norm.bias", "backbone2d.0.bottom_up.res2.2.conv3.norm.running_mean", "backbone2d.0.bottom_up.res2.2.conv3.norm.running_var", "backbone2d.0.bottom_up.res3.0.shortcut.weight", "backbone2d.0.bottom_up.res3.0.shortcut.norm.weight", "backbone2d.0.bottom_up.res3.0.shortcut.norm.bias", "backbone2d.0.bottom_up.res3.0.shortcut.norm.running_mean", "backbone2d.0.bottom_up.res3.0.shortcut.norm.running_var", "backbone2d.0.bottom_up.res3.0.conv1.weight", "backbone2d.0.bottom_up.res3.0.conv1.norm.weight", "backbone2d.0.bottom_up.res3.0.conv1.norm.bias", "backbone2d.0.bottom_up.res3.0.conv1.norm.running_mean", "backbone2d.0.bottom_up.res3.0.conv1.norm.running_var", "backbone2d.0.bottom_up.res3.0.conv2.weight", "backbone2d.0.bottom_up.res3.0.conv2.norm.weight", "backbone2d.0.bottom_up.res3.0.conv2.norm.bias", "backbone2d.0.bottom_up.res3.0.conv2.norm.running_mean", "backbone2d.0.bottom_up.res3.0.conv2.norm.running_var", "backbone2d.0.bottom_up.res3.0.conv3.weight", "backbone2d.0.bottom_up.res3.0.conv3.norm.weight", "backbone2d.0.bottom_up.res3.0.conv3.norm.bias", "backbone2d.0.bottom_up.res3.0.conv3.norm.running_mean", "backbone2d.0.bottom_up.res3.0.conv3.norm.running_var", "backbone2d.0.bottom_up.res3.1.conv1.weight", "backbone2d.0.bottom_up.res3.1.conv1.norm.weight", "backbone2d.0.bottom_up.res3.1.conv1.norm.bias", "backbone2d.0.bottom_up.res3.1.conv1.norm.running_mean", "backbone2d.0.bottom_up.res3.1.conv1.norm.running_var", "backbone2d.0.bottom_up.res3.1.conv2.weight", "backbone2d.0.bottom_up.res3.1.conv2.norm.weight", "backbone2d.0.bottom_up.res3.1.conv2.norm.bias", "backbone2d.0.bottom_up.res3.1.conv2.norm.running_mean", "backbone2d.0.bottom_up.res3.1.conv2.norm.running_var", "backbone2d.0.bottom_up.res3.1.conv3.weight", "backbone2d.0.bottom_up.res3.1.conv3.norm.weight", "backbone2d.0.bottom_up.res3.1.conv3.norm.bias", "backbone2d.0.bottom_up.res3.1.conv3.norm.running_mean", "backbone2d.0.bottom_up.res3.1.conv3.norm.running_var", "backbone2d.0.bottom_up.res3.2.conv1.weight", "backbone2d.0.bottom_up.res3.2.conv1.norm.weight", "backbone2d.0.bottom_up.res3.2.conv1.norm.bias", "backbone2d.0.bottom_up.res3.2.conv1.norm.running_mean", "backbone2d.0.bottom_up.res3.2.conv1.norm.running_var", "backbone2d.0.bottom_up.res3.2.conv2.weight", "backbone2d.0.bottom_up.res3.2.conv2.norm.weight", "backbone2d.0.bottom_up.res3.2.conv2.norm.bias", "backbone2d.0.bottom_up.res3.2.conv2.norm.running_mean", "backbone2d.0.bottom_up.res3.2.conv2.norm.running_var", "backbone2d.0.bottom_up.res3.2.conv3.weight", "backbone2d.0.bottom_up.res3.2.conv3.norm.weight", "backbone2d.0.bottom_up.res3.2.conv3.norm.bias", "backbone2d.0.bottom_up.res3.2.conv3.norm.running_mean", "backbone2d.0.bottom_up.res3.2.conv3.norm.running_var", "backbone2d.0.bottom_up.res3.3.conv1.weight", "backbone2d.0.bottom_up.res3.3.conv1.norm.weight", "backbone2d.0.bottom_up.res3.3.conv1.norm.bias", "backbone2d.0.bottom_up.res3.3.conv1.norm.running_mean", "backbone2d.0.bottom_up.res3.3.conv1.norm.running_var", "backbone2d.0.bottom_up.res3.3.conv2.weight", "backbone2d.0.bottom_up.res3.3.conv2.norm.weight", "backbone2d.0.bottom_up.res3.3.conv2.norm.bias", "backbone2d.0.bottom_up.res3.3.conv2.norm.running_mean", "backbone2d.0.bottom_up.res3.3.conv2.norm.running_var", "backbone2d.0.bottom_up.res3.3.conv3.weight", "backbone2d.0.bottom_up.res3.3.conv3.norm.weight", "backbone2d.0.bottom_up.res3.3.conv3.norm.bias", "backbone2d.0.bottom_up.res3.3.conv3.norm.running_mean", "backbone2d.0.bottom_up.res3.3.conv3.norm.running_var", "backbone2d.0.bottom_up.res4.0.shortcut.weight", "backbone2d.0.bottom_up.res4.0.shortcut.norm.weight", "backbone2d.0.bottom_up.res4.0.shortcut.norm.bias", "backbone2d.0.bottom_up.res4.0.shortcut.norm.running_mean", "backbone2d.0.bottom_up.res4.0.shortcut.norm.running_var", "backbone2d.0.bottom_up.res4.0.conv1.weight", "backbone2d.0.bottom_up.res4.0.conv1.norm.weight", "backbone2d.0.bottom_up.res4.0.conv1.norm.bias", "backbone2d.0.bottom_up.res4.0.conv1.norm.running_mean", "backbone2d.0.bottom_up.res4.0.conv1.norm.running_var", "backbone2d.0.bottom_up.res4.0.conv2.weight", "backbone2d.0.bottom_up.res4.0.conv2.norm.weight", "backbone2d.0.bottom_up.res4.0.conv2.norm.bias", "backbone2d.0.bottom_up.res4.0.conv2.norm.running_mean", "backbone2d.0.bottom_up.res4.0.conv2.norm.running_var", "backbone2d.0.bottom_up.res4.0.conv3.weight", "backbone2d.0.bottom_up.res4.0.conv3.norm.weight", "backbone2d.0.bottom_up.res4.0.conv3.norm.bias", "backbone2d.0.bottom_up.res4.0.conv3.norm.running_mean", "backbone2d.0.bottom_up.res4.0.conv3.norm.running_var", "backbone2d.0.bottom_up.res4.1.conv1.weight", "backbone2d.0.bottom_up.res4.1.conv1.norm.weight", "backbone2d.0.bottom_up.res4.1.conv1.norm.bias", "backbone2d.0.bottom_up.res4.1.conv1.norm.running_mean", "backbone2d.0.bottom_up.res4.1.conv1.norm.running_var", "backbone2d.0.bottom_up.res4.1.conv2.weight", "backbone2d.0.bottom_up.res4.1.conv2.norm.weight", "backbone2d.0.bottom_up.res4.1.conv2.norm.bias", "backbone2d.0.bottom_up.res4.1.conv2.norm.running_mean", "backbone2d.0.bottom_up.res4.1.conv2.norm.running_var", "backbone2d.0.bottom_up.res4.1.conv3.weight", "backbone2d.0.bottom_up.res4.1.conv3.norm.weight", "backbone2d.0.bottom_up.res4.1.conv3.norm.bias", "backbone2d.0.bottom_up.res4.1.conv3.norm.running_mean", "backbone2d.0.bottom_up.res4.1.conv3.norm.running_var", "backbone2d.0.bottom_up.res4.2.conv1.weight", "backbone2d.0.bottom_up.res4.2.conv1.norm.weight", "backbone2d.0.bottom_up.res4.2.conv1.norm.bias", "backbone2d.0.bottom_up.res4.2.conv1.norm.running_mean", "backbone2d.0.bottom_up.res4.2.conv1.norm.running_var", "backbone2d.0.bottom_up.res4.2.conv2.weight", "backbone2d.0.bottom_up.res4.2.conv2.norm.weight", "backbone2d.0.bottom_up.res4.2.conv2.norm.bias", "backbone2d.0.bottom_up.res4.2.conv2.norm.running_mean", "backbone2d.0.bottom_up.res4.2.conv2.norm.running_var", "backbone2d.0.bottom_up.res4.2.conv3.weight", "backbone2d.0.bottom_up.res4.2.conv3.norm.weight", "backbone2d.0.bottom_up.res4.2.conv3.norm.bias", "backbone2d.0.bottom_up.res4.2.conv3.norm.running_mean", "backbone2d.0.bottom_up.res4.2.conv3.norm.running_var", "backbone2d.0.bottom_up.res4.3.conv1.weight", "backbone2d.0.bottom_up.res4.3.conv1.norm.weight", "backbone2d.0.bottom_up.res4.3.conv1.norm.bias", "backbone2d.0.bottom_up.res4.3.conv1.norm.running_mean", "backbone2d.0.bottom_up.res4.3.conv1.norm.running_var", "backbone2d.0.bottom_up.res4.3.conv2.weight", "backbone2d.0.bottom_up.res4.3.conv2.norm.weight", "backbone2d.0.bottom_up.res4.3.conv2.norm.bias", "backbone2d.0.bottom_up.res4.3.conv2.norm.running_mean", "backbone2d.0.bottom_up.res4.3.conv2.norm.running_var", "backbone2d.0.bottom_up.res4.3.conv3.weight", "backbone2d.0.bottom_up.res4.3.conv3.norm.weight", "backbone2d.0.bottom_up.res4.3.conv3.norm.bias", "backbone2d.0.bottom_up.res4.3.conv3.norm.running_mean", "backbone2d.0.bottom_up.res4.3.conv3.norm.running_var", "backbone2d.0.bottom_up.res4.4.conv1.weight", "backbone2d.0.bottom_up.res4.4.conv1.norm.weight", "backbone2d.0.bottom_up.res4.4.conv1.norm.bias", "backbone2d.0.bottom_up.res4.4.conv1.norm.running_mean", "backbone2d.0.bottom_up.res4.4.conv1.norm.running_var", "backbone2d.0.bottom_up.res4.4.conv2.weight", "backbone2d.0.bottom_up.res4.4.conv2.norm.weight", "backbone2d.0.bottom_up.res4.4.conv2.norm.bias", "backbone2d.0.bottom_up.res4.4.conv2.norm.running_mean", "backbone2d.0.bottom_up.res4.4.conv2.norm.running_var", "backbone2d.0.bottom_up.res4.4.conv3.weight", "backbone2d.0.bottom_up.res4.4.conv3.norm.weight", "backbone2d.0.bottom_up.res4.4.conv3.norm.bias", "backbone2d.0.bottom_up.res4.4.conv3.norm.running_mean", "backbone2d.0.bottom_up.res4.4.conv3.norm.running_var", "backbone2d.0.bottom_up.res4.5.conv1.weight", "backbone2d.0.bottom_up.res4.5.conv1.norm.weight", "backbone2d.0.bottom_up.res4.5.conv1.norm.bias", "backbone2d.0.bottom_up.res4.5.conv1.norm.running_mean", "backbone2d.0.bottom_up.res4.5.conv1.norm.running_var", "backbone2d.0.bottom_up.res4.5.conv2.weight", "backbone2d.0.bottom_up.res4.5.conv2.norm.weight", "backbone2d.0.bottom_up.res4.5.conv2.norm.bias", "backbone2d.0.bottom_up.res4.5.conv2.norm.running_mean", "backbone2d.0.bottom_up.res4.5.conv2.norm.running_var", "backbone2d.0.bottom_up.res4.5.conv3.weight", "backbone2d.0.bottom_up.res4.5.conv3.norm.weight", "backbone2d.0.bottom_up.res4.5.conv3.norm.bias", "backbone2d.0.bottom_up.res4.5.conv3.norm.running_mean", "backbone2d.0.bottom_up.res4.5.conv3.norm.running_var", "backbone2d.0.bottom_up.res5.0.shortcut.weight", "backbone2d.0.bottom_up.res5.0.shortcut.norm.weight", "backbone2d.0.bottom_up.res5.0.shortcut.norm.bias", "backbone2d.0.bottom_up.res5.0.shortcut.norm.running_mean", "backbone2d.0.bottom_up.res5.0.shortcut.norm.running_var", "backbone2d.0.bottom_up.res5.0.conv1.weight", "backbone2d.0.bottom_up.res5.0.conv1.norm.weight", "backbone2d.0.bottom_up.res5.0.conv1.norm.bias", "backbone2d.0.bottom_up.res5.0.conv1.norm.running_mean", "backbone2d.0.bottom_up.res5.0.conv1.norm.running_var", "backbone2d.0.bottom_up.res5.0.conv2.weight", "backbone2d.0.bottom_up.res5.0.conv2.norm.weight", "backbone2d.0.bottom_up.res5.0.conv2.norm.bias", "backbone2d.0.bottom_up.res5.0.conv2.norm.running_mean", "backbone2d.0.bottom_up.res5.0.conv2.norm.running_var", "backbone2d.0.bottom_up.res5.0.conv3.weight", "backbone2d.0.bottom_up.res5.0.conv3.norm.weight", "backbone2d.0.bottom_up.res5.0.conv3.norm.bias", "backbone2d.0.bottom_up.res5.0.conv3.norm.running_mean", "backbone2d.0.bottom_up.res5.0.conv3.norm.running_var", "backbone2d.0.bottom_up.res5.1.conv1.weight", "backbone2d.0.bottom_up.res5.1.conv1.norm.weight", "backbone2d.0.bottom_up.res5.1.conv1.norm.bias", "backbone2d.0.bottom_up.res5.1.conv1.norm.running_mean", "backbone2d.0.bottom_up.res5.1.conv1.norm.running_var", "backbone2d.0.bottom_up.res5.1.conv2.weight", "backbone2d.0.bottom_up.res5.1.conv2.norm.weight", "backbone2d.0.bottom_up.res5.1.conv2.norm.bias", "backbone2d.0.bottom_up.res5.1.conv2.norm.running_mean", "backbone2d.0.bottom_up.res5.1.conv2.norm.running_var", "backbone2d.0.bottom_up.res5.1.conv3.weight", "backbone2d.0.bottom_up.res5.1.conv3.norm.weight", "backbone2d.0.bottom_up.res5.1.conv3.norm.bias", "backbone2d.0.bottom_up.res5.1.conv3.norm.running_mean", "backbone2d.0.bottom_up.res5.1.conv3.norm.running_var", "backbone2d.0.bottom_up.res5.2.conv1.weight", "backbone2d.0.bottom_up.res5.2.conv1.norm.weight", "backbone2d.0.bottom_up.res5.2.conv1.norm.bias", "backbone2d.0.bottom_up.res5.2.conv1.norm.running_mean", "backbone2d.0.bottom_up.res5.2.conv1.norm.running_var", "backbone2d.0.bottom_up.res5.2.conv2.weight", "backbone2d.0.bottom_up.res5.2.conv2.norm.weight", "backbone2d.0.bottom_up.res5.2.conv2.norm.bias", "backbone2d.0.bottom_up.res5.2.conv2.norm.running_mean", "backbone2d.0.bottom_up.res5.2.conv2.norm.running_var", "backbone2d.0.bottom_up.res5.2.conv3.weight", "backbone2d.0.bottom_up.res5.2.conv3.norm.weight", "backbone2d.0.bottom_up.res5.2.conv3.norm.bias", "backbone2d.0.bottom_up.res5.2.conv3.norm.running_mean", "backbone2d.0.bottom_up.res5.2.conv3.norm.running_var", "backbone2d.1.p2.0.weight", "backbone2d.1.p2.0.norm.weight", "backbone2d.1.p2.0.norm.bias", "backbone2d.1.p2.0.norm.running_mean", "backbone2d.1.p2.0.norm.running_var", "backbone2d.1.p3.0.weight", "backbone2d.1.p3.0.norm.weight", "backbone2d.1.p3.0.norm.bias", "backbone2d.1.p3.0.norm.running_mean", "backbone2d.1.p3.0.norm.running_var", "backbone2d.1.p4.0.weight", "backbone2d.1.p4.0.norm.weight", "backbone2d.1.p4.0.norm.bias", "backbone2d.1.p4.0.norm.running_mean", "backbone2d.1.p4.0.norm.running_var", "backbone2d.1.p4.2.weight", "backbone2d.1.p4.2.norm.weight", "backbone2d.1.p4.2.norm.bias", "backbone2d.1.p4.2.norm.running_mean", "backbone2d.1.p4.2.norm.running_var", "backbone2d.1.p5.0.weight", "backbone2d.1.p5.0.norm.weight", "backbone2d.1.p5.0.norm.bias", "backbone2d.1.p5.0.norm.running_mean", "backbone2d.1.p5.0.norm.running_var", "backbone2d.1.p5.2.weight", "backbone2d.1.p5.2.norm.weight", "backbone2d.1.p5.2.norm.bias", "backbone2d.1.p5.2.norm.running_mean", "backbone2d.1.p5.2.norm.running_var", "backbone2d.1.p5.4.weight", "backbone2d.1.p5.4.norm.weight", "backbone2d.1.p5.4.norm.bias", "backbone2d.1.p5.4.norm.running_mean", "backbone2d.1.p5.4.norm.running_var", "backbone3d.layers.0.0.conv1.weight", "backbone3d.layers.0.0.bn1.weight", "backbone3d.layers.0.0.bn1.bias", "backbone3d.layers.0.0.bn1.running_mean", "backbone3d.layers.0.0.bn1.running_var", "backbone3d.layers.0.0.conv2.weight", "backbone3d.layers.0.0.bn2.weight", "backbone3d.layers.0.0.bn2.bias", "backbone3d.layers.0.0.bn2.running_mean", "backbone3d.layers.0.0.bn2.running_var", "backbone3d.layers.0.0.downsample.0.weight", "backbone3d.layers.0.0.downsample.1.weight", "backbone3d.layers.0.0.downsample.1.bias", "backbone3d.layers.0.0.downsample.1.running_mean", "backbone3d.layers.0.0.downsample.1.running_var", "backbone3d.layers.1.0.weight", "backbone3d.layers.1.1.weight", "backbone3d.layers.1.1.bias", "backbone3d.layers.1.1.running_mean", "backbone3d.layers.1.1.running_var", "backbone3d.layers.1.4.conv1.weight", "backbone3d.layers.1.4.bn1.weight", "backbone3d.layers.1.4.bn1.bias", "backbone3d.layers.1.4.bn1.running_mean", "backbone3d.layers.1.4.bn1.running_var", "backbone3d.layers.1.4.conv2.weight", "backbone3d.layers.1.4.bn2.weight", "backbone3d.layers.1.4.bn2.bias", "backbone3d.layers.1.4.bn2.running_mean", "backbone3d.layers.1.4.bn2.running_var", "backbone3d.layers.1.5.conv1.weight", "backbone3d.layers.1.5.bn1.weight", "backbone3d.layers.1.5.bn1.bias", "backbone3d.layers.1.5.bn1.running_mean", "backbone3d.layers.1.5.bn1.running_var", "backbone3d.layers.1.5.conv2.weight", "backbone3d.layers.1.5.bn2.weight", "backbone3d.layers.1.5.bn2.bias", "backbone3d.layers.1.5.bn2.running_mean", "backbone3d.layers.1.5.bn2.running_var", "backbone3d.layers.2.0.weight", "backbone3d.layers.2.1.weight", "backbone3d.layers.2.1.bias", "backbone3d.layers.2.1.running_mean", "backbone3d.layers.2.1.running_var", "backbone3d.layers.2.4.conv1.weight", "backbone3d.layers.2.4.bn1.weight", "backbone3d.layers.2.4.bn1.bias", "backbone3d.layers.2.4.bn1.running_mean", "backbone3d.layers.2.4.bn1.running_var", "backbone3d.layers.2.4.conv2.weight", "backbone3d.layers.2.4.bn2.weight", "backbone3d.layers.2.4.bn2.bias", "backbone3d.layers.2.4.bn2.running_mean", "backbone3d.layers.2.4.bn2.running_var", "backbone3d.layers.2.5.conv1.weight", "backbone3d.layers.2.5.bn1.weight", "backbone3d.layers.2.5.bn1.bias", "backbone3d.layers.2.5.bn1.running_mean", "backbone3d.layers.2.5.bn1.running_var", "backbone3d.layers.2.5.conv2.weight", "backbone3d.layers.2.5.bn2.weight", "backbone3d.layers.2.5.bn2.bias", "backbone3d.layers.2.5.bn2.running_mean", "backbone3d.layers.2.5.bn2.running_var", "backbone3d.layers.2.6.conv1.weight", "backbone3d.layers.2.6.bn1.weight", "backbone3d.layers.2.6.bn1.bias", "backbone3d.layers.2.6.bn1.running_mean", "backbone3d.layers.2.6.bn1.running_var", "backbone3d.layers.2.6.conv2.weight", "backbone3d.layers.2.6.bn2.weight", "backbone3d.layers.2.6.bn2.bias", "backbone3d.layers.2.6.bn2.running_mean", "backbone3d.layers.2.6.bn2.running_var", "backbone3d.layers.3.0.weight", "backbone3d.layers.3.1.weight", "backbone3d.layers.3.1.bias", "backbone3d.layers.3.1.running_mean", "backbone3d.layers.3.1.running_var", "backbone3d.layers.3.4.conv1.weight", "backbone3d.layers.3.4.bn1.weight", "backbone3d.layers.3.4.bn1.bias", "backbone3d.layers.3.4.bn1.running_mean", "backbone3d.layers.3.4.bn1.running_var", "backbone3d.layers.3.4.conv2.weight", "backbone3d.layers.3.4.bn2.weight", "backbone3d.layers.3.4.bn2.bias", "backbone3d.layers.3.4.bn2.running_mean", "backbone3d.layers.3.4.bn2.running_var", "backbone3d.layers.3.5.conv1.weight", "backbone3d.layers.3.5.bn1.weight", "backbone3d.layers.3.5.bn1.bias", "backbone3d.layers.3.5.bn1.running_mean", "backbone3d.layers.3.5.bn1.running_var", "backbone3d.layers.3.5.conv2.weight", "backbone3d.layers.3.5.bn2.weight", "backbone3d.layers.3.5.bn2.bias", "backbone3d.layers.3.5.bn2.running_mean", "backbone3d.layers.3.5.bn2.running_var", "backbone3d.layers.3.6.conv1.weight", "backbone3d.layers.3.6.bn1.weight", "backbone3d.layers.3.6.bn1.bias", "backbone3d.layers.3.6.bn1.running_mean", "backbone3d.layers.3.6.bn1.running_var", "backbone3d.layers.3.6.conv2.weight", "backbone3d.layers.3.6.bn2.weight", "backbone3d.layers.3.6.bn2.bias", "backbone3d.layers.3.6.bn2.running_mean", "backbone3d.layers.3.6.bn2.running_var", "backbone3d.layers.3.7.conv1.weight", "backbone3d.layers.3.7.bn1.weight", "backbone3d.layers.3.7.bn1.bias", "backbone3d.layers.3.7.bn1.running_mean", "backbone3d.layers.3.7.bn1.running_var", "backbone3d.layers.3.7.conv2.weight", "backbone3d.layers.3.7.bn2.weight", "backbone3d.layers.3.7.bn2.bias", "backbone3d.layers.3.7.bn2.running_mean", "backbone3d.layers.3.7.bn2.running_var", "backbone3d.layers.4.0.weight", "backbone3d.layers.4.1.weight", "backbone3d.layers.4.1.bias", "backbone3d.layers.4.1.running_mean", "backbone3d.layers.4.1.running_var", "backbone3d.layers.4.4.conv1.weight", "backbone3d.layers.4.4.bn1.weight", "backbone3d.layers.4.4.bn1.bias", "backbone3d.layers.4.4.bn1.running_mean", "backbone3d.layers.4.4.bn1.running_var", "backbone3d.layers.4.4.conv2.weight", "backbone3d.layers.4.4.bn2.weight", "backbone3d.layers.4.4.bn2.bias", "backbone3d.layers.4.4.bn2.running_mean", "backbone3d.layers.4.4.bn2.running_var", "backbone3d.layers.4.5.conv1.weight", "backbone3d.layers.4.5.bn1.weight", "backbone3d.layers.4.5.bn1.bias", "backbone3d.layers.4.5.bn1.running_mean", "backbone3d.layers.4.5.bn1.running_var", "backbone3d.layers.4.5.conv2.weight", "backbone3d.layers.4.5.bn2.weight", "backbone3d.layers.4.5.bn2.bias", "backbone3d.layers.4.5.bn2.running_mean", "backbone3d.layers.4.5.bn2.running_var", "backbone3d.layers.4.6.conv1.weight", "backbone3d.layers.4.6.bn1.weight", "backbone3d.layers.4.6.bn1.bias", "backbone3d.layers.4.6.bn1.running_mean", "backbone3d.layers.4.6.bn1.running_var", "backbone3d.layers.4.6.conv2.weight", "backbone3d.layers.4.6.bn2.weight", "backbone3d.layers.4.6.bn2.bias", "backbone3d.layers.4.6.bn2.running_mean", "backbone3d.layers.4.6.bn2.running_var", "backbone3d.layers.4.7.conv1.weight", "backbone3d.layers.4.7.bn1.weight", "backbone3d.layers.4.7.bn1.bias", "backbone3d.layers.4.7.bn1.running_mean", "backbone3d.layers.4.7.bn1.running_var", "backbone3d.layers.4.7.conv2.weight", "backbone3d.layers.4.7.bn2.weight", "backbone3d.layers.4.7.bn2.bias", "backbone3d.layers.4.7.bn2.running_mean", "backbone3d.layers.4.7.bn2.running_var", "backbone3d.layers.4.8.conv1.weight", "backbone3d.layers.4.8.bn1.weight", "backbone3d.layers.4.8.bn1.bias", "backbone3d.layers.4.8.bn1.running_mean", "backbone3d.layers.4.8.bn1.running_var", "backbone3d.layers.4.8.conv2.weight", "backbone3d.layers.4.8.bn2.weight", "backbone3d.layers.4.8.bn2.bias", "backbone3d.layers.4.8.bn2.running_mean", "backbone3d.layers.4.8.bn2.running_var", "heads3d.pos_encoder.x_embed.weight", "heads3d.pos_encoder.y_embed.weight", "heads3d.pos_encoder.z_embed.weight", "heads3d.value_embedding.weight", "heads3d.query_coord_embedding.weight", "heads3d.query_pos_embedding.weight", "heads3d.linear.weight", "heads3d.linear.bias", "heads3d.decoder.layers.0.self_attn.in_proj_weight", "heads3d.decoder.layers.0.self_attn.in_proj_bias", "heads3d.decoder.layers.0.self_attn.out_proj.weight", "heads3d.decoder.layers.0.self_attn.out_proj.bias", "heads3d.decoder.layers.0.multihead_attn.in_proj_weight", "heads3d.decoder.layers.0.multihead_attn.in_proj_bias", "heads3d.decoder.layers.0.multihead_attn.out_proj.weight", "heads3d.decoder.layers.0.multihead_attn.out_proj.bias", "heads3d.decoder.layers.0.linear1.weight", "heads3d.decoder.layers.0.linear1.bias", "heads3d.decoder.layers.0.linear2.weight", "heads3d.decoder.layers.0.linear2.bias", "heads3d.decoder.layers.0.norm1.weight", "heads3d.decoder.layers.0.norm1.bias", "heads3d.decoder.layers.0.norm2.weight", "heads3d.decoder.layers.0.norm2.bias", "heads3d.decoder.layers.0.norm3.weight", "heads3d.decoder.layers.0.norm3.bias", "heads3d.decoder.layers.1.self_attn.in_proj_weight", "heads3d.decoder.layers.1.self_attn.in_proj_bias", "heads3d.decoder.layers.1.self_attn.out_proj.weight", "heads3d.decoder.layers.1.self_attn.out_proj.bias", "heads3d.decoder.layers.1.multihead_attn.in_proj_weight", "heads3d.decoder.layers.1.multihead_attn.in_proj_bias", "heads3d.decoder.layers.1.multihead_attn.out_proj.weight", "heads3d.decoder.layers.1.multihead_attn.out_proj.bias", "heads3d.decoder.layers.1.linear1.weight", "heads3d.decoder.layers.1.linear1.bias", "heads3d.decoder.layers.1.linear2.weight", "heads3d.decoder.layers.1.linear2.bias", "heads3d.decoder.layers.1.norm1.weight", "heads3d.decoder.layers.1.norm1.bias", "heads3d.decoder.layers.1.norm2.weight", "heads3d.decoder.layers.1.norm2.bias", "heads3d.decoder.layers.1.norm3.weight", "heads3d.decoder.layers.1.norm3.bias", "heads3d.decoder.layers.2.self_attn.in_proj_weight", "heads3d.decoder.layers.2.self_attn.in_proj_bias", "heads3d.decoder.layers.2.self_attn.out_proj.weight", "heads3d.decoder.layers.2.self_attn.out_proj.bias", "heads3d.decoder.layers.2.multihead_attn.in_proj_weight", "heads3d.decoder.layers.2.multihead_attn.in_proj_bias", "heads3d.decoder.layers.2.multihead_attn.out_proj.weight", "heads3d.decoder.layers.2.multihead_attn.out_proj.bias", "heads3d.decoder.layers.2.linear1.weight", "heads3d.decoder.layers.2.linear1.bias", "heads3d.decoder.layers.2.linear2.weight", "heads3d.decoder.layers.2.linear2.bias", "heads3d.decoder.layers.2.norm1.weight", "heads3d.decoder.layers.2.norm1.bias", "heads3d.decoder.layers.2.norm2.weight", "heads3d.decoder.layers.2.norm2.bias", "heads3d.decoder.layers.2.norm3.weight", "heads3d.decoder.layers.2.norm3.bias", "heads3d.decoder.layers.3.self_attn.in_proj_weight", "heads3d.decoder.layers.3.self_attn.in_proj_bias", "heads3d.decoder.layers.3.self_attn.out_proj.weight", "heads3d.decoder.layers.3.self_attn.out_proj.bias", "heads3d.decoder.layers.3.multihead_attn.in_proj_weight", "heads3d.decoder.layers.3.multihead_attn.in_proj_bias", "heads3d.decoder.layers.3.multihead_attn.out_proj.weight", "heads3d.decoder.layers.3.multihead_attn.out_proj.bias", "heads3d.decoder.layers.3.linear1.weight", "heads3d.decoder.layers.3.linear1.bias", "heads3d.decoder.layers.3.linear2.weight", "heads3d.decoder.layers.3.linear2.bias", "heads3d.decoder.layers.3.norm1.weight", "heads3d.decoder.layers.3.norm1.bias", "heads3d.decoder.layers.3.norm2.weight", "heads3d.decoder.layers.3.norm2.bias", "heads3d.decoder.layers.3.norm3.weight", "heads3d.decoder.layers.3.norm3.bias", "heads3d.decoder.layers.4.self_attn.in_proj_weight", "heads3d.decoder.layers.4.self_attn.in_proj_bias", "heads3d.decoder.layers.4.self_attn.out_proj.weight", "heads3d.decoder.layers.4.self_attn.out_proj.bias", "heads3d.decoder.layers.4.multihead_attn.in_proj_weight", "heads3d.decoder.layers.4.multihead_attn.in_proj_bias", "heads3d.decoder.layers.4.multihead_attn.out_proj.weight", "heads3d.decoder.layers.4.multihead_attn.out_proj.bias", "heads3d.decoder.layers.4.linear1.weight", "heads3d.decoder.layers.4.linear1.bias", "heads3d.decoder.layers.4.linear2.weight", "heads3d.decoder.layers.4.linear2.bias", "heads3d.decoder.layers.4.norm1.weight", "heads3d.decoder.layers.4.norm1.bias", "heads3d.decoder.layers.4.norm2.weight", "heads3d.decoder.layers.4.norm2.bias", "heads3d.decoder.layers.4.norm3.weight", "heads3d.decoder.layers.4.norm3.bias", "heads3d.decoder.layers.5.self_attn.in_proj_weight", "heads3d.decoder.layers.5.self_attn.in_proj_bias", "heads3d.decoder.layers.5.self_attn.out_proj.weight", "heads3d.decoder.layers.5.self_attn.out_proj.bias", "heads3d.decoder.layers.5.multihead_attn.in_proj_weight", "heads3d.decoder.layers.5.multihead_attn.in_proj_bias", "heads3d.decoder.layers.5.multihead_attn.out_proj.weight", "heads3d.decoder.layers.5.multihead_attn.out_proj.bias", "heads3d.decoder.layers.5.linear1.weight", "heads3d.decoder.layers.5.linear1.bias", "heads3d.decoder.layers.5.linear2.weight", "heads3d.decoder.layers.5.linear2.bias", "heads3d.decoder.layers.5.norm1.weight", "heads3d.decoder.layers.5.norm1.bias", "heads3d.decoder.layers.5.norm2.weight", "heads3d.decoder.layers.5.norm2.bias", "heads3d.decoder.layers.5.norm3.weight", "heads3d.decoder.layers.5.norm3.bias", "heads3d.decoder.norm.weight", "heads3d.decoder.norm.bias", "heads3d.vocab_head.weight", "heads3d.vocab_head.bias", "heads3d.pointer_head.weight", "heads3d.pointer_head.bias", "heads3d.switch_head.weight", "heads3d.switch_head.bias".
Unexpected key(s) in state_dict: "model.input_embeddings.input_value.weight", "model.input_embeddings.input_pos.weight", "model.input_embeddings.input_coord.weight", "model.input_embeddings.input_view.weight", "model.input_embeddings.input_type.weight", "model.query_coord_embedding.weight", "model.query_pos_embedding.weight", "model.encoder.layers.0.self_attn.in_proj_weight", "model.encoder.layers.0.self_attn.in_proj_bias", "model.encoder.layers.0.self_attn.out_proj.weight", "model.encoder.layers.0.self_attn.out_proj.bias", "model.encoder.layers.0.linear1.weight", "model.encoder.layers.0.linear1.bias", "model.encoder.layers.0.linear2.weight", "model.encoder.layers.0.linear2.bias", "model.encoder.layers.0.norm1.weight", "model.encoder.layers.0.norm1.bias", "model.encoder.layers.0.norm2.weight", "model.encoder.layers.0.norm2.bias", "model.encoder.layers.1.self_attn.in_proj_weight", "model.encoder.layers.1.self_attn.in_proj_bias", "model.encoder.layers.1.self_attn.out_proj.weight", "model.encoder.layers.1.self_attn.out_proj.bias", "model.encoder.layers.1.linear1.weight", "model.encoder.layers.1.linear1.bias", "model.encoder.layers.1.linear2.weight", "model.encoder.layers.1.linear2.bias", "model.encoder.layers.1.norm1.weight", "model.encoder.layers.1.norm1.bias", "model.encoder.layers.1.norm2.weight", "model.encoder.layers.1.norm2.bias", "model.encoder.layers.2.self_attn.in_proj_weight", "model.encoder.layers.2.self_attn.in_proj_bias", "model.encoder.layers.2.self_attn.out_proj.weight", "model.encoder.layers.2.self_attn.out_proj.bias", "model.encoder.layers.2.linear1.weight", "model.encoder.layers.2.linear1.bias", "model.encoder.layers.2.linear2.weight", "model.encoder.layers.2.linear2.bias", "model.encoder.layers.2.norm1.weight", "model.encoder.layers.2.norm1.bias", "model.encoder.layers.2.norm2.weight", "model.encoder.layers.2.norm2.bias", "model.encoder.layers.3.self_attn.in_proj_weight", "model.encoder.layers.3.self_attn.in_proj_bias", "model.encoder.layers.3.self_attn.out_proj.weight", "model.encoder.layers.3.self_attn.out_proj.bias", "model.encoder.layers.3.linear1.weight", "model.encoder.layers.3.linear1.bias", "model.encoder.layers.3.linear2.weight", "model.encoder.layers.3.linear2.bias", "model.encoder.layers.3.norm1.weight", "model.encoder.layers.3.norm1.bias", "model.encoder.layers.3.norm2.weight", "model.encoder.layers.3.norm2.bias", "model.encoder.layers.4.self_attn.in_proj_weight", "model.encoder.layers.4.self_attn.in_proj_bias", "model.encoder.layers.4.self_attn.out_proj.weight", "model.encoder.layers.4.self_attn.out_proj.bias", "model.encoder.layers.4.linear1.weight", "model.encoder.layers.4.linear1.bias", "model.encoder.layers.4.linear2.weight", "model.encoder.layers.4.linear2.bias", "model.encoder.layers.4.norm1.weight", "model.encoder.layers.4.norm1.bias", "model.encoder.layers.4.norm2.weight", "model.encoder.layers.4.norm2.bias", "model.encoder.layers.5.self_attn.in_proj_weight", "model.encoder.layers.5.self_attn.in_proj_bias", "model.encoder.layers.5.self_attn.out_proj.weight", "model.encoder.layers.5.self_attn.out_proj.bias", "model.encoder.layers.5.linear1.weight", "model.encoder.layers.5.linear1.bias", "model.encoder.layers.5.linear2.weight", "model.encoder.layers.5.linear2.bias", "model.encoder.layers.5.norm1.weight", "model.encoder.layers.5.norm1.bias", "model.encoder.layers.5.norm2.weight", "model.encoder.layers.5.norm2.bias", "model.encoder.norm.weight", "model.encoder.norm.bias", "model.decoder.layers.0.self_attn.in_proj_weight", "model.decoder.layers.0.self_attn.in_proj_bias", "model.decoder.layers.0.self_attn.out_proj.weight", "model.decoder.layers.0.self_attn.out_proj.bias", "model.decoder.layers.0.multihead_attn.in_proj_weight", "model.decoder.layers.0.multihead_attn.in_proj_bias", "model.decoder.layers.0.multihead_attn.out_proj.weight", "model.decoder.layers.0.multihead_attn.out_proj.bias", "model.decoder.layers.0.linear1.weight", "model.decoder.layers.0.linear1.bias", "model.decoder.layers.0.linear2.weight", "model.decoder.layers.0.linear2.bias", "model.decoder.layers.0.norm1.weight", "model.decoder.layers.0.norm1.bias", "model.decoder.layers.0.norm2.weight", "model.decoder.layers.0.norm2.bias", "model.decoder.layers.0.norm3.weight", "model.decoder.layers.0.norm3.bias", "model.decoder.layers.1.self_attn.in_proj_weight", "model.decoder.layers.1.self_attn.in_proj_bias", "model.decoder.layers.1.self_attn.out_proj.weight", "model.decoder.layers.1.self_attn.out_proj.bias", "model.decoder.layers.1.multihead_attn.in_proj_weight", "model.decoder.layers.1.multihead_attn.in_proj_bias", "model.decoder.layers.1.multihead_attn.out_proj.weight", "model.decoder.layers.1.multihead_attn.out_proj.bias", "model.decoder.layers.1.linear1.weight", "model.decoder.layers.1.linear1.bias", "model.decoder.layers.1.linear2.weight", "model.decoder.layers.1.linear2.bias", "model.decoder.layers.1.norm1.weight", "model.decoder.layers.1.norm1.bias", "model.decoder.layers.1.norm2.weight", "model.decoder.layers.1.norm2.bias", "model.decoder.layers.1.norm3.weight", "model.decoder.layers.1.norm3.bias", "model.decoder.layers.2.self_attn.in_proj_weight", "model.decoder.layers.2.self_attn.in_proj_bias", "model.decoder.layers.2.self_attn.out_proj.weight", "model.decoder.layers.2.self_attn.out_proj.bias", "model.decoder.layers.2.multihead_attn.in_proj_weight", "model.decoder.layers.2.multihead_attn.in_proj_bias", "model.decoder.layers.2.multihead_attn.out_proj.weight", "model.decoder.layers.2.multihead_attn.out_proj.bias", "model.decoder.layers.2.linear1.weight", "model.decoder.layers.2.linear1.bias", "model.decoder.layers.2.linear2.weight", "model.decoder.layers.2.linear2.bias", "model.decoder.layers.2.norm1.weight", "model.decoder.layers.2.norm1.bias", "model.decoder.layers.2.norm2.weight", "model.decoder.layers.2.norm2.bias", "model.decoder.layers.2.norm3.weight", "model.decoder.layers.2.norm3.bias", "model.decoder.layers.3.self_attn.in_proj_weight", "model.decoder.layers.3.self_attn.in_proj_bias", "model.decoder.layers.3.self_attn.out_proj.weight", "model.decoder.layers.3.self_attn.out_proj.bias", "model.decoder.layers.3.multihead_attn.in_proj_weight", "model.decoder.layers.3.multihead_attn.in_proj_bias", "model.decoder.layers.3.multihead_attn.out_proj.weight", "model.decoder.layers.3.multihead_attn.out_proj.bias", "model.decoder.layers.3.linear1.weight", "model.decoder.layers.3.linear1.bias", "model.decoder.layers.3.linear2.weight", "model.decoder.layers.3.linear2.bias", "model.decoder.layers.3.norm1.weight", "model.decoder.layers.3.norm1.bias", "model.decoder.layers.3.norm2.weight", "model.decoder.layers.3.norm2.bias", "model.decoder.layers.3.norm3.weight", "model.decoder.layers.3.norm3.bias", "model.decoder.layers.4.self_attn.in_proj_weight", "model.decoder.layers.4.self_attn.in_proj_bias", "model.decoder.layers.4.self_attn.out_proj.weight", "model.decoder.layers.4.self_attn.out_proj.bias", "model.decoder.layers.4.multihead_attn.in_proj_weight", "model.decoder.layers.4.multihead_attn.in_proj_bias", "model.decoder.layers.4.multihead_attn.out_proj.weight", "model.decoder.layers.4.multihead_attn.out_proj.bias", "model.decoder.layers.4.linear1.weight", "model.decoder.layers.4.linear1.bias", "model.decoder.layers.4.linear2.weight", "model.decoder.layers.4.linear2.bias", "model.decoder.layers.4.norm1.weight", "model.decoder.layers.4.norm1.bias", "model.decoder.layers.4.norm2.weight", "model.decoder.layers.4.norm2.bias", "model.decoder.layers.4.norm3.weight", "model.decoder.layers.4.norm3.bias", "model.decoder.layers.5.self_attn.in_proj_weight", "model.decoder.layers.5.self_attn.in_proj_bias", "model.decoder.layers.5.self_attn.out_proj.weight", "model.decoder.layers.5.self_attn.out_proj.bias", "model.decoder.layers.5.multihead_attn.in_proj_weight", "model.decoder.layers.5.multihead_attn.in_proj_bias", "model.decoder.layers.5.multihead_attn.out_proj.weight", "model.decoder.layers.5.multihead_attn.out_proj.bias", "model.decoder.layers.5.linear1.weight", "model.decoder.layers.5.linear1.bias", "model.decoder.layers.5.linear2.weight", "model.decoder.layers.5.linear2.bias", "model.decoder.layers.5.norm1.weight", "model.decoder.layers.5.norm1.bias", "model.decoder.layers.5.norm2.weight", "model.decoder.layers.5.norm2.bias", "model.decoder.layers.5.norm3.weight", "model.decoder.layers.5.norm3.bias", "model.decoder.norm.weight", "model.decoder.norm.bias", "model.vocab_head.weight", "model.vocab_head.bias", "model.pointer_head.weight", "model.pointer_head.bias", "model.switch_head.weight", "model.switch_head.bias".

use SVG files as input

I aim to input SVG files representing the top, bottom, and side orthographic projections of an object. I want to transform these SVG files into a data format suitable for model input data, to reconstruct 3d model

can you give me code reference to do so

系统问题

作者你好。想请问下您是在什么系统下跑的？因为我在训练得那一步出了点问题

curve space

This repo also works on curves?