threedle/textdeformer

textdeformer's Issues

Question on parameters for human faces

Thanks for your great work and for putting this nice code together!

I have a question regarding finding proper parameters in the .yml. When I'm trying to go from Max Planck to Einstein/Obama the results I obtain are not as smooth as what's shown in the paper but instead the output has rough edges (providing one example below where batch_size = 50, epochs=3500, train_res=512, lr=0.005). I'm wondering what's the trick that I'm missing and whether it's a matter of parameters or different desciptions ("bust of Albert Einstein" vs "Einstein" for example). Happy to get your insights!

Thanks again!

Error using max_planck.obj

I do not know whether you meet this problem when replacing shape with max_planck.obj in the YAML file.

Nan in Poisson Solve Backward

Hello, I'm wondering have you seen this error and know what's possibly wrong?

  File "/home/**/Documents/projects/3d-detailization/TextDeformer/NeuralJacobianFields/PoissonSystem.py", line 476, in backward
    assert not torch.isnan(grad).any(),  "Nan in the backward pass of the POISSON SOLVE"
AssertionError: Nan in the backward pass of the POISSON SOLVE

I'm simply replacing the mesh in the example script with my own mesh that has around 10k vertices and only has one connected component.

Question for Jacobian regularization

I print the Jacobian regularization and its not eye matrix at first. Does the regualrization work?

fatal error: EGL/egl.h: No such file or directory

Hi,

Thanks for sharing your really nice work!
When I run main.py, I met this error related to nvdiffrast. Please let me know if you have any ideas to address this problem. Many thanks.

Unable to use mu model

Thanks for your work. But when I use my model, I get ValueError: Matrix is not positive definite! Error, could you give me some suggestions?
eagle-obj.zip

CholeskySolverD: Matrix is not positive definite

Thanks for this great codebase. One quick question:

Given some new triangle meshes as input, CholeskySolverD returns an error that the matrix is not positive definite. Did you encounter this at any point and if so do you know how to fix it? I've already double-checked that the normals are properly oriented.

Thanks!

[F glutil.cpp:332] eglGetDisplay() failed

Hi, Thanks for your great work!

I was trying to run the command: python main.py --config example_config.yml. However, I got the following error. How do I fix this issue? Thanks a lot!

Warning:
Unable to load the following plugins:

	libio_e57.so: libio_e57.so does not seem to be a Qt Plugin.

Cannot load library /home/user/anaconda3/envs/TextDeformer/lib/python3.9/site-packages/pymeshlab/lib/plugins/libio_e57.so: (/lib/x86_64-linux-gnu/libp11-kit.so.0: undefined symbol: ffi_type_pointer, version LIBFFI_BASE_7.0)

Using /home/user/.cache/torch_extensions/py39_cu116 as PyTorch extensions root...
Detected CUDA files, patching ldflags
Emitting ninja build file /home/user/.cache/torch_extensions/py39_cu116/renderutils_plugin/build.ninja...
Building extension module renderutils_plugin...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
ninja: no work to do.
Loading extension module renderutils_plugin...
adapt_dist: 1
aug_bkg: 1
aug_light: 1
aug_loc: 1
azim_max: 360.0
azim_min: 0.0
base_text_prompt: a cow
batch_size: 25
bsdf: diffuse
clip_model: ViT-B/32
clip_weight: 1.0
config: example_config.yml
consistency_azim_filter: 20
consistency_clip_model: ViT-B/32
consistency_elev_filter: 30
consistency_loss_weight: 0.5
consistency_vit_layer: 11
consistency_vit_stride: 8
delta_clip_weight: 1.0
dist_max: 3.5
dist_min: 2.5
elev_alpha: 1.0
elev_beta: 5.0
elev_max: 60.0
epochs: 2500
fov_max: 90.0
fov_min: 30.0
gpu: 0
light_power: 5.0
log_dist: 3.0
log_elev: 30.0
log_fov: 60.0
log_interval: 5
log_interval_im: 150
log_light_power: 3.0
log_res: 512
lr: 0.0025
mesh: ./meshes/spot.obj
output_path: ./outputs
regularize_jacobians_weight: 0.5
resize_method: cubic
retriangulate: 0
seed: 99
text_prompt: a giraffe
train_res: 512

Output directory ./outputs created
Loading CLIP Models
[F glutil.cpp:332] eglGetDisplay() failed
Aborted (core dumped)

Unable to reproduce chair results from paper

Thanks for the cool work!

I'm having trouble reproducing the chair examples in Fig. 9 and Fig. 11 of the paper. I'm running the following commands:

python main.py --config example_config.yml \
    --mesh meshes/chair1.obj \
    --text_prompt 'gaming chair' \
    --base_text_prompt 'a chair'

python main.py --config example_config.yml \
    --mesh meshes/chair2.obj \
    --text_prompt 'a comfortable chair' \
    --base_text_prompt 'a chair'

Here are my tensorboard visualizations:

I'm able to reproduce the cow->giraffe and hand->octopus examples just fine with the following commands:

python main.py --config example_config.yml
python main.py --config example_config.yml --mesh meshes/hand.obj \
    --text_prompt 'an octupus' --base_text_prompt 'a hand'

Unstable Fitting

Could you give me some suggestions that why there are NaN losses in the fitting procedure.

Error Running Example

Hello! I'm trying to get the code from this paper running, and am getting the following error:

Traceback (most recent call last):
  File "/content/TextDeformer/NeuralJacobianFields/MeshProcessor.py", line 153, in get_centroids
    self.load_centroids()
  File "/content/TextDeformer/NeuralJacobianFields/MeshProcessor.py", line 105, in load_centroids
    self.centroids.points_and_normals = np.load(os.path.join(self.source_dir, "centroids_and_normals.npy"))
  File "/usr/local/lib/python3.10/dist-packages/numpy/lib/npyio.py", line 405, in load
    fid = stack.enter_context(open(os_fspath(file), "rb"))
FileNotFoundError: [Errno 2] No such file or directory: 'outputs/tmp/centroids_and_normals.npy'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/content/TextDeformer/NeuralJacobianFields/MeshProcessor.py", line 113, in get_samples
    self.load_samples()
  File "/content/TextDeformer/NeuralJacobianFields/MeshProcessor.py", line 121, in load_samples
    self.samples.xyz = np.load(os.path.join(self.source_dir, 'samples.npy'))
  File "/usr/local/lib/python3.10/dist-packages/numpy/lib/npyio.py", line 405, in load
    fid = stack.enter_context(open(os_fspath(file), "rb"))
FileNotFoundError: [Errno 2] No such file or directory: 'outputs/tmp/samples.npy'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/content/TextDeformer/main.py", line 92, in <module>
    main()
  File "/content/TextDeformer/main.py", line 88, in main
    loop(cfg)
  File "/content/TextDeformer/loop.py", line 111, in loop
    jacobian_source.load()
  File "/content/TextDeformer/NeuralJacobianFields/SourceMesh.py", line 130, in load
    self.__init_from_mesh_data()
  File "/content/TextDeformer/NeuralJacobianFields/SourceMesh.py", line 90, in __init_from_mesh_data
    centroids = self.mesh_processor.get_centroids()
  File "/content/TextDeformer/NeuralJacobianFields/MeshProcessor.py", line 155, in get_centroids
    self.compute_centroids()
  File "/content/TextDeformer/NeuralJacobianFields/MeshProcessor.py", line 162, in compute_centroids
    self.get_samples()# this is to compute WKS for centroids
  File "/content/TextDeformer/NeuralJacobianFields/MeshProcessor.py", line 115, in get_samples
    self.compute_samples()
  File "/content/TextDeformer/NeuralJacobianFields/MeshProcessor.py", line 143, in compute_samples
    pt_samples, normals_samples, wks_samples, bary = self.sample_points( self.num_samples)
  File "/content/TextDeformer/NeuralJacobianFields/MeshProcessor.py", line 292, in sample_points
    bary, found_faces = igl.random_points_on_mesh(n, self.vertices, self.faces)
ValueError: too many values to unpack (expected 2)

Any insights would be helpful, thank you!