camenduru/panohead-colab

🐣 Please follow me for new updates https://twitter.com/camenduru
🔥 Please join our discord server https://discord.gg/k5BwmmvJJU
🥳 Please join my patreon community https://patreon.com/camenduru

🦒 Colab

🚦 WIP 🚦

Colab	Info
	PanoHead_colab (generates 3D models from generated images)
	PanoHead_custom_colab (generates 3D models from custom images)

Tutorial

https://www.youtube.com/watch?v=QwPAo-f5UgM

For .ply output please add --shapes=True

After running the first cell, add your custom image named test.jpg to the in folder.

Maybe for better 3D we can play with
max_batch = 3000000 voxel_resolution = 512 --num-steps=300 --num-steps-pti=300

--num-steps=100 --num-steps-pti=100 max_batch = 100000

Free Colab Limit

--num-steps=200 --num-steps-pti=200 max_batch = 100000

Main Repo

https://github.com/SizheAn/PanoHead

Page

https://sizhean.github.io/panohead

Paper

https://arxiv.org/abs/2303.13071

Output

--num-steps=300 --num-steps-pti=300

2023-06-29.15-38-28.mp4

panohead-colab's Issues

Custom image colab endless loop

I started the custom image colab on a free plan.
Uploaded 512*512 jpg face
First it went into at 402 steps.
Then i decreased both steps down to 400.
After doing the 400 steps the colab says 'Saving optimization progress video "/content/output/easy-khair-180-gpc0.8-trans10-025000.pkl/0/proj.mp4"' and looks like stucking in some loop, the cell processing icon is running but nothing is happening for an hour or so already

dataset.json extension mismatch

The dataset.json leaves the original extension, but the file is converted into a jpeg which results in a keyerror

Loading networks from "/content/PanoHead/models/easy-khair-180-gpc0.8-trans10-025000.pkl"... Traceback (most recent call last): File "/content/PanoHead/projector_withseg.py", line 439, in <module> run_projection() # pylint: disable=no-value-for-parameter File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1130, in __call__ return self.main(*args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1055, in main rv = self.invoke(ctx) File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1404, in invoke return ctx.invoke(self.callback, **ctx.params) File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 760, in invoke return __callback(*args, **kwargs) File "/content/PanoHead/projector_withseg.py", line 325, in run_projection c = torch.from_numpy(dataset._get_raw_labels()[idx:idx+1]).to(device) File "/content/PanoHead/training/dataset.py", line 59, in _get_raw_labels self._raw_labels = self._load_raw_labels() if self._use_labels else None File "/content/PanoHead/training/dataset.py", line 244, in _load_raw_labels labels = [labels[fname.replace('\\', '/')] for fname in self._image_fnames] File "/content/PanoHead/training/dataset.py", line 244, in <listcomp> labels = [labels[fname.replace('\\', '/')] for fname in self._image_fnames] KeyError: '[YOURFILENAME].jpg'

Is it possible to save textures along with geometry?

Not an issue, rather a question.
I see in your demo video the textured head rotating which makes me ask this question.

Generating Shapes on Free Colab

I couldn't generate .ply in colab, even with the provided settings for free colab limit.
I removed the save_video part in projector_withseg.py, ran again, and it did generate .ply.
Why is that?

Readme Asset