thu-lyj-lab/t3bench

T3Bench: Benchmarking Current Progress in Text-to-3D Generation

t3bench's Issues

Submit results on t3bench

Hello, thank you for providing a comprehensive benchmark for 3D generation. I am the author of GaussianDreamer and I would like to know how to submit our results on T3bench. Here are our results.

Single Obj.	** Single w/ Surr.**	Multi Obj.	Average
54.0	48.6	34.5	45.7

Generation time

Hello, this is a wonderful job. I would like to know the generation time of these methods (ProlificDreamer, SJC, Fantasia3D, LatentNeRF, Magic3D, Dreamfusion) in the paper?

Question regarding text prompts

Hello! Thanks for your interesting work. I have a question regarding the text prompts you release in this benchmark.
I am wondering if you are providing triplets of the increasingly descriptive text prompts for every object. or if every text prompt is referring to a random object.

For example:

• single object : "A red fire hydrant"
• single object with surroundings: "A red fire hydrant on a sidewalk"
• multiple objects: "A red fire hydrant is on a sidewalk close to a large tree".

There's a conflict in requirements.txt

When using pip install for requirement.txt, salesforce-lavis, and image-reward are in conflict when sharing the transformer. They have different requirements for the version of the transformer.

Release human annotation?

Would it be possible to also release the human annotations? Thanks!

Question regarding the alignment scores

If I get an average alignment score 3,
then the normalized score is 3/5x100 = 60
or (3-1)/4x100 = 50 ?
Currently based on your paper sec 4.2, I assume the second calculation is correct.
Is that right?