Comments (3)
I encountered some of the problems you raised. Here are some suggestions for each of them:
-
No WC in the bathroom: I feel this is a rare case. It may happen, but usually, the toilet is the most important object in the bathroom, and the holodeck should be able to generate it first.
-
microwave or pans on the floor in the kitchen: I also noticed this microwave issue. One thing you can do is to add another check for the object list here You can have another LLM to verify if this plan is reasonable and you can fix some mistakes here, e.g., the microwave should be on the top of table not on the floor.
-
wardrobes or shelves in front of the windows: you can change this variable to True. In this case, we will treat windows as objects, so the object placement will consider the collision with windows.
-
no isle in the kitchen although I manually specified this constraint to ChatGPT in the prompt: This is due to the lack of good isle assets in our subset of objaverse, but I think most of the time, if the kitchen is large enough, there should be an isle or kitchen counter.
-
huge, but quite empty, bedrooms: you can specify the size of the room in the prompt, for example, "5m x 5m". Or if you want more objects in the room, you can increase this value. This value controls the capacity ratio of objects for each room.
-
missing specified objects (e.g., a book on the table or the remote controller on the sofa): adding them in the prompt can help; however, since scene generation is a long and complicated process, it's hard to ensure every detail can be satisfied.
from holodeck.
Thanks for the exhaustive answer!
missing specified objects (e.g., a book on the table or the remote controller on the sofa): adding them in the prompt can help; however, since scene generation is a long and complicated process, it's hard to ensure every detail can be satisfied.
is there a way to force (or at least to control) the generation and placement of objects? I am interested in creating some realistic scenarios with a few constraints, but I feel like most of the time the constraints I am adding are not satisfied. I have a few examples:
- a book on the dining table
- a chair near the sofa
- a bottle far from the kitchen
- a blanket (wherever in the environment)
I would like to make sure that at least these constraints are met. I can also modify the code if necessary, but I would at least appreciate some indication of where I should start to modify it.
from holodeck.
Same way as the microwave solution, adding another step to double-check the object selection plan here. You can implement a function taking in the original object plan and your prompt, and let a LLM to modify the original plan to satisfy your requirement.
from holodeck.
Related Issues (20)
- Unable to Connect to Unity After Scene Generation HOT 3
- Converting .pkl.gz to .glb files, incorrect material HOT 3
- Which attribute represents the rotation angle that ensure the object face as expected before loading HOT 5
- Runtime error: Only available on CPU when generating environment HOT 2
- No module named objathor.dataset.download_holodeck_metadata HOT 3
- Unity compile error HOT 2
- Exception in setup and the first use HOT 1
- Invalid Load Key HOT 4
- Can you give an example of complex query? HOT 5
- Objects Disappear After Unity Simulation HOT 9
- Output Stuck after "Selecting objects for living room..." HOT 2
- Question about adjusting the spatial relationship of objects HOT 4
- ValueError: Invalid action: CreateRuntimeAsset HOT 5
- Why XZ plane is always used for placement? HOT 3
- Generate the JSON file normally, but only one picture of the pilot view HOT 7
- Vulkan API driver missing. HOT 1
- A dataset problem
- issue with connect_to_unity --scene <SCENE_JSON_FILE_PATH> HOT 1
- Are there examples of spatial relational constraints generated by large language models mentioned in the paper included in the GitHub code? HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from holodeck.