Comments (9)
@Lucaweihs Could you help to answer this?
from holodeck.
Hi @BlackJ36,
Can I ask how you'd like objectType
to work? Do you just want to have the type returned with the object metadata or is there some AI2-THOR action that you want to use that uses the objectType
as input?
@AlvaroHG @winthos - how hard would it be to let people specify the object type as a string (e.g. on the call to CreateObjectPrefab
)?
from holodeck.
@Lucaweihs Hi, thanks to your reply. To be specific, I want to create some item small object in holodeck. The item
objectType
may be different from what I defined, so I have to try to modify the objectType
. Then I want to execute the function in RoboThor like get_shortest_path_to_object_type
which take objectType
as the parameter.
Or I just want to place all the book
type (the type I specify) objecect randomly by PlaceObjectAtPoint
and GetSpawnCoordinatesAboveReceptacle
.
from holodeck.
Got it, so the get_shortest_path_to_object_type
function won't work for now but you should be able to implement something similar by specifying the objectId
and using the GetShortestPath
action:
controller.step(
action="GetShortestPath",
objectId=object_id,
position=agent_position,
)
Similarly, for the moment you'll have to manually track which object ids correspond to books and use that for your PlaceObjectAtPoint
/GetSpawnCoordinatesAboveReceptacle
applications.
We'll look into supporting custom object types more natively in AI2-THOR but for now it's probably best to work around it.
from holodeck.
Sure, I have tried get_shortest_path_to_object
which implement in the /utils/metrics . There is another trouble I met.
ValueError: Unable to find shortest path for objectId 'tv stand-0 (living room)' due to error 'InvalidOperationException: No point on NavMesh near startPosition (3.5, 0.0, 2.0).
I plot the ReachablePosition
of robot, it seems like the initialization of the agent encounter an error.Thus, it can't find the waypoint in the function get_shortest_path_to_object
.Further more, the robot can not finish the movement thought the Action
like Teleport
and MoveAhead
.
Is there any possible way to solve this problem? I have check the API document but still in stuck.
from holodeck.
Could you try initializing the agent in a different position without collision with objects?
from holodeck.
Thanks for the picture, that's very helpful! Yes, as @YueYANG1996 said, you'll need to teleport the agent to a collision-free starting position. Here's some code I've used in the past for this (for ProcTHOR houses but I believe the same thing should work here); in particular, see the try_find_collision_free_starting_position
function. After teleporting, you might need to also run controller.step("BakeNavMesh")
.
from typing import Dict, Any
import ai2thor.controller
from shapely import Polygon
from shapely.ops import triangulate
def get_rooms_polymap(house: Dict[str, Any]):
room_poly_map = {}
# NOTE: Map the rooms
for i, room in enumerate(house["rooms"]):
room_poly_map[room["id"]] = Polygon(
[(p["x"], p["z"]) for p in room["floorPolygon"]]
)
return room_poly_map
def get_candidate_points_in_room(
room_id: str,
room_poly_map: Dict[str, Polygon],
):
polygon = room_poly_map[room_id]
room_triangles = triangulate(polygon)
candidate_points = [
((t.centroid.x, t.centroid.y), t.area) for t in room_triangles # type:ignore
]
# We sort the triangles by size so we try to go to the center of the largest triangle first
candidate_points.sort(key=lambda x: x[1], reverse=True)
candidate_points = [p[0] for p in candidate_points]
# The centroid of the whole room polygon need not be in the room when the room is concave. If it is,
# let's make it the first point we try to navigate to.
if polygon.contains(polygon.centroid):
candidate_points.insert(0, (polygon.centroid.x, polygon.centroid.y))
return candidate_points
def try_find_collision_free_starting_position(
house: Dict[str, Any],
controller: ai2thor.controller.Controller,
room_poly_map: Dict[str, Polygon],
):
teleport_success = False
for room_id in sorted(room_poly_map.keys()):
candidate_points = get_candidate_points_in_room(room_id, room_poly_map)
for cand in candidate_points:
event = controller.step(
action="TeleportFull",
position={
"x": float(cand[0]),
"y": house["metadata"]["agent"]["position"]["y"],
"z": float(cand[1]),
},
rotation=house["metadata"]["agent"]["rotation"],
standing=True,
horizon=30,
)
if event:
teleport_success = True
break
if teleport_success:
break
if teleport_success:
return True
else:
return False
from holodeck.
Thanks for your assistance. I tried your method in the scene, unfortunatly it didn't work well. The result of try_find_collision_free_starting_position
is False .
I think the problem I met can only be solved in the initalization instead of the postproc
ess? The teleoport action can't be done although there are three candidate points returned by get_candidate_points_in_room
.
By the way, the function return False in the scene generated by the ProcThor's example.py. So, I think it happens ocasionally, I tested my other scene generated by Holodeck and Robot work well. Thank's for your detailed reply again!
Attachment: The Trouble
a_living_room_with_a_book.json
from holodeck.
You could update the JSON file to make sure the robot is in a collision-free position.
Holodeck/modules/empty_house.json
Line 6 in 7094a01
from holodeck.
Related Issues (20)
- How can I export the room model to Blender HOT 3
- how holodeck search models from the dataset? HOT 1
- How to map the material files back to the assets? HOT 3
- Can't open ai2thor/unity/Assets/Scenes/Procedural/Procedural.unity HOT 5
- Floor reachable points abnormality after importing to ai2thor HOT 2
- get_top_down_frame error HOT 4
- Unity process has exited HOT 13
- Error trying to open generated scene in Unity. HOT 7
- Using new materials and objects HOT 4
- Inquiry about the structure and field meanings of `scene` dictionary HOT 13
- Share Experiment Prompts or Selections in Paper HOT 4
- Sharing the details of experiments on NoveltyTHOR HOT 4
- Unable to Connect to Unity After Scene Generation HOT 3
- Converting .pkl.gz to .glb files, incorrect material HOT 3
- Which attribute represents the rotation angle that ensure the object face as expected before loading HOT 5
- Runtime error: Only available on CPU when generating environment HOT 2
- No module named objathor.dataset.download_holodeck_metadata HOT 3
- Issue in generating scenarios that make sense and are as realistic as possible HOT 3
- Unity compile error HOT 1
- Exception in setup and the first use HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from holodeck.