matterport/habitat-matterport-3dresearch

habitat-matterport-3dresearch's Issues

Difference between the format of GLC and Habitat

Dear authors,

Thanks for your great work!

After checking your paper and the website, I still do not quite understand the difference between GLC format and Habitat format, could you please explain this?

Thanks a lot!

HM3D semantic annotations?

Where are the annotations mentioned here: https://aihabitat.org/datasets/hm3d-semantics/ ?

There are some links to annotation files in the README of some branches, but they are broken.

The difference between hm3d v0.1 and v0.2

Hi, could you please illustrate that the difference between the new version and old version of the hm3d semantic dataset? So we can select a suitable version for our task. Thanks!

Working with Habitat Simulator

Hello guys, many thanks for the dataset!

I have used Matterport3D dataset before this. But for HM3D dataset, on loading the GLB files provided for various scenes in the tarballs, I noticed that I am not able to save the ground truth poses of the agent during navigation. I also notice the absence of navigation support i.e, no physical boundaries to constrain the agent while moving. This makes moving the agent up or down staircases a bit unnatural and odd compared to Matterport3D. I see a .navmesh file and tried to load it along with the GLB using the habitat-viewer tool but it does not work for some reason. Why has it been provided?

Am I missing something? How was this dataset intended to be used with Habitat simulator?
I would love it if someone can help!
Regards.

Where is the metadata?

Hello,

Here(https://aihabitat.org/datasets/hm3d/), it is mentioned that you have meta data for splits, reviewer scores, etc. Where can I find this? Sorry; I have trouble finding this.

How to organize the dataset?

I have some problems with organizing the dataset, the raw downloaded files seems not to be right?

In the dataset.md file, the suggested file structure is :

ls
hm3d_annotated_basis.scene_dataset_config.json train/
hm3d_basis.scene_dataset_config.json val/
minival/

ls /minival
00800-TEEsavR23oF/ 00806-tQ5s4ShP627/
00801-HaxA7YrQdEC/ 00807-rsggHU7g7dh/
00802-wcojb4TFT35/ 00808-y9hTuugGdiq/
00803-k1cupFYWXJ6/ 00809-Qpor2mEya8F/
00804-BHXhpBwSMLh/ hm3d_annotated_minival_basis.scene_dataset_config.json
00805-SUHsP6z2gcJ/ hm3d_minival_basis.scene_dataset_config.json

ls /minival/00800-TEEsavR23oF/
TEEsavR23oF.basis.glb TEEsavR23oF.basis.navmesh
TEEsavR23oF.semantic.glb TEEsavR23oF.semantic.txt

My question is :

1、hm3d_basis.scene_dataset_config.json seems not to be provided?
2、 hm3d_minival_basis.scene_dataset_config.json is also not provided?
3、not every scene is provided with (TEEsavR23oF.semantic.glb TEEsavR23oF.semantic.txt
) because the number of files in hm3d-*-semantic-annots-v0.2 is less than that in hm3d-*-habitat-v0.2

Semantic Scene of HM3D

I follow the method in issue to get semantic annotation of HM3D as followed:

self._sim_config = get_config("exp_config/ddppo_imagenav_hm3d_v1.yaml")
self._sim_config.habitat.simulator['scene'] = 'data/scene_datasets/hm3d/train/00006HkseAnWCgqk/HkseAnWCgqk.basis.glb'
self._sim_config.habitat.simulator['scene_dataset'] = 'data/scene_datasets/hm3d/train/hm3d_annotated_basis.scene_dataset_config.json'
self._sim = make_sim(id_sim=self._sim_config.habitat.simulator.type,
config=self._sim_config.habitat.simulator)
ssa = self._sim.semantic_scene

The semantic annotation value 'ssa' which has region attribute in ssa.regions. However, each region in regions attribute has no center which is 0. and sizes which is inf annotation .
Can you please tell me how to get the region bound annotation as the paper says ''Region annotations are derived from reasonable estimates of room boundaries within the scenes'' in Appendix Annotation Inferences part

Thank you very much!

Bounding box of the annotations?

Is there a smart way to extract the bounding box of the annotated objects in the scene?

Filter semantic annotation in HM3D dataset

Habitat-Lab: v0.2.4
Habitat-Sim: v0.2.4

Hi!

I want to filtering out some classes in HM3D dataset. But edit %Datasetname%.semantic.txt doesn't work

Dataset
data/scene_datasets.hm3d/minival/00800-TEEsavR23oF/ there are four files

TEEsavR23oF.basis.glb
TEEsavR23oF.basis.navmesh
TEEsavR23oF.semantic.glb
TEEsavR23oF.semantic.txt
And I would like to filter just remain ceiling annotation.
When I delete other annotations in TEEsavR23oF.semantic.txt, habitat_sim.Simulator generate empty class and printed like
[21:50:57:044332]:[Metadata] AttributesManagerBase.h(357)::buildAttrSrcPathsFromJSONAndLoad : No Glob path result for ./data/scene_datasets/hm3d/train/00017-oEPjPNSPmzL/*.basis.scene_instance.json

And edit annotations to other class in TEEsavR23oF.semantic.txt also doesn't work.

I think It is related in *.basis.scene_instance.json in hm3d_annotated_basis.scene_dataset_config.json but I can't find those .basis.scene_instance.json file locally.

Is there any method to filter annotations??
Thanks for your help in advance !

Sematic annotation file typo

In the annotation file nS8T59Aw3sf.semantic.txt in the train split there is a typo on the color annotation of a radiator.

v2 and v1 of this dataset had no info on region centers. The habitat version of the original matterport data does

Thanks for your excellent work.
Due to the lack of center coordinates, I am not able to extract the images from all regions of a scene for semantic segmentation. When I use the habitat version of the original matterport dataset that is downloaded from download_mp.py, it has the centers of regions. Is there any other way to get centers, or to get all regions somehow without the center?

I tried doing things like using the coords door, ceiling, etc to find a center, but all were bad, as I couldn't point to which direction it should move, in order to reach the center. I think a workaround would be to detect all walls and ceilings, then compute the centers that way, but I doubt this will work for all images. Is there any other workaround?

This is the result from minival/00800-TEEsavR23oF and hm3d_annotated_minival_basis.scene_dataset_config.json

House has 0 levels, 14 regions and 661 objects
House center:[0. 0. 0.] dims:[-inf -inf -inf]
Region id:_-1, category:None, center:[0. 0. 0.], dims:[-inf -inf -inf]
Region id:_1, category:None, center:[0. 0. 0.], dims:[-inf -inf -inf]
Region id:_2, category:None, center:[0. 0. 0.], dims:[-inf -inf -inf]
Region id:_3, category:None, center:[0. 0. 0.], dims:[-inf -inf -inf]
Region id:_4, category:None, center:[0. 0. 0.], dims:[-inf -inf -inf]
Region id:_5, category:None, center:[0. 0. 0.], dims:[-inf -inf -inf]
Region id:_6, category:None, center:[0. 0. 0.], dims:[-inf -inf -inf]
Region id:_7, category:None, center:[0. 0. 0.], dims:[-inf -inf -inf]
Region id:_8, category:None, center:[0. 0. 0.], dims:[-inf -inf -inf]
Region id:_9, category:None, center:[0. 0. 0.], dims:[-inf -inf -inf]
Region id:_10, category:None, center:[0. 0. 0.], dims:[-inf -inf -inf]
Region id:_11, category:None, center:[0. 0. 0.], dims:[-inf -inf -inf]
Region id:_12, category:None, center:[0. 0. 0.], dims:[-inf -inf -inf]
Region id:_13, category:None, center:[0. 0. 0.], dims:[-inf -inf -inf]

This is from mp3d/1LXtFkjw3qL and mp3d.scene_dataset_config.json

Region id:2_0, category:bedroom, center:[-0.696105 5.876904 7.47785 ], dims:[5.82755 5.0632906 4.642979 ]
Region id:2_1, category:bedroom, center:[-0.73821497 5.869445 13.285263 ], dims:[5.69943 5.0556297 6.9206686]
Region id:2_2, category:bathroom, center:[-1.5556049 5.855009 16.014853 ], dims:[3.92283 4.8258 2.7374992]
Region id:2_3, category:toilet, center:[-0.1834985 4.628419 15.450899 ], dims:[1.273465 2.3871403 1.513999 ]
Region id:2_4, category:spa/sauna, center:[-1.038781 4.380009 4.0081596], dims:[3.648958 1.9890797 2.3295202]
Region id:1_5, category:porch/terrace/deck, center:[-5.63754 1.6281538 11.358154 ], dims:[ 3.8106003 3.0849504 11.70929 ]
Region id:1_6, category:kitchen, center:[-1.6988602 4.5161204 -0.6340549], dims:[7.36716 7.813819 7.249289]
Region id:1_7, category:hallway, center:[-4.595725 1.6373835 4.9533896], dims:[1.73205 2.9854317 3.9453 ]
Region id:1_8, category:stairs, center:[1.414272 3.6792169 2.6212902], dims:[1.354676 5.7876062 5.110919 ]
Region id:1_9, category:stairs, center:[-4.5807953 1.6682456 3.214875 ], dims:[1.8436699 2.884709 0.5157094]
Region id:1_10, category:office, center:[-0.82038 1.656003 15.7973 ], dims:[5.5746 3.0550919 2.8691978]
Region id:1_11, category:living room, center:[-0.803185 1.6077166 9.955522 ], dims:[5.45431 2.960826 8.798361]
Region id:1_12, category:hallway, center:[1.9568756 2.5135574 6.4436903], dims:[4.127609 5.003624 9.56222 ]
Region id:0_13, category:closet, center:[ 0.539799 -1.6975722 4.8729296], dims:[0.6300919 2.7176151 0.72369957]
Region id:1_14, category:hallway, center:[ 3.03258 1.5403118 14.23005 ], dims:[1.9445 3.0837345 6.0061016]
Region id:0_15, category:bathroom, center:[ 0.5301975 -1.68649 14.789148 ], dims:[2.610045 2.730423 4.872699]
Region id:0_16, category:bedroom, center:[-4.1186237 -1.6176748 15.797 ], dims:[6.673073 2.6894484 2.7996006]
Region id:0_17, category:bedroom, center:[-4.6803603 -1.6154566 12.89305 ], dims:[5.579221 2.685926 2.9934998]
Region id:0_18, category:bedroom, center:[-4.6226 -1.61621 9.868441], dims:[5.5316 2.6871195 3.0303192]
Region id:0_19, category:bedroom, center:[-4.574565 -1.6151962 6.9728546], dims:[5.44477 2.6855097 2.7280297]
Region id:0_20, category:hallway, center:[-0.46183947 -1.6429033 8.91875 ], dims:[ 2.5349011 2.6403122 10.9487 ]
Region id:0_21, category:toilet, center:[ 1.0019575 -1.6828442 5.910165 ], dims:[1.413585 2.723012 1.0809898]
Region id:0_22, category:closet, center:[ 0.884981 -1.6886387 8.2303705], dims:[1.2529579 2.7319617 2.2224197]
Region id:0_23, category:closet, center:[ 0.895914 -1.6896095 11.102665 ], dims:[1.311772 2.7334614 2.2736702]
Region id:0_24, category:stairs, center:[-0.43271753 -2.6956449 3.1824796 ], dims:[2.453765 0.36514997 0.60734034]
Region id:0_25, category:hallway, center:[ 0.34897995 -1.0808079 0.7692351 ], dims:[3.0130801 2.8645244 4.5243893]
Region id:0_26, category:bathroom, center:[ 1.0604529 -1.028461 -2.856915 ], dims:[1.6081738 2.7615576 2.7168703]
Region id:1_27, category:porch/terrace/deck, center:[ 3.139145 2.605465 -1.3372247], dims:[1.8337297 5.35499 6.0002294]
Region id:0_28, category:stairs, center:[ 1.280018 -1.1482306 1.670922 ], dims:[1.015024 2.3518791 2.130136 ]
Region id:0_29, category:workout/gym/exercise, center:[-3.04459 -1.0788939 0.6934099], dims:[4.3276005 2.8613925 4.1135197]
Region id:0_30, category:bedroom, center:[-2.4535055 -1.0308 -2.7611852], dims:[5.3881693 2.7652996 2.7803698]

Problems about connection with Matterport3D

Hi all, I am really appreciate your excellent work, but I'm curious about is there any connection with the previous work Matterport3D dataset? To the beast of my knowledge, the only way I can have access to Habitat M3D is on the github. It seems that there is only a textured mesh in each scene of Habitat M3D, I cannot find more useful data such as camera poses, panorama, semantic segmentation and depth images, which is available in Matterport3D. You know, only textured mesh will limit the applications in the research community, we prefer more informative data in Habitat M3D.

I am looking forward to your reply, thanks in advance!
@haeric @aclegg3 @apatel-matterport @srama2512

Scale of topdown views for each floor

Is there a way to get the scale of the topdown views for each floor (from the online dataset browser), e.g. the length / width of a particular room, or a particular object (e.g. the dining table), in some real world units like meters?

How can I get access to these two datasets in Habitat-Matterport 3D?

Hi, I have I registered an account of Matterport , filled out the form and submitted the request, when and how can I get access to these two datasets in Habitat-Matterport 3D?(https://api.matterport.com/resources/habitat/hm3d-train-habitat-v0.2.tar, https://api.matterport.com/resources/habitat/hm3d-val-habitat-v0.2.tar)

Issue with extracting contents of example data

I'm trying to extract the content of the example data, but I am running into the following errors:

~/habitat-matterport-3dresearch/example$ tar -xvf hm3d-example-configs.tar
tar: This does not look like a tar archive
tar: Exiting with failure status due to previous errors

~/habitat-matterport-3dresearch/example$ tar -zxvf hm3d-example-obj+mtl.tar.gz
gzip: stdin: not in gzip format
tar: Child returned status 1
tar: Error is not recoverable: exiting now

Any ideas why these files don't seem to be in the expected .tar and .tar.gz formats?

Accessing HM3DSem Test Split?

Hello,

Is the test split referenced by hm3d_annotated_basis.scene_dataset_config.json available? If so, how do I access it?

"test/00900-XRJ2muKkwAV/*.basis.glb",
"test/00901-aJoC8Qw6xQ5/*.basis.glb",
"test/00904-S9CUp5RsFY9/*.basis.glb",
"test/00905-9v5HzFAB1cm/*.basis.glb",
"test/00907-WGxjrSp71zX/*.basis.glb",
"test/00909-MrKys1cv4jD/*.basis.glb",
"test/00910-jE7Qzu3FyJD/*.basis.glb",
"test/00911-a8h4mZT8buN/*.basis.glb",
"test/00913-URu5ybF8PrH/*.basis.glb",
"test/00914-TxyEAAVD4iD/*.basis.glb",
"test/00915-ieavb8rbmA9/*.basis.glb",
"test/00918-NQX2SZD1TMJ/*.basis.glb",
"test/00921-nhtTwUrWv7v/*.basis.glb",
"test/00925-4QUpJVDQ1DN/*.basis.glb",
"test/00926-egVUByjgpHU/*.basis.glb",
"test/00927-kucQY8TbKkz/*.basis.glb",
"test/00929-azb2LDaMrGZ/*.basis.glb",
"test/00939-VedHjk858Vu/*.basis.glb",
"test/00940-oHcTqmvveM7/*.basis.glb",
"test/00947-fyZC1Whhj9i/*.basis.glb",
"test/00951-RGvKsZAErEk/*.basis.glb",
"test/00952-ogAZhviACbN/*.basis.glb",
"test/00955-5TxBa2qkcAh/*.basis.glb",
"test/00956-oNiVkiFKkAr/*.basis.glb",
"test/00957-DowRMDn4MVe/*.basis.glb",
"test/00961-dU7KLdn2Kcd/*.basis.glb",
"test/00963-awRQoguC7TN/*.basis.glb",
"test/00967-XwXCfkthawu/*.basis.glb",
"test/00970-kUmc6DP2tff/*.basis.glb",
"test/00977-1cQ7WUQgXJ4/*.basis.glb",
"test/00978-sZJ5zo6FrEe/*.basis.glb",
"test/00981-Rp8JNfstaew/*.basis.glb",
"test/00985-3ZJjFFjJKqZ/*.basis.glb",
"test/00989-t9gvgFMoH4R/*.basis.glb",
"test/00992-G3AqUkGnykx/*.basis.glb",

Format mismatch in hm3d-example-semantic-configs-v0.2

Hello, the filename format inside hm3d-example-semantic-configs-v0.2 doesn't match the filename format in hm3d-example-semantic-annots-v0.2.

Dataset: hm3d-example-semantic-annots-v0.2
Filenames:

• GLAQ4DNUx5U.semantic.txt
• GLAQ4DNUx5U.semantic.glb

Dataset: hm3d-example-semantic-configs-v0.2
Filename: hm3d_annotated_example_basis
Line 25-26

"semantic_descriptor_filename": "%%CONFIG_NAME_AS_ASSET_FILENAME%%_semantic.txt",
"semantic_asset": "%%CONFIG_NAME_AS_ASSET_FILENAME%%_semantic.ply",

Can be changed into

"semantic_descriptor_filename": "%%CONFIG_NAME_AS_ASSET_FILENAME%%.semantic.txt",
"semantic_asset": "%%CONFIG_NAME_AS_ASSET_FILENAME%%.semantic.glb",

Thank you!

Semantic annotations

I just downloaded the semantic annotations and GLAQ4DNUx5U.semantic.txt seems to contain rows of the form:

313,23694C,"refrigerator cabinet",9

What does this mean? Is there a way to get the location of the refrigerator cabinet in the scene?

Thanks!

Data download failed, no datasource named hm3d_minival-v0.2

Hi, thank you for sharing the dataset.

However, I'm facing the issue with download the data from the following command:
python -m habitat_sim.utils.datasets_download --username xxx --password xxx --uids hm3d_minival-v0.2
and following error:
Data download failed, no datasource named hm3d_minival-v0.2

When I remove "-v0.2" (--uids hm3d_minival), it works.
Is there any problem with my command for downloading?

Fail to get scene instance data

Hi, I encountered another problem when I use habitat-sim to render the textured meshes of HM3D.
Firstly, I downloaded the following data sequentially:

• hm3d-minival-habitat-v0.2

• hm3d-minival-semantic-annots-v0.2

• hm3d-minival-semantic-configs-v0.2
  Then, I put \*.basis.glb \*.scene_dataset_config.json, \*.semantic.glb and \*.semantic.txt into the same scene folder respectively. Next, I run the demo code , everything is ok when I run in the scene with the habitat simulator, I can render color image, semantic image and depth image successfully. But when the code run into print_scene_recur() function

  it seems some problems related about objects in the scene:
  House has 0 levels, 14 regions and 661 objects
House center:[0,0,0] dims:[-inf -inf -inf]

I wonder if this phenomena is caused by the absence of \*basis.scene_instance.json, because the scene_dataset_config.json has specified the location of those instances' file:

But I fail to have access to the scene_instance.json files, only the semantic annotation and config files is available now.

Im looking forward your help, best regards! @dhruvbatra @aclegg3 @haeric

Unable to obtain Habitat - Matterport 3D Research Dataset

Hello, I submitted application to Matterport three weeks ago but still cannot get an approval. As a result we are getting stuck due to lack of training data. Do you have any information why the process is getting so slow? Or is there a way to work around?

Where can I find the semantic annotations?

Hello,
I'm trying to use semantic sensor in habitat-lab to get the ground truth semantic image. I've downloaded this dataset: hm3d-minival-habitat.tar. And I think the scene is correctly loaded, but the semantic image is black (zeros), and it shows 'The active scene does not contain semantic annotations.'

[12:10:20:220323]:[Sim] Simulator.cpp(457)::instanceStageForSceneAttributes :
---
The active scene does not contain semantic annotations.
---

Does that mean that I need to download semantic annotations from somewhere else?

Physical units of scans

Are the scans (e.g. when loaded into habitat-sim) in meters?

pointnav task dataset for hm3d v0.2?

Hi there,

I look through the [https://github.com/facebookresearch/habitat-lab/blob/main/DATASETS.md](task dataset table) but there is no pointnav task dataset for hm3d v0.2. Is there any resource about it? Or can I just use v0.1? Thanks!