ankanbansal/isvqa-dataset

This repository contains the dataset from the paper "Visual Question Answering on Image Sets"

isvqa-dataset's Issues

Error while generating the dataset

Hi,
Please provide clear documentation on what gibson data/habitat env has to be setup for generating the data.
Otherwise, please share the image-based dataset.
I am getting the following unintrepretable error while trying to run generate_image_sets_from_info_files.py

Please help!!!

[01:55:42:942460]:[Metadata] MetadataMediator.cpp(297)::getSceneInstanceAttributesByName : Dataset : default has no preloaded SceneInstanceAttributes or StageAttributes named : /data1/abhiram/ISVQA-Dataset/gibson/gibson/Crandon.glb so loading/creating a new StageAttributes with this name, and then creating a SceneInstanceAttributes with the same name that references this stage.
[01:55:42:942488]:[Metadata] AttributesManagerBase.h(380)::createFromJsonOrDefaultInternal : <Stage Template>: Proposing JSON name : /data1/abhiram/ISVQA-Dataset/gibson/gibson/Crandon.stage_config.json from original name : /data1/abhiram/ISVQA-Dataset/gibson/gibson/Crandon.glb| This file does not exist.
[01:55:42:942544]:[Metadata] AbstractObjectAttributesManagerBase.h(184)::createObject : File (/data1/abhiram/ISVQA-Dataset/gibson/gibson/Crandon.glb) exists but is not a recognized config filename extension, so new default Stage Template attributes created and registered.
[01:55:42:942579]:[Metadata] SceneDatasetAttributes.cpp(47)::addNewSceneInstanceToDataset : Dataset : 'default' : Stage Attributes '/data1/abhiram/ISVQA-Dataset/gibson/gibson/Crandon.glb' specified in Scene Attributes exists in dataset library.
[01:55:42:942586]:[Metadata] SceneDatasetAttributes.cpp(84)::addNewSceneInstanceToDataset : Dataset : 'default' : Lighting Layout Attributes 'no_lights' specified in Scene Attributes but does not exist in dataset, so creating.
[01:55:42:942599]:[Metadata] AttributesManagerBase.h(380)::createFromJsonOrDefaultInternal : <Lighting Layout>: Proposing JSON name : no_lights.lighting_config.json from original name : no_lights| This file does not exist.
[01:55:42:942614]:[Metadata] LightLayoutAttributesManager.cpp(36)::createObject : File (no_lights) not found, so new default light layout attributes created and registered.
[01:55:42:942623]:[Sim] Simulator.cpp(237)::createSceneInstance : Navmesh file location in scene instance : /data1/abhiram/ISVQA-Dataset/gibson/gibson/Crandon.navmesh
[01:55:42:942636]:[Sim] Simulator.cpp(242)::createSceneInstance : Loading navmesh from /data1/abhiram/ISVQA-Dataset/gibson/gibson/Crandon.navmesh
[01:55:42:942652]:[Nav] PathFinder.cpp(852)::loadNavMesh : NavMeshSettings aren't present, guessing that they are the default
[01:55:42:942722]:[Sim] Simulator.cpp(244)::createSceneInstance : Navmesh Loaded.
[01:55:42:942743]:[Scene] SceneGraph.h(85)::createDrawableGroup : Created DrawableGroup: 
[01:55:42:942752]:[Assets] ResourceManager.cpp(297)::loadSemanticSceneDescriptor : SceneInstance : /data1/abhiram/ISVQA-Dataset/gibson/gibson/Crandon.glb proposed Semantic Scene Descriptor filename : /data1/abhiram/ISVQA-Dataset/gibson/gibson/Crandon.scn
[01:55:42:942762]:[Assets] ResourceManager.cpp(344)::loadSemanticSceneDescriptor : SSD File Naming Issue! Neither SceneInstanceAttributes-provided name : /data1/abhiram/ISVQA-Dataset/gibson/gibson/Crandon.scn  nor constructed filename : /data1/abhiram/ISVQA-Dataset/gibson/gibson/info_semantic.json exist on disk.
[01:55:42:942791]:[Assets] ResourceManager.cpp(1652)::ConfigureImporterManagerGLExtensions : Importing Basis files as BC7.
[01:55:42:943612]:[Sim] Simulator.cpp(302)::createSceneInstance : Using scene instance-specified Light key : - no_lights -
[01:55:42:943625]:[Metadata] MetadataMediator.cpp(66)::setSimulatorConfiguration : Set new simulator config for scene/stage : /data1/abhiram/ISVQA-Dataset/gibson/gibson/Crandon.glb and dataset : default which is currently active dataset.
[01:55:42:943642]:[Sim] Simulator.cpp(412)::instanceStageForSceneAttributes : Start to load stage named : /data1/abhiram/ISVQA-Dataset/gibson/gibson/Crandon.glb with render asset : /data1/abhiram/ISVQA-Dataset/gibson/gibson/Crandon.glb and collision asset : /data1/abhiram/ISVQA-Dataset/gibson/gibson/Crandon.glb
[01:55:42:943675]:[Assets] ResourceManager.cpp(691)::createStageAssetInfosFromAttributes : Frame :{"up":[0,0,1],"front":[0,1,0],"origin":[0,0,0]} for render mesh named : /data1/abhiram/ISVQA-Dataset/gibson/gibson/Crandon.glb|{"up":[0,0,1],"front":[0,1,0],"origin":[0,0,0]} for semantic mesh named :  with type specified as semantic|Semantic Txtrs : False
[01:55:42:943686]:[Assets] ResourceManager.cpp(507)::loadStage : Start load render asset /data1/abhiram/ISVQA-Dataset/gibson/gibson/Crandon.glb .
[01:55:42:943692]:[Assets] ResourceManager.cpp(973)::loadStageInternal : Attempting to load stage /data1/abhiram/ISVQA-Dataset/gibson/gibson/Crandon.glb 
[01:55:42:943700]:[Assets] ResourceManager.cpp(855)::loadRenderAsset : Loading general asset named: /data1/abhiram/ISVQA-Dataset/gibson/gibson/Crandon.glb
[01:55:42:943706]:[Assets] ResourceManager.cpp(1652)::ConfigureImporterManagerGLExtensions : Importing Basis files as BC7.
[01:55:45:497075]:[Assets] ResourceManager.cpp(2089)::loadMaterials : Building 2 materials for asset named 'Crandon.glb' : 
[01:55:45:497163]:[Assets] ResourceManager.cpp(2194)::loadMaterials : Idx 00:Flat.
[01:55:45:497183]:[Assets] ResourceManager.cpp(2194)::loadMaterials : Idx 01:Flat.
[01:55:45:534417]:[Sim] Simulator.cpp(439)::instanceStageForSceneAttributes : Successfully loaded stage named : /data1/abhiram/ISVQA-Dataset/gibson/gibson/Crandon.glb
[01:55:45:534450]:[Sim] Simulator.cpp(471)::instanceStageForSceneAttributes : 
---
The active scene does not contain semantic annotations : activeSemanticSceneID_ = 0  
---
[01:55:45:534471]:[Sim] Simulator.cpp(205)::reconfigure : CreateSceneInstance success == true for active scene name : /data1/abhiram/ISVQA-Dataset/gibson/gibson/Crandon.glb  with renderer.
2022-11-10 01:55:45,537 Initializing task Nav-v0
Traceback (most recent call last):
  File "generate_image_sets_from_info_files.py", line 101, in <module>
    generate_and_save_image_sets(pointnav_file_dir, train_scene_names, config, save_dir, 'train')
  File "generate_image_sets_from_info_files.py", line 72, in generate_and_save_image_sets
    episode=env.current_episode, task=env.task
  File "/data1/abhiram/ISVQA-Dataset/gibson/data_generation/habitat-lab/habitat/core/embodied_task.py", line 162, in reset_measures
    measure.reset_metric(*args, **kwargs)
  File "/data1/abhiram/ISVQA-Dataset/gibson/data_generation/habitat-lab/habitat/tasks/nav/nav.py", line 590, in reset_metric
    self.uuid, [DistanceToGoal.cls_uuid, Success.cls_uuid]
  File "/data1/abhiram/ISVQA-Dataset/gibson/data_generation/habitat-lab/habitat/core/embodied_task.py", line 191, in check_measure_dependencies
    listed in the measures list in the config."""
AssertionError: spl measure requires distance_to_goal
                listed in the measures list in the config.
[01:55:45:771607]:[Physics] PhysicsManager.cpp(50)::~PhysicsManager : Deconstructing PhysicsManager
[01:55:45:771674]:[Scene] SceneManager.h(25)::~SceneManager : Deconstructing SceneManager
[01:55:45:771680]:[Scene] SceneGraph.h(25)::~SceneGraph : Deconstructing SceneGraph
[01:55:45:771755]:[Sensor] Sensor.cpp(69)::~Sensor : Deconstructing Sensor
[01:55:45:771794]:[Scene] SemanticScene.h(47)::~SemanticScene : Deconstructing SemanticScene
[01:55:45:826003]:[Gfx] Renderer.cpp(72)::~Impl : Deconstructing Renderer
[01:55:45:826055]:[Gfx] WindowlessContext.h(17)::~WindowlessContext : Deconstructing WindowlessContext
[01:55:45:830442]:[Sim] Simulator.cpp(66)::~Simulator : Deconstructing Simulator

A possible bug in generate_image_sets_from_info_files.py

Thanks for your great dataset first.

The bug

I am not sure which version of habitat you use, but when I run the code of generate_image_sets_from_info_files.py with habitat-0.1.6 and habitat-sim-0.1.6, all sub-folders of the same scene (from 0 to 150 for train) get the same images. E.g. md5sum(info_files/images/train/Avonia/0/0.png) == md5sum(info_files/images/train/Avonia/1/0.png)

Reasons

After some explorement, I find it is because this line of code:
https://github.com/ankanbansal/ISVQA-Dataset/blob/master/gibson/data_generation/generate_image_sets_from_info_files.py#L64

Actually, env.reset() does return the next episode, but the iterator is not from env.episodes, so the assignment to env.episodes is furitless.

Fix it

I replace:

observations = env.reset()

with the following code:

env._reset_stats()
assert len(env.episodes) > 0, "Episodes list is empty"
if env._current_episode is not None:
    env._current_episode._shortest_path_cache = None
env._current_episode = temp[i]
env.reconfigure(env._config)
observations = env.task.reset(episode=env.current_episode)
env._task.measurements.reset_measures(
    episode=env.current_episode, task=env.task
)

and it seems works well. The reference is from here -- the source code of habitat. In one word, we should set env._current_episode and leave all other things to original API.