Giter Site home page Giter Site logo

ONNX Model Availability about spiga HOT 15 CLOSED

john09282922 avatar john09282922 commented on May 28, 2024
ONNX Model Availability

from spiga.

Comments (15)

andresprados avatar andresprados commented on May 28, 2024

Hi @john09282922,

Not yet, I believe that it shouldn't be a complex conversion since most of the layers are common ones. Maybe the visual feature cropping used as GNN input could not be supported but is the same method used by well knonw models like MaskRCNN.

Depending on the impact of the repository, we will consider further improvements of it surch as releasing ONNX models. In the meantime, feel free to submit a PR if you do the conversion.

Best,
Andrés

from spiga.

john09282922 avatar john09282922 commented on May 28, 2024

from spiga.

andresprados avatar andresprados commented on May 28, 2024

Sry Jungmin, it is just a guess of a possible problem since it is a sort of "dynamic operation" and ONNX usually doesnt like them. I havent try to convert the model so I cannot know if it is even a problem.

Let me know which exact error are you facing. Maybe I can help.

from spiga.

john09282922 avatar john09282922 commented on May 28, 2024

from spiga.

andresprados avatar andresprados commented on May 28, 2024

Jackpot! Your are facing a conversion problem related to the affine transformation used to crop the visual features of the GNN input. This operation is used by Spatial Transformer Networks, take a look at this issue, it seems to be already solved.

from spiga.

john09282922 avatar john09282922 commented on May 28, 2024

from spiga.

john09282922 avatar john09282922 commented on May 28, 2024

from spiga.

john09282922 avatar john09282922 commented on May 28, 2024

from spiga.

edmosRovi avatar edmosRovi commented on May 28, 2024

Hello,

I am running into the same problem, and I used the affine_grad reimplementation from pytorch/pytorch#30563.

I was able to convert the model to ONNX using opset 16, but when I used it with onnxruntime, the session creation failed:
"Fail: [ONNXRuntimeError] : 1 : FAIL : Load model from model.onnx failed:Node (/Squeeze) Op (Squeeze) [ShapeInferenceError] Dimension of input 1 must be 1 instead of 2".

Any idea how I can work with the model in onnxruntime? My ultimate goal is to generate a tensorrt engine from the model, but I am doing it step by step first.

Thanks.

from spiga.

john09282922 avatar john09282922 commented on May 28, 2024

Hello,

I am running into the same problem, and I used the affine_grad reimplementation from pytorch/pytorch#30563.

I was able to convert the model to ONNX using opset 16, but when I used it with onnxruntime, the session creation failed: "Fail: [ONNXRuntimeError] : 1 : FAIL : Load model from model.onnx failed:Node (/Squeeze) Op (Squeeze) [ShapeInferenceError] Dimension of input 1 must be 1 instead of 2".

Any idea how I can work with the model in onnxruntime? My ultimate goal is to generate a tensorrt engine from the model, but I am doing it step by step first.

Thanks.

Hi, could you share how to change affine grid reimplementation?
and did you set up input correctly?

from spiga.

edmosRovi avatar edmosRovi commented on May 28, 2024

Hi,

To change affine grid, I did it in "spiga\models\spiga.py", in "extract_visual_embedded", replaces "grid = torch.nn.functional.affine_grid(theta, (B * L, C, self.kwindow, self.kwindow))" with the call to the reimplemented method.

To set up the input, I did this:

model_cfg = ModelConfig('merlrav')
loader_3DM = AddModel3D(model_cfg.dataset.ldm_ids,
ftmap_size=model_cfg.ftmap_size,
focal_ratio=model_cfg.focal_ratio,
totensor=True)
params_3DM = _data2device(loader_3DM())
model3d = params_3DM['model3d']
cam_matrix = params_3DM['cam_matrix']
image = cv2.imread('img.jpg')
bboxes = [[274.84974931074015, 128.89023861421734, 203.57020487055968, 295.73514067406427]]
inputs, crop_bboxes = pretreat(image, bboxes)

Unfortunately, I don't think this is a proper way to do it, as I can't use the onnx model in any way. Neither with OpenCV DNN, nor with ONNXRuntime, nor TensorRT.

Any idea how it could be done?

from spiga.

john09282922 avatar john09282922 commented on May 28, 2024

Hi,

To change affine grid, I did it in "spiga\models\spiga.py", in "extract_visual_embedded", replaces "grid = torch.nn.functional.affine_grid(theta, (B * L, C, self.kwindow, self.kwindow))" with the call to the reimplemented method.

To set up the input, I did this:

model_cfg = ModelConfig('merlrav') loader_3DM = AddModel3D(model_cfg.dataset.ldm_ids, ftmap_size=model_cfg.ftmap_size, focal_ratio=model_cfg.focal_ratio, totensor=True) params_3DM = _data2device(loader_3DM()) model3d = params_3DM['model3d'] cam_matrix = params_3DM['cam_matrix'] image = cv2.imread('img.jpg') bboxes = [[274.84974931074015, 128.89023861421734, 203.57020487055968, 295.73514067406427]] inputs, crop_bboxes = pretreat(image, bboxes)

Unfortunately, I don't think this is a proper way to do it, as I can't use the onnx model in any way. Neither with OpenCV DNN, nor with ONNXRuntime, nor TensorRT.

Any idea how it could be done?

did u check that original code's input and your input size is same?
anyway, which lines did you change to export spiga model to onnx ?

from spiga.

SaShukla090 avatar SaShukla090 commented on May 28, 2024

Did anybody able to generate the ONNX model of SPIGA, which successfully runs on ONNX runtime or TensorRT?. Please reply and let us know your changes to convert the model to ONNX type. Any help will be appreciated!

from spiga.

lokeycookie avatar lokeycookie commented on May 28, 2024

Someone posted to use dynamo_export here. However, i got the following error:

SPIGA model loaded!
/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/nn/functional.py:4358: UserWarning: Default grid_sample and affine_grid behavior has changed to align_corners=False since 1.3.0. Please specify align_corners=True if the old behavior is desired. See the documentation of grid_sample for details.
warnings.warn(
/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/nn/functional.py:4296: UserWarning: Default grid_sample and affine_grid behavior has changed to align_corners=False since 1.3.0. Please specify align_corners=True if the old behavior is desired. See the documentation of grid_sample for details.
warnings.warn(
EXPORTING!!!!!!!
/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/onnx/_internal/exporter.py:130: UserWarning: torch.onnx.dynamo_export only implements opset version 18 for now. If you need to use a different opset version, please register them with register_custom_op.
warnings.warn(
/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/nn/functional.py:4358: UserWarning: Default grid_sample and affine_grid behavior has changed to align_corners=False since 1.3.0. Please specify align_corners=True if the old behavior is desired. See the documentation of grid_sample for details.
warnings.warn(
/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/nn/functional.py:4296: UserWarning: Default grid_sample and affine_grid behavior has changed to align_corners=False since 1.3.0. Please specify align_corners=True if the old behavior is desired. See the documentation of grid_sample for details.
warnings.warn(
Traceback (most recent call last):
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 1413, in run_node
return getattr(args[0], node.target)(*args[1:], **kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/utils/_stats.py", line 20, in wrapper
return fn(*args, **kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_subclasses/fake_tensor.py", line 1299, in torch_dispatch
return self.dispatch(func, types, args, kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_subclasses/fake_tensor.py", line 1504, in dispatch
return decomposition_table[func](args, **kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_refs/init.py", line 4445, in view
return _reshape_view_helper(a, shape, allow_copy=False)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_refs/init.py", line 3578, in reshape_view_helper
shape = utils.infer_size(shape, a.numel())
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/prims_common/init.py", line 813, in infer_size
numel == newsize or (dim is not None and newsize > 0 and numel % newsize == 0),
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/init.py", line 352, in bool
return self.node.bool
()
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/fx/experimental/symbolic_shapes.py", line 1003, in bool

return self.guard_bool("", 0)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/fx/experimental/symbolic_shapes.py", line 985, in guard_bool
r = self.shape_env.evaluate_expr(self.expr, self.hint, fx_node=self.fx_node)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/fx/experimental/symbolic_shapes.py", line 3512, in evaluate_expr
concrete_val = self.size_hint(orig_expr)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/fx/experimental/symbolic_shapes.py", line 3313, in size_hint
raise self._make_data_dependent_error(result_expr, expr)
torch.fx.experimental.symbolic_shapes.GuardOnDataDependentSymNode: It appears that you're trying to get a value out of symbolic int/float whose value is data-dependent (and thus we do not know the true value.) The expression we were trying to evaluate is Eq(2
i0, 98) (unhinted: Eq(2
i0, 98)). Scroll up to see where each of these data-dependent accesses originally occurred.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 1350, in get_fake_value
return wrap_fake_exception(
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 925, in wrap_fake_exception
return fn()
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 1351, in
lambda: run_node(tx.output, node, args, kwargs, nnmodule)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 1430, in run_node
raise RuntimeError(fn_str + str(e)).with_traceback(e.traceback) from e
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 1413, in run_node
return getattr(args[0], node.target)(*args[1:], **kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/utils/_stats.py", line 20, in wrapper
return fn(args, **kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_subclasses/fake_tensor.py", line 1299, in torch_dispatch
return self.dispatch(func, types, args, kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_subclasses/fake_tensor.py", line 1504, in dispatch
return decomposition_table[func](args, **kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_refs/init.py", line 4445, in view
return _reshape_view_helper(a, shape, allow_copy=False)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_refs/init.py", line 3578, in reshape_view_helper
shape = utils.infer_size(shape, a.numel())
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/prims_common/init.py", line 813, in infer_size
numel == newsize or (dim is not None and newsize > 0 and numel % newsize == 0),
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/init.py", line 352, in bool
return self.node.bool
()
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/fx/experimental/symbolic_shapes.py", line 1003, in bool

return self.guard_bool("", 0)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/fx/experimental/symbolic_shapes.py", line 985, in guard_bool
r = self.shape_env.evaluate_expr(self.expr, self.hint, fx_node=self.fx_node)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/fx/experimental/symbolic_shapes.py", line 3512, in evaluate_expr
concrete_val = self.size_hint(orig_expr)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/fx/experimental/symbolic_shapes.py", line 3313, in size_hint
raise self._make_data_dependent_error(result_expr, expr)
RuntimeError: Failed running call_method reshape(
(FakeTensor(..., device='cuda:0', size=(1, i0, 2), grad_fn=), 1, 98, -1), **{}):
It appears that you're trying to get a value out of symbolic int/float whose value is data-dependent (and thus we do not know the true value.) The expression we were trying to evaluate is Eq(2i0, 98) (unhinted: Eq(2i0, 98)). Scroll up to see where each of these data-dependent accesses originally occurred.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/onnx/_internal/exporter.py", line 1195, in dynamo_export
).export()
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/onnx/_internal/exporter.py", line 941, in export
graph_module = self.options.fx_tracer.generate_fx(
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/onnx/_internal/fx/dynamo_graph_extractor.py", line 199, in generate_fx
graph_module, graph_guard = torch._dynamo.export(
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 1150, in inner
result_traced = opt_f(*args, **kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 338, in _fn
return fn(*args, **kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/onnx/_internal/fx/dynamo_graph_extractor.py", line 154, in wrapped
return output_adapter.apply(model_func(*args, **kwargs))
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 500, in catch_errors
return callback(frame, cache_entry, hooks, frame_state)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 140, in _fn
return fn(*args, **kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 380, in _convert_frame_assert
return _compile(
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 561, in _compile
guarded_code = compile_inner(code, one_graph, hooks, transform)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 197, in time_wrapper
r = func(*args, **kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 483, in compile_inner
out_code = transform_code_object(code, transform)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/bytecode_transformation.py", line 1028, in transform_code_object
transformations(instructions, code_options)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 449, in transform
tracer.run()
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2093, in run
super().run()
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 733, in run
and self.step()
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 696, in step
getattr(self, inst.opname)(inst)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 397, in wrapper
return inner_fn(self, inst)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 1124, in CALL_FUNCTION
self.call_function(fn, args, {})
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 570, in call_function
self.push(fn.call_function(self, args, kwargs))
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 307, in call_function
return super().call_function(tx, args, kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 261, in call_function
return super().call_function(tx, args, kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 90, in call_function
return tx.inline_user_function_return(
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 606, in inline_user_function_return
result = InliningInstructionTranslator.inline_call(self, fn, args, kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/dynamo/symbolic_convert.py", line 2210, in inline_call
return cls.inline_call
(parent, func, args, kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/dynamo/symbolic_convert.py", line 2326, in inline_call
tracer.run()
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 733, in run
and self.step()
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 696, in step
getattr(self, inst.opname)(inst)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 397, in wrapper
return inner_fn(self, inst)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 1124, in CALL_FUNCTION
self.call_function(fn, args, {})
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 570, in call_function
self.push(fn.call_function(self, args, kwargs))
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 307, in call_function
return super().call_function(tx, args, kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 261, in call_function
return super().call_function(tx, args, kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 90, in call_function
return tx.inline_user_function_return(
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 606, in inline_user_function_return
result = InliningInstructionTranslator.inline_call(self, fn, args, kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/dynamo/symbolic_convert.py", line 2210, in inline_call
return cls.inline_call
(parent, func, args, kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/dynamo/symbolic_convert.py", line 2326, in inline_call
tracer.run()
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 733, in run
and self.step()
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 696, in step
getattr(self, inst.opname)(inst)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 397, in wrapper
return inner_fn(self, inst)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 1124, in CALL_FUNCTION
self.call_function(fn, args, {})
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 570, in call_function
self.push(fn.call_function(self, args, kwargs))
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/variables/misc.py", line 594, in call_function
return self.obj.call_method(tx, self.name, args, kwargs).add_options(self)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/variables/tensor.py", line 652, in call_method
return wrap_fx_proxy(
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/variables/builder.py", line 1198, in wrap_fx_proxy
return wrap_fx_proxy_cls(
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/variables/builder.py", line 1285, in wrap_fx_proxy_cls
example_value = get_fake_value(proxy.node, tx)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 1379, in get_fake_value
raise UserError(
torch._dynamo.exc.UserError: Tried to use data-dependent value in the subsequent computation. This can happen when we encounter unbounded dynamic value that is unknown during tracing time.You will need to explicitly give hint to the compiler. Please take a look at constrain_as_value OR constrain_as_size APIs

from user code:
File "/home/evas/Downloads/SPIGA/SPIGA/spiga/models/spiga.py", line 82, in forward
embedded_ft = self.extract_embedded(pts_proj, visual_field, step)
File "/home/evas/Downloads/SPIGA/SPIGA/spiga/models/spiga.py", line 124, in extract_embedded
shape_ft = self.calculate_distances(pts_proj)
File "/home/evas/Downloads/SPIGA/SPIGA/spiga/models/spiga.py", line 163, in calculate_distances
dist_wo_self = dist[:, self.diagonal_mask, :].reshape(B, L, -1)

Set TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 for more information

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/home/evas/Downloads/SPIGA/SPIGA/app_CH.py", line 73, in
tracked_obj = processor.process_frame(frame, [faces[0]])
File "/home/evas/Downloads/SPIGA/SPIGA/spiga/demo/analyze/extract/spiga_processor.py", line 35, in process_frame
features = self.processor.inference(frame, bboxes)
File "/home/evas/Downloads/SPIGA/SPIGA/spiga/inference/framework.py", line 68, in inference
outputs = self.net_forward(batch_crops)
File "/home/evas/Downloads/SPIGA/SPIGA/spiga/inference/framework.py", line 97, in net_forward
torch.onnx.dynamo_export(self.model, inputs).save("wflw.onnx")
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/onnx/_internal/exporter.py", line 1206, in dynamo_export
raise OnnxExporterError(

what i add to inference/framework.py

def net_forward(self, inputs):
        outputs = self.model(inputs)
        print("EXPORTING!!!!!!!")
        export_options = torch.onnx.ExportOptions(dynamic_shapes=True)
        torch.onnx.dynamo_export(self.model, inputs).save("wflw.onnx")
        return outputs

from spiga.

zhenyi1998 avatar zhenyi1998 commented on May 28, 2024

Someone posted to use dynamo_export here. However, i got the following error:

SPIGA model loaded!
/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/nn/functional.py:4358: UserWarning: Default grid_sample and affine_grid behavior has changed to align_corners=False since 1.3.0. Please specify align_corners=True if the old behavior is desired. See the documentation of grid_sample for details.
warnings.warn(
/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/nn/functional.py:4296: UserWarning: Default grid_sample and affine_grid behavior has changed to align_corners=False since 1.3.0. Please specify align_corners=True if the old behavior is desired. See the documentation of grid_sample for details.
warnings.warn(
EXPORTING!!!!!!!
/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/onnx/_internal/exporter.py:130: UserWarning: torch.onnx.dynamo_export only implements opset version 18 for now. If you need to use a different opset version, please register them with register_custom_op.
warnings.warn(
/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/nn/functional.py:4358: UserWarning: Default grid_sample and affine_grid behavior has changed to align_corners=False since 1.3.0. Please specify align_corners=True if the old behavior is desired. See the documentation of grid_sample for details.
warnings.warn(
/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/nn/functional.py:4296: UserWarning: Default grid_sample and affine_grid behavior has changed to align_corners=False since 1.3.0. Please specify align_corners=True if the old behavior is desired. See the documentation of grid_sample for details.
warnings.warn(
Traceback (most recent call last):
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 1413, in run_node
return getattr(args[0], node.target)(*args[1:], **kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/utils/_stats.py", line 20, in wrapper
return fn(*args, **kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_subclasses/fake_tensor.py", line 1299, in torch_dispatch
return self.dispatch(func, types, args, kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_subclasses/fake_tensor.py", line 1504, in dispatch
return decomposition_table[func](_args, **kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_refs/init.py", line 4445, in view
return _reshape_view_helper(a, _shape, allow_copy=False)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_refs/init.py", line 3578, in reshape_view_helper
shape = utils.infer_size(shape, a.numel())
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/prims_common/init.py", line 813, in infer_size
numel == newsize or (dim is not None and newsize > 0 and numel % newsize == 0),
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/init.py", line 352, in bool
return self.node.bool
()
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/fx/experimental/symbolic_shapes.py", line 1003, in bool

return self.guard_bool("", 0)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/fx/experimental/symbolic_shapes.py", line 985, in guard_bool
r = self.shape_env.evaluate_expr(self.expr, self.hint, fx_node=self.fx_node)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/fx/experimental/symbolic_shapes.py", line 3512, in evaluate_expr
concrete_val = self.size_hint(orig_expr)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/fx/experimental/symbolic_shapes.py", line 3313, in size_hint
raise self._make_data_dependent_error(result_expr, expr)
torch.fx.experimental.symbolic_shapes.GuardOnDataDependentSymNode: It appears that you're trying to get a value out of symbolic int/float whose value is data-dependent (and thus we do not know the true value.) The expression we were trying to evaluate is Eq(2_i0, 98) (unhinted: Eq(2_i0, 98)). Scroll up to see where each of these data-dependent accesses originally occurred.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 1350, in get_fake_value
return wrap_fake_exception(
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 925, in wrap_fake_exception
return fn()
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 1351, in
lambda: run_node(tx.output, node, args, kwargs, nnmodule)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 1430, in run_node
raise RuntimeError(fn_str + str(e)).with_traceback(e.traceback) from e
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 1413, in run_node
return getattr(args[0], node.target)(*args[1:], **kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/utils/_stats.py", line 20, in wrapper
return fn(_args, **kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_subclasses/fake_tensor.py", line 1299, in torch_dispatch
return self.dispatch(func, types, args, kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_subclasses/fake_tensor.py", line 1504, in dispatch
return decomposition_table[func](_args, **kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_refs/init.py", line 4445, in view
return _reshape_view_helper(a, shape, allow_copy=False)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/refs/init.py", line 3578, in reshape_view_helper
shape = utils.infer_size(shape, a.numel())
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/prims_common/init.py", line 813, in infer_size
numel == newsize or (dim is not None and newsize > 0 and numel % newsize == 0),
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/init.py", line 352, in bool
return self.node.bool
()
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/fx/experimental/symbolic_shapes.py", line 1003, in bool

return self.guard_bool("", 0)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/fx/experimental/symbolic_shapes.py", line 985, in guard_bool
r = self.shape_env.evaluate_expr(self.expr, self.hint, fx_node=self.fx_node)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/fx/experimental/symbolic_shapes.py", line 3512, in evaluate_expr
concrete_val = self.size_hint(orig_expr)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/fx/experimental/symbolic_shapes.py", line 3313, in size_hint
raise self.make_data_dependent_error(result_expr, expr)
RuntimeError: Failed running call_method reshape(
(FakeTensor(..., device='cuda:0', size=(1, i0, 2), grad_fn=), 1, 98, -1), **{}):
It appears that you're trying to get a value out of symbolic int/float whose value is data-dependent (and thus we do not know the true value.) The expression we were trying to evaluate is Eq(2i0, 98) (unhinted: Eq(2i0, 98)). Scroll up to see where each of these data-dependent accesses originally occurred.

During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/onnx/_internal/exporter.py", line 1195, in dynamo_export
).export()
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/onnx/_internal/exporter.py", line 941, in export
graph_module = self.options.fx_tracer.generate_fx(
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/onnx/_internal/fx/dynamo_graph_extractor.py", line 199, in generate_fx
graph_module, graph_guard = torch._dynamo.export(
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 1150, in inner
result_traced = opt_f(*args, **kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 338, in _fn
return fn(*args, **kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/onnx/_internal/fx/dynamo_graph_extractor.py", line 154, in wrapped
return output_adapter.apply(model_func(*args, **kwargs))
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 500, in catch_errors
return callback(frame, cache_entry, hooks, frame_state)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 140, in _fn
return fn(*args, **kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 380, in _convert_frame_assert
return _compile(
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 561, in _compile
guarded_code = compile_inner(code, one_graph, hooks, transform)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 197, in time_wrapper
r = func(*args, **kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 483, in compile_inner
out_code = transform_code_object(code, transform)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/bytecode_transformation.py", line 1028, in transform_code_object
transformations(instructions, code_options)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 449, in transform
tracer.run()
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2093, in run
super().run()
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 733, in run
and self.step()
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 696, in step
getattr(self, inst.opname)(inst)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 397, in wrapper
return inner_fn(self, inst)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 1124, in CALL_FUNCTION
self.call_function(fn, args, {})
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 570, in call_function
self.push(fn.call_function(self, args, kwargs))
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 307, in call_function
return super().call_function(tx, args, kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 261, in call_function
return super().call_function(tx, args, kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 90, in call_function
return tx.inline_user_function_return(
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 606, in inline_user_function_return
result = InliningInstructionTranslator.inline_call(self, fn, args, kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/dynamo/symbolic_convert.py", line 2210, in inline_call
return cls.inline_call
(parent, func, args, kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/dynamo/symbolic_convert.py", line 2326, in inline_call
tracer.run()
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 733, in run
and self.step()
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 696, in step
getattr(self, inst.opname)(inst)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 397, in wrapper
return inner_fn(self, inst)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 1124, in CALL_FUNCTION
self.call_function(fn, args, {})
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 570, in call_function
self.push(fn.call_function(self, args, kwargs))
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 307, in call_function
return super().call_function(tx, args, kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 261, in call_function
return super().call_function(tx, args, kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 90, in call_function
return tx.inline_user_function_return(
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 606, in inline_user_function_return
result = InliningInstructionTranslator.inline_call(self, fn, args, kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/dynamo/symbolic_convert.py", line 2210, in inline_call
return cls.inline_call
(parent, func, args, kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/dynamo/symbolic_convert.py", line 2326, in inline_call
tracer.run()
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 733, in run
and self.step()
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 696, in step
getattr(self, inst.opname)(inst)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 397, in wrapper
return inner_fn(self, inst)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 1124, in CALL_FUNCTION
self.call_function(fn, args, {})
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 570, in call_function
self.push(fn.call_function(self, args, kwargs))
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/variables/misc.py", line 594, in call_function
return self.obj.call_method(tx, self.name, args, kwargs).add_options(self)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/variables/tensor.py", line 652, in call_method
return wrap_fx_proxy(
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/variables/builder.py", line 1198, in wrap_fx_proxy
return wrap_fx_proxy_cls(
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/variables/builder.py", line 1285, in wrap_fx_proxy_cls
example_value = get_fake_value(proxy.node, tx)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 1379, in get_fake_value
raise UserError(
torch._dynamo.exc.UserError: Tried to use data-dependent value in the subsequent computation. This can happen when we encounter unbounded dynamic value that is unknown during tracing time.You will need to explicitly give hint to the compiler. Please take a look at constrain_as_value OR constrain_as_size APIs
from user code:
File "/home/evas/Downloads/SPIGA/SPIGA/spiga/models/spiga.py", line 82, in forward
embedded_ft = self.extract_embedded(pts_proj, visual_field, step)
File "/home/evas/Downloads/SPIGA/SPIGA/spiga/models/spiga.py", line 124, in extract_embedded
shape_ft = self.calculate_distances(pts_proj)
File "/home/evas/Downloads/SPIGA/SPIGA/spiga/models/spiga.py", line 163, in calculate_distances
dist_wo_self = dist[:, self.diagonal_mask, :].reshape(B, L, -1)
Set TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 for more information
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/evas/Downloads/SPIGA/SPIGA/app_CH.py", line 73, in
tracked_obj = processor.process_frame(frame, [faces[0]])
File "/home/evas/Downloads/SPIGA/SPIGA/spiga/demo/analyze/extract/spiga_processor.py", line 35, in process_frame
features = self.processor.inference(frame, bboxes)
File "/home/evas/Downloads/SPIGA/SPIGA/spiga/inference/framework.py", line 68, in inference
outputs = self.net_forward(batch_crops)
File "/home/evas/Downloads/SPIGA/SPIGA/spiga/inference/framework.py", line 97, in net_forward
torch.onnx.dynamo_export(self.model, inputs).save("wflw.onnx")
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/onnx/_internal/exporter.py", line 1206, in dynamo_export
raise OnnxExporterError(

what i add to inference/framework.py

def net_forward(self, inputs):
        outputs = self.model(inputs)
        print("EXPORTING!!!!!!!")
        export_options = torch.onnx.ExportOptions(dynamic_shapes=True)
        torch.onnx.dynamo_export(self.model, inputs).save("wflw.onnx")
        return outputs

Have u managed to solve this issue?

from spiga.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.