Comments (15)
Hi @john09282922,
Not yet, I believe that it shouldn't be a complex conversion since most of the layers are common ones. Maybe the visual feature cropping used as GNN input could not be supported but is the same method used by well knonw models like MaskRCNN.
Depending on the impact of the repository, we will consider further improvements of it surch as releasing ONNX models. In the meantime, feel free to submit a PR if you do the conversion.
Best,
Andrés
from spiga.
from spiga.
Sry Jungmin, it is just a guess of a possible problem since it is a sort of "dynamic operation" and ONNX usually doesnt like them. I havent try to convert the model so I cannot know if it is even a problem.
Let me know which exact error are you facing. Maybe I can help.
from spiga.
from spiga.
Jackpot! Your are facing a conversion problem related to the affine transformation used to crop the visual features of the GNN input. This operation is used by Spatial Transformer Networks, take a look at this issue, it seems to be already solved.
from spiga.
from spiga.
from spiga.
from spiga.
Hello,
I am running into the same problem, and I used the affine_grad reimplementation from pytorch/pytorch#30563.
I was able to convert the model to ONNX using opset 16, but when I used it with onnxruntime, the session creation failed:
"Fail: [ONNXRuntimeError] : 1 : FAIL : Load model from model.onnx failed:Node (/Squeeze) Op (Squeeze) [ShapeInferenceError] Dimension of input 1 must be 1 instead of 2".
Any idea how I can work with the model in onnxruntime? My ultimate goal is to generate a tensorrt engine from the model, but I am doing it step by step first.
Thanks.
from spiga.
Hello,
I am running into the same problem, and I used the affine_grad reimplementation from pytorch/pytorch#30563.
I was able to convert the model to ONNX using opset 16, but when I used it with onnxruntime, the session creation failed: "Fail: [ONNXRuntimeError] : 1 : FAIL : Load model from model.onnx failed:Node (/Squeeze) Op (Squeeze) [ShapeInferenceError] Dimension of input 1 must be 1 instead of 2".
Any idea how I can work with the model in onnxruntime? My ultimate goal is to generate a tensorrt engine from the model, but I am doing it step by step first.
Thanks.
Hi, could you share how to change affine grid reimplementation?
and did you set up input correctly?
from spiga.
Hi,
To change affine grid, I did it in "spiga\models\spiga.py", in "extract_visual_embedded", replaces "grid = torch.nn.functional.affine_grid(theta, (B * L, C, self.kwindow, self.kwindow))" with the call to the reimplemented method.
To set up the input, I did this:
model_cfg = ModelConfig('merlrav')
loader_3DM = AddModel3D(model_cfg.dataset.ldm_ids,
ftmap_size=model_cfg.ftmap_size,
focal_ratio=model_cfg.focal_ratio,
totensor=True)
params_3DM = _data2device(loader_3DM())
model3d = params_3DM['model3d']
cam_matrix = params_3DM['cam_matrix']
image = cv2.imread('img.jpg')
bboxes = [[274.84974931074015, 128.89023861421734, 203.57020487055968, 295.73514067406427]]
inputs, crop_bboxes = pretreat(image, bboxes)
Unfortunately, I don't think this is a proper way to do it, as I can't use the onnx model in any way. Neither with OpenCV DNN, nor with ONNXRuntime, nor TensorRT.
Any idea how it could be done?
from spiga.
Hi,
To change affine grid, I did it in "spiga\models\spiga.py", in "extract_visual_embedded", replaces "grid = torch.nn.functional.affine_grid(theta, (B * L, C, self.kwindow, self.kwindow))" with the call to the reimplemented method.
To set up the input, I did this:
model_cfg = ModelConfig('merlrav') loader_3DM = AddModel3D(model_cfg.dataset.ldm_ids, ftmap_size=model_cfg.ftmap_size, focal_ratio=model_cfg.focal_ratio, totensor=True) params_3DM = _data2device(loader_3DM()) model3d = params_3DM['model3d'] cam_matrix = params_3DM['cam_matrix'] image = cv2.imread('img.jpg') bboxes = [[274.84974931074015, 128.89023861421734, 203.57020487055968, 295.73514067406427]] inputs, crop_bboxes = pretreat(image, bboxes)
Unfortunately, I don't think this is a proper way to do it, as I can't use the onnx model in any way. Neither with OpenCV DNN, nor with ONNXRuntime, nor TensorRT.
Any idea how it could be done?
did u check that original code's input and your input size is same?
anyway, which lines did you change to export spiga model to onnx ?
from spiga.
Did anybody able to generate the ONNX model of SPIGA, which successfully runs on ONNX runtime or TensorRT?. Please reply and let us know your changes to convert the model to ONNX type. Any help will be appreciated!
from spiga.
Someone posted to use dynamo_export
here. However, i got the following error:
SPIGA model loaded!
/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/nn/functional.py:4358: UserWarning: Default grid_sample and affine_grid behavior has changed to align_corners=False since 1.3.0. Please specify align_corners=True if the old behavior is desired. See the documentation of grid_sample for details.
warnings.warn(
/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/nn/functional.py:4296: UserWarning: Default grid_sample and affine_grid behavior has changed to align_corners=False since 1.3.0. Please specify align_corners=True if the old behavior is desired. See the documentation of grid_sample for details.
warnings.warn(
EXPORTING!!!!!!!
/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/onnx/_internal/exporter.py:130: UserWarning: torch.onnx.dynamo_export only implements opset version 18 for now. If you need to use a different opset version, please register them with register_custom_op.
warnings.warn(
/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/nn/functional.py:4358: UserWarning: Default grid_sample and affine_grid behavior has changed to align_corners=False since 1.3.0. Please specify align_corners=True if the old behavior is desired. See the documentation of grid_sample for details.
warnings.warn(
/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/nn/functional.py:4296: UserWarning: Default grid_sample and affine_grid behavior has changed to align_corners=False since 1.3.0. Please specify align_corners=True if the old behavior is desired. See the documentation of grid_sample for details.
warnings.warn(
Traceback (most recent call last):
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 1413, in run_node
return getattr(args[0], node.target)(*args[1:], **kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/utils/_stats.py", line 20, in wrapper
return fn(*args, **kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_subclasses/fake_tensor.py", line 1299, in torch_dispatch
return self.dispatch(func, types, args, kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_subclasses/fake_tensor.py", line 1504, in dispatch
return decomposition_table[func](args, **kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_refs/init.py", line 4445, in view
return _reshape_view_helper(a, shape, allow_copy=False)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_refs/init.py", line 3578, in reshape_view_helper
shape = utils.infer_size(shape, a.numel())
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/prims_common/init.py", line 813, in infer_size
numel == newsize or (dim is not None and newsize > 0 and numel % newsize == 0),
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/init.py", line 352, in bool
return self.node.bool()
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/fx/experimental/symbolic_shapes.py", line 1003, in bool
return self.guard_bool("", 0)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/fx/experimental/symbolic_shapes.py", line 985, in guard_bool
r = self.shape_env.evaluate_expr(self.expr, self.hint, fx_node=self.fx_node)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/fx/experimental/symbolic_shapes.py", line 3512, in evaluate_expr
concrete_val = self.size_hint(orig_expr)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/fx/experimental/symbolic_shapes.py", line 3313, in size_hint
raise self._make_data_dependent_error(result_expr, expr)
torch.fx.experimental.symbolic_shapes.GuardOnDataDependentSymNode: It appears that you're trying to get a value out of symbolic int/float whose value is data-dependent (and thus we do not know the true value.) The expression we were trying to evaluate is Eq(2i0, 98) (unhinted: Eq(2i0, 98)). Scroll up to see where each of these data-dependent accesses originally occurred.The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 1350, in get_fake_value
return wrap_fake_exception(
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 925, in wrap_fake_exception
return fn()
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 1351, in
lambda: run_node(tx.output, node, args, kwargs, nnmodule)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 1430, in run_node
raise RuntimeError(fn_str + str(e)).with_traceback(e.traceback) from e
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 1413, in run_node
return getattr(args[0], node.target)(*args[1:], **kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/utils/_stats.py", line 20, in wrapper
return fn(args, **kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_subclasses/fake_tensor.py", line 1299, in torch_dispatch
return self.dispatch(func, types, args, kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_subclasses/fake_tensor.py", line 1504, in dispatch
return decomposition_table[func](args, **kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_refs/init.py", line 4445, in view
return _reshape_view_helper(a, shape, allow_copy=False)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_refs/init.py", line 3578, in reshape_view_helper
shape = utils.infer_size(shape, a.numel())
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/prims_common/init.py", line 813, in infer_size
numel == newsize or (dim is not None and newsize > 0 and numel % newsize == 0),
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/init.py", line 352, in bool
return self.node.bool()
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/fx/experimental/symbolic_shapes.py", line 1003, in bool
return self.guard_bool("", 0)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/fx/experimental/symbolic_shapes.py", line 985, in guard_bool
r = self.shape_env.evaluate_expr(self.expr, self.hint, fx_node=self.fx_node)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/fx/experimental/symbolic_shapes.py", line 3512, in evaluate_expr
concrete_val = self.size_hint(orig_expr)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/fx/experimental/symbolic_shapes.py", line 3313, in size_hint
raise self._make_data_dependent_error(result_expr, expr)
RuntimeError: Failed running call_method reshape((FakeTensor(..., device='cuda:0', size=(1, i0, 2), grad_fn=), 1, 98, -1), **{}):
It appears that you're trying to get a value out of symbolic int/float whose value is data-dependent (and thus we do not know the true value.) The expression we were trying to evaluate is Eq(2i0, 98) (unhinted: Eq(2i0, 98)). Scroll up to see where each of these data-dependent accesses originally occurred.During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/onnx/_internal/exporter.py", line 1195, in dynamo_export
).export()
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/onnx/_internal/exporter.py", line 941, in export
graph_module = self.options.fx_tracer.generate_fx(
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/onnx/_internal/fx/dynamo_graph_extractor.py", line 199, in generate_fx
graph_module, graph_guard = torch._dynamo.export(
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 1150, in inner
result_traced = opt_f(*args, **kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 338, in _fn
return fn(*args, **kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/onnx/_internal/fx/dynamo_graph_extractor.py", line 154, in wrapped
return output_adapter.apply(model_func(*args, **kwargs))
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 500, in catch_errors
return callback(frame, cache_entry, hooks, frame_state)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 140, in _fn
return fn(*args, **kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 380, in _convert_frame_assert
return _compile(
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 561, in _compile
guarded_code = compile_inner(code, one_graph, hooks, transform)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 197, in time_wrapper
r = func(*args, **kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 483, in compile_inner
out_code = transform_code_object(code, transform)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/bytecode_transformation.py", line 1028, in transform_code_object
transformations(instructions, code_options)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 449, in transform
tracer.run()
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2093, in run
super().run()
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 733, in run
and self.step()
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 696, in step
getattr(self, inst.opname)(inst)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 397, in wrapper
return inner_fn(self, inst)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 1124, in CALL_FUNCTION
self.call_function(fn, args, {})
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 570, in call_function
self.push(fn.call_function(self, args, kwargs))
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 307, in call_function
return super().call_function(tx, args, kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 261, in call_function
return super().call_function(tx, args, kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 90, in call_function
return tx.inline_user_function_return(
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 606, in inline_user_function_return
result = InliningInstructionTranslator.inline_call(self, fn, args, kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/dynamo/symbolic_convert.py", line 2210, in inline_call
return cls.inline_call(parent, func, args, kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/dynamo/symbolic_convert.py", line 2326, in inline_call
tracer.run()
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 733, in run
and self.step()
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 696, in step
getattr(self, inst.opname)(inst)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 397, in wrapper
return inner_fn(self, inst)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 1124, in CALL_FUNCTION
self.call_function(fn, args, {})
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 570, in call_function
self.push(fn.call_function(self, args, kwargs))
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 307, in call_function
return super().call_function(tx, args, kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 261, in call_function
return super().call_function(tx, args, kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 90, in call_function
return tx.inline_user_function_return(
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 606, in inline_user_function_return
result = InliningInstructionTranslator.inline_call(self, fn, args, kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/dynamo/symbolic_convert.py", line 2210, in inline_call
return cls.inline_call(parent, func, args, kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/dynamo/symbolic_convert.py", line 2326, in inline_call
tracer.run()
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 733, in run
and self.step()
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 696, in step
getattr(self, inst.opname)(inst)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 397, in wrapper
return inner_fn(self, inst)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 1124, in CALL_FUNCTION
self.call_function(fn, args, {})
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 570, in call_function
self.push(fn.call_function(self, args, kwargs))
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/variables/misc.py", line 594, in call_function
return self.obj.call_method(tx, self.name, args, kwargs).add_options(self)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/variables/tensor.py", line 652, in call_method
return wrap_fx_proxy(
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/variables/builder.py", line 1198, in wrap_fx_proxy
return wrap_fx_proxy_cls(
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/variables/builder.py", line 1285, in wrap_fx_proxy_cls
example_value = get_fake_value(proxy.node, tx)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 1379, in get_fake_value
raise UserError(
torch._dynamo.exc.UserError: Tried to use data-dependent value in the subsequent computation. This can happen when we encounter unbounded dynamic value that is unknown during tracing time.You will need to explicitly give hint to the compiler. Please take a look at constrain_as_value OR constrain_as_size APIsfrom user code:
File "/home/evas/Downloads/SPIGA/SPIGA/spiga/models/spiga.py", line 82, in forward
embedded_ft = self.extract_embedded(pts_proj, visual_field, step)
File "/home/evas/Downloads/SPIGA/SPIGA/spiga/models/spiga.py", line 124, in extract_embedded
shape_ft = self.calculate_distances(pts_proj)
File "/home/evas/Downloads/SPIGA/SPIGA/spiga/models/spiga.py", line 163, in calculate_distances
dist_wo_self = dist[:, self.diagonal_mask, :].reshape(B, L, -1)Set TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 for more information
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/evas/Downloads/SPIGA/SPIGA/app_CH.py", line 73, in
tracked_obj = processor.process_frame(frame, [faces[0]])
File "/home/evas/Downloads/SPIGA/SPIGA/spiga/demo/analyze/extract/spiga_processor.py", line 35, in process_frame
features = self.processor.inference(frame, bboxes)
File "/home/evas/Downloads/SPIGA/SPIGA/spiga/inference/framework.py", line 68, in inference
outputs = self.net_forward(batch_crops)
File "/home/evas/Downloads/SPIGA/SPIGA/spiga/inference/framework.py", line 97, in net_forward
torch.onnx.dynamo_export(self.model, inputs).save("wflw.onnx")
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/onnx/_internal/exporter.py", line 1206, in dynamo_export
raise OnnxExporterError(
what i add to inference/framework.py
def net_forward(self, inputs):
outputs = self.model(inputs)
print("EXPORTING!!!!!!!")
export_options = torch.onnx.ExportOptions(dynamic_shapes=True)
torch.onnx.dynamo_export(self.model, inputs).save("wflw.onnx")
return outputs
from spiga.
Someone posted to use
dynamo_export
here. However, i got the following error:SPIGA model loaded!
/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/nn/functional.py:4358: UserWarning: Default grid_sample and affine_grid behavior has changed to align_corners=False since 1.3.0. Please specify align_corners=True if the old behavior is desired. See the documentation of grid_sample for details.
warnings.warn(
/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/nn/functional.py:4296: UserWarning: Default grid_sample and affine_grid behavior has changed to align_corners=False since 1.3.0. Please specify align_corners=True if the old behavior is desired. See the documentation of grid_sample for details.
warnings.warn(
EXPORTING!!!!!!!
/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/onnx/_internal/exporter.py:130: UserWarning: torch.onnx.dynamo_export only implements opset version 18 for now. If you need to use a different opset version, please register them with register_custom_op.
warnings.warn(
/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/nn/functional.py:4358: UserWarning: Default grid_sample and affine_grid behavior has changed to align_corners=False since 1.3.0. Please specify align_corners=True if the old behavior is desired. See the documentation of grid_sample for details.
warnings.warn(
/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/nn/functional.py:4296: UserWarning: Default grid_sample and affine_grid behavior has changed to align_corners=False since 1.3.0. Please specify align_corners=True if the old behavior is desired. See the documentation of grid_sample for details.
warnings.warn(
Traceback (most recent call last):
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 1413, in run_node
return getattr(args[0], node.target)(*args[1:], **kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/utils/_stats.py", line 20, in wrapper
return fn(*args, **kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_subclasses/fake_tensor.py", line 1299, in torch_dispatch
return self.dispatch(func, types, args, kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_subclasses/fake_tensor.py", line 1504, in dispatch
return decomposition_table[func](_args, **kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_refs/init.py", line 4445, in view
return _reshape_view_helper(a, _shape, allow_copy=False)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_refs/init.py", line 3578, in reshape_view_helper
shape = utils.infer_size(shape, a.numel())
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/prims_common/init.py", line 813, in infer_size
numel == newsize or (dim is not None and newsize > 0 and numel % newsize == 0),
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/init.py", line 352, in bool
return self.node.bool()
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/fx/experimental/symbolic_shapes.py", line 1003, in bool
return self.guard_bool("", 0)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/fx/experimental/symbolic_shapes.py", line 985, in guard_bool
r = self.shape_env.evaluate_expr(self.expr, self.hint, fx_node=self.fx_node)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/fx/experimental/symbolic_shapes.py", line 3512, in evaluate_expr
concrete_val = self.size_hint(orig_expr)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/fx/experimental/symbolic_shapes.py", line 3313, in size_hint
raise self._make_data_dependent_error(result_expr, expr)
torch.fx.experimental.symbolic_shapes.GuardOnDataDependentSymNode: It appears that you're trying to get a value out of symbolic int/float whose value is data-dependent (and thus we do not know the true value.) The expression we were trying to evaluate is Eq(2_i0, 98) (unhinted: Eq(2_i0, 98)). Scroll up to see where each of these data-dependent accesses originally occurred.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 1350, in get_fake_value
return wrap_fake_exception(
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 925, in wrap_fake_exception
return fn()
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 1351, in
lambda: run_node(tx.output, node, args, kwargs, nnmodule)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 1430, in run_node
raise RuntimeError(fn_str + str(e)).with_traceback(e.traceback) from e
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 1413, in run_node
return getattr(args[0], node.target)(*args[1:], **kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/utils/_stats.py", line 20, in wrapper
return fn(_args, **kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_subclasses/fake_tensor.py", line 1299, in torch_dispatch
return self.dispatch(func, types, args, kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_subclasses/fake_tensor.py", line 1504, in dispatch
return decomposition_table[func](_args, **kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_refs/init.py", line 4445, in view
return _reshape_view_helper(a, shape, allow_copy=False)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/refs/init.py", line 3578, in reshape_view_helper
shape = utils.infer_size(shape, a.numel())
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/prims_common/init.py", line 813, in infer_size
numel == newsize or (dim is not None and newsize > 0 and numel % newsize == 0),
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/init.py", line 352, in bool
return self.node.bool()
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/fx/experimental/symbolic_shapes.py", line 1003, in bool
return self.guard_bool("", 0)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/fx/experimental/symbolic_shapes.py", line 985, in guard_bool
r = self.shape_env.evaluate_expr(self.expr, self.hint, fx_node=self.fx_node)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/fx/experimental/symbolic_shapes.py", line 3512, in evaluate_expr
concrete_val = self.size_hint(orig_expr)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/fx/experimental/symbolic_shapes.py", line 3313, in size_hint
raise self.make_data_dependent_error(result_expr, expr)
RuntimeError: Failed running call_method reshape((FakeTensor(..., device='cuda:0', size=(1, i0, 2), grad_fn=), 1, 98, -1), **{}):
It appears that you're trying to get a value out of symbolic int/float whose value is data-dependent (and thus we do not know the true value.) The expression we were trying to evaluate is Eq(2i0, 98) (unhinted: Eq(2i0, 98)). Scroll up to see where each of these data-dependent accesses originally occurred.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/onnx/_internal/exporter.py", line 1195, in dynamo_export
).export()
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/onnx/_internal/exporter.py", line 941, in export
graph_module = self.options.fx_tracer.generate_fx(
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/onnx/_internal/fx/dynamo_graph_extractor.py", line 199, in generate_fx
graph_module, graph_guard = torch._dynamo.export(
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 1150, in inner
result_traced = opt_f(*args, **kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 338, in _fn
return fn(*args, **kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/onnx/_internal/fx/dynamo_graph_extractor.py", line 154, in wrapped
return output_adapter.apply(model_func(*args, **kwargs))
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 500, in catch_errors
return callback(frame, cache_entry, hooks, frame_state)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 140, in _fn
return fn(*args, **kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 380, in _convert_frame_assert
return _compile(
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 561, in _compile
guarded_code = compile_inner(code, one_graph, hooks, transform)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 197, in time_wrapper
r = func(*args, **kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 483, in compile_inner
out_code = transform_code_object(code, transform)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/bytecode_transformation.py", line 1028, in transform_code_object
transformations(instructions, code_options)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 449, in transform
tracer.run()
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2093, in run
super().run()
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 733, in run
and self.step()
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 696, in step
getattr(self, inst.opname)(inst)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 397, in wrapper
return inner_fn(self, inst)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 1124, in CALL_FUNCTION
self.call_function(fn, args, {})
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 570, in call_function
self.push(fn.call_function(self, args, kwargs))
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 307, in call_function
return super().call_function(tx, args, kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 261, in call_function
return super().call_function(tx, args, kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 90, in call_function
return tx.inline_user_function_return(
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 606, in inline_user_function_return
result = InliningInstructionTranslator.inline_call(self, fn, args, kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/dynamo/symbolic_convert.py", line 2210, in inline_call
return cls.inline_call(parent, func, args, kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/dynamo/symbolic_convert.py", line 2326, in inline_call
tracer.run()
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 733, in run
and self.step()
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 696, in step
getattr(self, inst.opname)(inst)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 397, in wrapper
return inner_fn(self, inst)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 1124, in CALL_FUNCTION
self.call_function(fn, args, {})
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 570, in call_function
self.push(fn.call_function(self, args, kwargs))
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 307, in call_function
return super().call_function(tx, args, kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 261, in call_function
return super().call_function(tx, args, kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 90, in call_function
return tx.inline_user_function_return(
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 606, in inline_user_function_return
result = InliningInstructionTranslator.inline_call(self, fn, args, kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/dynamo/symbolic_convert.py", line 2210, in inline_call
return cls.inline_call(parent, func, args, kwargs)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/dynamo/symbolic_convert.py", line 2326, in inline_call
tracer.run()
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 733, in run
and self.step()
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 696, in step
getattr(self, inst.opname)(inst)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 397, in wrapper
return inner_fn(self, inst)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 1124, in CALL_FUNCTION
self.call_function(fn, args, {})
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 570, in call_function
self.push(fn.call_function(self, args, kwargs))
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/variables/misc.py", line 594, in call_function
return self.obj.call_method(tx, self.name, args, kwargs).add_options(self)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/variables/tensor.py", line 652, in call_method
return wrap_fx_proxy(
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/variables/builder.py", line 1198, in wrap_fx_proxy
return wrap_fx_proxy_cls(
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/variables/builder.py", line 1285, in wrap_fx_proxy_cls
example_value = get_fake_value(proxy.node, tx)
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 1379, in get_fake_value
raise UserError(
torch._dynamo.exc.UserError: Tried to use data-dependent value in the subsequent computation. This can happen when we encounter unbounded dynamic value that is unknown during tracing time.You will need to explicitly give hint to the compiler. Please take a look at constrain_as_value OR constrain_as_size APIs
from user code:
File "/home/evas/Downloads/SPIGA/SPIGA/spiga/models/spiga.py", line 82, in forward
embedded_ft = self.extract_embedded(pts_proj, visual_field, step)
File "/home/evas/Downloads/SPIGA/SPIGA/spiga/models/spiga.py", line 124, in extract_embedded
shape_ft = self.calculate_distances(pts_proj)
File "/home/evas/Downloads/SPIGA/SPIGA/spiga/models/spiga.py", line 163, in calculate_distances
dist_wo_self = dist[:, self.diagonal_mask, :].reshape(B, L, -1)
Set TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 for more information
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/evas/Downloads/SPIGA/SPIGA/app_CH.py", line 73, in
tracked_obj = processor.process_frame(frame, [faces[0]])
File "/home/evas/Downloads/SPIGA/SPIGA/spiga/demo/analyze/extract/spiga_processor.py", line 35, in process_frame
features = self.processor.inference(frame, bboxes)
File "/home/evas/Downloads/SPIGA/SPIGA/spiga/inference/framework.py", line 68, in inference
outputs = self.net_forward(batch_crops)
File "/home/evas/Downloads/SPIGA/SPIGA/spiga/inference/framework.py", line 97, in net_forward
torch.onnx.dynamo_export(self.model, inputs).save("wflw.onnx")
File "/home/evas/anaconda3/envs/CH_test/lib/python3.10/site-packages/torch/onnx/_internal/exporter.py", line 1206, in dynamo_export
raise OnnxExporterError(what i add to inference/framework.py
def net_forward(self, inputs): outputs = self.model(inputs) print("EXPORTING!!!!!!!") export_options = torch.onnx.ExportOptions(dynamic_shapes=True) torch.onnx.dynamo_export(self.model, inputs).save("wflw.onnx") return outputs
Have u managed to solve this issue?
from spiga.
Related Issues (20)
- Onnx output and pytorch output are inconsistent HOT 6
- Fit FLAME model to Landmarks
- Performance improvement
- RuntimeError: Found no NVIDIA driver on your system HOT 1
- Importing retinaface causes all torch tensors to lack gradient function
- showing only the face with highest confidence HOT 2
- How to convert to onxx/tensorRT? HOT 1
- Running on CPU without using GPU. In case there is no CUDA compatibility. HOT 1
- On the issue of converting ONNX HOT 1
- Problems running in a thread on Linux/Mac (with solution)
- Have a problem in colab, UnpicklingError: invalid load key, '<' HOT 2
- UnpicklingError: While running in Local and Colab HOT 5
- 3D Head Translation
- Unable to generate an ONNX that works in ONNXRuntime HOT 5
- Live demo on HF
- Dog Face Cases (requesting the training script)
- Head pose estimation Only
- sort_tracker and retinaface not found HOT 1
- How to test an image by model? HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from spiga.