Paddle如何支持Resnet半精度推理？

[xiaoyewww]

Paddle如何支持Resnet半精度推理？

[vivienfanghuagood]

你好，关于fp16的resnet推理，请参考这个demo：https://github.com/PaddlePaddle/Paddle-Inference-Demo/tree/master/c%2B%2B/gpu/gpu_fp16

[Zheng-Bicheng]

你好，关于fp16的resnet推理，请参考这个demo：https://github.com/PaddlePaddle/Paddle-Inference-Demo/tree/master/c%2B%2B/gpu/gpu_fp16

大佬，只能用C++做推理吗？没有Python的API接口来进行推理吗？

[xiaoyewww]

@vivienfanghuagood 大佬，这个问题还能解答一下吗，我发现这里没法跑fp16的原因应该是type不统一，python上详细的接口提供吗？

[vivienfanghuagood]

你好，关于fp16的resnet推理，请参考这个demo：https://github.com/PaddlePaddle/Paddle-Inference-Demo/tree/master/c%2B%2B/gpu/gpu_fp16

大佬，只能用C++做推理吗？没有Python的API接口来进行推理吗？

有的，python端的demo: https://github.com/PaddlePaddle/Paddle-Inference-Demo/blob/master/python/gpu/resnet50/infer_resnet.py#L34

[vivienfanghuagood]

https://www.paddlepaddle.org.cn/inference/v2.6/guides/nv_gpu_infer/gpu_mixed_precision.html#paddle-gpu 也可以参考这个文档

[xiaoyewww]

https://www.paddlepaddle.org.cn/inference/v2.6/guides/nv_gpu_infer/gpu_mixed_precision.html#paddle-gpu 也可以参考这个文档

@vivienfanghuagood 感谢，可以转换出fp16的模型，但又有一个问题，我没法通过paddle.jit.load来加载这个模型吗：

# download resnet model
if not os.path.exists("ResNet50_infer"):
    os.system("wget https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNet50_infer.tar && tar -xf ResNet50_infer.tar && rm -rf ResNet50_infer.tar")

# generate fp16 model
path = "ResNet50_infer"
src_model = os.path.join(path,"inference.pdmodel")
src_params = os.path.join(path,"inference.pdiparams")
dst_model = os.path.join(path,"inference_fp16.pdmodel")
dst_params = os.path.join(path,"inference_fp16.pdiparams")
black_list = set()

convert_to_mixed_precision(
    src_model,     # fp32模型文件路径
    src_params,    # fp32权重文件路径
    dst_model,     # 混合精度模型文件保存路径
    dst_params,    # 混合精度权重文件保存路径
    PrecisionType.Half, # 转换精度，如 PrecisionType.Half
    PlaceType.GPU,      # 后端，如 PlaceType.GPU
    False,               # 保留输入输出精度信息，若为 True 则输入输出保留为 fp32 类型，否则转为 precision 类型
    black_list          # 黑名单列表，哪些 op 不需要进行精度类型转换
)

# paddle.set_device("gpu")
path_fp16 = os.path.join(path, "inference_fp16")
print(f"path_fp16: {path_fp16}")
model = paddle.jit.load(path_fp16)
model.eval()

报错如下：

/root/miniconda3/envs/paddle_onnx/lib/python3.9/site-packages/decorator.py:232: in fun
    return caller(func, *(extras + args), **kw)
/root/miniconda3/envs/paddle_onnx/lib/python3.9/site-packages/paddle/base/wrapped_decorator.py:26: in __impl__
    return wrapped_func(*args, **kwargs)
/root/miniconda3/envs/paddle_onnx/lib/python3.9/site-packages/paddle/base/framework.py:593: in __impl__
    return func(*args, **kwargs)
/root/miniconda3/envs/paddle_onnx/lib/python3.9/site-packages/paddle/jit/api.py:1584: in load
    return TranslatedLayer._construct(model_path, config)
/root/miniconda3/envs/paddle_onnx/lib/python3.9/site-packages/decorator.py:232: in fun
    return caller(func, *(extras + args), **kw)
/root/miniconda3/envs/paddle_onnx/lib/python3.9/site-packages/paddle/base/wrapped_decorator.py:26: in __impl__
    return wrapped_func(*args, **kwargs)
/root/miniconda3/envs/paddle_onnx/lib/python3.9/site-packages/paddle/base/framework.py:593: in __impl__
    return func(*args, **kwargs)
/root/miniconda3/envs/paddle_onnx/lib/python3.9/site-packages/paddle/jit/translated_layer.py:1444: in _construct
    persistable_vars = _construct_params_and_buffers(
/root/miniconda3/envs/paddle_onnx/lib/python3.9/site-packages/paddle/jit/translated_layer.py:874: in _construct_params_and_buffers
    var_dict = _load_persistable_vars_by_program(
/root/miniconda3/envs/paddle_onnx/lib/python3.9/site-packages/paddle/jit/translated_layer.py:714: in _load_persistable_vars_by_program
    all_var_names = _get_all_var_names(program_holder.train_program)
/root/miniconda3/envs/paddle_onnx/lib/python3.9/site-packages/paddle/jit/dy2static/partial_program.py:110: in __get__
    val = self.function(instance)
/root/miniconda3/envs/paddle_onnx/lib/python3.9/site-packages/paddle/jit/translated_layer.py:389: in train_program
    return self._append_backward_desc(self._infer_program_desc)
/root/miniconda3/envs/paddle_onnx/lib/python3.9/site-packages/decorator.py:232: in fun
    return caller(func, *(extras + args), **kw)
/root/miniconda3/envs/paddle_onnx/lib/python3.9/site-packages/paddle/base/wrapped_decorator.py:26: in __impl__
    return wrapped_func(*args, **kwargs)
/root/miniconda3/envs/paddle_onnx/lib/python3.9/site-packages/paddle/base/dygraph/base.py:68: in __impl__
    return func(*args, **kwargs)
/root/miniconda3/envs/paddle_onnx/lib/python3.9/site-packages/paddle/jit/translated_layer.py:604: in _append_backward_desc
    grad_info_map = backward.calc_gradient_helper(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

targets = [], inputs = [], target_gradients = [], no_grad_set = None

    def calc_gradient_helper(
        targets, inputs, target_gradients=None, no_grad_set=None
    ):
        '''
        Calculate gradient and return grad_info_map
        '''
        targets = _as_list(targets)
        inputs = _as_list(inputs)
        target_gradients = _as_list(target_gradients)
    
>       block = targets[0].block
E       IndexError: list index out of range

/root/miniconda3/envs/paddle_onnx/lib/python3.9/site-packages/paddle/base/backward.py:2472: IndexError

看上去没反向信息就不能加载了

[xiaoyewww]

@vivienfanghuagood 感谢，已经没有问题了。