airockchip / rknn-toolkit2 Goto Github PK
View Code? Open in Web Editor NEWLicense: BSD 3-Clause "New" or "Revised" License
rknn-toolkit2's People
Contributors
Stargazers
Watchers
Forkers
cuteld habibzadeh askovia how2flow tigersean xudingyi312 linuxfanfan toujifenzi mbohun ljx0305 dfyuan leslievon kotenev magicrodri cqhqp trq-up marsangus gaokangdaodi1 pelochus tang799319844 dongfobao zhenligod gziren bobe-wang scholtde matiandan dengmingd bjia56 noahbear digidecode lindsayshuo itoledo yuyun2000 yingjiancode hugoobox armsom 111wzg knifecutter palmip joyboycz suranyamirknn-toolkit2's Issues
OpenCV版本和RKNN_TOOLKIT2版本问题
requirements文件上显示opencv_python应不低于4.5.5.64，基于Python3.8我安装了opencv_python4.6.0.66，并成功安装rknn_toolkit2。但当我尝试导入RKNN时，发现报错：module ‘cv2‘ has no attribute ‘gapi_wip_gst_GStreamerPipeline‘。对此，我将opencv_python降级至4.2.0.34，并再次尝试导入RKNN，这次成功了。请问rknn_toolkit2对于opencv_python的版本要求究竟应该是多少？
When will version 2.0 be released
When will version 2.0 be released
no support yolov8-cls
I have a problem that instead of using yolov8 dectection, I want to use yolov8 cls. But I can't find any conversion instructions or anything related to yolov8-cls. I don't know if the toolkit supports it. Does it support conversion or deployment?
使用Python库RKNNLite推理视频， 应该经常调用rknn.release() 吗？
我使用RKNNLite库去推理一个视频流， 进行一次rknn.init_runtime 然后不断的推理，发现内存会一直上涨直到崩溃。
后来我改为每一帧都rknn.init_runtime， 推理完这一帧后release， 内存问题泄漏就没有了，
就是应该这样做吗？
怎样把rk3588 npu三个核的占用率拉满
我想测试下rk3588的npu三个核在占用率拉满的情况下的性能，请问跑什么程序可以做到把三个npu占用率全部拉满，目前最高尝试跑大语言模型，三个核最高占用为70%
自定义GPU算子测试时RKNN内部报错 (layout convert failed)
原issue: rockchip-linux/rknn-toolkit2#301
更新到2.0.0b0后报错内容发生变化：
firefly@zt-firefly ~/r/n/2.0.0b0 [SIGINT]> sudo cp ./librknnrt.so /usr/lib/;sudo ./rknn_server
start rknn server, version:2.0.0b0 (18eacd0 build@2024-03-22T14:07:19)
I NPUTransfer: Starting NPU Transfer Server, Transfer version 2.1.0 (b5861e7@2020-11-23T11:50:51)
arm_release_ver: g13p0-01eac0, rk_so_ver: 6
E RKNN: [19:25:37.399] OpenCL error with code CL_BUILD_PROGRAM_FAILURE.
E RKNN: [19:25:37.399] Program build error: <source>:47:10: error: use of undeclared identifier 'To8'
E RKNN: [19:25:37.399] vstore8((To8)(val[0].x, val[1].x, val[2].x, val[3].x, val[4].x, val[5].x, val[6].x, val[7].x), out_off,
E RKNN: [19:25:37.399] ^
E RKNN: [19:25:37.399] <source>:50:10: error: use of undeclared identifier 'To8'
E RKNN: [19:25:37.399] vstore8((To8)(val[0].y, val[1].y, val[2].y, val[3].y, val[4].y, val[5].y, val[6].y, val[7].y), out_off + out_str,
E RKNN: [19:25:37.399] ^
E RKNN: [19:25:37.399] <source>:54:10: error: use of undeclared identifier 'To8'
E RKNN: [19:25:37.399] vstore8((To8)(val[0].z, val[1].z, val[2].z, val[3].z, val[4].z, val[5].z, val[6].z, val[7].z),
E RKNN: [19:25:37.399] ^
E RKNN: [19:25:37.399] <source>:58:10: error: use of undeclared identifier 'To8'
E RKNN: [19:25:37.399] vstore8((To8)(val[0].w, val[1].w, val[2].w, val[3].w, val[4].w, val[5].w, val[6].w, val[7].w),
E RKNN: [19:25:37.399] ^
E RKNN: [19:25:37.399] <source>:111:10: error: use of undeclared identifier 'To8'
E RKNN: [19:25:37.399] vstore8((To8)(val[0].x, val[1].x, val[2].x, val[3].x, val[4].x, val[5].x, val[6].x, val[7].x), out_off,
E RKNN: [19:25:37.399] ^
E RKNN: [19:25:37.399] <source>:114:10: error: use of undeclared identifier 'To8'
E RKNN: [19:25:37.399] vstore8((To8)(val[0].y, val[1].y, val[2].y, val[3].y, val[4].y, val[5].y, val[6].y, val[7].y), out_off + out_str,
E RKNN: [19:25:37.399] ^
E RKNN: [19:25:37.399] <source>:118:10: error: use of undeclared identifier 'To8'
E RKNN: [19:25:37.399] vstore8((To8)(val[0].z, val[1].z, val[2].z, val[3].z, val[4].z, val[5].z, val[6].z, val[7].z),
E RKNN: [19:25:37.399] ^
E RKNN: [19:25:37.399] <source>:122:10: error: use of undeclared identifier 'To8'
E RKNN: [19:25:37.399] vstore8((To8)(val[0].w, val[1].w, val[2].w, val[3].w, val[4].w, val[5].w, val[6].w, val[7].w),
E RKNN: [19:25:37.399] ^
E RKNN: [19:25:37.399] error: Compiler frontend failed (error code 63)
E RKNN: [19:25:37.399] OpenCL error with code CL_INVALID_PROGRAM_EXECUTABLE.
E RKNN: [19:25:37.399] Add pack gpu op kernel failed, file_name = buffer/pack_nchw_to_nchwc8_buf.cl, kernel_name = pack_nchw_to_nchwc8, build_options = -D Ti=float -D Ti4=float4 -D To=half -D To4=half4
E RKNN: [19:25:37.399] layout convert failed
E RKNN: [19:25:37.399] Op: 'cstInstanceNormalization_1_256_64_64:/model/model.8/InstanceNormalization' init failed with a return value of -1
E RKNN: [19:25:37.400] Op type: 'cstInstanceNormalization_1_256_64_64' register custom op failed for gpu device !
10132 SERVER init(207): rknn_register_custom_ops fail! ret = -1
10132 SERVER process_msg_init(379): Client 0 init model fail!
这次是pack_nchw_to_nchwc8_buf.cl报错了。希望能尽快修复。
可否提供一个1.6的分支
对于stable版本，可否提供一个1.6的分支
Request for method to access input dtypes, name, shape for rknnlite
I know the Cpp implementation we are able to query the name, dims, shape, n_elems, size, fmt etc. I would like rknnlite.api to have this method too.
FastSAM convert failed
convert FastSAM-s.onnx of https://github.com/CASIA-IVA-Lab/FastSAM failed.
error message
...
W RKNN: [17:34:40.242] Failed to config layer: 'Conv:/model.16/conv/Conv' using 3Core fallback to single core mode,
W RKNN: [17:34:40.242] core_num 3 ori_Ih 80 ori_Iw 80 ori_Ic 128 ori_Ib 1
W RKNN: [17:34:40.242] ori_Kh 3 ori_Kw 3 ori_Kk 128 ori_Kc 128 ori_Ksx 2 ori_Ksy 2
W RKNN: [17:34:40.242] ori_Oh 40 oriOw 40 oriOc 128 pad_t 1 pad_b 0 pad_l 1 pad_r 0,
W RKNN: [17:34:40.242] Please help report this bug!
D RKNN: [17:34:40.245] DatainEntries overflow, need to tiling more, datain_entries: 4200, limitation: 2048
...
E RKNN: [17:34:40.458] failed to config argb mode layer!
Trouble with install req.txt and whl file for my RK3588
Hello, may i ask about help - when i try install lib for working in future with yolov7 on step install requirements - i have trouble and don`t know what to do. I try diff version of python, updated diff lib etc.
i have ubuntu 20.04, python 64 (not 32), rk 3588 aarch64
Building wheel for onnxoptimizer (pyproject.toml) ... error
error: subprocess-exited-with-error
× Building wheel for onnxoptimizer (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [82 lines of output]
fatal: not a git repository (or any parent up to mount point /)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
/tmp/pip-build-env-0zivm9u6/overlay/lib/python3.10/site-packages/setuptools/dist.py:476: SetuptoolsDeprecationWarning: Invalid dash-separated options
!!
********************************************************************************
Usage of dash-separated 'license-file' will not be supported in future
versions. Please use the underscore name 'license_file' instead.
By 2024-Sep-26, you need to update your project and remove deprecated calls
or your builds will no longer be supported.
See https://setuptools.pypa.io/en/latest/userguide/declarative_config.html for details.
********************************************************************************
!!
opt = self.warn_dash_deprecation(opt, section)
running bdist_wheel
running build
running build_py
running create_version
running cmake_build
CMake Error at CMakeLists.txt:1 (cmake_minimum_required):
CMake 3.22 or higher is required. You are running version 3.16.3
-- Configuring incomplete, errors occurred!
Traceback (most recent call last):
File "/home/orangepi/test_rknn/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
main()
File "/home/orangepi/test_rknn/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "/home/orangepi/test_rknn/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 251, in build_wheel
return _build_backend().build_wheel(wheel_directory, config_settings,
File "/tmp/pip-build-env-0zivm9u6/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 410, in build_wheel
return self._build_with_temp_dir(
File "/tmp/pip-build-env-0zivm9u6/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 395, in _build_with_temp_dir
self.run_setup()
File "/tmp/pip-build-env-0zivm9u6/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 487, in run_setup
super().run_setup(setup_script=setup_script)
File "/tmp/pip-build-env-0zivm9u6/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 311, in run_setup
exec(code, locals())
File "<string>", line 320, in <module>
File "/tmp/pip-build-env-0zivm9u6/overlay/lib/python3.10/site-packages/setuptools/__init__.py", line 104, in setup
return distutils.core.setup(**attrs)
File "/tmp/pip-build-env-0zivm9u6/overlay/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 185, in setup
return run_commands(dist)
File "/tmp/pip-build-env-0zivm9u6/overlay/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
dist.run_commands()
File "/tmp/pip-build-env-0zivm9u6/overlay/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
self.run_command(cmd)
File "/tmp/pip-build-env-0zivm9u6/overlay/lib/python3.10/site-packages/setuptools/dist.py", line 967, in run_command
super().run_command(command)
File "/tmp/pip-build-env-0zivm9u6/overlay/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
cmd_obj.run()
File "/tmp/pip-build-env-0zivm9u6/normal/lib/python3.10/site-packages/wheel/bdist_wheel.py", line 368, in run
self.run_command("build")
File "/tmp/pip-build-env-0zivm9u6/overlay/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
self.distribution.run_command(command)
File "/tmp/pip-build-env-0zivm9u6/overlay/lib/python3.10/site-packages/setuptools/dist.py", line 967, in run_command
super().run_command(command)
File "/tmp/pip-build-env-0zivm9u6/overlay/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
cmd_obj.run()
File "/tmp/pip-build-env-0zivm9u6/overlay/lib/python3.10/site-packages/setuptools/_distutils/command/build.py", line 131, in run
self.run_command(cmd_name)
File "/tmp/pip-build-env-0zivm9u6/overlay/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
self.distribution.run_command(command)
File "/tmp/pip-build-env-0zivm9u6/overlay/lib/python3.10/site-packages/setuptools/dist.py", line 967, in run_command
super().run_command(command)
File "/tmp/pip-build-env-0zivm9u6/overlay/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
cmd_obj.run()
File "<string>", line 216, in run
File "/tmp/pip-build-env-0zivm9u6/overlay/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
self.distribution.run_command(command)
File "/tmp/pip-build-env-0zivm9u6/overlay/lib/python3.10/site-packages/setuptools/dist.py", line 967, in run_command
super().run_command(command)
File "/tmp/pip-build-env-0zivm9u6/overlay/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
cmd_obj.run()
File "<string>", line 202, in run
File "/usr/lib/python3.10/subprocess.py", line 369, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/usr/bin/cmake', '-DPython_INCLUDE_DIR=/usr/include/python3.10', '-DPython_EXECUTABLE=/home/orangepi/test_rknn/bin/python3.10', '-DBUILD_ONNX_PYTHON=ON', '-DONNX_USE_LITE_PROTO=ON', '-DCMAKE_EXPORT_COMPILE_COMMANDS=ON', '-DONNX_NAMESPACE=onnx', '-DPY_EXT_SUFFIX=.cpython-310-aarch64-linux-gnu.so', '-DONNX_OPT_USE_SYSTEM_PROTOBUF=OFF', '-DCMAKE_BUILD_TYPE=Release', '-DONNX_ML=1', '/tmp/pip-install-eahlw5iq/onnxoptimizer_3ffb076dd8c4437882f687ded2028415']' returned non-zero exit status 1.
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for onnxoptimizer
Building wheel for psutil (pyproject.toml) ... error
error: subprocess-exited-with-error
× Building wheel for psutil (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [50 lines of output]
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-aarch64-cpython-310
creating build/lib.linux-aarch64-cpython-310/psutil
copying psutil/_pswindows.py -> build/lib.linux-aarch64-cpython-310/psutil
copying psutil/_pssunos.py -> build/lib.linux-aarch64-cpython-310/psutil
copying psutil/_psposix.py -> build/lib.linux-aarch64-cpython-310/psutil
copying psutil/_psosx.py -> build/lib.linux-aarch64-cpython-310/psutil
copying psutil/_pslinux.py -> build/lib.linux-aarch64-cpython-310/psutil
copying psutil/_psbsd.py -> build/lib.linux-aarch64-cpython-310/psutil
copying psutil/_psaix.py -> build/lib.linux-aarch64-cpython-310/psutil
copying psutil/_compat.py -> build/lib.linux-aarch64-cpython-310/psutil
copying psutil/_common.py -> build/lib.linux-aarch64-cpython-310/psutil
copying psutil/init.py -> build/lib.linux-aarch64-cpython-310/psutil
creating build/lib.linux-aarch64-cpython-310/psutil/tests
copying psutil/tests/test_windows.py -> build/lib.linux-aarch64-cpython-310/psutil/tests
copying psutil/tests/test_unicode.py -> build/lib.linux-aarch64-cpython-310/psutil/tests
copying psutil/tests/test_testutils.py -> build/lib.linux-aarch64-cpython-310/psutil/tests
copying psutil/tests/test_system.py -> build/lib.linux-aarch64-cpython-310/psutil/tests
copying psutil/tests/test_sunos.py -> build/lib.linux-aarch64-cpython-310/psutil/tests
copying psutil/tests/test_process_all.py -> build/lib.linux-aarch64-cpython-310/psutil/tests
copying psutil/tests/test_process.py -> build/lib.linux-aarch64-cpython-310/psutil/tests
copying psutil/tests/test_posix.py -> build/lib.linux-aarch64-cpython-310/psutil/tests
copying psutil/tests/test_osx.py -> build/lib.linux-aarch64-cpython-310/psutil/tests
copying psutil/tests/test_misc.py -> build/lib.linux-aarch64-cpython-310/psutil/tests
copying psutil/tests/test_memleaks.py -> build/lib.linux-aarch64-cpython-310/psutil/tests
copying psutil/tests/test_linux.py -> build/lib.linux-aarch64-cpython-310/psutil/tests
copying psutil/tests/test_contracts.py -> build/lib.linux-aarch64-cpython-310/psutil/tests
copying psutil/tests/test_connections.py -> build/lib.linux-aarch64-cpython-310/psutil/tests
copying psutil/tests/test_bsd.py -> build/lib.linux-aarch64-cpython-310/psutil/tests
copying psutil/tests/test_aix.py -> build/lib.linux-aarch64-cpython-310/psutil/tests
copying psutil/tests/runner.py -> build/lib.linux-aarch64-cpython-310/psutil/tests
copying psutil/tests/main.py -> build/lib.linux-aarch64-cpython-310/psutil/tests
copying psutil/tests/init.py -> build/lib.linux-aarch64-cpython-310/psutil/tests
running build_ext
building 'psutil._psutil_linux' extension
creating build/temp.linux-aarch64-cpython-310
creating build/temp.linux-aarch64-cpython-310/psutil
creating build/temp.linux-aarch64-cpython-310/psutil/arch
creating build/temp.linux-aarch64-cpython-310/psutil/arch/linux
aarch64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_SIZEOF_PID_T=4 -DPSUTIL_VERSION=598 -DPy_LIMITED_API=0x03060000 -DPSUTIL_LINUX=1 -I/home/orangepi/test_rknn/include -I/usr/include/python3.10 -c psutil/_psutil_common.c -o build/temp.linux-aarch64-cpython-310/psutil/_psutil_common.o
psutil/_psutil_common.c:9:10: fatal error: Python.h: No such file or directory
9 | #include <Python.h>
| ^~~~~~~~~~
compilation terminated.
psutil could not be installed from sources. Perhaps Python header files are not installed. Try running:
sudo apt-get install gcc python3-dev
error: command '/usr/bin/aarch64-linux-gnu-gcc' failed with exit code 1
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for psutil
Failed to build onnxoptimizer psutil
ERROR: Could not build wheels for onnxoptimizer, psutil, which is required to install pyproject.toml-based projects
where to find RKNPU kernel driver ?
where to find RKNPU kernel ?
especialy for 6.1+ kernel ?
rknn_input 结构体中的pass_through 释义有误
释义中解释到：
pass_through
/* 直通模式。
如果为 TRUE，则 buf 数据将直接传递到 rknn 模型的输入节点
无需任何转换。无需设置以下变量。
如果为 FALSE，则将 buf 数据转换为与模型一致的输入
根据以下类型和 FMT。所以以下变量(type,fmt)
需要设置。*/
但实际上不论pass_through是否为FALSE，type,fmt都需要设置，否则rknn_inputs_set接口会报错
可以提供opencv的C++，cv::mat格式的图像输入的示例代码？
使用2.0版本的rknn，关于图像输入，有没有opencv的C++版本？如下的yolov8_seg例子中的输入是一个自定义的结构体image_buffer_t ，可以提供cv::mat格式的图像输入的示例代码吗？
image_buffer_t src_image;
memset(&src_image, 0, sizeof(image_buffer_t));
ret = read_image(image_path, &src_image);
object_detect_result_list od_results;
ret = inference_yolov8_seg_model(&rknn_app_ctx, &src_image, &od_results);
Broken documentation link
Hello,
The link to the error codes is broken in the english pdf [1].
It works fine on the CN version of the same pdf [2]
[1] https://github.com/airockchip/rknn-toolkit2/blob/master/doc/03_Rockchip_RKNPU_API_Reference_RKNN_Toolkit2_V2.0.0beta0_EN.pdf
[2] https://github.com/airockchip/rknn-toolkit2/blob/master/doc/03_Rockchip_RKNPU_API_Reference_RKNN_Toolkit2_V2.0.0beta0_CN.pdf
Have a nice day !
"import pytorch" gives error on rk3588 board.
description
Running python in rk3588 gives error:
Traceback (most recent call last):
File "yolov8_lite_video.py", line 8, in
import torch
File "/home/khadas/miniconda3/envs/npu-env-1.4/lib/python3.8/site-packages/torch/init.py", line 1829, in
from torch import export as export
File "/home/khadas/miniconda3/envs/npu-env-1.4/lib/python3.8/site-packages/torch/export/init.py", line 29, in
from torch.fx.passes.infra.pass_base import PassResult
File "/home/khadas/miniconda3/envs/npu-env-1.4/lib/python3.8/site-packages/torch/fx/passes/init.py", line 3, in
from . import net_min_base
File "/home/khadas/miniconda3/envs/npu-env-1.4/lib/python3.8/site-packages/torch/fx/passes/net_min_base.py", line 11, in
from .split_utils import split_by_tags
File "/home/khadas/miniconda3/envs/npu-env-1.4/lib/python3.8/site-packages/torch/fx/passes/split_utils.py", line 8, in
from torch.fx.passes.utils import HolderModule, lift_subgraph_as_module
File "/home/khadas/miniconda3/envs/npu-env-1.4/lib/python3.8/site-packages/torch/fx/passes/utils/init.py", line 1, in
from .common import lift_subgraph_as_module, HolderModule, compare_graphs
File "/home/khadas/miniconda3/envs/npu-env-1.4/lib/python3.8/site-packages/torch/fx/passes/utils/common.py", line 7, in
from torch.fx.passes.utils.matcher_utils import SubgraphMatcher
File "/home/khadas/miniconda3/envs/npu-env-1.4/lib/python3.8/site-packages/torch/fx/passes/utils/matcher_utils.py", line 31, in
logger = _init_logger()
File "/home/khadas/miniconda3/envs/npu-env-1.4/lib/python3.8/site-packages/torch/fx/passes/utils/matcher_utils.py", line 21, in _init_logger
logger.setLevel(level)
File "/home/khadas/miniconda3/envs/npu-env-1.4/lib/python3.8/logging/init.py", line 1421, in setLevel
self.level = _checkLevel(level)
File "/home/khadas/miniconda3/envs/npu-env-1.4/lib/python3.8/logging/init.py", line 198, in _checkLevel
raise ValueError("Unknown level: %r" % level)
ValueError: Unknown level: 'WARNING'
The first line of the python code is as follows:
import cv2
import sys
import argparse
import numpy as np
import platform
from rknnlite.api import RKNNLite
import torch
I have tried to search for the error but still have no idea what's the potential solution. I try to downgrade python from version 3.9 to 3.8, but the error still occurs.
environment
conda list gives following result:
# Name Version Build Channel
_libgcc_mutex 0.1 main defaults
_openmp_mutex 5.1 51_gnu defaults
blas 1.0 openblas defaults
brotli-python 1.0.9 py38h419075a_7 defaults
bzip2 1.0.8 h998d150_5 defaults
c-ares 1.19.1 h998d150_0 defaults
ca-certificates 2024.3.11 hd43f75c_0 defaults
cairo 1.16.0 h537eab0_5 defaults
certifi 2024.2.2 py38hd43f75c_0 defaults
cffi 1.16.0 py38h998d150_0 defaults
charset-normalizer 2.0.4 pyhd3eb1b0_0 defaults
cyrus-sasl 2.1.28 h647bc0d_1 defaults
dbus 1.13.18 h821dc26_0 defaults
eigen 3.4.0 hb8fdbf2_0 defaults
expat 2.5.0 h419075a_0 defaults
ffmpeg 4.2.2 hdfaaa67_0 defaults
filelock 3.13.4 pypi_0 pypi
fontconfig 2.14.1 h652894d_2 defaults
freeglut 3.0.0 h22f4aa5_5 defaults
freetype 2.12.1 h6df46f4_0 defaults
fsspec 2024.3.1 pypi_0 pypi
future 0.18.3 py38hd43f75c_0 defaults
gettext 0.21.0 h0cce8dc_1 defaults
giflib 5.2.1 h998d150_3 defaults
glib 2.78.4 h419075a_0 defaults
glib-tools 2.78.4 h419075a_0 defaults
gmp 6.2.1 h22f4aa5_3 defaults
gnutls 3.6.15 hc6589d6_0 defaults
graphite2 1.3.14 h22f4aa5_1 defaults
gst-plugins-base 1.22.3 h94b7715_0 defaults
gst-plugins-good 1.22.3 h419075a_0 defaults
gstreamer 1.22.3 h998d150_0 defaults
harfbuzz 2.8.0 h905054b_0 defaults
hdf5 1.10.6 h8b20701_1 defaults
icu 68.1 h22f4aa5_0 defaults
idna 3.4 py38hd43f75c_0 defaults
jasper 2.0.14 hf85ac85_2 defaults
jinja2 3.1.3 pypi_0 pypi
joblib 1.2.0 py38hd43f75c_0 defaults
jpeg 9e h998d150_1 defaults
krb5 1.20.1 h2e2fba8_1 defaults
lame 3.100 hfd63f10_0 defaults
ld_impl_linux-aarch64 2.38 h8131f2d_1 defaults
lerc 3.0 h22f4aa5_0 defaults
libclang 14.0.6 default_hd3a980f_1 defaults
libclang13 14.0.6 default_h5e70c7c_1 defaults
libcups 2.4.2 hb788212_1 defaults
libcurl 8.5.0 hfa2bbb0_0 defaults
libdeflate 1.17 h998d150_1 defaults
libedit 3.1.20230828 h998d150_0 defaults
libev 4.33 hfd63f10_1 defaults
libevent 2.1.12 h6ac735f_1 defaults
libffi 3.4.4 h419075a_0 defaults
libgcc-ng 11.2.0 h1234567_1 defaults
libgfortran-ng 11.2.0 h6e398d7_1 defaults
libgfortran5 11.2.0 h1234567_1 defaults
libglib 2.78.4 hd439bcf_0 defaults
libglu 9.0.0 h22f4aa5_1 defaults
libgomp 11.2.0 h1234567_1 defaults [40/281]
libiconv 1.16 h2f4d8fa_2 defaults
libidn2 2.3.4 h998d150_0 defaults
libllvm14 14.0.6 hb8fdbf2_3 defaults
libnghttp2 1.57.0 hb788212_0 defaults
libogg 1.3.5 h2f4d8fa_1 defaults
libopenblas 0.3.21 hc2e42e2_0 defaults
libopencv 4.5.2 py38h3114f89_0 defaults
libopus 1.3.1 h2f4d8fa_0 defaults
libpng 1.6.39 h998d150_0 defaults
libpq 12.17 h6ac735f_0 defaults
libprotobuf 3.14.0 h549d06d_0 defaults
libssh2 1.10.0 h6ac735f_2 defaults
libstdcxx-ng 11.2.0 h1234567_1 defaults
libtasn1 4.19.0 h998d150_0 defaults
libtiff 4.5.1 h419075a_0 defaults
libunistring 0.9.10 h2f4d8fa_0 defaults
libuuid 1.41.5 h998d150_0 defaults
libvorbis 1.3.7 hfd63f10_0 defaults
libvpx 1.8.2 h7c1a80f_0 defaults
libwebp 1.3.2 he1bfee4_0 defaults
libwebp-base 1.3.2 h998d150_0 defaults
libxcb 1.15 h2f4d8fa_0 defaults
libxkbcommon 1.0.1 h998d150_1 defaults
libxml2 2.10.4 hd6958ba_0 defaults
libxslt 1.1.37 h4d22567_0 defaults
lz4-c 1.9.4 h419075a_0 defaults
markupsafe 2.1.5 pypi_0 pypi
mpg123 1.30.0 h419075a_1000 defaults
mpmath 1.3.0 pypi_0 pypi
mysql 5.7.24 h3140d82_2 defaults
ncurses 6.4 h419075a_0 defaults
nettle 3.7.3 h82288b7_1 defaults
networkx 3.1 pypi_0 pypi
ninja 1.10.2 hd43f75c_5 defaults
ninja-base 1.10.2 h59a28a9_5 defaults
nspr 4.35 h419075a_0 defaults
nss 3.89.1 h419075a_0 defaults
numpy 1.24.3 py38h8708280_0 defaults
numpy-base 1.24.3 py38h4a83355_0 defaults
opencv 4.5.2 py38hd43f75c_0 defaults
openh264 1.8.0 h22f4aa5_0 defaults
openssl 3.0.13 h2f4d8fa_0 defaults
packaging 23.2 py38hd43f75c_0 defaults
pcre2 10.42 hcfaa891_0 defaults
pip 23.3.1 py38hd43f75c_0 defaults
pixman 0.40.0 h2f4d8fa_1 defaults
platformdirs 3.10.0 py38hd43f75c_0 defaults
pooch 1.7.0 py38hd43f75c_0 defaults
psutil 5.9.8 pypi_0 pypi
py-opencv 4.5.2 py38h7100cfe_0 defaults
pycparser 2.21 pyhd3eb1b0_0 defaults
pysocks 1.7.1 py38hd43f75c_0 defaults
python 3.8.19 h4bb2201_0 defaults
pyyaml 6.0.1 py38h998d150_0 defaults
readline 8.2 h998d150_0 defaults
requests 2.31.0 py38hd43f75c_1 defaults
rknn-toolkit-lite2 2.0.0b0 pypi_0 pypi
ruamel-yaml 0.18.6 pypi_0 pypi
ruamel-yaml-clib 0.2.8 pypi_0 pypi
scikit-learn 1.3.0 py38hfb1e5ee_1 defaults
scipy 1.10.1 py38h7caaa05_1 defaults
setuptools 68.2.2 py38hd43f75c_0 defaults
sqlite 3.41.2 h998d150_0 defaults
sympy 1.12 pypi_0 pypi
threadpoolctl 2.2.0 pyh0d69192_0 defaults
tk 8.6.12 h241ca14_0 defaults
torch 2.2.2 pypi_0 pypi
typing-extensions 4.9.0 py38hd43f75c_1 defaults
typing_extensions 4.9.0 py38hd43f75c_1 defaults
tzdata 2024a h04d1e81_0 defaults
urllib3 2.1.0 py38hd43f75c_1 defaults
wheel 0.41.2 py38hd43f75c_0 defaults
x264 1!152.20180806 h2f4d8fa_0 defaults
xz 5.4.6 h998d150_0 defaults
yaml 0.2.5 hfd63f10_0 defaults
zlib 1.2.13 h998d150_0 defaults
zstd 1.5.5 h6a09583_0 defaults
and command
strings /usr/lib/librknnrt.so | grep -i "librknnrt"
gives
librknnrt.so
librknnrt version: 2.0.0b0 (35a6907d79@2024-03-24T10:31:14)
, but current librknnrt.so is support model version <=
Unsupport GPU op: %s in this librknnrt.so, please try to register custom op by calling rknn_register_custom_ops or %s
Unsupport CPU op: %s in this librknnrt.so, please try to register custom op by callingrknn_register_custom_ops or %s
Unsupport GPU op: %s in this librknnrt.so, %s
rknn_toolkit_lite2 feature requests
Hello! Thanks for the awesome inference library and NPU. I have recently been working on adding Rockchip NPU real-time object detection support to the Scrypted home video platform.
A couple feature requests that would help with rknn usability on Linux:
- It appears that rknn-toolkit queries
/proc/device-tree/compatiblefor the CPU model. In Docker, this path is only accessible if the container is created in privileged mode. Can this check be optional or done some other way to relax the privileged Docker requirement? - It appears that rknn-toolkit requires
librknnrt.soto exist under/usr/lib, my guess its loading is done with a dynamic dlopen somewhere in the Python native extensions. Can this library be bundled into rknn_toolkit_lite2 wheels (such as withauditwheel) so it doesn't need to be downloaded separately? Alternatively, can this library be loaded fromLD_LIBRARY_PATHso a process could be pointed to a different directory? Non-Docker Scrypted installations are not guaranteed to have write access to/usr/lib.
rknn_matmul_set_core_mask failed on RK3588
I'm running rknn_matmul_api_demo, I tried to run the demo in NPU core0 and core1 on RK3588, but failed
modified as below
int ret = rknn_matmul_create(&ctx, &info, &io_attr);
if (ret < 0)
{
fprintf(stderr, "rknn_matmul_create fail! ret=%d\n", ret);
return -1;
}
ret = rknn_matmul_set_core_mask(ctx, RKNN_NPU_CORE_0_1);
if (ret < 0)
{
fprintf(stderr, "rknn_matmul_coremask fail! ret=%d\n", ret);
return -1;
}
result
E RKNN: [18:09:24.926] Not support core mask: 3, fallback to single core auto mode
E RKNN: [18:09:24.926] NN Compiler/Model Version is 0.0.0 now
E RKNN: [18:09:24.926] rknn_set_core_mask: failed to set core mask: 3
rknn_matmul_coremask fail! ret=-1
同时使用动态输入和自定义算子功能时，模型会无法加载，且无任何报错信息，直接给出segmentation fault
如题，我所使用的模型如下
custom.zip
模型初始化时即报错，且不通过ret正常退出，C端报错如下
而这两个功能单独使用并不会报错，请修复
能不能提供openharmony平台对应的闭源so呢
rknn_query_cmd 结构体中的指令RKNN_QUERY_DEVICE_MEM_INFO，文档中没有相关任何结构体信息，rknn_query_cmd拿不到返回值
OK3588 rknn-toolkit2的docker源文件？
这些不同版本的docker在哪里呀？ CSDN上提供的百度网盘都消失了呀
rknn支持随机森林吗，完整的例子吗
rknn支持随机森林吗，有完整的例子吗
关于RKNNLite中inference 的input类型
在接口中看到
:param inputs: Input data List (ndarray list)
这个Inputs 是一个ndarray list， 这个ndarray 是一个int8 的 mat吗？ 或者其他类型？ 或者有什么文档能查？
The following operators are not implemented: ['aten::scaled_dot_product_attention']
I'm attempting to convert a large (really large, 26GB model) to RKNN and I came across this:
E load_pytorch: Traceback (most recent call last):
E load_pytorch: File "rknn/api/rknn_base.py", line 1616, in rknn.api.rknn_base.RKNNBase.load_pytorch
E load_pytorch: File "rknn/base/convertor/torch2onnx/pytorch.py", line 4574, in rknn.base.convertor.torch2onnx.pytorch.from_pytorch
E load_pytorch: File "rknn/base/convertor/torch2onnx/pytorch.py", line 3657, in rknn.base.convertor.torch2onnx.pytorch.PyTorchOpConverter.report_missing_conversion
E load_pytorch: File "rknn/api/rknn_log.py", line 92, in rknn.api.rknn_log.RKNNLog.e
E load_pytorch: ValueError: The following operators are not implemented: ['aten::scaled_dot_product_attention']
Is this supported by the RK3588? Will it be supported by the toolkit?
Support for RK3582
Is the RK3582 that is used in the Rock 5C Lite is compatible with rknn-toolkit2?
Hopefully someone here knows more about this.
我在调用rknn.load_pytorch时遇到了问题：E load_pytorch: ValueError: The following operators are not implemented: ['aten::copy_']
RK3588 YOLOv8-seg量化异常
RK3588板环境
系统版本：ubuntu 22.04
rknpu2-dev: 1.5.2
PC环境：
系统版本：ubuntu 22.04
python: 3.8
rknn-toolkit2: 1.5.2+b642f30c
问题描述：
1）YOLOv8-seg模型量化精度异常 2）rknn-toolkit2配置未生效
复现步骤：
模型：官方yolov8s-seg.pt模型https://github.com/ultralytics/assets/releases/download/v8.1.0/yolov8s-seg.pt
转换脚本：
from rknn.api import RKNN
# COCO
ONNX_MODEL = './yolov8s-seg.onnx'
RKNN_MODEL = './yolov8s-seg.rknn'
DATASET = './dataset/COCO/coco_subset_10.txt'
ANALYSIS_DATASET = ['./dataset/COCO/bus.jpg']
def step1():
# Create RKNN object
rknn = RKNN(verbose=True)
# pre-process config
print('--> Config model')
rknn.config(mean_values=[[0, 0, 0]],
std_values=[[255, 255, 255]],
quantized_algorithm='kl_divergence',
quantized_method='channel',
optimization_level=0,
target_platform="rk3588")
print('done')
# Load ONNX model
print('--> Loading model')
ret = rknn.load_onnx(model=ONNX_MODEL)
#ret = rknn.load_onnx(model=ONNX_MODEL, outputs=['326', '372', '418'])
if ret != 0:
print('Load model failed!')
exit(ret)
print('done')
# 调用 hybrid_quantization_step1 产生量化配置文件
print('Hybrid quantization step1')
ret = rknn.hybrid_quantization_step1(dataset=DATASET, rknn_batch_size=1, proposal=False)
if ret != 0:
print('Hybrid quantization step1 failed!')
exit(ret)
print('done')
rknn.release()
def step2():
# Create RKNN object
rknn = RKNN(verbose=True)
# Call hybrid_quantization_step2 to generate hybrid quantized RKNN model
print('Hybrid quantization step2')
ret = rknn.hybrid_quantization_step2(model_input='./yolov8s-seg.model',
data_input='yolov8s-seg.data',
model_quantization_cfg='yolov8s-seg.quantization.cfg')
if ret != 0:
print('Hybrid quantization step2 failed!')
exit(ret)
print('done')
# Export RKNN model
print('--> Export rknn model')
ret = rknn.export_rknn(RKNN_MODEL)
if ret != 0:
print('Export rknn model failed!')
exit(ret)
print('done')
# Accuracy analysis
print('--> Accuracy analysis')
ret = rknn.accuracy_analysis(inputs=ANALYSIS_DATASET,
target="RK3588", device_id="90ce0632eb5338b5")
if ret != 0:
print('Accuracy analysis failed!')
print('done')
rknn.release()
if __name__ == '__main__':
step1()
# step2()
混合量化步骤一输出日志：
╰─ /home/stardust/miniconda3/envs/rknn/bin/python /home/stardust/stardust_project/rknn-deploy/workspace/hybrid_convert.py
W __init__: rknn-toolkit2 version: 1.5.2+b642f30c
--> Config model
done
--> Loading model
Loading : 100%|████████████████████████████████████████████████| 162/162 [00:00<00:00, 17165.45it/s]
done
Hybrid quantization step1
I base_optimize ...
I base_optimize done.
I
I fold_constant ...
I fold_constant done.
I
I correct_ops ...
I correct_ops done.
I
I fuse_ops ...
I fuse_ops results:
I replace_exswish: remove node = ['/model.0/act/Sigmoid', '/model.0/act/Mul'], add node = ['/model.0/act/Sigmoid_2swish']
I replace_exswish: remove node = ['/model.1/act/Sigmoid', '/model.1/act/Mul'], add node = ['/model.1/act/Sigmoid_2swish']
I replace_exswish: remove node = ['/model.2/cv1/act/Sigmoid', '/model.2/cv1/act/Mul'], add node = ['/model.2/cv1/act/Sigmoid_2swish']
I replace_exswish: remove node = ['/model.2/m.0/cv1/act/Sigmoid', '/model.2/m.0/cv1/act/Mul'], add node = ['/model.2/m.0/cv1/act/Sigmoid_2swish']
I replace_exswish: remove node = ['/model.2/m.0/cv2/act/Sigmoid', '/model.2/m.0/cv2/act/Mul'], add node = ['/model.2/m.0/cv2/act/Sigmoid_2swish']
I replace_exswish: remove node = ['/model.2/cv2/act/Sigmoid', '/model.2/cv2/act/Mul'], add node = ['/model.2/cv2/act/Sigmoid_2swish']
I replace_exswish: remove node = ['/model.3/act/Sigmoid', '/model.3/act/Mul'], add node = ['/model.3/act/Sigmoid_2swish']
I replace_exswish: remove node = ['/model.4/cv1/act/Sigmoid', '/model.4/cv1/act/Mul'], add node = ['/model.4/cv1/act/Sigmoid_2swish']
I replace_exswish: remove node = ['/model.4/m.0/cv1/act/Sigmoid', '/model.4/m.0/cv1/act/Mul'], add node = ['/model.4/m.0/cv1/act/Sigmoid_2swish']
I replace_exswish: remove node = ['/model.4/m.0/cv2/act/Sigmoid', '/model.4/m.0/cv2/act/Mul'], add node = ['/model.4/m.0/cv2/act/Sigmoid_2swish']
I replace_exswish: remove node = ['/model.4/m.1/cv1/act/Sigmoid', '/model.4/m.1/cv1/act/Mul'], add node = ['/model.4/m.1/cv1/act/Sigmoid_2swish']
I replace_exswish: remove node = ['/model.4/m.1/cv2/act/Sigmoid', '/model.4/m.1/cv2/act/Mul'], add node = ['/model.4/m.1/cv2/act/Sigmoid_2swish']
I replace_exswish: remove node = ['/model.4/cv2/act/Sigmoid', '/model.4/cv2/act/Mul'], add node = ['/model.4/cv2/act/Sigmoid_2swish']
I replace_exswish: remove node = ['/model.5/act/Sigmoid', '/model.5/act/Mul'], add node = ['/model.5/act/Sigmoid_2swish']
I replace_exswish: remove node = ['/model.6/cv1/act/Sigmoid', '/model.6/cv1/act/Mul'], add node = ['/model.6/cv1/act/Sigmoid_2swish']
I replace_exswish: remove node = ['/model.6/m.0/cv1/act/Sigmoid', '/model.6/m.0/cv1/act/Mul'], add node = ['/model.6/m.0/cv1/act/Sigmoid_2swish']
I replace_exswish: remove node = ['/model.6/m.0/cv2/act/Sigmoid', '/model.6/m.0/cv2/act/Mul'], add node = ['/model.6/m.0/cv2/act/Sigmoid_2swish']
I replace_exswish: remove node = ['/model.6/m.1/cv1/act/Sigmoid', '/model.6/m.1/cv1/act/Mul'], add node = ['/model.6/m.1/cv1/act/Sigmoid_2swish']
I replace_exswish: remove node = ['/model.6/m.1/cv2/act/Sigmoid', '/model.6/m.1/cv2/act/Mul'], add node = ['/model.6/m.1/cv2/act/Sigmoid_2swish']
I replace_exswish: remove node = ['/model.6/cv2/act/Sigmoid', '/model.6/cv2/act/Mul'], add node = ['/model.6/cv2/act/Sigmoid_2swish']
I replace_exswish: remove node = ['/model.7/act/Sigmoid', '/model.7/act/Mul'], add node = ['/model.7/act/Sigmoid_2swish']
I replace_exswish: remove node = ['/model.8/cv1/act/Sigmoid', '/model.8/cv1/act/Mul'], add node = ['/model.8/cv1/act/Sigmoid_2swish']
I replace_exswish: remove node = ['/model.8/m.0/cv1/act/Sigmoid', '/model.8/m.0/cv1/act/Mul'], add node = ['/model.8/m.0/cv1/act/Sigmoid_2swish']
I replace_exswish: remove node = ['/model.8/m.0/cv2/act/Sigmoid', '/model.8/m.0/cv2/act/Mul'], add node = ['/model.8/m.0/cv2/act/Sigmoid_2swish']
I replace_exswish: remove node = ['/model.8/cv2/act/Sigmoid', '/model.8/cv2/act/Mul'], add node = ['/model.8/cv2/act/Sigmoid_2swish']
I replace_exswish: remove node = ['/model.9/cv1/act/Sigmoid', '/model.9/cv1/act/Mul'], add node = ['/model.9/cv1/act/Sigmoid_2swish']
I replace_exswish: remove node = ['/model.9/cv2/act/Sigmoid', '/model.9/cv2/act/Mul'], add node = ['/model.9/cv2/act/Sigmoid_2swish']
I replace_exswish: remove node = ['/model.12/cv1/act/Sigmoid', '/model.12/cv1/act/Mul'], add node = ['/model.12/cv1/act/Sigmoid_2swish']
I replace_exswish: remove node = ['/model.12/m.0/cv1/act/Sigmoid', '/model.12/m.0/cv1/act/Mul'], add node = ['/model.12/m.0/cv1/act/Sigmoid_2swish']
I replace_exswish: remove node = ['/model.12/m.0/cv2/act/Sigmoid', '/model.12/m.0/cv2/act/Mul'], add node = ['/model.12/m.0/cv2/act/Sigmoid_2swish']
I replace_exswish: remove node = ['/model.12/cv2/act/Sigmoid', '/model.12/cv2/act/Mul'], add node = ['/model.12/cv2/act/Sigmoid_2swish']
I replace_exswish: remove node = ['/model.15/cv1/act/Sigmoid', '/model.15/cv1/act/Mul'], add node = ['/model.15/cv1/act/Sigmoid_2swish']
I replace_exswish: remove node = ['/model.15/m.0/cv1/act/Sigmoid', '/model.15/m.0/cv1/act/Mul'], add node = ['/model.15/m.0/cv1/act/Sigmoid_2swish']
I replace_exswish: remove node = ['/model.15/m.0/cv2/act/Sigmoid', '/model.15/m.0/cv2/act/Mul'], add node = ['/model.15/m.0/cv2/act/Sigmoid_2swish']
I replace_exswish: remove node = ['/model.15/cv2/act/Sigmoid', '/model.15/cv2/act/Mul'], add node = ['/model.15/cv2/act/Sigmoid_2swish']
I replace_exswish: remove node = ['/model.16/act/Sigmoid', '/model.16/act/Mul'], add node = ['/model.16/act/Sigmoid_2swish']
I replace_exswish: remove node = ['/model.18/cv1/act/Sigmoid', '/model.18/cv1/act/Mul'], add node = ['/model.18/cv1/act/Sigmoid_2swish']
I replace_exswish: remove node = ['/model.18/m.0/cv1/act/Sigmoid', '/model.18/m.0/cv1/act/Mul'], add node = ['/model.18/m.0/cv1/act/Sigmoid_2swish']
I replace_exswish: remove node = ['/model.18/m.0/cv2/act/Sigmoid', '/model.18/m.0/cv2/act/Mul'], add node = ['/model.18/m.0/cv2/act/Sigmoid_2swish']
I replace_exswish: remove node = ['/model.18/cv2/act/Sigmoid', '/model.18/cv2/act/Mul'], add node = ['/model.18/cv2/act/Sigmoid_2swish']
I replace_exswish: remove node = ['/model.19/act/Sigmoid', '/model.19/act/Mul'], add node = ['/model.19/act/Sigmoid_2swish']
I replace_exswish: remove node = ['/model.21/cv1/act/Sigmoid', '/model.21/cv1/act/Mul'], add node = ['/model.21/cv1/act/Sigmoid_2swish']
I replace_exswish: remove node = ['/model.21/m.0/cv1/act/Sigmoid', '/model.21/m.0/cv1/act/Mul'], add node = ['/model.21/m.0/cv1/act/Sigmoid_2swish']
I replace_exswish: remove node = ['/model.21/m.0/cv2/act/Sigmoid', '/model.21/m.0/cv2/act/Mul'], add node = ['/model.21/m.0/cv2/act/Sigmoid_2swish']
I replace_exswish: remove node = ['/model.21/cv2/act/Sigmoid', '/model.21/cv2/act/Mul'], add node = ['/model.21/cv2/act/Sigmoid_2swish']
I replace_exswish: remove node = ['/model.22/cv4.2/cv4.2.0/act/Sigmoid', '/model.22/cv4.2/cv4.2.0/act/Mul'], add node = ['/model.22/cv4.2/cv4.2.0/act/Sigmoid_2swish']
I replace_exswish: remove node = ['/model.22/cv4.2/cv4.2.1/act/Sigmoid', '/model.22/cv4.2/cv4.2.1/act/Mul'], add node = ['/model.22/cv4.2/cv4.2.1/act/Sigmoid_2swish']
I replace_exswish: remove node = ['/model.22/cv2.2/cv2.2.0/act/Sigmoid', '/model.22/cv2.2/cv2.2.0/act/Mul'], add node = ['/model.22/cv2.2/cv2.2.0/act/Sigmoid_2swish']
I replace_exswish: remove node = ['/model.22/cv2.2/cv2.2.1/act/Sigmoid', '/model.22/cv2.2/cv2.2.1/act/Mul'], add node = ['/model.22/cv2.2/cv2.2.1/act/Sigmoid_2swish']
I replace_exswish: remove node = ['/model.22/cv3.2/cv3.2.0/act/Sigmoid', '/model.22/cv3.2/cv3.2.0/act/Mul'], add node = ['/model.22/cv3.2/cv3.2.0/act/Sigmoid_2swish']
I replace_exswish: remove node = ['/model.22/cv3.2/cv3.2.1/act/Sigmoid', '/model.22/cv3.2/cv3.2.1/act/Mul'], add node = ['/model.22/cv3.2/cv3.2.1/act/Sigmoid_2swish']
I replace_exswish: remove node = ['/model.22/cv4.1/cv4.1.0/act/Sigmoid', '/model.22/cv4.1/cv4.1.0/act/Mul'], add node = ['/model.22/cv4.1/cv4.1.0/act/Sigmoid_2swish']
I replace_exswish: remove node = ['/model.22/cv4.1/cv4.1.1/act/Sigmoid', '/model.22/cv4.1/cv4.1.1/act/Mul'], add node = ['/model.22/cv4.1/cv4.1.1/act/Sigmoid_2swish']
I replace_exswish: remove node = ['/model.22/cv2.1/cv2.1.0/act/Sigmoid', '/model.22/cv2.1/cv2.1.0/act/Mul'], add node = ['/model.22/cv2.1/cv2.1.0/act/Sigmoid_2swish']
I replace_exswish: remove node = ['/model.22/cv2.1/cv2.1.1/act/Sigmoid', '/model.22/cv2.1/cv2.1.1/act/Mul'], add node = ['/model.22/cv2.1/cv2.1.1/act/Sigmoid_2swish']
I replace_exswish: remove node = ['/model.22/cv3.1/cv3.1.0/act/Sigmoid', '/model.22/cv3.1/cv3.1.0/act/Mul'], add node = ['/model.22/cv3.1/cv3.1.0/act/Sigmoid_2swish']
I replace_exswish: remove node = ['/model.22/cv3.1/cv3.1.1/act/Sigmoid', '/model.22/cv3.1/cv3.1.1/act/Mul'], add node = ['/model.22/cv3.1/cv3.1.1/act/Sigmoid_2swish']
I replace_exswish: remove node = ['/model.22/proto/cv1/act/Sigmoid', '/model.22/proto/cv1/act/Mul'], add node = ['/model.22/proto/cv1/act/Sigmoid_2swish']
I replace_exswish: remove node = ['/model.22/proto/cv2/act/Sigmoid', '/model.22/proto/cv2/act/Mul'], add node = ['/model.22/proto/cv2/act/Sigmoid_2swish']
I replace_exswish: remove node = ['/model.22/proto/cv3/act/Sigmoid', '/model.22/proto/cv3/act/Mul'], add node = ['/model.22/proto/cv3/act/Sigmoid_2swish']
I replace_exswish: remove node = ['/model.22/cv4.0/cv4.0.0/act/Sigmoid', '/model.22/cv4.0/cv4.0.0/act/Mul'], add node = ['/model.22/cv4.0/cv4.0.0/act/Sigmoid_2swish']
I replace_exswish: remove node = ['/model.22/cv4.0/cv4.0.1/act/Sigmoid', '/model.22/cv4.0/cv4.0.1/act/Mul'], add node = ['/model.22/cv4.0/cv4.0.1/act/Sigmoid_2swish']
I replace_exswish: remove node = ['/model.22/cv2.0/cv2.0.0/act/Sigmoid', '/model.22/cv2.0/cv2.0.0/act/Mul'], add node = ['/model.22/cv2.0/cv2.0.0/act/Sigmoid_2swish']
I replace_exswish: remove node = ['/model.22/cv2.0/cv2.0.1/act/Sigmoid', '/model.22/cv2.0/cv2.0.1/act/Mul'], add node = ['/model.22/cv2.0/cv2.0.1/act/Sigmoid_2swish']
I replace_exswish: remove node = ['/model.22/cv3.0/cv3.0.0/act/Sigmoid', '/model.22/cv3.0/cv3.0.0/act/Mul'], add node = ['/model.22/cv3.0/cv3.0.0/act/Sigmoid_2swish']
I replace_exswish: remove node = ['/model.22/cv3.0/cv3.0.1/act/Sigmoid', '/model.22/cv3.0/cv3.0.1/act/Mul'], add node = ['/model.22/cv3.0/cv3.0.1/act/Sigmoid_2swish']
I convert_reduce_sum_to_conv: remove node = ['/model.22/ReduceSum_2'], add node = ['/model.22/ReduceSum_2_2conv']
I convert_reduce_sum_to_conv: remove node = ['/model.22/ReduceSum_1'], add node = ['/model.22/ReduceSum_1_2conv']
I convert_reduce_sum_to_conv: remove node = ['/model.22/ReduceSum'], add node = ['/model.22/ReduceSum_2conv']
I fold_constant ...
I fold_constant done.
I fuse_ops done.
I
I sparse_weight ...
I sparse_weight done.
I
Analysing : 100%|██████████████████████████████████████████████| 183/183 [00:00<00:00, 11048.45it/s]
Quantizating : 100%|██████████████████████████████████████████████| 183/183 [00:42<00:00, 4.33it/s]
I
I quant_optimizer ...
I quant_optimizer results:
I adjust_tanh_sigmoid: ['/model.22/Sigmoid', '/model.22/Sigmoid_1', '/model.22/Sigmoid_2']
I adjust_concat_split: ['/model.21/Concat', '/model.21/Split', '/model.18/Concat', '/model.17/Concat', '/model.15/Concat', '/model.12/Concat', '/model.8/Concat', '/model.6/Concat', '/model.4/Concat']
I adjust_no_change_node: ['/model.9/m_2/MaxPool', '/model.9/m_1/MaxPool', '/model.9/m/MaxPool']
I quant_optimizer done.
I
done
输出的yolov8s-seg.quantization.cfg文件内容：
yolov8s-seg.quantization.txt
混合量化步骤二输出的连板验证精度输出：
# simulator_error: calculate the simulator errors.
# entire: errors between 'golden' and 'simulator'.
# single: single layer errors. (compare to 'entire', the input of each layer is come from 'golden')!
# runtime_error: calculate the runtime errors.
# entire: errors between 'golden' and 'runtime'.
# single_sim: single layer errors between 'simulator' and 'runtime'.
# ('nan' means that tensor are 'all zeros', or 'all equal', or 'large values', etc)
layer_name simulator_error runtime_error
entire single entire single_sim
---------------------------------------------------------------------------------------------------------------------
[Input] images 1.000000 1.000000 1.000000 1.000000
[exDataConvert] images_int8__float16 1.000000 1.000000
[Conv] /model.0/conv/Conv_output_0 1.000000 1.000000 1.000000 1.000000
[exSwish] /model.0/act/Mul_output_0 1.000000 1.000000 0.999998 0.999998
[Conv] /model.1/conv/Conv_output_0 0.999999 1.000000 0.999998 1.000000
[exSwish] /model.1/act/Mul_output_0 1.000000 1.000000 0.999997 0.999999
[Conv] /model.2/cv1/conv/Conv_output_0 0.999999 1.000000 0.999997 1.000000
[exSwish] /model.2/cv1/act/Mul_output_0 0.999999 1.000000 0.999993 0.999997
[Split] /model.2/Split_output_0 0.999999 1.000000 0.999993 1.000000
[Split] /model.2/Split_output_1 0.999999 1.000000 0.999993 1.000000
[Conv] /model.2/m.0/cv1/conv/Conv_output_0 0.999999 1.000000 0.999992 1.000000
[exSwish] /model.2/m.0/cv1/act/Mul_output_0 0.999998 1.000000 0.999992 0.999999
[Conv] /model.2/m.0/cv2/conv/Conv_output_0 0.999998 1.000000 0.999990 1.000000
[exSwish] /model.2/m.0/cv2/act/Mul_output_0 0.999998 1.000000 0.999992 0.999999
[Add] /model.2/m.0/Add_output_0 0.999999 1.000000 0.999994 1.000000
[Concat] /model.2/Concat_output_0 0.999998 1.000000 0.999993 1.000000
[Conv] /model.2/cv2/conv/Conv_output_0 0.999998 1.000000 0.999990 1.000000
[exSwish] /model.2/cv2/act/Mul_output_0 0.999998 1.000000 0.999991 1.000000
[Conv] /model.3/conv/Conv_output_0 0.999999 1.000000 0.999994 1.000000
[exDataConvert] /model.3/conv/Conv_output_0__int8 0.999904 0.999905 0.999900 1.000000
[exSwish] /model.3/act/Mul_output_0 0.999712 0.999713 0.999707 0.999995
[Conv] /model.4/cv1/conv/Conv_output_0 0.999697 0.999869
[exSwish] /model.4/cv1/act/Mul_output_0 0.999538 0.999876 0.999534 0.999998
[Split] /model.4/Split_output_0 0.999461 0.999849 0.999455 1.000000
[Split] /model.4/Split_output_1 0.999717 0.999913 0.999716 1.000000
[Conv] /model.4/m.0/cv1/conv/Conv_output_0 0.999601 0.999875
[exSwish] /model.4/m.0/cv1/act/Mul_output_0 0.998799 0.999507 0.998789 0.999995
[Conv] /model.4/m.0/cv2/conv/Conv_output_0 0.999532 0.999829
[exSwish] /model.4/m.0/cv2/act/Mul_output_0 0.999391 0.999910 0.999391 0.999996
[Add] /model.4/m.0/Add_output_0 0.999536 0.999900 0.999536 1.000000
[Conv] /model.4/m.1/cv1/conv/Conv_output_0 0.999740 0.999958
[exSwish] /model.4/m.1/cv1/act/Mul_output_0 0.997372 0.997848 0.997376 0.999996
[Conv] /model.4/m.1/cv2/conv/Conv_output_0 0.997706 0.998230
[exSwish] /model.4/m.1/cv2/act/Mul_output_0 0.997658 0.999706 0.997659 0.999997
[Add] /model.4/m.1/Add_output_0 0.998741 0.999814 0.998743 1.000000
[Concat] /model.4/Concat_output_0 0.999081 0.999883 0.999080 1.000000
[Conv] /model.4/cv2/conv/Conv_output_0 0.998995 0.999900
[exSwish] /model.4/cv2/act/Mul_output_0 0.998275 0.999533 0.998271 0.999997
[Conv] /model.5/conv/Conv_output_0 0.999134 0.999786
[exSwish] /model.5/act/Mul_output_0 0.996467 0.997897 0.996458 0.999996
[Conv] /model.6/cv1/conv/Conv_output_0 0.997913 0.999051
[exSwish] /model.6/cv1/act/Mul_output_0 0.997111 0.999616 0.997101 0.999997
[exDataConvert] /model.6/cv1/act/Mul_output_0__float16 0.997112 0.999616 0.997101 1.000000
[Split] /model.6/Split_output_0 0.996765 1.000000 0.996751 1.000000
[Split] /model.6/Split_output_1 0.998072 1.000000 0.998069 1.000000
[exDataConvert] /model.6/Split_output_1__int8 0.997777 0.999602 0.997773 1.000000
[Conv] /model.6/m.0/cv1/conv/Conv_output_0 0.998935 0.999854
[exSwish] /model.6/m.0/cv1/act/Mul_output_0 0.994676 0.997817 0.994685 0.999992
[Conv] /model.6/m.0/cv2/conv/Conv_output_0 0.997526 0.999247
[exSwish] /model.6/m.0/cv2/act/Mul_output_0 0.994799 0.998849 0.994800 0.999993
[Add] /model.6/m.0/Add_output_0 0.996220 0.999134 0.996218 1.000000
[Conv] /model.6/m.1/cv1/conv/Conv_output_0 0.998646 0.999942
[exSwish] /model.6/m.1/cv1/act/Mul_output_0 0.993792 0.996710 0.993789 0.999994
[Conv] /model.6/m.1/cv2/conv/Conv_output_0 0.995344 0.997829 0.995322 0.999998
[exDataConvert] /model.6/m.1/cv2/conv/Conv_output_0__float16 0.995344 0.999828 0.995321 1.000000
[exSwish] /model.6/m.1/cv2/act/Mul_output_0 0.994559 1.000000 0.994526 1.000000
[exDataConvert] /model.6/m.0/Add_output_0__float16 0.996220 0.999814 0.996218 1.000000
[Add] /model.6/m.1/Add_output_0 0.995340 1.000000 0.995316 1.000000
[Concat] /model.6/Concat_output_0 0.996092 1.000000 0.996076 1.000000
[exDataConvert] /model.6/Concat_output_0__int8 0.995836 0.999583 0.995823 0.999998
[Conv] /model.6/cv2/conv/Conv_output_0 0.997972 0.999872
[exSwish] /model.6/cv2/act/Mul_output_0 0.994244 0.998145 0.994243 0.999993
[Conv] /model.7/conv/Conv_output_0 0.997883 0.999449
[exSwish] /model.7/act/Mul_output_0 0.993304 0.999162 0.993281 0.999995
[Conv] /model.8/cv1/conv/Conv_output_0 0.995852 0.999654
[exSwish] /model.8/cv1/act/Mul_output_0 0.992441 0.999049 0.992438 0.999996
[Split] /model.8/Split_output_0 0.991735 0.998857 0.991741 1.000000
[Split] /model.8/Split_output_1 0.989582 0.994118 0.989560 1.000000
[Conv] /model.8/m.0/cv1/conv/Conv_output_0 0.994679 0.997575
[exSwish] /model.8/m.0/cv1/act/Mul_output_0 0.985564 0.995179 0.985530 0.999996
[Conv] /model.8/m.0/cv2/conv/Conv_output_0 0.990769 0.997637 0.990715 0.999997
[exDataConvert] /model.8/m.0/cv2/conv/Conv_output_0__float16 0.990770 0.999782 0.990715 1.000000
[exSwish] /model.8/m.0/cv2/act/Mul_output_0 0.988347 1.000000 0.988264 1.000000
[exDataConvert] /model.8/Split_output_1__float16 0.989589 0.994368 0.989558 1.000000
[Add] /model.8/m.0/Add_output_0 0.987352 1.000000 0.987258 1.000000
[exDataConvert] /model.8/m.0/Add_output_0__int8 0.987235 0.999853 0.987155 0.999996
[Concat] /model.8/Concat_output_0 0.988545 0.998956 0.988492 1.000000
[Conv] /model.8/cv2/conv/Conv_output_0 0.993268 0.999934
[exSwish] /model.8/cv2/act/Mul_output_0 0.988431 0.998617 0.988410 0.999995
[Conv] /model.9/cv1/conv/Conv_output_0 0.995147 0.999600
[exSwish] /model.9/cv1/act/Mul_output_0 0.995686 0.999977 0.995701 0.999998
[MaxPool] /model.9/m/MaxPool_output_0 0.998157 0.999993 0.998176 1.000000
[MaxPool] /model.9/m_1/MaxPool_output_0 0.998345 0.999995 0.998374 1.000000
[MaxPool] /model.9/m_2/MaxPool_output_0 0.998438 0.999996 0.998475 1.000000
[Concat] /model.9/Concat_output_0 0.998165 0.999988 0.998191 1.000000
[Conv] /model.9/cv2/conv/Conv_output_0 0.996734 0.999909
[exSwish] /model.9/cv2/act/Mul_output_0 0.983371 0.991884 0.983471 0.999995
[Resize] /model.10/Resize_output_0 0.983371 0.991884 0.983471 1.000000
[Concat] /model.11/Concat_output_0 0.987593 0.994258 0.987647 1.000000
[Conv] /model.12/cv1/conv/Conv_output_0 0.991737 0.998124
[exSwish] /model.12/cv1/act/Mul_output_0 0.987277 0.998000 0.987263 0.999996
[Split] /model.12/Split_output_0 0.983949 0.996749 0.983951 1.000000
[Split] /model.12/Split_output_1 0.989459 0.998788 0.989435 1.000000
[Conv] /model.12/m.0/cv1/conv/Conv_output_0 0.991305 0.999938
[exSwish] /model.12/m.0/cv1/act/Mul_output_0 0.982684 0.999581 0.982592 0.999993
[Conv] /model.12/m.0/cv2/conv/Conv_output_0 0.982501 0.999761
[exSwish] /model.12/m.0/cv2/act/Mul_output_0 0.972088 0.990556 0.971959 0.999997
[Concat] /model.12/Concat_output_0 0.981439 0.995096 0.981388 1.000000
[Conv] /model.12/cv2/conv/Conv_output_0 0.987011 0.997520
[exSwish] /model.12/cv2/act/Mul_output_0 0.975684 0.997228 0.975644 0.999996
[Resize] /model.13/Resize_output_0 0.975685 0.997228 0.975644 1.000000
[Concat] /model.14/Concat_output_0 0.984781 0.997112 0.984755 0.999999
[Conv] /model.15/cv1/conv/Conv_output_0 0.994747 0.999449
[exSwish] /model.15/cv1/act/Mul_output_0 0.995976 0.999960 0.995956 0.999997
[Split] /model.15/Split_output_0 0.996633 0.999970 0.996601 1.000000
[Split] /model.15/Split_output_1 0.994637 0.999879 0.994642 1.000000
[Conv] /model.15/m.0/cv1/conv/Conv_output_0 0.993847 0.999878
[exSwish] /model.15/m.0/cv1/act/Mul_output_0 0.991842 0.999939 0.991862 0.999994
[Conv] /model.15/m.0/cv2/conv/Conv_output_0 0.990124 0.999900
[exSwish] /model.15/m.0/cv2/act/Mul_output_0 0.992060 0.999958 0.992110 0.999997
[Concat] /model.15/Concat_output_0 0.994582 0.999928 0.994586 1.000000
[Conv] /model.15/cv2/conv/Conv_output_0 0.993808 0.999937
[exSwish] /model.15/cv2/act/Mul_output_0 0.993252 0.999942 0.993281 0.999993
[Conv] /model.16/conv/Conv_output_0 0.984581 0.999898
[exSwish] /model.16/act/Mul_output_0 0.965541 0.994624 0.965579 0.999996
[Concat] /model.17/Concat_output_0 0.972043 0.996269 0.972031 1.000000
[Conv] /model.18/cv1/conv/Conv_output_0 0.981650 0.998201
[exSwish] /model.18/cv1/act/Mul_output_0 0.971250 0.999479 0.971254 0.999996
[Split] /model.18/Split_output_0 0.973918 0.999562 0.973978 1.000000
[Split] /model.18/Split_output_1 0.964332 0.994438 0.964221 1.000000
[Conv] /model.18/m.0/cv1/conv/Conv_output_0 0.985269 0.998521
[exSwish] /model.18/m.0/cv1/act/Mul_output_0 0.975605 0.999534 0.975545 0.999995
[Conv] /model.18/m.0/cv2/conv/Conv_output_0 0.963114 0.999551
[exSwish] /model.18/m.0/cv2/act/Mul_output_0 0.953899 0.997655 0.954108 0.999997
[Concat] /model.18/Concat_output_0 0.961866 0.997476 0.961958 1.000000
[Conv] /model.18/cv2/conv/Conv_output_0 0.980369 0.999478
[exSwish] /model.18/cv2/act/Mul_output_0 0.967018 0.998585 0.967117 0.999994
[Conv] /model.19/conv/Conv_output_0 0.978109 0.999521
[exSwish] /model.19/act/Mul_output_0 0.955933 0.988509 0.956107 0.999996
[Concat] /model.20/Concat_output_0 0.971698 0.990355 0.971838 1.000000
[Conv] /model.21/cv1/conv/Conv_output_0 0.985268 0.996546
[exSwish] /model.21/cv1/act/Mul_output_0 0.970007 0.997934 0.970007 0.999995
[Split] /model.21/Split_output_0 0.970068 0.999353 0.969994 1.000000
[Split] /model.21/Split_output_1 0.967899 0.992915 0.967956 1.000000
[Conv] /model.21/m.0/cv1/conv/Conv_output_0 0.988204 0.998082
[exSwish] /model.21/m.0/cv1/act/Mul_output_0 0.959128 0.989179 0.959249 0.999994
[Conv] /model.21/m.0/cv2/conv/Conv_output_0 0.978439 0.994900
[exSwish] /model.21/m.0/cv2/act/Mul_output_0 0.941659 0.963073 0.941747 0.999998
[Concat] /model.21/Concat_output_0 0.951232 0.975574 0.951287 1.000000
[Conv] /model.21/cv2/conv/Conv_output_0 0.967255 0.981855
[exSwish] /model.21/cv2/act/Mul_output_0 0.946344 0.995443 0.946423 0.999993
[Conv] /model.22/cv4.2/cv4.2.0/conv/Conv_output_0 0.976794 0.998878 0.976687 0.999997
[exDataConvert] /model.22/cv4.2/cv4.2.0/conv/Conv_output_0__float16 0.976793 0.999902 0.976688 1.000000
[exSwish] /model.22/cv4.2/cv4.2.0/act/Mul_output_0 0.979537 1.000000 0.979627 1.000000
[Conv] /model.22/cv4.2/cv4.2.1/conv/Conv_output_0 0.984081 1.000000 0.984063 1.000000
[exSwish] /model.22/cv4.2/cv4.2.1/act/Mul_output_0 0.986374 1.000000 0.986365 1.000000
[Conv] output11_int8 0.992544 1.000000
[exDataConvert] output11 0.992544 1.000000 0.992501 1.000000
[Conv] /model.22/cv2.2/cv2.2.0/conv/Conv_output_0 0.966998 0.997992 0.967220 0.999998
[exDataConvert] /model.22/cv2.2/cv2.2.0/conv/Conv_output_0__float16 0.966999 0.999892 0.967220 1.000000
[exSwish] /model.22/cv2.2/cv2.2.0/act/Mul_output_0 0.972475 1.000000 0.972623 1.000000
[Conv] /model.22/cv2.2/cv2.2.1/conv/Conv_output_0 0.978953 1.000000 0.978722 1.000000
[exSwish] /model.22/cv2.2/cv2.2.1/act/Mul_output_0 0.983686 1.000000 0.983433 1.000000
[Conv] output8_int8 0.992880 1.000000
[exDataConvert] output8 0.992880 1.000000 0.992856 1.000000
[Conv] /model.22/cv3.2/cv3.2.0/conv/Conv_output_0 0.969897 0.996682 0.970212 0.999995
[exDataConvert] /model.22/cv3.2/cv3.2.0/conv/Conv_output_0__float16 0.969894 0.999736 0.970213 1.000000
[exSwish] /model.22/cv3.2/cv3.2.0/act/Mul_output_0 0.967017 1.000000 0.967431 1.000000
[Conv] /model.22/cv3.2/cv3.2.1/conv/Conv_output_0 0.973027 1.000000 0.973418 1.000000
[exSwish] /model.22/cv3.2/cv3.2.1/act/Mul_output_0 0.978745 1.000000 0.979017 1.000000
[Conv] /model.22/cv3.2/cv3.2.2/Conv_output_0 0.999228 1.000000 0.999227 1.000000
[Sigmoid] output9_int8 0.993253 1.000000
[exDataConvert] output9 0.993253 1.000000 0.991211 0.997793
[Conv] /model.22/ReduceSum_2_output_0 0.993216 1.000000
[Clip] output10_int8 0.993216 1.000000
[exDataConvert] output10 0.993216 1.000000 0.902946 0.907305
[Conv] /model.22/cv4.1/cv4.1.0/conv/Conv_output_0 0.974635 0.999637
[exSwish] /model.22/cv4.1/cv4.1.0/act/Mul_output_0 0.978413 0.999659 0.978607 0.999996
[Conv] /model.22/cv4.1/cv4.1.1/conv/Conv_output_0 0.983883 0.999745 0.983974 0.999998
[exDataConvert] /model.22/cv4.1/cv4.1.1/conv/Conv_output_0__float16 0.983883 0.999912 0.983974 1.000000
[exSwish] /model.22/cv4.1/cv4.1.1/act/Mul_output_0 0.983129 1.000000 0.983230 1.000000
[Conv] output7_int8 0.992122 1.000000
[exDataConvert] output7 0.992122 1.000000 0.992180 1.000000
[Conv] /model.22/cv2.1/cv2.1.0/conv/Conv_output_0 0.959621 0.999012 0.959668 0.999996
[exDataConvert] /model.22/cv2.1/cv2.1.0/conv/Conv_output_0__float16 0.959621 0.999758 0.959668 1.000000
[exSwish] /model.22/cv2.1/cv2.1.0/act/Mul_output_0 0.961373 1.000000 0.961553 1.000000
[Conv] /model.22/cv2.1/cv2.1.1/conv/Conv_output_0 0.971303 1.000000 0.971525 1.000000
[exSwish] /model.22/cv2.1/cv2.1.1/act/Mul_output_0 0.977228 1.000000 0.977464 1.000000
[Conv] output4_int8 0.993265 1.000000
[exDataConvert] output4 0.993265 1.000000 0.993345 1.000000
[Conv] /model.22/cv3.1/cv3.1.0/conv/Conv_output_0 0.969095 0.999391
[exSwish] /model.22/cv3.1/cv3.1.0/act/Mul_output_0 0.953357 0.998573 0.953245 0.999995
[Conv] /model.22/cv3.1/cv3.1.1/conv/Conv_output_0 0.977318 0.998247 0.977310 0.999988
[exDataConvert] /model.22/cv3.1/cv3.1.1/conv/Conv_output_0__float16 0.977316 0.999430 0.977311 1.000000
[exSwish] /model.22/cv3.1/cv3.1.1/act/Mul_output_0 0.981851 1.000000 0.981897 1.000000
[Conv] /model.22/cv3.1/cv3.1.2/Conv_output_0 0.999710 1.000000 0.999714 1.000000
[Sigmoid] output5_int8 0.986039 1.000000
[exDataConvert] output5 0.986039 1.000000 0.736498 0.747560
[Conv] /model.22/ReduceSum_1_output_0 0.989157 1.000000
[Clip] output6_int8 0.989157 1.000000
[exDataConvert] output6 0.989157 1.000000 0.223930 0.227319
[Conv] /model.22/proto/cv1/conv/Conv_output_0 0.991815 0.999881
[exSwish] /model.22/proto/cv1/act/Mul_output_0 0.985442 0.998241 0.985507 0.999995
[ConvTranspose] /model.22/proto/upsample/ConvTranspose_output_0 0.993589 0.999328 0.993615 0.999999
[Conv] /model.22/proto/cv2/conv/Conv_output_0 0.996772 0.999993
[exSwish] /model.22/proto/cv2/act/Mul_output_0 0.996644 0.999980 0.996637 0.999996
[Conv] /model.22/proto/cv3/conv/Conv_output_0 0.988931 0.999829 0.988919 0.999998
[exDataConvert] /model.22/proto/cv3/conv/Conv_output_0__float16 0.988931 0.999921 0.988919 1.000000
[exSwish] proto_int8 0.989512 1.000000
[exDataConvert] proto 0.989512 1.000000 0.989494 1.000000
[Conv] /model.22/cv4.0/cv4.0.0/conv/Conv_output_0 0.985720 0.999909
[exSwish] /model.22/cv4.0/cv4.0.0/act/Mul_output_0 0.986436 0.999769 0.986434 0.999997
[Conv] /model.22/cv4.0/cv4.0.1/conv/Conv_output_0 0.986299 0.999826 0.986237 1.000000
[exDataConvert] /model.22/cv4.0/cv4.0.1/conv/Conv_output_0__float16 0.986299 0.999935 0.986238 1.000000
[exSwish] /model.22/cv4.0/cv4.0.1/act/Mul_output_0 0.988153 1.000000 0.988096 1.000000
[Conv] output3_int8 0.995409 1.000000
[exDataConvert] output3 0.995409 1.000000 0.995372 1.000000
[Conv] /model.22/cv2.0/cv2.0.0/conv/Conv_output_0 0.989050 0.999792
[exSwish] /model.22/cv2.0/cv2.0.0/act/Mul_output_0 0.986228 0.999830 0.986218 0.999990
[exDataConvert] /model.22/cv2.0/cv2.0.0/act/Mul_output_0__float16 0.986228 0.999830 0.986219 1.000000
[Conv] /model.22/cv2.0/cv2.0.1/conv/Conv_output_0 0.984343 1.000000 0.984311 1.000000
[exSwish] /model.22/cv2.0/cv2.0.1/act/Mul_output_0 0.986249 1.000000 0.986199 1.000000
[Conv] output0_int8 0.995273 1.000000
[exDataConvert] output0 0.995273 1.000000 0.995252 1.000000
[Conv] /model.22/cv3.0/cv3.0.0/conv/Conv_output_0 0.989440 0.999885
[exSwish] /model.22/cv3.0/cv3.0.0/act/Mul_output_0 0.971564 0.993903 0.971541 0.999995
[Conv] /model.22/cv3.0/cv3.0.1/conv/Conv_output_0 0.980204 0.991985 0.980069 0.999993
[exDataConvert] /model.22/cv3.0/cv3.0.1/conv/Conv_output_0__float16 0.980204 0.999633 0.980068 1.000000
[exSwish] /model.22/cv3.0/cv3.0.1/act/Mul_output_0 0.985248 1.000000 0.985197 1.000000
[Conv] /model.22/cv3.0/cv3.0.2/Conv_output_0 0.999838 1.000000 0.999837 1.000000
[Sigmoid] output1_int8 0.706008 1.000000
[exDataConvert] output1 0.706008 1.000000 0.125208 0.181133
[Conv] /model.22/ReduceSum_output_0 0.818402 1.000000
[Clip] output2_int8 0.818402 1.000000
[exDataConvert] output2 0.818402 1.000000 0.131679 0.159088
问题一：
混合量化步骤一没有使用所用的参数.
1）设置了optimization_level=0，但是仍然替换了很多算子
2）设置了quantized_method为channel，但yolov8s-seg.quantization.cfg仍然显示为layer
3）设置了proposal=False，但yolov8s-seg.quantization.cfg仍然给出了混合量化的建议
问题二：
精度损失巨大
| output1 | 0.125208 |
| output2 | 0.131679 |
| output5 | 0.736498 |
| output6 | 0.223930 |
转模型过程中出现算子错误
我尝试用最新的rknn-toolkit2将自己的onnx模型转为rk模型，是遇到一个错误，并且我无法通过这个报错追溯到源码的位置，具体报错日志如下
--> Building model
Build model failed!
E build: Catch exception when building RKNN model!
E build: Traceback (most recent call last):
E build: File "rknn/api/rknn_base.py", line 1979, in rknn.api.rknn_base.RKNNBase.build
E build: File "rknn/api/graph_optimizer.py", line 1365, in rknn.api.graph_optimizer.GraphOptimizer.correct_ops
E build: TypeError: '>' not supported between instances of 'numpy.ndarray' and 'str'
RKNN failed to submit!, op id: 1, op name: Conv:MobilenetV1/MobilenetV1/Conv2d_0/Relu6_prequant
Testing on the Radxa Rock 5B, their Debian Bulleye (Linux 5.10) image has the following version of RKNN driver and Toolkit installed.
driver version: 0.8.2, API version: 1.6.0 (9a7b5d24c@2023-12-13T17:31:11)
When running the MobileNet demo it completes successfully.
While testing the Debian Bookworm (Linux 6.1) image it has the following version of RKNN driver and Toolkit installed.
driver version: 0.9.3, API version: 1.6.0 (9a7b5d24c@2023-12-13T17:31:11)
When running the above MobileNet demo it fails with the following error;
model input num: 1, output num: 1
input tensors:
index=0, name=input, n_dims=4, dims=[1, 224, 224, 3], n_elems=150528, size=150528, fmt=NHWC, type=INT8, qnt_type=AFFINE, zp=0, scale=0.007812
output tensors:
index=0, name=MobilenetV1/Predictions/Reshape_1, n_dims=2, dims=[1, 1001, 0, 0], n_elems=1001, size=1001, fmt=UNDEFINED, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003906
rknn_run
E RKNN: [05:03:32.244] failed to submit!, op id: 1, op name: Conv:MobilenetV1/MobilenetV1/Conv2d_0/Relu6_prequant, flags: 0x5, task start: 0, task number: 38, run task counter: 0, int status: 0, please try updating to the latest version of the toolkit2 and runtime from: https://console.zbox.filez.com/l/I00fc3 (PWD: rknn)
rknn_run fail! ret=-1
This must mean there is a bug in the new driver version 0.9.3?
当图像输入和模型w_stride不一致时，通用api和零拷贝api结果不一致
RK1106 的NPU支持 单通道模型，必须要多通道吗
目前的模型shape 中的 NHWC C=1，RK支持吗。我试了好几次都是报错，无法运行。
Why not supported in ONNXruntime?
This page says only old chips supported:
https://onnxruntime.ai/docs/execution-providers/community-maintained/RKNPU-ExecutionProvider.html#supported-platform
Here is current EP:
https://github.com/microsoft/onnxruntime/tree/main/onnxruntime/core/providers/rknpu
Not sure if it can support newer chips, I need RK3588 support
Official sample question of rknn-tool-lite2
When I run the official SDK on rock5a test.py in the rknn-tool-lite2 /example/dynamic_shape
I got this
(py310) rock@rock-5a:~/dynamic_shape$ python test.py
--> Load RKNN model
done
--> Init runtime environment
I RKNN: [14:09:40.197] RKNN Runtime Information, librknnrt version: 2.0.0b0 (35a6907d79@2024-03-24T10:31:14)
I RKNN: [14:09:40.197] RKNN Driver Information, version: 0.8.2
W RKNN: [14:09:40.197] Current driver version: 0.8.2, recommend to upgrade the driver to the new version: >= 0.8.8
I RKNN: [14:09:40.197] RKNN Model Information, version: 6, toolkit version: 2.0.0b0+9bab5682(compiler version: 2.0.0b0 (35a6907d79@2024-03-24T02:34:11)), target: RKNPU v2, target platform: rk3588, framework name: Caffe, framework layout: NCHW, model inference type: dynamic_shape
done
--> Running model
model: mobilenet_v2
input shape: 1,3,224,224
W The input[0] need NHWC data format, but NCHW set, the data format and data buffer will be changed to NHWC.
Aborted (core dumped)
Does anyone know why？
Could not find tensorflow==2.8.0 for python 3.10 venv
File requirements_cp310-2.0.0b0.txt has requirement tensorflow==2.8.0. I have python 3.10 venv and try to install dependencies:
(venv)$pip install -r requirements_cp310-2.0.0b0.txt
All packages collected correctly, but tensorflow gives an error:
ERROR: Could not find a version that satisfies the requirement tensorflow==2.8.0 (from versions: 2.10.0rc0, 2.10.0rc1, ....
not implemented: ['aten::copy_']
我在调用rknn.load_pytorch时遇到了问题：E load_pytorch: ValueError: The following operators are 
在build模型的时候，遇到核心转储
我的模型有两个输入，分别是input：[1, 256, 6, 6]，input1：[1, 256, 26, 26]，我通过torch库随机构造200个上述输入维度的数据并保存为npy文件后，成功设置rknn.build(do_quantization= True, dataset = "./npy/data.txt")后，在模型量化过程中核心转储。其中通过gdb展示的堆栈信息如下：
#0 0x00007f25a0a0c762 in ?? () from /usr/local/lib/python3.8/dist-packages/rknn/api/lib/linux-x86_64/cp38/librknnc.so
#1 0x00007f25a0a0d7b0 in ?? () from /usr/local/lib/python3.8/dist-packages/rknn/api/lib/linux-x86_64/cp38/librknnc.so
#2 0x00007f25a0a0e200 in ?? () from /usr/local/lib/python3.8/dist-packages/rknn/api/lib/linux-x86_64/cp38/librknnc.so
#3 0x00007f25a09cd523 in ?? () from /usr/local/lib/python3.8/dist-packages/rknn/api/lib/linux-x86_64/cp38/librknnc.so
#4 0x00007f25a09cd6dd in ?? () from /usr/local/lib/python3.8/dist-packages/rknn/api/lib/linux-x86_64/cp38/librknnc.so
#5 0x00007f25a09cde66 in ?? () from /usr/local/lib/python3.8/dist-packages/rknn/api/lib/linux-x86_64/cp38/librknnc.so
#6 0x00007f25a097f6c1 in rknn::RKNNCompiler::build() () from /usr/local/lib/python3.8/dist-packages/rknn/api/lib/linux-x86_64/cp38/librknnc.so
#7 0x00007f25a099666c in ?? () from /usr/local/lib/python3.8/dist-packages/rknn/api/lib/linux-x86_64/cp38/librknnc.so
#8 0x00007f25a09a86a5 in ?? () from /usr/local/lib/python3.8/dist-packages/rknn/api/lib/linux-x86_64/cp38/librknnc.so
#9 0x00007f25a09a38ba in ?? () from /usr/local/lib/python3.8/dist-packages/rknn/api/lib/linux-x86_64/cp38/librknnc.so
#10 0x00000000005d5499 in PyCFunction_Call ()
#11 0x00007f268fce8486 in ?? () from /usr/local/lib/python3.8/dist-packages/rknn/api/rknn_base.cpython-38-x86_64-linux-gnu.so
#12 0x00007f268fe0552b in ?? () from /usr/local/lib/python3.8/dist-packages/rknn/api/rknn_base.cpython-38-x86_64-linux-gnu.so
#13 0x00007f268fd7395d in ?? () from /usr/local/lib/python3.8/dist-packages/rknn/api/rknn_base.cpython-38-x86_64-linux-gnu.so
#14 0x00007f268fd7989d in ?? () from /usr/local/lib/python3.8/dist-packages/rknn/api/rknn_base.cpython-38-x86_64-linux-gnu.so
#15 0x00007f268fceb799 in ?? () from /usr/local/lib/python3.8/dist-packages/rknn/api/rknn_base.cpython-38-x86_64-linux-gnu.so
#16 0x00007f268fdb6697 in ?? () from /usr/local/lib/python3.8/dist-packages/rknn/api/rknn_base.cpython-38-x86_64-linux-gnu.so
#17 0x00007f268fdbc3d3 in ?? () from /usr/local/lib/python3.8/dist-packages/rknn/api/rknn_base.cpython-38-x86_64-linux-gnu.so
#18 0x00000000005d6066 in _PyObject_MakeTpCall ()
#19 0x00000000004e22b3 in ?? ()
#20 0x00000000005483b6 in _PyEval_EvalFrameDefault ()
#21 0x000000000054552a in _PyEval_EvalCodeWithName ()
#22 0x00000000004e1bd0 in ?? ()
#23 0x00000000005483b6 in _PyEval_EvalFrameDefault ()
#24 0x000000000054552a in _PyEval_EvalCodeWithName ()
#25 0x0000000000684327 in PyEval_EvalCode ()
#26 0x0000000000673a41 in ?? ()
#27 0x0000000000673abb in ?? ()
#28 0x0000000000673b61 in ?? ()
#29 0x00000000006747e7 in PyRun_SimpleFileExFlags ()
#30 0x00000000006b4072 in Py_RunMain ()
#31 0x00000000006b43fd in Py_BytesMain ()
#32 0x00007f269337a083 in __libc_start_main (main=0x4c4510 <main>, argc=2, argv=0x7fffad24e4f8, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffad24e4e8) at ../csu/libc-start.c:308
#33 0x00000000005da67e in _start ()
部分执行过程中输出如下：
I RKNN: [10:09:08.256] AppointLayout: t->setNativeLayout(64), tname:[2778]
I RKNN: [10:09:08.256] AppointLayout: t->setNativeLayout(64), tname:[4703]
I RKNN: [10:09:08.256] AppointLayout: t->setNativeLayout(64), tname:[4703_conv_out254.1_Concat]
I RKNN: [10:09:08.256] AppointLayout: t->setNativeLayout(64), tname:[2779]
I RKNN: [10:09:08.256] AppointLayout: t->setNativeLayout(64), tname:[4710]
I RKNN: [10:09:08.256] AppointLayout: t->setNativeLayout(64), tname:[4710_conv_out254.1_Concat]
I RKNN: [10:09:08.256] AppointLayout: t->setNativeLayout(64), tname:[2780]
I RKNN: [10:09:08.256] AppointLayout: t->setNativeLayout(64), tname:[4717]
I RKNN: [10:09:08.256] AppointLayout: t->setNativeLayout(64), tname:[4717_conv_out254.1_Concat]
I RKNN: [10:09:08.256] AppointLayout: t->setNativeLayout(64), tname:[2781]
I RKNN: [10:09:08.256] AppointLayout: t->setNativeLayout(64), tname:[4724]
I RKNN: [10:09:08.256] AppointLayout: t->setNativeLayout(64), tname:[4724_conv_out254.1_Concat]
I RKNN: [10:09:08.256] AppointLayout: t->setNativeLayout(64), tname:[2782]
I RKNN: [10:09:08.256] AppointLayout: t->setNativeLayout(64), tname:[4731]
I RKNN: [10:09:08.256] AppointLayout: t->setNativeLayout(64), tname:[4731_conv_out254.1_Concat]
I RKNN: [10:09:08.256] AppointLayout: t->setNativeLayout(64), tname:[2783]
I RKNN: [10:09:08.256] AppointLayout: t->setNativeLayout(64), tname:[4738]
I RKNN: [10:09:08.256] AppointLayout: t->setNativeLayout(64), tname:[4738_conv_out254.1_Concat]
I RKNN: [10:09:08.256] AppointLayout: t->setNativeLayout(64), tname:[2784]
I RKNN: [10:09:08.256] AppointLayout: t->setNativeLayout(64), tname:[4745]
I RKNN: [10:09:08.256] AppointLayout: t->setNativeLayout(64), tname:[4745_conv_out254.1_Concat]
I RKNN: [10:09:08.256] AppointLayout: t->setNativeLayout(64), tname:[2785]
I RKNN: [10:09:08.256] AppointLayout: t->setNativeLayout(64), tname:[4752]
I RKNN: [10:09:08.256] AppointLayout: t->setNativeLayout(64), tname:[4752_conv_out254.1_Concat]
I RKNN: [10:09:08.256] AppointLayout: t->setNativeLayout(64), tname:[2786]
I RKNN: [10:09:08.256] AppointLayout: t->setNativeLayout(64), tname:[4759]
I RKNN: [10:09:08.256] AppointLayout: t->setNativeLayout(64), tname:[4759_conv_out254.1_Concat]
I RKNN: [10:09:08.256] AppointLayout: t->setNativeLayout(64), tname:[2787]
I RKNN: [10:09:08.256] AppointLayout: t->setNativeLayout(64), tname:[4766]
I RKNN: [10:09:08.256] AppointLayout: t->setNativeLayout(64), tname:[4766_conv_out254.1_Concat]
I RKNN: [10:09:08.256] AppointLayout: t->setNativeLayout(64), tname:[2788]
I RKNN: [10:09:08.256] AppointLayout: t->setNativeLayout(64), tname:[4773]
I RKNN: [10:09:08.256] AppointLayout: t->setNativeLayout(64), tname:[4773_conv_out254.1_Concat]
I RKNN: [10:09:08.256] AppointLayout: t->setNativeLayout(64), tname:[2789]
I RKNN: [10:09:08.256] AppointLayout: t->setNativeLayout(64), tname:[4780]
I RKNN: [10:09:08.256] AppointLayout: t->setNativeLayout(64), tname:[4780_conv_out254.1_Concat]
I RKNN: [10:09:08.256] AppointLayout: t->setNativeLayout(64), tname:[2790]
I RKNN: [10:09:08.256] AppointLayout: t->setNativeLayout(64), tname:[4787]
I RKNN: [10:09:08.256] AppointLayout: t->setNativeLayout(64), tname:[4787_conv_out254.1_Concat]
I RKNN: [10:09:08.256] AppointLayout: t->setNativeLayout(64), tname:[2791]
I RKNN: [10:09:08.256] AppointLayout: t->setNativeLayout(64), tname:[4794]
I RKNN: [10:09:08.256] AppointLayout: t->setNativeLayout(64), tname:[4794_conv_out254.1_Concat]
I RKNN: [10:09:08.256] AppointLayout: t->setNativeLayout(64), tname:[2792]
I RKNN: [10:09:08.256] AppointLayout: t->setNativeLayout(64), tname:[4801]
I RKNN: [10:09:08.256] AppointLayout: t->setNativeLayout(64), tname:[4801_conv_out254.1_Concat]
I RKNN: [10:09:08.256] AppointLayout: t->setNativeLayout(64), tname:[2793]
I RKNN: [10:09:08.256] AppointLayout: t->setNativeLayout(64), tname:[4808]
I RKNN: [10:09:08.256] AppointLayout: t->setNativeLayout(64), tname:[4808_conv_out254.1_Concat]
I RKNN: [10:09:08.256] AppointLayout: t->setNativeLayout(64), tname:[2794]
I RKNN: [10:09:08.256] AppointLayout: t->setNativeLayout(64), tname:[4815]
I RKNN: [10:09:08.256] AppointLayout: t->setNativeLayout(64), tname:[4815_conv_out254.1_Concat]
I RKNN: [10:09:08.256] AppointLayout: t->setNativeLayout(64), tname:[2795]
I RKNN: [10:09:08.256] AppointLayout: t->setNativeLayout(64), tname:[4822]
I RKNN: [10:09:08.256] AppointLayout: t->setNativeLayout(64), tname:[4822_conv_out254.1_Concat]
I RKNN: [10:09:08.256] AppointLayout: t->setNativeLayout(64), tname:[2796]
I RKNN: [10:09:08.256] AppointLayout: t->setNativeLayout(64), tname:[4829]
I RKNN: [10:09:08.256] AppointLayout: t->setNativeLayout(64), tname:[4829_conv_out254.1_Concat]
I RKNN: [10:09:08.256] AppointLayout: t->setNativeLayout(64), tname:[2797]
I RKNN: [10:09:08.256] AppointLayout: t->setNativeLayout(64), tname:[4836]
I RKNN: [10:09:08.256] AppointLayout: t->setNativeLayout(64), tname:[4836_conv_out254.1_Concat]
I RKNN: [10:09:08.256] AppointLayout: t->setNativeLayout(64), tname:[4843]
I RKNN: [10:09:08.256] AppointLayout: t->setNativeLayout(64), tname:[out254.1_before_conv]
I RKNN: [10:09:08.256] AppointLayout: t->setNativeLayout(64), tname:[out254.1]
I RKNN: [10:09:08.256] AppointLayout: t->setNativeLayout(64), tname:[2446]
I RKNN: [10:09:08.256] AppointLayout: t->setNativeLayout(64), tname:[4878]
D RKNN: [10:09:08.256] <<<<<<<< end: N4rknn19RKNNLayoutMatchPassE
D RKNN: [10:09:08.256] >>>>>> start: N4rknn20RKNNAddSecondaryNodeE
D RKNN: [10:09:08.258] <<<<<<<< end: N4rknn20RKNNAddSecondaryNodeE
D RKNN: [10:09:08.258] >>>>>> start: OpEmit
D RKNN: [10:09:10.106] <<<<<<<< end: OpEmit
D RKNN: [10:09:10.106] >>>>>> start: N4rknn23RKNNProfileAnalysisPassE
D RKNN: [10:09:10.111] node: Reshape:109_Reshape, Target: CPU
D RKNN: [10:09:10.111] node: Reshape:2541_Reshape, Target: CPU
D RKNN: [10:09:10.112] <<<<<<<< end: N4rknn23RKNNProfileAnalysisPassE
D RKNN: [10:09:10.985] >>>>>> start: N4rknn21RKNNOperatorIdGenPassE
D RKNN: [10:09:10.989] <<<<<<<< end: N4rknn21RKNNOperatorIdGenPassE
D RKNN: [10:09:10.989] >>>>>> start: N4rknn23RKNNWeightTransposePassE
Segmentation fault (core dumped)
我的模型量化代码如下：
from rknn.api import RKNN
model = '326/siam_rpn326.pt'
input_size_list = [[1, 256, 6, 6], [1, 256, 26, 26]]
# Create RKNN object
rknn = RKNN(verbose=True)
# Pre-process config
print('--> Config model')
# 注意要将target_platform指定清楚
rknn.config(mean_values=[[0]*256, [0]*256], std_values=[[1]*256, [1]*256], target_platform="rk3588")
print('done')
# Load model
print('--> Loading model')
ret = rknn.load_pytorch(model=model, input_size_list=input_size_list)
if ret != 0:
print('Load model failed!')
exit(ret)
print('done')
# Build model
print('--> Building model')
ret = rknn.build(do_quantization= True, dataset = "./npy/data.txt")
if ret != 0:
print('Build model failed!')
exit(ret)
print('done')
# Export rknn model
print('--> Export rknn model')
ret = rknn.export_rknn('326/siam_rpn327.rknn')
if ret != 0:
print('Export rknn model failed!')
exit(ret)
rknn.release()
如何解决？量化过程可能存在什么问题？
Python 推理，结果正常，但是一直有warning
用Python测试推理yolov5， 结果正常，但是一直有warning，这个应该怎么处理？
W RKNN: [09:44:39.467] Output(output0): size_with_stride larger than model origin size, if need run OutputOperator in NPU, please call rknn_create_memory using size_with_stride.
W RKNN: [09:44:39.468] Output(275): size_with_stride larger than model origin size, if need run OutputOperator in NPU, please call rknn_create_memory using size_with_stride.
W RKNN: [09:44:39.468] Output(277): size_with_stride larger than model origin size, if need run OutputOperator in NPU, please call rknn_create_memory using size_with_stride.
Crash on custom yolov8 conversion
Source: Custom trained yolov8n.pt model with 2 classes. (Another model with 3 classes converts, but acts weird, still investigating).
Runtime 2.0.0b0
I rknn-toolkit2 version: 2.0.0b0+9bab5682
Configuring with {'target_platform': 'rk3588'}
Loading onnx with {}
I Loading : 100%|██████████████████████████████████████████████| 149/149 [00:00<00:00, 67796.84it/s]
W load_onnx: The config.mean_values is None, zeros will be set for input 0!
W load_onnx: The config.std_values is None, ones will be set for input 0!
D base_optimize ...
D base_optimize done.
D
D fold_constant ...
D fold_constant done.
D fold_constant remove nodes = ['/model.22/Mul_1', '/model.22/Mul', '/model.22/Div', '/model.22/Add', '/model.22/Gather', '/model.22/Shape']
D
D correct_ops ...
D correct_ops done.
D
D fuse_ops ...
D fuse_ops results:
D replace_exswish: remove node = ['/model.0/act/Sigmoid', '/model.0/act/Mul'], add node = ['/model.0/act/Sigmoid_2swish']
D replace_exswish: remove node = ['/model.1/act/Sigmoid', '/model.1/act/Mul'], add node = ['/model.1/act/Sigmoid_2swish']
D replace_exswish: remove node = ['/model.2/cv1/act/Sigmoid', '/model.2/cv1/act/Mul'], add node = ['/model.2/cv1/act/Sigmoid_2swish']
D replace_exswish: remove node = ['/model.2/m.0/cv1/act/Sigmoid', '/model.2/m.0/cv1/act/Mul'], add node = ['/model.2/m.0/cv1/act/Sigmoid_2swish']
D replace_exswish: remove node = ['/model.2/m.0/cv2/act/Sigmoid', '/model.2/m.0/cv2/act/Mul'], add node = ['/model.2/m.0/cv2/act/Sigmoid_2swish']
D replace_exswish: remove node = ['/model.2/cv2/act/Sigmoid', '/model.2/cv2/act/Mul'], add node = ['/model.2/cv2/act/Sigmoid_2swish']
D replace_exswish: remove node = ['/model.3/act/Sigmoid', '/model.3/act/Mul'], add node = ['/model.3/act/Sigmoid_2swish']
D replace_exswish: remove node = ['/model.4/cv1/act/Sigmoid', '/model.4/cv1/act/Mul'], add node = ['/model.4/cv1/act/Sigmoid_2swish']
D replace_exswish: remove node = ['/model.4/m.0/cv1/act/Sigmoid', '/model.4/m.0/cv1/act/Mul'], add node = ['/model.4/m.0/cv1/act/Sigmoid_2swish']
D replace_exswish: remove node = ['/model.4/m.0/cv2/act/Sigmoid', '/model.4/m.0/cv2/act/Mul'], add node = ['/model.4/m.0/cv2/act/Sigmoid_2swish']
D replace_exswish: remove node = ['/model.4/m.1/cv1/act/Sigmoid', '/model.4/m.1/cv1/act/Mul'], add node = ['/model.4/m.1/cv1/act/Sigmoid_2swish']
D replace_exswish: remove node = ['/model.4/m.1/cv2/act/Sigmoid', '/model.4/m.1/cv2/act/Mul'], add node = ['/model.4/m.1/cv2/act/Sigmoid_2swish']
D replace_exswish: remove node = ['/model.4/cv2/act/Sigmoid', '/model.4/cv2/act/Mul'], add node = ['/model.4/cv2/act/Sigmoid_2swish']
D replace_exswish: remove node = ['/model.5/act/Sigmoid', '/model.5/act/Mul'], add node = ['/model.5/act/Sigmoid_2swish']
D replace_exswish: remove node = ['/model.6/cv1/act/Sigmoid', '/model.6/cv1/act/Mul'], add node = ['/model.6/cv1/act/Sigmoid_2swish']
D replace_exswish: remove node = ['/model.6/m.0/cv1/act/Sigmoid', '/model.6/m.0/cv1/act/Mul'], add node = ['/model.6/m.0/cv1/act/Sigmoid_2swish']
D replace_exswish: remove node = ['/model.6/m.0/cv2/act/Sigmoid', '/model.6/m.0/cv2/act/Mul'], add node = ['/model.6/m.0/cv2/act/Sigmoid_2swish']
D replace_exswish: remove node = ['/model.6/m.1/cv1/act/Sigmoid', '/model.6/m.1/cv1/act/Mul'], add node = ['/model.6/m.1/cv1/act/Sigmoid_2swish']
D replace_exswish: remove node = ['/model.6/m.1/cv2/act/Sigmoid', '/model.6/m.1/cv2/act/Mul'], add node = ['/model.6/m.1/cv2/act/Sigmoid_2swish']
D replace_exswish: remove node = ['/model.6/cv2/act/Sigmoid', '/model.6/cv2/act/Mul'], add node = ['/model.6/cv2/act/Sigmoid_2swish']
D replace_exswish: remove node = ['/model.7/act/Sigmoid', '/model.7/act/Mul'], add node = ['/model.7/act/Sigmoid_2swish']
D replace_exswish: remove node = ['/model.8/cv1/act/Sigmoid', '/model.8/cv1/act/Mul'], add node = ['/model.8/cv1/act/Sigmoid_2swish']
D replace_exswish: remove node = ['/model.8/m.0/cv1/act/Sigmoid', '/model.8/m.0/cv1/act/Mul'], add node = ['/model.8/m.0/cv1/act/Sigmoid_2swish']
D replace_exswish: remove node = ['/model.8/m.0/cv2/act/Sigmoid', '/model.8/m.0/cv2/act/Mul'], add node = ['/model.8/m.0/cv2/act/Sigmoid_2swish']
D replace_exswish: remove node = ['/model.8/cv2/act/Sigmoid', '/model.8/cv2/act/Mul'], add node = ['/model.8/cv2/act/Sigmoid_2swish']
D replace_exswish: remove node = ['/model.9/cv1/act/Sigmoid', '/model.9/cv1/act/Mul'], add node = ['/model.9/cv1/act/Sigmoid_2swish']
D replace_exswish: remove node = ['/model.9/cv2/act/Sigmoid', '/model.9/cv2/act/Mul'], add node = ['/model.9/cv2/act/Sigmoid_2swish']
D replace_exswish: remove node = ['/model.12/cv1/act/Sigmoid', '/model.12/cv1/act/Mul'], add node = ['/model.12/cv1/act/Sigmoid_2swish']
D replace_exswish: remove node = ['/model.12/m.0/cv1/act/Sigmoid', '/model.12/m.0/cv1/act/Mul'], add node = ['/model.12/m.0/cv1/act/Sigmoid_2swish']
D replace_exswish: remove node = ['/model.12/m.0/cv2/act/Sigmoid', '/model.12/m.0/cv2/act/Mul'], add node = ['/model.12/m.0/cv2/act/Sigmoid_2swish']
D replace_exswish: remove node = ['/model.12/cv2/act/Sigmoid', '/model.12/cv2/act/Mul'], add node = ['/model.12/cv2/act/Sigmoid_2swish']
D replace_exswish: remove node = ['/model.15/cv1/act/Sigmoid', '/model.15/cv1/act/Mul'], add node = ['/model.15/cv1/act/Sigmoid_2swish']
D replace_exswish: remove node = ['/model.15/m.0/cv1/act/Sigmoid', '/model.15/m.0/cv1/act/Mul'], add node = ['/model.15/m.0/cv1/act/Sigmoid_2swish']
D replace_exswish: remove node = ['/model.15/m.0/cv2/act/Sigmoid', '/model.15/m.0/cv2/act/Mul'], add node = ['/model.15/m.0/cv2/act/Sigmoid_2swish']
D replace_exswish: remove node = ['/model.15/cv2/act/Sigmoid', '/model.15/cv2/act/Mul'], add node = ['/model.15/cv2/act/Sigmoid_2swish']
D replace_exswish: remove node = ['/model.16/act/Sigmoid', '/model.16/act/Mul'], add node = ['/model.16/act/Sigmoid_2swish']
D replace_exswish: remove node = ['/model.18/cv1/act/Sigmoid', '/model.18/cv1/act/Mul'], add node = ['/model.18/cv1/act/Sigmoid_2swish']
D replace_exswish: remove node = ['/model.18/m.0/cv1/act/Sigmoid', '/model.18/m.0/cv1/act/Mul'], add node = ['/model.18/m.0/cv1/act/Sigmoid_2swish']
D replace_exswish: remove node = ['/model.18/m.0/cv2/act/Sigmoid', '/model.18/m.0/cv2/act/Mul'], add node = ['/model.18/m.0/cv2/act/Sigmoid_2swish']
D replace_exswish: remove node = ['/model.18/cv2/act/Sigmoid', '/model.18/cv2/act/Mul'], add node = ['/model.18/cv2/act/Sigmoid_2swish']
D replace_exswish: remove node = ['/model.19/act/Sigmoid', '/model.19/act/Mul'], add node = ['/model.19/act/Sigmoid_2swish']
D replace_exswish: remove node = ['/model.21/cv1/act/Sigmoid', '/model.21/cv1/act/Mul'], add node = ['/model.21/cv1/act/Sigmoid_2swish']
D replace_exswish: remove node = ['/model.21/m.0/cv1/act/Sigmoid', '/model.21/m.0/cv1/act/Mul'], add node = ['/model.21/m.0/cv1/act/Sigmoid_2swish']
D replace_exswish: remove node = ['/model.21/m.0/cv2/act/Sigmoid', '/model.21/m.0/cv2/act/Mul'], add node = ['/model.21/m.0/cv2/act/Sigmoid_2swish']
D replace_exswish: remove node = ['/model.21/cv2/act/Sigmoid', '/model.21/cv2/act/Mul'], add node = ['/model.21/cv2/act/Sigmoid_2swish']
D replace_exswish: remove node = ['/model.22/cv2.2/cv2.2.0/act/Sigmoid', '/model.22/cv2.2/cv2.2.0/act/Mul'], add node = ['/model.22/cv2.2/cv2.2.0/act/Sigmoid_2swish']
D replace_exswish: remove node = ['/model.22/cv2.2/cv2.2.1/act/Sigmoid', '/model.22/cv2.2/cv2.2.1/act/Mul'], add node = ['/model.22/cv2.2/cv2.2.1/act/Sigmoid_2swish']
D replace_exswish: remove node = ['/model.22/cv3.2/cv3.2.0/act/Sigmoid', '/model.22/cv3.2/cv3.2.0/act/Mul'], add node = ['/model.22/cv3.2/cv3.2.0/act/Sigmoid_2swish']
D replace_exswish: remove node = ['/model.22/cv3.2/cv3.2.1/act/Sigmoid', '/model.22/cv3.2/cv3.2.1/act/Mul'], add node = ['/model.22/cv3.2/cv3.2.1/act/Sigmoid_2swish']
D replace_exswish: remove node = ['/model.22/cv2.1/cv2.1.0/act/Sigmoid', '/model.22/cv2.1/cv2.1.0/act/Mul'], add node = ['/model.22/cv2.1/cv2.1.0/act/Sigmoid_2swish']
D replace_exswish: remove node = ['/model.22/cv2.1/cv2.1.1/act/Sigmoid', '/model.22/cv2.1/cv2.1.1/act/Mul'], add node = ['/model.22/cv2.1/cv2.1.1/act/Sigmoid_2swish']
D replace_exswish: remove node = ['/model.22/cv3.1/cv3.1.0/act/Sigmoid', '/model.22/cv3.1/cv3.1.0/act/Mul'], add node = ['/model.22/cv3.1/cv3.1.0/act/Sigmoid_2swish']
D replace_exswish: remove node = ['/model.22/cv3.1/cv3.1.1/act/Sigmoid', '/model.22/cv3.1/cv3.1.1/act/Mul'], add node = ['/model.22/cv3.1/cv3.1.1/act/Sigmoid_2swish']
D replace_exswish: remove node = ['/model.22/cv2.0/cv2.0.0/act/Sigmoid', '/model.22/cv2.0/cv2.0.0/act/Mul'], add node = ['/model.22/cv2.0/cv2.0.0/act/Sigmoid_2swish']
D replace_exswish: remove node = ['/model.22/cv2.0/cv2.0.1/act/Sigmoid', '/model.22/cv2.0/cv2.0.1/act/Mul'], add node = ['/model.22/cv2.0/cv2.0.1/act/Sigmoid_2swish']
D replace_exswish: remove node = ['/model.22/cv3.0/cv3.0.0/act/Sigmoid', '/model.22/cv3.0/cv3.0.0/act/Mul'], add node = ['/model.22/cv3.0/cv3.0.0/act/Sigmoid_2swish']
D replace_exswish: remove node = ['/model.22/cv3.0/cv3.0.1/act/Sigmoid', '/model.22/cv3.0/cv3.0.1/act/Mul'], add node = ['/model.22/cv3.0/cv3.0.1/act/Sigmoid_2swish']
D replace_parallel_slice_by_split: remove node = ['/model.22/Slice', '/model.22/Slice_1'], add node = ['/model.22/Slice_2split']
D unsqueeze_to_4d_concat: remove node = [], add node = ['/model.22/Concat_3_0_unsqueeze0', '/model.22/Concat_3_1_unsqueeze0', '/model.22/Concat_3_2_unsqueeze0', '/model.22/Concat_3_0_unsqueeze1']
D unsqueeze_to_4d_split: remove node = [], add node = ['/model.22/Split_0_unsqueeze0', '/model.22/Split_0_unsqueeze1', '/model.22/Split_1_unsqueeze1']
D convert_softmax_to_exsoftmax13: remove node = ['/model.22/dfl/Softmax'], add node = ['/model.22/dfl/Softmax']
D unsqueeze_to_4d_split: remove node = [], add node = ['/model.22/Slice_2split_0_unsqueeze0', '/model.22/Slice_2split_0_unsqueeze1', '/model.22/Slice_2split_1_unsqueeze1']
D unsqueeze_to_4d_sub: remove node = [], add node = ['/model.22/Sub_1_unsqueeze0', '/model.22/Sub_0_unsqueeze1']
D unsqueeze_to_4d_add: remove node = [], add node = ['/model.22/Add_1_1_unsqueeze0', '/model.22/Add_1_0_unsqueeze1']
D unsqueeze_to_4d_add: remove node = [], add node = ['/model.22/Add_2_0_unsqueeze0', '/model.22/Add_2_1_unsqueeze0', '/model.22/Add_2_0_unsqueeze1']
D convert_div_to_mul: remove node = ['/model.22/Div_1'], add node = ['/model.22/Div_1_2mul']
D unsqueeze_to_4d_sub: remove node = [], add node = ['/model.22/Sub_1_0_unsqueeze0', '/model.22/Sub_1_1_unsqueeze0', '/model.22/Sub_1_0_unsqueeze1']
D unsqueeze_to_4d_concat: remove node = [], add node = ['/model.22/Concat_4_0_unsqueeze0', '/model.22/Concat_4_1_unsqueeze0', '/model.22/Concat_4_0_unsqueeze1']
D unsqueeze_to_4d_mul: remove node = [], add node = ['/model.22/Mul_2_0_unsqueeze0', '/model.22/Mul_2_0_unsqueeze1']
D unsqueeze_to_4d_sigmoid: remove node = [], add node = ['/model.22/Sigmoid_0_unsqueeze0', '/model.22/Sigmoid_0_unsqueeze1']
D unsqueeze_to_4d_concat: remove node = [], add node = ['/model.22/Concat_5_0_unsqueeze0', '/model.22/Concat_5_1_unsqueeze0', '/model.22/Concat_5_0_unsqueeze1']
D fuse_two_reshape: remove node = ['/model.22/Reshape_2', '/model.22/Reshape_1', '/model.22/Reshape', '/model.22/Concat_3_0_unsqueeze1', '/model.22/Split_0_unsqueeze1', '/model.22/dfl/Reshape_1', '/model.22/Slice_2split_0_unsqueeze1']
D remove_parallel_reshape: remove node = ['/model.22/Add_2_0_unsqueeze0']
D fuse_two_reshape: remove node = ['/model.22/Slice_2split_1_unsqueeze1']
D remove_parallel_reshape: remove node = ['/model.22/Add_2_1_unsqueeze0']
D unsqueeze_to_4d_mul: remove node = [], add node = ['/model.22/Div_1_2mul_0_unsqueeze0', '/model.22/Div_1_2mul_0_unsqueeze1']
D swap_concat_axis_avoid_channel_concat: remove node = [], add node = ['/model.22/Concat_4_swap_concat_reshape_i0_out', '/model.22/Concat_4_swap_concat_reshape_i1_out', '/model.22/Concat_4_swap_concat_reshape_o0_out']
D fuse_two_reshape: remove node = ['/model.22/Concat_4_0_unsqueeze1']
D input_align_4D_mul: remove node = ['/model.22/Mul_2'], add node = ['/model.22/Mul_2']
D fuse_two_reshape: remove node = ['/model.22/Mul_2_0_unsqueeze1', '/model.22/Split_1_unsqueeze1', '/model.22/Sigmoid_0_unsqueeze1']
D swap_concat_axis_avoid_channel_concat: remove node = [], add node = ['/model.22/Concat_5_swap_concat_reshape_i0_out', '/model.22/Concat_5_swap_concat_reshape_i1_out', '/model.22/Concat_5_swap_concat_reshape_o0_out']
D remove_invalid_reshape: remove node = ['/model.22/Split_0_unsqueeze0', '/model.22/Sub_1_unsqueeze0']
D fuse_two_reshape: remove node = ['/model.22/Sub_0_unsqueeze1']
D remove_invalid_reshape: remove node = ['/model.22/Add_1_1_unsqueeze0']
D fuse_two_reshape: remove node = ['/model.22/Add_1_0_unsqueeze1', '/model.22/Sub_1_0_unsqueeze1', '/model.22/Add_2_0_unsqueeze1', '/model.22/Div_1_2mul_0_unsqueeze1', '/model.22/Concat_4_swap_concat_reshape_o0_out', '/model.22/Concat_5_0_unsqueeze0']
D remove_invalid_reshape: remove node = ['/model.22/Sigmoid_0_unsqueeze0']
D fuse_two_reshape: remove node = ['/model.22/Concat_5_1_unsqueeze0', '/model.22/Concat_5_swap_concat_reshape_o0_out']
D remove_invalid_reshape: remove node = ['/model.22/Sub_1_1_unsqueeze0', '/model.22/Sub_1_0_unsqueeze0']
D fuse_two_reshape: remove node = ['/model.22/Concat_4_1_unsqueeze0']
D remove_invalid_reshape: remove node = ['/model.22/Div_1_2mul_0_unsqueeze0']
D fuse_two_reshape: remove node = ['/model.22/Concat_4_0_unsqueeze0']
D replace_batch_shuffle_transpose_by_gather_after_reshape: remove node = ['/model.22/dfl/Reshape', '/model.22/dfl/Transpose'], add node = ['/model.22/dfl/Reshape', '/model.22/dfl/Transpose_2_gather']
D convert_reshape_to_transpose: remove node = ['/model.22/Slice_2split_0_unsqueeze0'], add node = ['/model.22/Slice_2split_0_unsqueeze0']
D convert_reshape_to_transpose: remove node = ['/model.22/Concat_4_swap_concat_reshape_i1_out'], add node = ['/model.22/Concat_4_swap_concat_reshape_i1_out']
D convert_reshape_to_transpose: remove node = ['/model.22/Concat_4_swap_concat_reshape_i0_out'], add node = ['/model.22/Concat_4_swap_concat_reshape_i0_out']
D reduce_transpose_op_around_concat: remove node = ['/model.22/Concat_4'], add node = ['/model.22/Concat_4_swap_concat_reshape_i0_out_tp_4/model.22/Concat_4', '/model.22/Concat_4_swap_concat_reshape_i1_out_tp_4/model.22/Concat_4', '/model.22/Concat_4', '/model.22/Concat_4_output_0_shape4_tp_4/model.22/Concat_4']
D convert_reshape_to_transpose: remove node = ['/model.22/Mul_2_0_unsqueeze0'], add node = ['/model.22/Mul_2_0_unsqueeze0']
D convert_reshape_to_transpose: remove node = ['/model.22/Concat_5_swap_concat_reshape_i0_out'], add node = ['/model.22/Concat_5_swap_concat_reshape_i0_out']
D convert_reshape_to_transpose: remove node = ['/model.22/Concat_5_swap_concat_reshape_i1_out'], add node = ['/model.22/Concat_5_swap_concat_reshape_i1_out']
D reduce_transpose_op_around_concat: remove node = ['/model.22/Concat_5'], add node = ['/model.22/Concat_5_swap_concat_reshape_i0_out_tp_4/model.22/Concat_5', '/model.22/Concat_5_swap_concat_reshape_i1_out_tp_4/model.22/Concat_5', '/model.22/Concat_5', 'output0_shape4_tp_4/model.22/Concat_5']
D convert_transpose_to_reshape: remove node = ['/model.22/Slice_2split_0_unsqueeze0'], add node = ['/model.22/Slice_2split_0_unsqueeze0_2reshape']
D bypass_two_transpose: remove node = ['/model.22/Concat_4_swap_concat_reshape_i1_out_tp_4/model.22/Concat_4', '/model.22/Concat_4_swap_concat_reshape_i1_out', '/model.22/Concat_4_swap_concat_reshape_i0_out_tp_4/model.22/Concat_4', '/model.22/Concat_4_swap_concat_reshape_i0_out', '/model.22/Mul_2_0_unsqueeze0', '/model.22/Concat_4_output_0_shape4_tp_4/model.22/Concat_4', '/model.22/Concat_5_swap_concat_reshape_i0_out_tp_4/model.22/Concat_5', '/model.22/Concat_5_swap_concat_reshape_i0_out', '/model.22/Concat_5_swap_concat_reshape_i1_out_tp_4/model.22/Concat_5', '/model.22/Concat_5_swap_concat_reshape_i1_out']
D convert_transpose_to_reshape: remove node = ['output0_shape4_tp_4/model.22/Concat_5'], add node = ['output0_shape4_tp_4/model.22/Concat_5_2reshape']
D convert_reshape_to_transpose: remove node = ['/model.22/Slice_2split_0_unsqueeze0_2reshape'], add node = ['/model.22/Slice_2split_0_unsqueeze0_2reshape']
D fuse_two_reshape: remove node = ['output0_shape4_tp_4/model.22/Concat_5_2reshape']
D fold_constant ...
D fold_constant done.
D fuse_ops done.
D
D sparse_weight ...
D sparse_weight done.
D
D recover_const_share ...
D recover_const_share done.
D
I rknn building ...
I RKNN: [18:20:16.296] compress = 0, conv_eltwise_activation_fuse = 1, global_fuse = 1, multi-core-model-mode = 7, output_optimize = 1, layout_match = 1, enable_argb_group = 0
I RKNN: librknnc version: 2.0.0b0 (35a6907d79@2024-03-24T02:34:11)
D RKNN: [18:20:16.320] RKNN is invoked
W RKNN: [18:20:16.432] Model initializer tensor data is empty, name: empty_placeholder_0
D RKNN: [18:20:16.433] >>>>>> start: rknn::RKNNExtractCustomOpAttrs
D RKNN: [18:20:16.433] <<<<<<<< end: rknn::RKNNExtractCustomOpAttrs
D RKNN: [18:20:16.434] >>>>>> start: rknn::RKNNSetOpTargetPass
D RKNN: [18:20:16.434] <<<<<<<< end: rknn::RKNNSetOpTargetPass
D RKNN: [18:20:16.434] >>>>>> start: rknn::RKNNBindNorm
D RKNN: [18:20:16.434] <<<<<<<< end: rknn::RKNNBindNorm
D RKNN: [18:20:16.434] >>>>>> start: rknn::RKNNAddFirstConv
D RKNN: [18:20:16.434] <<<<<<<< end: rknn::RKNNAddFirstConv
D RKNN: [18:20:16.434] >>>>>> start: rknn::RKNNEliminateQATDataConvert
D RKNN: [18:20:16.434] <<<<<<<< end: rknn::RKNNEliminateQATDataConvert
D RKNN: [18:20:16.434] >>>>>> start: rknn::RKNNTileGroupConv
D RKNN: [18:20:16.434] <<<<<<<< end: rknn::RKNNTileGroupConv
D RKNN: [18:20:16.434] >>>>>> start: rknn::RKNNAddConvBias
D RKNN: [18:20:16.434] <<<<<<<< end: rknn::RKNNAddConvBias
D RKNN: [18:20:16.434] >>>>>> start: rknn::RKNNTileChannel
D RKNN: [18:20:16.434] <<<<<<<< end: rknn::RKNNTileChannel
D RKNN: [18:20:16.434] >>>>>> start: rknn::RKNNPerChannelPrep
D RKNN: [18:20:16.434] <<<<<<<< end: rknn::RKNNPerChannelPrep
D RKNN: [18:20:16.434] >>>>>> start: rknn::RKNNBnQuant
D RKNN: [18:20:16.434] <<<<<<<< end: rknn::RKNNBnQuant
D RKNN: [18:20:16.434] >>>>>> start: rknn::RKNNFuseOptimizerPass
D RKNN: [18:20:16.448] <<<<<<<< end: rknn::RKNNFuseOptimizerPass
D RKNN: [18:20:16.448] >>>>>> start: rknn::RKNNTurnAutoPad
D RKNN: [18:20:16.448] <<<<<<<< end: rknn::RKNNTurnAutoPad
D RKNN: [18:20:16.448] >>>>>> start: rknn::RKNNInitRNNConst
D RKNN: [18:20:16.448] <<<<<<<< end: rknn::RKNNInitRNNConst
D RKNN: [18:20:16.448] >>>>>> start: rknn::RKNNInitCastConst
D RKNN: [18:20:16.448] <<<<<<<< end: rknn::RKNNInitCastConst
D RKNN: [18:20:16.448] >>>>>> start: rknn::RKNNMultiSurfacePass
D RKNN: [18:20:16.448] <<<<<<<< end: rknn::RKNNMultiSurfacePass
D RKNN: [18:20:16.448] >>>>>> start: rknn::RKNNReplaceConstantTensorPass
D RKNN: [18:20:16.449] <<<<<<<< end: rknn::RKNNReplaceConstantTensorPass
D RKNN: [18:20:16.449] >>>>>> start: rknn::RKNNTilingPass
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 4200, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 4200, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 4200, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2100, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 4200, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 4200, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 4200, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2100, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2100, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2101, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2101, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2102, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2102, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2103, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2103, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2104, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2104, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2105, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2105, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2106, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2106, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2107, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2107, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2108, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2108, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2109, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2109, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2110, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2110, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2111, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2111, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2112, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2112, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2113, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2113, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2114, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2114, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2115, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2115, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2116, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2116, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2117, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2117, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2118, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2118, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2119, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2119, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2120, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2120, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2121, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2121, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2122, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2122, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2123, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2123, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2124, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2124, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2125, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2125, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2126, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2126, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2127, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2127, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2128, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2128, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2129, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2129, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2130, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2130, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2131, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2131, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2132, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2132, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2133, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2133, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2134, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2134, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2135, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2135, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2136, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2136, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2137, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2137, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2138, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2138, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2139, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2139, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2140, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2140, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2141, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2141, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2142, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2142, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2143, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2143, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2144, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2144, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2145, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2145, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2146, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2146, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2147, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2147, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2148, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2148, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2149, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2149, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2150, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2150, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2151, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2151, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2152, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2152, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2153, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2153, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2154, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2154, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2155, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2155, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2156, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2156, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2157, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2157, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2158, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2158, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2159, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2159, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2160, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2160, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2161, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2161, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2162, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2162, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2163, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2163, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2164, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2164, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2165, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2165, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2166, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2166, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2167, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2167, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2168, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2168, limitation: 2048
D RKNN: [18:20:16.450] DatainEntries overflow, need to tiling more, datain_entries: 2169, limitation: 2048
[ ....................... Repeated ...........................]
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2757, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2757, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2758, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2758, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2759, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2759, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2760, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2760, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2761, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2761, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2762, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2762, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2763, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2763, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2764, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2764, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2765, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2765, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2766, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2766, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2767, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2767, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2768, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2768, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2769, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2769, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2770, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2770, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2771, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2771, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2772, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2772, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2773, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2773, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2774, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2774, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2775, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2775, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2776, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2776, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2777, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2777, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2778, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2778, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2779, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2779, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2780, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2780, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2781, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2781, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2782, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2782, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2783, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2783, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2784, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2784, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2785, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2785, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2786, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2786, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2787, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2787, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2788, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2788, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2789, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2789, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2790, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2790, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2791, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2791, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2792, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2792, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2793, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2793, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2794, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2794, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2795, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2795, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2796, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2796, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2797, limitation: 2048
D RKNN: [18:20:16.482] DatainEntries overflow, need to tiling more, datain_entries: 2797, limitation: 2048
D RKNN: [18:20:16.483] DatainEntries overflow, need to tiling more, datain_entries: 2798, limitation: 2048
D RKNN: [18:20:16.483] DatainEntries overflow, need to tiling more, datain_entries: 2798, limitation: 2048
D RKNN: [18:20:16.483] DatainEntries overflow, need to tiling more, datain_entries: 2799, limitation: 2048
D RKNN: [18:20:16.483] DatainEntries overflow, need to tiling more, datain_entries: 2799, limitation: 2048
D RKNN: [18:20:16.483] DatainEntries overflow, need to tiling more, datain_entries: 2800, limitation: 2048
D RKNN: [18:20:16.483] DatainEntries overflow, need to tiling more, datain_entries: 2800, limitation: 2048
D RKNN: [18:20:16.483] DatainEntries overflow, need to tiling more, datain_entries: 2801, limitation: 2048
D RKNN: [18:20:16.483] DatainEntries overflow, need to tiling more, datain_entries: 2801, limitation: 2048
D RKNN: [18:20:16.483] DatainEntries overflow, need to tiling more, datain_entries: 2802, limitation: 2048
D RKNN: [18:20:16.483] DatainEntries overflow, need to tiling more, datain_entries: 2802, limitation: 2048
D RKNN: [18:20:16.483] DatainEntries overflow, need to tiling more, datain_entries: 2803, limitation: 2048
D RKNN: [18:20:16.483] DatainEntries overflow, need to tiling more, datain_entries: 2803, limitation: 2048
D RKNN: [18:20:16.483] DatainEntries overflow, need to tiling more, datain_entries: 2804, limitation: 2048
D RKNN: [18:20:16.483] DatainEntries overflow, need to tiling more, datain_entries: 2804, limitation: 2048
D RKNN: [18:20:16.483] DatainEntries overflow, need to tiling more, datain_entries: 2805, limitation: 2048
D RKNN: [18:20:16.483] DatainEntries overflow, need to tiling more, datain_entries: 2805, limitation: 2048
D RKNN: [18:20:16.483] DatainEntries overflow, need to tiling more, datain_entries: 2806, limitation: 2048
D RKNN: [18:20:16.483] DatainEntries overflow, need to tiling more, datain_entries: 2806, limitation: 2048
D RKNN: [18:20:16.483] DatainEntries overflow, need to tiling more, datain_entries: 2807, limitation: 2048
D RKNN: [18:20:16.483] DatainEntries overflow, need to tiling more, datain_entries: 2807, limitation: 2048
D RKNN: [18:20:16.483] DatainEntries overflow, need to tiling more, datain_entries: 2808, limitation: 2048
D RKNN: [18:20:16.483] DatainEntries overflow, need to tiling more, datain_entries: 2808, limitation: 2048
D RKNN: [18:20:16.483] DatainEntries overflow, need to tiling more, datain_entries: 2809, limitation: 2048
D RKNN: [18:20:16.483] DatainEntries overflow, need to tiling more, datain_entries: 2809, limitation: 2048
D RKNN: [18:20:16.483] DatainEntries overflow, need to tiling more, datain_entries: 2810, limitation: 2048
D RKNN: [18:20:16.483] DatainEntries overflow, need to tiling more, datain_entries: 2810, limitation: 2048
D RKNN: [18:20:16.483] DatainEntries overflow, need to tiling more, datain_entries: 2811, limitation: 2048
D RKNN: [18:20:16.483] DatainEntries overflow, need to tiling more, datain_entries: 2811, limitation: 2048
D RKNN: [18:20:16.483] DatainEntries overflow, need to tiling more, datain_entries: 2812, limitation: 2048
D RKNN: [18:20:16.483] DatainEntries overflow, need to tiling more, datain_entries: 2812, limitation: 2048
D RKNN: [18:20:16.483] DatainEntries overflow, need to tiling more, datain_entries: 2813, limitation: 2048
D RKNN: [18:20:16.483] DatainEntries overflow, need to tiling more, datain_entries: 2813, limitation: 2048
D RKNN: [18:20:16.483] DatainEntries overflow, need to tiling more, datain_entries: 2814, limitation: 2048
D RKNN: [18:20:16.483] DatainEntries overflow, need to tiling more, datain_entries: 2814, limitation: 2048
D RKNN: [18:20:16.483] DatainEntries overflow, need to tiling more, datain_entries: 2815, limitation: 2048
D RKNN: [18:20:16.483] DatainEntries overflow, need to tiling more, datain_entries: 2815, limitation: 2048
D RKNN: [18:20:16.483] DatainEntries overflow, need to tiling more, datain_entries: 2816, limitation: 2048
D RKNN: [18:20:16.483] DatainEntries overflow, need to tiling more, datain_entries: 4200, limitation: 2048
D RKNN: [18:20:16.483] DatainEntries overflow, need to tiling more, datain_entries: 2100, limitation: 2048
D RKNN: [18:20:16.483] DatainEntries overflow, need to tiling more, datain_entries: 4200, limitation: 2048
D RKNN: [18:20:16.483] DatainEntries overflow, need to tiling more, datain_entries: 2100, limitation: 2048
D RKNN: [18:20:16.483] DatainEntries overflow, need to tiling more, datain_entries: 4200, limitation: 2048
D RKNN: [18:20:16.483] DatainEntries overflow, need to tiling more, datain_entries: 2100, limitation: 2048
D RKNN: [18:20:16.483] <<<<<<<< end: rknn::RKNNTilingPass
D RKNN: [18:20:16.483] >>>>>> start: rknn::RKNNSubgraphManager
D RKNN: [18:20:16.483] <<<<<<<< end: rknn::RKNNSubgraphManager
D RKNN: [18:20:16.483] >>>>>> start: OpEmit
W RKNN: [18:20:16.484] Transpose will fallback to CPU, because input shape has exceeded the max limit, height(8) * width(8400) = 67200, required product no larger than 8192!
D RKNN: [18:20:16.484] <<<<<<<< end: OpEmit
D RKNN: [18:20:16.484] >>>>>> start: rknn::RKNNLayoutMatchPass
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[images]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.0/act/Mul_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.1/act/Mul_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.2/cv1/act/Mul_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.2/Split_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.2/Split_output_1]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.2/m.0/cv1/act/Mul_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.2/m.0/cv2/act/Mul_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.2/m.0/Add_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.2/Concat_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.2/cv2/act/Mul_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.3/act/Mul_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.4/cv1/act/Mul_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.4/Split_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.4/Split_output_1]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.4/m.0/cv1/act/Mul_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.4/m.0/cv2/act/Mul_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.4/m.0/Add_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.4/m.1/cv1/act/Mul_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.4/m.1/cv2/act/Mul_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.4/m.1/Add_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.4/Concat_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.5/act/Mul_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.6/cv1/act/Mul_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.6/Split_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.6/Split_output_1]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.6/m.0/cv1/act/Mul_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.6/m.0/cv2/act/Mul_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.6/m.0/Add_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.6/m.1/cv1/act/Mul_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.6/m.1/cv2/act/Mul_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.6/m.1/Add_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.6/Concat_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.7/act/Mul_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.8/cv1/act/Mul_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.8/Split_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.8/Split_output_1]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.8/m.0/cv1/act/Mul_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.8/m.0/cv2/act/Mul_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.8/m.0/Add_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.8/Concat_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.8/cv2/act/Mul_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.9/cv1/act/Mul_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.9/m/MaxPool_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.9/m_1/MaxPool_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.9/m_2/MaxPool_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.9/Concat_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.12/cv1/act/Mul_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.12/Split_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.12/Split_output_1]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.12/m.0/cv1/act/Mul_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.12/m.0/cv2/act/Mul_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.12/Concat_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.15/cv1/act/Mul_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.15/Split_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.15/Split_output_1]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.15/m.0/cv1/act/Mul_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.15/m.0/cv2/act/Mul_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.15/Concat_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.15/cv2/act/Mul_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.22/cv2.0/cv2.0.0/act/Mul_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.22/cv3.0/cv3.0.0/act/Mul_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.18/cv1/act/Mul_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.18/Split_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.18/Split_output_1]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.22/cv2.0/cv2.0.1/act/Mul_output_0]
W RKNN: [18:20:16.484] LayoutMatchManager: recursion_depth=3, Logic is Dangerous, Will Force layout to native.
W RKNN: [18:20:16.484] LayoutMatchManager: recursion_depth=3, Logic is Dangerous, Will Force layout to native.
W RKNN: [18:20:16.484] LayoutMatchManager: recursion_depth=3, Logic is Dangerous, Will Force layout to native.
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.22/cv2.0/cv2.0.2/Conv_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.22/cv3.0/cv3.0.1/act/Mul_output_0]
W RKNN: [18:20:16.484] LayoutMatchManager: recursion_depth=3, Logic is Dangerous, Will Force layout to native.
W RKNN: [18:20:16.484] LayoutMatchManager: recursion_depth=3, Logic is Dangerous, Will Force layout to native.
W RKNN: [18:20:16.484] LayoutMatchManager: recursion_depth=3, Logic is Dangerous, Will Force layout to native.
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.22/cv3.0/cv3.0.2/Conv_output_0]
W RKNN: [18:20:16.484] LayoutMatchManager: recursion_depth=3, Logic is Dangerous, Will Force layout to native.
W RKNN: [18:20:16.484] LayoutMatchManager: recursion_depth=3, Logic is Dangerous, Will Force layout to native.
W RKNN: [18:20:16.484] LayoutMatchManager: recursion_depth=3, Logic is Dangerous, Will Force layout to native.
W RKNN: [18:20:16.484] LayoutMatchManager: recursion_depth=3, Logic is Dangerous, Will Force layout to native.
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.22/Concat_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.22/Reshape_output_0_shape4_/model.22/Concat_3]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.18/m.0/cv1/act/Mul_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.18/m.0/cv2/act/Mul_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.18/Concat_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.18/cv2/act/Mul_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.22/cv2.1/cv2.1.0/act/Mul_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.22/cv3.1/cv3.1.0/act/Mul_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.21/cv1/act/Mul_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.21/Split_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.21/Split_output_1]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.22/cv2.1/cv2.1.1/act/Mul_output_0]
W RKNN: [18:20:16.484] LayoutMatchManager: recursion_depth=3, Logic is Dangerous, Will Force layout to native.
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.22/cv2.1/cv2.1.2/Conv_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.22/cv3.1/cv3.1.1/act/Mul_output_0]
W RKNN: [18:20:16.484] LayoutMatchManager: recursion_depth=3, Logic is Dangerous, Will Force layout to native.
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.22/cv3.1/cv3.1.2/Conv_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.22/Concat_1_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.22/Reshape_1_output_0_shape4_/model.22/Concat_3]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.21/m.0/cv1/act/Mul_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.21/m.0/cv2/act/Mul_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.21/Concat_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.21/cv2/act/Mul_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.22/cv2.2/cv2.2.0/act/Mul_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.22/cv3.2/cv3.2.0/act/Mul_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.22/cv2.2/cv2.2.1/act/Mul_output_0]
W RKNN: [18:20:16.484] LayoutMatchManager: recursion_depth=3, Logic is Dangerous, Will Force layout to native.
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.22/cv2.2/cv2.2.2/Conv_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.22/cv3.2/cv3.2.1/act/Mul_output_0]
W RKNN: [18:20:16.484] LayoutMatchManager: recursion_depth=3, Logic is Dangerous, Will Force layout to native.
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.22/cv3.2/cv3.2.2/Conv_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.22/Concat_2_output_0]
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.22/Reshape_2_output_0_shape4_/model.22/Concat_3]
W RKNN: [18:20:16.484] LayoutMatchManager: recursion_depth=3, Logic is Dangerous, Will Force layout to native.
W RKNN: [18:20:16.484] LayoutMatchManager: recursion_depth=3, Logic is Dangerous, Will Force layout to native.
W RKNN: [18:20:16.484] LayoutMatchManager: recursion_depth=3, Logic is Dangerous, Will Force layout to native.
W RKNN: [18:20:16.484] LayoutMatchManager: recursion_depth=3, Logic is Dangerous, Will Force layout to native.
W RKNN: [18:20:16.484] LayoutMatchManager: recursion_depth=3, Logic is Dangerous, Will Force layout to native.
W RKNN: [18:20:16.484] LayoutMatchManager: recursion_depth=3, Logic is Dangerous, Will Force layout to native.
I RKNN: [18:20:16.484] AppointLayout: t->setNativeLayout(64), tname:[/model.22/Concat_3_output_0]
W RKNN: [18:20:16.485] LayoutMatchManager: recursion_depth=3, Logic is Dangerous, Will Force layout to native.
I RKNN: [18:20:16.485] AppointLayout: t->setNativeLayout(64), tname:[/model.22/dfl/Softmax_output_0]
I RKNN: [18:20:16.485] AppointLayout: t->setNativeLayout(64), tname:[/model.22/Slice_output_0]
I RKNN: [18:20:16.485] AppointLayout: t->setNativeLayout(64), tname:[/model.22/Slice_1_output_0]
I RKNN: [18:20:16.485] AppointLayout: t->setNativeLayout(64), tname:[/model.22/Sub_output_0]
I RKNN: [18:20:16.485] AppointLayout: t->setNativeLayout(64), tname:[/model.22/Add_1_output_0]
I RKNN: [18:20:16.485] AppointLayout: t->setNativeLayout(64), tname:[/model.22/Sub_1_output_0]
I RKNN: [18:20:16.485] AppointLayout: t->setNativeLayout(64), tname:[/model.22/Add_2_output_0]
I RKNN: [18:20:16.485] AppointLayout: t->setNativeLayout(64), tname:[/model.22/Div_1_output_0]
I RKNN: [18:20:16.485] AppointLayout: t->setNativeLayout(64), tname:[/model.22/Concat_4_output_0_shape4_tp_4/model.22/Concat_4]
W RKNN: [18:20:16.485] LayoutMatchManager: recursion_depth=3, Logic is Dangerous, Will Force layout to native.
D RKNN: [18:20:16.485] <<<<<<<< end: rknn::RKNNLayoutMatchPass
D RKNN: [18:20:16.485] >>>>>> start: rknn::RKNNAddSecondaryNode
D RKNN: [18:20:16.485] <<<<<<<< end: rknn::RKNNAddSecondaryNode
D RKNN: [18:20:16.485] >>>>>> start: OpEmit
D RKNN: [18:20:16.511] finish initComputeZoneMapByStepsVector
D RKNN: [18:20:16.511] finish initComputeZoneMapByStepsVector
D RKNN: [18:20:16.511] finish initComputeZoneMapByStepsVector
D RKNN: [18:20:16.511] finish initComputeZoneMapByStepsVector
D RKNN: [18:20:16.511] not need tranpose
D RKNN: [18:20:16.511] not need tranpose
D RKNN: [18:20:16.511] finish initComputeZoneMap
D RKNN: [18:20:16.511] emit max
E RKNN: [18:20:16.512] failed to config argb mode layer!
make: *** [Makefile:13: output/rk3588/yolov8n_custom.rknn] Aborted
RKNN自定义算子的性能非常差/提供零拷贝的自定义算子api？
rknn默认的InstanceNorm算子实现性能太差了，因此我打算使用自定义算子功能替换成高效的实现
然而尽管我实现的算子的计算时间约为7ms，但RKNN的性能统计中这个自定义算子的计算时间高达13.6ms，性能非常糟糕，不知道这6.6ms差距是怎么来的？
是否是拷贝带来的开销？能否提供零拷贝的api？
这次的算子是在CPU上实现的，GPU上也测试过但无法使用：#15
算子计算实现:
int compute_cb(rknn_custom_op_context* op_ctx, rknn_custom_op_tensor* inputs, uint32_t n_inputs, rknn_custom_op_tensor* outputs, uint32_t n_outputs){
std::chrono::steady_clock::time_point begin = std::chrono::steady_clock::now();
auto res = static_cast<tvm_op_resources*>(op_ctx->priv_data);
auto input = to_tvm_ndarray(inputs[0]);
res->out_buf = res->op_func(input);
//复制到output那里
res->out_buf.CopyToBytes((uint8_t*)outputs[0].mem.virt_addr + outputs[0].mem.offset, outputs[0].mem.size);
decltype(累计时间) deltaTime = std::chrono::steady_clock::now() - begin;
累计时间 += deltaTime;
std::cout << "InstanceNormalization_1_256_64_64 耗时: " << deltaTime.count() * 1000 << "ms" << "\n";
return 0;
}算子库的输出日志:
InstanceNormalization_1_256_64_64 耗时: 7.08817ms
InstanceNormalization_1_256_64_64 耗时: 7.15846ms
InstanceNormalization_1_256_64_64 耗时: 7.02225ms
InstanceNormalization_1_256_64_64 耗时: 6.97034ms
InstanceNormalization_1_256_64_64 耗时: 7.12084ms
InstanceNormalization_1_256_64_64 耗时: 6.94788ms
InstanceNormalization_1_256_64_64 耗时: 6.95167ms
InstanceNormalization_1_256_64_64 耗时: 6.95692ms
InstanceNormalization_1_256_64_64 耗时: 6.8875ms
...
RKNN的性能统计:
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
ID OpType DataType Target InputShape OutputShape Cycles(DDR/NPU/Total) Time(us) MacUsage(%) WorkLoad(0/1/2) RW(KB) FullName
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
1 InputOperator FLOAT16 CPU \ (1,3,256,256) 0/0/0 3 0.0%/0.0%/0.0% 0 InputOperator:input
2 Conv FLOAT16 NPU (1,3,256,256),(3,3,1,1) (1,3,256,256) 60956/131072/131072 639 0.18/0.00/0.00 100.0%/0.0%/0.0% 384
3 Pad FLOAT16 CPU (1,3,256,256),(8) (1,3,262,262) 0/0/0 1122 0.0%/0.0%/0.0% 1024 Pad:/model/model.0/Pad
4 Conv FLOAT16 NPU (1,3,262,262),(64,3,7,7),(64) (1,64,256,256) 403203/12845056/12845056 10158 11.85/0.00/0.00 100.0%/0.0%/0.0% 1121 Conv:/model/model.1/Conv
5 Reshape FLOAT16 NPU (1,64,256,256),(4) (64,65536,1,1) 0/0/0 3097 100.0%/0.0%/0.0% 8192 Reshape:/model/model.2/InstanceNormalization_2ln_reshape1
6 exLayerNorm FLOAT16 NPU (64,65536,1,1),(1,65536,1,1) (64,65536,1,1) 0/0/0 20473 100.0%/0.0%/0.0% 8320 exLayerNorm:/model/model.2/InstanceNormalization_2ln
7 Relu FLOAT16 NPU (64,65536,1,1) (64,65536,1,1) 0/0/0 4939 100.0%/0.0%/0.0% 8192 Relu:/model/model.3/Relu
8 Reshape FLOAT16 NPU (64,65536,1,1),(4) (1,64,256,256) 0/0/0 6844 100.0%/0.0%/0.0% 8192 Reshape:/model/model.2/InstanceNormalization_2ln_reshape2
9 Conv FLOAT16 NPU (1,64,256,256),(128,64,3,3),(128) (1,128,128,128) 538213/2359296/2359296 2824 83.54/0.00/0.00 100.0%/0.0%/0.0% 8336 Conv:/model/model.4/Conv
10 Reshape FLOAT16 NPU (1,128,128,128),(4) (128,16384,1,1) 0/0/0 948 100.0%/0.0%/0.0% 4096 Reshape:/model/model.5/InstanceNormalization_2ln_reshape1
11 exLayerNorm FLOAT16 NPU (128,16384,1,1),(1,16384,1,1) (128,16384,1,1) 0/0/0 10580 100.0%/0.0%/0.0% 4128 exLayerNorm:/model/model.5/InstanceNormalization_2ln
12 Relu FLOAT16 NPU (128,16384,1,1) (128,16384,1,1) 0/0/0 2515 100.0%/0.0%/0.0% 4096 Relu:/model/model.6/Relu
13 Reshape FLOAT16 NPU (128,16384,1,1),(4) (1,128,128,128) 0/0/0 3420 100.0%/0.0%/0.0% 4096 Reshape:/model/model.5/InstanceNormalization_2ln_reshape2
14 Conv FLOAT16 NPU (1,128,128,128),(256,128,3,3),(256) (1,256,64,64) 290958/2359296/2359296 2385 98.92/0.00/0.00 100.0%/0.0%/0.0% 4673 Conv:/model/model.7/Conv
15 cstInstanceNormalization_1_256_64_64 FLOAT16 CPU (1,256,64,64),(256),(256) (1,256,64,64) 0/0/0 25335 0.0%/0.0%/0.0% 2050 cstInstanceNormalization_1_256_64_64:/model/model.8/InstanceNormalization
16 Relu FLOAT16 NPU (1,256,64,64) (1,256,64,64) 0/0/0 352 100.0%/0.0%/0.0% 2048 Relu:/model/model.9/Relu
17 Pad FLOAT16 CPU (1,256,64,64),(8) (1,256,66,66) 0/0/0 5031 0.0%/0.0%/0.0% 2048 Pad:/model/model.10/conv_block/conv_block.0/Pad
18 Conv FLOAT16 NPU (1,256,66,66),(256,256,3,3),(256) (1,256,64,64) 232862/4718592/4718592 4234 99.00/0.00/0.00 100.0%/0.0%/0.0% 3331 Conv:/model/model.10/conv_block/conv_block.1/Conv
19 cstInstanceNormalization_1_256_64_64 FLOAT16 CPU (1,256,64,64),(256),(256) (1,256,64,64) 0/0/0 14105 0.0%/0.0%/0.0% 2050 cstInstanceNormalization_1_256_64_64:/model/model.10/conv_block/conv_block.2/InstanceNormalization
20 Relu FLOAT16 NPU (1,256,64,64) (1,256,64,64) 0/0/0 301 100.0%/0.0%/0.0% 2048 Relu:/model/model.10/conv_block/conv_block.3/Relu
21 Pad FLOAT16 CPU (1,256,64,64),(8) (1,256,66,66) 0/0/0 4561 0.0%/0.0%/0.0% 2048 Pad:/model/model.10/conv_block/conv_block.4/Pad
22 Conv FLOAT16 NPU (1,256,66,66),(256,256,3,3),(256) (1,256,64,64) 232862/4718592/4718592 4186 99.00/0.00/0.00 100.0%/0.0%/0.0% 3331 Conv:/model/model.10/conv_block/conv_block.5/Conv
23 cstInstanceNormalization_1_256_64_64 FLOAT16 CPU (1,256,64,64),(256),(256) (1,256,64,64) 0/0/0 13991 0.0%/0.0%/0.0% 2050 cstInstanceNormalization_1_256_64_64:/model/model.10/conv_block/conv_block.6/InstanceNormalization
24 Add FLOAT16 NPU (1,256,64,64),(1,256,64,64) (1,256,64,64) 0/0/0 452 100.0%/0.0%/0.0% 4096 Add:/model/model.10/Add
25 Pad FLOAT16 CPU (1,256,64,64),(8) (1,256,66,66) 0/0/0 2920 0.0%/0.0%/0.0% 2048 Pad:/model/model.11/conv_block/conv_block.0/Pad
26 Conv FLOAT16 NPU (1,256,66,66),(256,256,3,3),(256) (1,256,64,64) 232862/4718592/4718592 4193 99.00/0.00/0.00 100.0%/0.0%/0.0% 3331 Conv:/model/model.11/conv_block/conv_block.1/Conv
27 cstInstanceNormalization_1_256_64_64 FLOAT16 CPU (1,256,64,64),(256),(256) (1,256,64,64) 0/0/0 14112 0.0%/0.0%/0.0% 2050 cstInstanceNormalization_1_256_64_64:/model/model.11/conv_block/conv_block.2/InstanceNormalization
28 Relu FLOAT16 NPU (1,256,64,64) (1,256,64,64) 0/0/0 325 100.0%/0.0%/0.0% 2048 Relu:/model/model.11/conv_block/conv_block.3/Relu
29 Pad FLOAT16 CPU (1,256,64,64),(8) (1,256,66,66) 0/0/0 2848 0.0%/0.0%/0.0% 2048 Pad:/model/model.11/conv_block/conv_block.4/Pad
30 Conv FLOAT16 NPU (1,256,66,66),(256,256,3,3),(256) (1,256,64,64) 232862/4718592/4718592 4190 99.00/0.00/0.00 100.0%/0.0%/0.0% 3331 Conv:/model/model.11/conv_block/conv_block.5/Conv
31 cstInstanceNormalization_1_256_64_64 FLOAT16 CPU (1,256,64,64),(256),(256) (1,256,64,64) 0/0/0 13780 0.0%/0.0%/0.0% 2050 cstInstanceNormalization_1_256_64_64:/model/model.11/conv_block/conv_block.6/InstanceNormalization
32 Add FLOAT16 NPU (1,256,64,64),(1,256,64,64) (1,256,64,64) 0/0/0 506 100.0%/0.0%/0.0% 4096 Add:/model/model.11/Add
33 Pad FLOAT16 CPU (1,256,64,64),(8) (1,256,66,66) 0/0/0 2894 0.0%/0.0%/0.0% 2048 Pad:/model/model.12/conv_block/conv_block.0/Pad
34 Conv FLOAT16 NPU (1,256,66,66),(256,256,3,3),(256) (1,256,64,64) 232862/4718592/4718592 4240 99.00/0.00/0.00 100.0%/0.0%/0.0% 3331 Conv:/model/model.12/conv_block/conv_block.1/Conv
35 cstInstanceNormalization_1_256_64_64 FLOAT16 CPU (1,256,64,64),(256),(256) (1,256,64,64) 0/0/0 13783 0.0%/0.0%/0.0% 2050 cstInstanceNormalization_1_256_64_64:/model/model.12/conv_block/conv_block.2/InstanceNormalization
36 Relu FLOAT16 NPU (1,256,64,64) (1,256,64,64) 0/0/0 313 100.0%/0.0%/0.0% 2048 Relu:/model/model.12/conv_block/conv_block.3/Relu
37 Pad FLOAT16 CPU (1,256,64,64),(8) (1,256,66,66) 0/0/0 2892 0.0%/0.0%/0.0% 2048 Pad:/model/model.12/conv_block/conv_block.4/Pad
38 Conv FLOAT16 NPU (1,256,66,66),(256,256,3,3),(256) (1,256,64,64) 232862/4718592/4718592 4238 99.00/0.00/0.00 100.0%/0.0%/0.0% 3331 Conv:/model/model.12/conv_block/conv_block.5/Conv
39 cstInstanceNormalization_1_256_64_64 FLOAT16 CPU (1,256,64,64),(256),(256) (1,256,64,64) 0/0/0 13935 0.0%/0.0%/0.0% 2050 cstInstanceNormalization_1_256_64_64:/model/model.12/conv_block/conv_block.6/InstanceNormalization
40 Add FLOAT16 NPU (1,256,64,64),(1,256,64,64) (1,256,64,64) 0/0/0 454 100.0%/0.0%/0.0% 4096 Add:/model/model.12/Add
41 Pad FLOAT16 CPU (1,256,64,64),(8) (1,256,66,66) 0/0/0 2888 0.0%/0.0%/0.0% 2048 Pad:/model/model.13/conv_block/conv_block.0/Pad
42 Conv FLOAT16 NPU (1,256,66,66),(256,256,3,3),(256) (1,256,64,64) 232862/4718592/4718592 4238 99.00/0.00/0.00 100.0%/0.0%/0.0% 3331 Conv:/model/model.13/conv_block/conv_block.1/Conv
43 cstInstanceNormalization_1_256_64_64 FLOAT16 CPU (1,256,64,64),(256),(256) (1,256,64,64) 0/0/0 13753 0.0%/0.0%/0.0% 2050 cstInstanceNormalization_1_256_64_64:/model/model.13/conv_block/conv_block.2/InstanceNormalization
44 Relu FLOAT16 NPU (1,256,64,64) (1,256,64,64) 0/0/0 303 100.0%/0.0%/0.0% 2048 Relu:/model/model.13/conv_block/conv_block.3/Relu
45 Pad FLOAT16 CPU (1,256,64,64),(8) (1,256,66,66) 0/0/0 2830 0.0%/0.0%/0.0% 2048 Pad:/model/model.13/conv_block/conv_block.4/Pad
46 Conv FLOAT16 NPU (1,256,66,66),(256,256,3,3),(256) (1,256,64,64) 232862/4718592/4718592 4235 99.00/0.00/0.00 100.0%/0.0%/0.0% 3331 Conv:/model/model.13/conv_block/conv_block.5/Conv
47 cstInstanceNormalization_1_256_64_64 FLOAT16 CPU (1,256,64,64),(256),(256) (1,256,64,64) 0/0/0 13713 0.0%/0.0%/0.0% 2050 cstInstanceNormalization_1_256_64_64:/model/model.13/conv_block/conv_block.6/InstanceNormalization
48 Add FLOAT16 NPU (1,256,64,64),(1,256,64,64) (1,256,64,64) 0/0/0 495 100.0%/0.0%/0.0% 4096 Add:/model/model.13/Add
49 Pad FLOAT16 CPU (1,256,64,64),(8) (1,256,66,66) 0/0/0 2834 0.0%/0.0%/0.0% 2048 Pad:/model/model.14/conv_block/conv_block.0/Pad
50 Conv FLOAT16 NPU (1,256,66,66),(256,256,3,3),(256) (1,256,64,64) 232862/4718592/4718592 4189 99.00/0.00/0.00 100.0%/0.0%/0.0% 3331 Conv:/model/model.14/conv_block/conv_block.1/Conv
51 cstInstanceNormalization_1_256_64_64 FLOAT16 CPU (1,256,64,64),(256),(256) (1,256,64,64) 0/0/0 13763 0.0%/0.0%/0.0% 2050 cstInstanceNormalization_1_256_64_64:/model/model.14/conv_block/conv_block.2/InstanceNormalization
52 Relu FLOAT16 NPU (1,256,64,64) (1,256,64,64) 0/0/0 352 100.0%/0.0%/0.0% 2048 Relu:/model/model.14/conv_block/conv_block.3/Relu
53 Pad FLOAT16 CPU (1,256,64,64),(8) (1,256,66,66) 0/0/0 2858 0.0%/0.0%/0.0% 2048 Pad:/model/model.14/conv_block/conv_block.4/Pad
54 Conv FLOAT16 NPU (1,256,66,66),(256,256,3,3),(256) (1,256,64,64) 232862/4718592/4718592 4235 99.00/0.00/0.00 100.0%/0.0%/0.0% 3331 Conv:/model/model.14/conv_block/conv_block.5/Conv
55 cstInstanceNormalization_1_256_64_64 FLOAT16 CPU (1,256,64,64),(256),(256) (1,256,64,64) 0/0/0 13577 0.0%/0.0%/0.0% 2050 cstInstanceNormalization_1_256_64_64:/model/model.14/conv_block/conv_block.6/InstanceNormalization
56 Add FLOAT16 NPU (1,256,64,64),(1,256,64,64) (1,256,64,64) 0/0/0 502 100.0%/0.0%/0.0% 4096 Add:/model/model.14/Add
57 Pad FLOAT16 CPU (1,256,64,64),(8) (1,256,66,66) 0/0/0 2869 0.0%/0.0%/0.0% 2048 Pad:/model/model.15/conv_block/conv_block.0/Pad
58 Conv FLOAT16 NPU (1,256,66,66),(256,256,3,3),(256) (1,256,64,64) 232862/4718592/4718592 4238 99.00/0.00/0.00 100.0%/0.0%/0.0% 3331 Conv:/model/model.15/conv_block/conv_block.1/Conv
59 cstInstanceNormalization_1_256_64_64 FLOAT16 CPU (1,256,64,64),(256),(256) (1,256,64,64) 0/0/0 13534 0.0%/0.0%/0.0% 2050 cstInstanceNormalization_1_256_64_64:/model/model.15/conv_block/conv_block.2/InstanceNormalization
60 Relu FLOAT16 NPU (1,256,64,64) (1,256,64,64) 0/0/0 304 100.0%/0.0%/0.0% 2048 Relu:/model/model.15/conv_block/conv_block.3/Relu
61 Pad FLOAT16 CPU (1,256,64,64),(8) (1,256,66,66) 0/0/0 2831 0.0%/0.0%/0.0% 2048 Pad:/model/model.15/conv_block/conv_block.4/Pad
62 Conv FLOAT16 NPU (1,256,66,66),(256,256,3,3),(256) (1,256,64,64) 232862/4718592/4718592 4238 99.00/0.00/0.00 100.0%/0.0%/0.0% 3331 Conv:/model/model.15/conv_block/conv_block.5/Conv
63 cstInstanceNormalization_1_256_64_64 FLOAT16 CPU (1,256,64,64),(256),(256) (1,256,64,64) 0/0/0 13467 0.0%/0.0%/0.0% 2050 cstInstanceNormalization_1_256_64_64:/model/model.15/conv_block/conv_block.6/InstanceNormalization
64 Add FLOAT16 NPU (1,256,64,64),(1,256,64,64) (1,256,64,64) 0/0/0 494 100.0%/0.0%/0.0% 4096 Add:/model/model.15/Add
65 Pad FLOAT16 CPU (1,256,64,64),(8) (1,256,66,66) 0/0/0 2853 0.0%/0.0%/0.0% 2048 Pad:/model/model.16/conv_block/conv_block.0/Pad
66 Conv FLOAT16 NPU (1,256,66,66),(256,256,3,3),(256) (1,256,64,64) 232862/4718592/4718592 4241 99.00/0.00/0.00 100.0%/0.0%/0.0% 3331 Conv:/model/model.16/conv_block/conv_block.1/Conv
67 cstInstanceNormalization_1_256_64_64 FLOAT16 CPU (1,256,64,64),(256),(256) (1,256,64,64) 0/0/0 13477 0.0%/0.0%/0.0% 2050 cstInstanceNormalization_1_256_64_64:/model/model.16/conv_block/conv_block.2/InstanceNormalization
68 Relu FLOAT16 NPU (1,256,64,64) (1,256,64,64) 0/0/0 351 100.0%/0.0%/0.0% 2048 Relu:/model/model.16/conv_block/conv_block.3/Relu
69 Pad FLOAT16 CPU (1,256,64,64),(8) (1,256,66,66) 0/0/0 2879 0.0%/0.0%/0.0% 2048 Pad:/model/model.16/conv_block/conv_block.4/Pad
70 Conv FLOAT16 NPU (1,256,66,66),(256,256,3,3),(256) (1,256,64,64) 232862/4718592/4718592 4238 99.00/0.00/0.00 100.0%/0.0%/0.0% 3331 Conv:/model/model.16/conv_block/conv_block.5/Conv
71 cstInstanceNormalization_1_256_64_64 FLOAT16 CPU (1,256,64,64),(256),(256) (1,256,64,64) 0/0/0 13348 0.0%/0.0%/0.0% 2050 cstInstanceNormalization_1_256_64_64:/model/model.16/conv_block/conv_block.6/InstanceNormalization
72 Add FLOAT16 NPU (1,256,64,64),(1,256,64,64) (1,256,64,64) 0/0/0 498 100.0%/0.0%/0.0% 4096 Add:/model/model.16/Add
73 Pad FLOAT16 CPU (1,256,64,64),(8) (1,256,66,66) 0/0/0 2860 0.0%/0.0%/0.0% 2048 Pad:/model/model.17/conv_block/conv_block.0/Pad
74 Conv FLOAT16 NPU (1,256,66,66),(256,256,3,3),(256) (1,256,64,64) 232862/4718592/4718592 4239 99.00/0.00/0.00 100.0%/0.0%/0.0% 3331 Conv:/model/model.17/conv_block/conv_block.1/Conv
75 cstInstanceNormalization_1_256_64_64 FLOAT16 CPU (1,256,64,64),(256),(256) (1,256,64,64) 0/0/0 13502 0.0%/0.0%/0.0% 2050 cstInstanceNormalization_1_256_64_64:/model/model.17/conv_block/conv_block.2/InstanceNormalization
76 Relu FLOAT16 NPU (1,256,64,64) (1,256,64,64) 0/0/0 305 100.0%/0.0%/0.0% 2048 Relu:/model/model.17/conv_block/conv_block.3/Relu
77 Pad FLOAT16 CPU (1,256,64,64),(8) (1,256,66,66) 0/0/0 2853 0.0%/0.0%/0.0% 2048 Pad:/model/model.17/conv_block/conv_block.4/Pad
78 Conv FLOAT16 NPU (1,256,66,66),(256,256,3,3),(256) (1,256,64,64) 232862/4718592/4718592 4235 99.00/0.00/0.00 100.0%/0.0%/0.0% 3331 Conv:/model/model.17/conv_block/conv_block.5/Conv
79 cstInstanceNormalization_1_256_64_64 FLOAT16 CPU (1,256,64,64),(256),(256) (1,256,64,64) 0/0/0 13467 0.0%/0.0%/0.0% 2050 cstInstanceNormalization_1_256_64_64:/model/model.17/conv_block/conv_block.6/InstanceNormalization
80 Add FLOAT16 NPU (1,256,64,64),(1,256,64,64) (1,256,64,64) 0/0/0 495 100.0%/0.0%/0.0% 4096 Add:/model/model.17/Add
81 Pad FLOAT16 CPU (1,256,64,64),(8) (1,256,66,66) 0/0/0 2857 0.0%/0.0%/0.0% 2048 Pad:/model/model.18/conv_block/conv_block.0/Pad
82 Conv FLOAT16 NPU (1,256,66,66),(256,256,3,3),(256) (1,256,64,64) 232862/4718592/4718592 4239 99.00/0.00/0.00 100.0%/0.0%/0.0% 3331 Conv:/model/model.18/conv_block/conv_block.1/Conv
83 cstInstanceNormalization_1_256_64_64 FLOAT16 CPU (1,256,64,64),(256),(256) (1,256,64,64) 0/0/0 13535 0.0%/0.0%/0.0% 2050 cstInstanceNormalization_1_256_64_64:/model/model.18/conv_block/conv_block.2/InstanceNormalization
84 Relu FLOAT16 NPU (1,256,64,64) (1,256,64,64) 0/0/0 352 100.0%/0.0%/0.0% 2048 Relu:/model/model.18/conv_block/conv_block.3/Relu
85 Pad FLOAT16 CPU (1,256,64,64),(8) (1,256,66,66) 0/0/0 2849 0.0%/0.0%/0.0% 2048 Pad:/model/model.18/conv_block/conv_block.4/Pad
86 Conv FLOAT16 NPU (1,256,66,66),(256,256,3,3),(256) (1,256,64,64) 232862/4718592/4718592 4233 99.00/0.00/0.00 100.0%/0.0%/0.0% 3331 Conv:/model/model.18/conv_block/conv_block.5/Conv
87 cstInstanceNormalization_1_256_64_64 FLOAT16 CPU (1,256,64,64),(256),(256) (1,256,64,64) 0/0/0 13532 0.0%/0.0%/0.0% 2050 cstInstanceNormalization_1_256_64_64:/model/model.18/conv_block/conv_block.6/InstanceNormalization
88 Add FLOAT16 NPU (1,256,64,64),(1,256,64,64) (1,256,64,64) 0/0/0 497 100.0%/0.0%/0.0% 4096 Add:/model/model.18/Add
89 ConvTranspose FLOAT16 NPU (1,256,64,64),(256,128,3,3),(128) (1,128,128,128) 290936/9437184/9437184 7667 99.00/0.00/0.00 100.0%/0.0%/0.0% 2624 ConvTranspose:/model/model.19/ConvTranspose
90 Reshape FLOAT16 NPU (1,128,128,128),(4) (128,16384,1,1) 0/0/0 984 100.0%/0.0%/0.0% 4096 Reshape:/model/model.20/InstanceNormalization_2ln_reshape1
91 exLayerNorm FLOAT16 NPU (128,16384,1,1),(1,16384,1,1) (128,16384,1,1) 0/0/0 10601 100.0%/0.0%/0.0% 4128 exLayerNorm:/model/model.20/InstanceNormalization_2ln
92 Relu FLOAT16 NPU (128,16384,1,1) (128,16384,1,1) 0/0/0 2505 100.0%/0.0%/0.0% 4096 Relu:/model/model.21/Relu
93 Reshape FLOAT16 NPU (128,16384,1,1),(4) (1,128,128,128) 0/0/0 3452 100.0%/0.0%/0.0% 4096 Reshape:/model/model.20/InstanceNormalization_2ln_reshape2
94 ConvTranspose FLOAT16 NPU (1,128,128,128),(128,64,3,3),(64) (1,64,256,256) 538202/9437184/9437184 7763 99.00/0.00/0.00 100.0%/0.0%/0.0% 4240 ConvTranspose:/model/model.22/ConvTranspose
95 Reshape FLOAT16 NPU (1,64,256,256),(4) (64,65536,1,1) 0/0/0 3085 100.0%/0.0%/0.0% 8192 Reshape:/model/model.23/InstanceNormalization_2ln_reshape1
96 exLayerNorm FLOAT16 NPU (64,65536,1,1),(1,65536,1,1) (64,65536,1,1) 0/0/0 19358 100.0%/0.0%/0.0% 8320 exLayerNorm:/model/model.23/InstanceNormalization_2ln
97 Relu FLOAT16 NPU (64,65536,1,1) (64,65536,1,1) 0/0/0 4924 100.0%/0.0%/0.0% 8192 Relu:/model/model.24/Relu
98 Reshape FLOAT16 NPU (64,65536,1,1),(4) (1,64,256,256) 0/0/0 6823 100.0%/0.0%/0.0% 8192 Reshape:/model/model.23/InstanceNormalization_2ln_reshape2
99 Pad FLOAT16 CPU (1,64,256,256),(8) (1,64,262,262) 0/0/0 20454 0.0%/0.0%/0.0% 8192 Pad:/model/model.25/Pad
100 ConvTanh FLOAT16 NPU (1,64,262,262),(1,64,7,7),(1) (1,1,256,256) 549044/6422528/6422528 7074 5.67/0.00/0.00 100.0%/0.0%/0.0% 8586 Conv:/model/model.26/Conv
101 OutputOperator FLOAT16 CPU (1,1,256,256) \ 0/0/0 189 0.0%/0.0%/0.0% 4096 OutputOperator:output
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total Operator Elapsed Per Frame Time(us): 575672
invalid ELF header
Hi!
I am trying to run this new version of RKNN.lite. At the moment, I am trying to call for ret = rknn_lite.init_runtime(core_mask=RKNNLite.NPU_CORE_0) I get this error:
OSError: /usr/lib/librknnrt.so: invalid ELF header
I used this lib to put it into "/usr/lib/" - "https://github.com/rockchip-linux/rknn-toolkit2/tree/master/rknpu2/runtime/Linux/librknn_api/aarch64/librknnrt.so"
Did you change the installation process somehow?
I am using - friendlyelec, nanopc-t6rockchip, rk3588
Query the sdk version need unnecessary parameter
使用rknn_query()查询sdk版本时，必须传入inited context，使用空的context会返回-7错误码。context在查询sdk时应该是非必需的，希望支持空context查询sdk
RgaCollorFill(1819) RGA_COLORFILL fail: Invalid argument
test example yolov8 demo, when input image width=640, height=360, it will warn RgaCollorFill(1819) RGA_COLORFILL fail: Invalid argument, how to solve this problem? Full outputs as follow:
load lable ./model/coco_80_labels_list.txt
Open ./model/coco_80_labels_list.txt fail!
model input num: 1, output num: 9
input tensors:
index=0, name=images, n_dims=4, dims=[1, 640, 640, 3], n_elems=1228800, size=1228800, fmt=NHWC, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922
output tensors:
index=0, name=318, n_dims=4, dims=[1, 64, 80, 80], n_elems=409600, size=409600, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-58, scale=0.117659
index=1, name=onnx::ReduceSum_326, n_dims=4, dims=[1, 80, 80, 80], n_elems=512000, size=512000, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003104
index=2, name=331, n_dims=4, dims=[1, 1, 80, 80], n_elems=6400, size=6400, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003173
index=3, name=338, n_dims=4, dims=[1, 64, 40, 40], n_elems=102400, size=102400, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-45, scale=0.093747
index=4, name=onnx::ReduceSum_346, n_dims=4, dims=[1, 80, 40, 40], n_elems=128000, size=128000, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003594
index=5, name=350, n_dims=4, dims=[1, 1, 40, 40], n_elems=1600, size=1600, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003627
index=6, name=357, n_dims=4, dims=[1, 64, 20, 20], n_elems=25600, size=25600, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-34, scale=0.083036
index=7, name=onnx::ReduceSum_365, n_dims=4, dims=[1, 80, 20, 20], n_elems=32000, size=32000, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003874
index=8, name=369, n_dims=4, dims=[1, 1, 20, 20], n_elems=400, size=400, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922
model is NHWC input fmt
model input height=640, width=640, channel=3
origin size=640x360 crop size=640x352
input image: 640 x 360, subsampling: 4:2:0, colorspace: YCbCr, orientation: 1
scale=1.000000 dst_box=(0 140 639 499) allow_slight_change=1 _left_offset=0 top_offset=140 padding_w=0 padding_h=280
src width=640 height=360 fmt=0x1 virAddr=0x0x55a47c7f30 fd=0
dst width=640 height=640 fmt=0x1 virAddr=0x0x55a4870b40 fd=0
src_box=(0 0 639 359)
dst_box=(0 140 639 499)
color=0x72
rga_api version 1.10.1[0]
fill dst image (x y w h)=(0 0 640 640) with color=0x72727272
RgaCollorFill(1819) RGA_COLORFILL fail: Invalid argument
RgaCollorFill(1820) RGA_COLORFILL fail: Invalid argument
23 im2d_rga_impl rga_task_submit(2171): Failed to call RockChipRga interface, please use 'dmesg' command to view driver error log.
23 im2d_rga_impl rga_dump_channel_info(1500): src_channel:
rect[x,y,w,h] = [0, 0, 0, 0]
image[w,h,ws,hs,f] = [0, 0, 0, 0, rgba8888]
buffer[handle,fd,va,pa] = [0, 0, 0, 0]
color_space = 0x0, global_alpha = 0x0, rd_mode = 0x0
23 im2d_rga_impl rga_dump_channel_info(1500): dst_channel:
rect[x,y,w,h] = [0, 0, 640, 640]
image[w,h,ws,hs,f] = [640, 640, 640, 640, rgb888]
buffer[handle,fd,va,pa] = [7264, 0, 0, 0]
color_space = 0x0, global_alpha = 0xff, rd_mode = 0x1
23 im2d_rga_impl rga_dump_opt(1550): opt version[0x0]:
23 im2d_rga_impl rga_dump_opt(1551): set_core[0x0], priority[0]
23 im2d_rga_impl rga_dump_opt(1554): color[0x72727272]
23 im2d_rga_impl rga_dump_opt(1563):
23 im2d_rga_impl rga_task_submit(2180): acquir_fence[-1], release_fence_ptr[0x0], usage[0x280000]
rknn_run
null @ (110 129 223 302) 0.883
null @ (477 126 560 289) 0.841
null @ (212 133 282 287) 0.820
null @ (93 77 554 247) 0.775
null @ (79 183 122 290) 0.413
write_image path: out.png width=640 height=360 channel=3 data=0x55a47c7f30
为什么1.6版本文档写的支持 ReduceL2，推理的时候不能用
jupyter导入rknn报错， NameError: name 'exit' is not defined
----> 2 from rknn.api import RKNN
3 rknn = RKNN(verbose=True, verbose_file='./mobilenet_build.log')
4 rknn = RKNN(verbose=True)
File ~/Documents/anaconda3/envs/RKNN-Toolkit2/lib/python3.8/site-packages/rknn/api/init.py:1
----> 1 from rknn.api.rknn import RKNN
File ~/Documents/anaconda3/envs/RKNN-Toolkit2/lib/python3.8/site-packages/rknn/api/rknn.py:7
5 from .rknn_log import set_log_level_and_file_path
6 from .rknn_platform import get_host_os_platform, get_librknn_api_require_dll_dir
----> 7 from .rknn_base import RKNNBase
8 from argparse import Namespace
10 already_imported = False
File rknn/api/rknn_base.py:24, in init rknn.api.rknn_base()
File rknn/api/graph_optimizer.py:1, in init rknn.api.graph_optimizer()
NameError: name 'exit' is not defined
模型加密解密问题
平台：
RV1106G3
平台版本信息：
rknn_api/rknnrt version: 1.6.0 (9a7b5d24c@2023-12-13T17:33:10), driver version: 0.8.2
模型未加密前可以正常推理，加密后报invalid rknn model magic。使用md5sum 对比模型是完整的。
请问RV1106是否支持推理加密模型
加密脚本
from rknn.api import RKNN
import sys
if len(sys.argv) < 2:
print("args error")
exit(-1)
model = sys.argv[1]
rknn = RKNN()
ret = rknn.export_encrypted_rknn_model(model)
if ret != 0:
print('Encrypt RKNN model failed.')
exit(ret)
rknn.release()
板端报错输出
E RKNN: invalid rknn model magic: 4e4e4b5254505943
terminate called after throwing an instance of 'std::runtime_error'
what(): rknn_init fail!
Aborted (core dumped)
FastSAM convert failed
convert FastSAM-s.onnx of https://github.com/CASIA-IVA-Lab/FastSAM failed.
error message
...
W RKNN: [17:34:40.242] Failed to config layer: 'Conv:/model.16/conv/Conv' using 3Core fallback to single core mode,
W RKNN: [17:34:40.242] core_num 3 ori_Ih 80 ori_Iw 80 ori_Ic 128 ori_Ib 1
W RKNN: [17:34:40.242] ori_Kh 3 ori_Kw 3 ori_Kk 128 ori_Kc 128 ori_Ksx 2 ori_Ksy 2
W RKNN: [17:34:40.242] ori_Oh 40 oriOw 40 oriOc 128 pad_t 1 pad_b 0 pad_l 1 pad_r 0,
W RKNN: [17:34:40.242] Please help report this bug!
D RKNN: [17:34:40.245] DatainEntries overflow, need to tiling more, datain_entries: 4200, limitation: 2048
...
E RKNN: [17:34:40.458] failed to config argb mode layer!
使用香橙派rk3588平台运行transformer语音转录，安装完toolkit-lite之后会出现E RKNN: [21:05:29.879] failed to submit!的错误
错误是如下所示，会一直显示这个错误，直到开发板强制退出ssh连接。是aarch64架构，所以用的是toolkit-lite，请教一下这是为什么
E RKNN: [21:05:29.879] failed to submit!, op id: 0, op name: MatMul, flags: 0x5, task start: 0, task number: 16, run task counter: 0, int status: 0, please try updating to the latest version of the toolkit2 and runtime from: https://eyun.baidu.com/s/3eTDMk6Y (PWD: rknn)
Toolkit-lite2
i want to know the toolkitlite2 only support load rknn model? i try to directly import .pth model in my RKXXXdevice
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.