arm-software/cmsis-nn

CMSIS-NN Library

CMSIS NN

CMSIS NN software library is a collection of efficient neural network kernels developed to maximize the performance and minimize the memory footprint of neural networks on Arm Cortex-M processors.

Supported Framework

The library follows the int8 and int16 quantization specification of TensorFlow Lite for Microcontrollers. This means CMSIS-NN is bit-exact with Tensorflow Lite reference kernels. In some cases TFL and TFLM reference kernels may not be bit-exact. In that case CMSIS-NN follows TFLM reference kernels. The unit test readme provides an overview.

Branches and Tags

There is a single branch called 'main'. Tags are created during a release. Two releases are planned to be done in a year. The releases can be found here .

Current Operator Support

In general optimizations are written for an architecture feature. This falls into one of the following categories. Based on feature flags for a processor or architecture provided to the compiler, the right implementation is picked.

Pure C

There is always a pure C implementation for an operator. This is used for processors like Arm Cortex-M0 or Cortex-M3.

DSP Extension

Processors with DSP extension uses Single Instruction Multiple Data(SIMD) instructions for optimization. Examples of processors here are Cortex-M4 or a Cortex-M33 configured with optional DSP extension.

MVE Extension

Processors with Arm Helium Technology use the Arm M-profile Vector Extension(MVE) instructions for optimization. Examples are Cortex-M55 or Cortex-M85 configured with MVE.

Operator	C int8	C int16	C int4*	DSP int8	DSP int16	DSP int4*	MVE int8	MVE int16	MVE int4*
Conv2D	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
DepthwiseConv2D	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
TransposeConv2D	Yes	No	No	Yes	No	No	Yes	No	No
Fully Connected	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Add	Yes	Yes	N/A	Yes	Yes	N/A	Yes	Yes	N/A
Mul	Yes	Yes	N/A	Yes	Yes	N/A	Yes	Yes	N/A
MaxPooling	Yes	Yes	N/A	Yes	Yes	N/A	Yes	Yes	N/A
AvgPooling	Yes	Yes	N/A	Yes	Yes	N/A	Yes	Yes	N/A
Softmax	Yes	Yes	N/A	Yes	Yes	N/A	Yes	No	N/A
LSTM	Yes	NA	No	Yes	NA	No	Yes	NA	No
SVDF	Yes	No	No	Yes	No	No	Yes	No	No

• int4 weights + int8 activations

Contribution Guideline

First, a thank you for the contribution. Here are some guidelines and good to know information to get started.

Coding Guideline

By default, follow the style used in the file. You'll soon start noticing a pattern like

• Variable and function names are lower case with an underscore separator.
• Hungarian notation is not used. Well, almost.
• If the variable names don't convey the action, then add comments.

New Files

One function per file is followed in most places. In those cases, the file name must match the function name. Connect the function to an appropriate Doxygen group as well.

Doxygen

Function prototypes must have a detailed comment header in Doxygen format. You can execute the doxygen document generation script in the Documentation/Doxygen folder to check that no errors are introduced.

Unit Tests

For any new features and bug fixes, new unit tests are needed. Improvements have to be verifed by unit tests. If you do not have the means to execute the tests, you can still make the PR and comment that you need help in completing/executing the unit tests.

Version & Date

Each File has a version number and a date field that must be updated when making any change to that file. The versioning follows Semantic Versioning 2.0.0 format. For details check: https://semver.org/

Building CMSIS-NN as a library

It is recommended to use toolchain files from Arm Ethos-U Core Platform project. These are supporting TARGET_CPU, which is a required argument. Note that if not specifying TARGET_CPU, these toolchains will set some default. The format must be TARGET_CPU=cortex-mXX, see examples below.

Here is an example:

cd </path/to/CMSIS_NN>
mkdir build
cd build
cmake .. -DCMAKE_TOOLCHAIN_FILE=</path/to/ethos-u-core-platform>/cmake/toolchain/arm-none-eabi-gcc.cmake -DTARGET_CPU=cortex-m55
make

Some more examples:

cmake .. -DCMAKE_TOOLCHAIN_FILE=</path/to/ethos-u-core-platform>/cmake/toolchain/armclang.cmake -DTARGET_CPU=cortex-m55
cmake .. -DCMAKE_TOOLCHAIN_FILE=</path/to/ethos-u-core-platform>/cmake/toolchain/arm-none-eabi-gcc.cmake -DTARGET_CPU=cortex-m7
cmake .. -DCMAKE_TOOLCHAIN_FILE=</path/to/ethos-u-core-platform>/cmake/toolchain/armclang.cmake -DTARGET_CPU=cortex-m3

Compiler Options

Default optimization level is set at Ofast. This can be overwritten with CMake on command line by using "-DCMSIS_OPTIMIZATION_LEVEL". Please change according to project needs. Just bear in mind this can impact performance. With only optimization level -O0, ARM_MATH_AUTOVECTORIZE needs to be defined for processors with Helium Technology.

The compiler option '-fomit-frame-pointer' is enabled by default at -O and higher. When no optimization level is specified, you may need to specify '-fomit-frame-pointer'.

The compiler option '-fno-builtin' does not utilize optimized implementations of e.g. memcpy and memset, which are heavily used by CMSIS-NN. It can significantly downgrade performance. So this should be avoided. The compiler option '-ffreestanding' should also be avoided as it enables '-fno-builtin' implicitly.

Another option is to enable CMSIS_NN_USE_SINGLE_ROUNDING. This may affect the output. If enabling this the equivalent flag should be enabled in TFL/TFLM.

Supported Compilers

• CMSIS-NN is tested on Arm Compiler 6 and on Arm GNU Toolchain.
• IAR compiler is not tested and there can be compilation and/or performance issues.
• Compilation for Host is not supported out of the box. It should be possible to use the C implementation and compile for host with minor stubbing effort.

Inclusive Language

This product confirms to Arm’s inclusive language policy and, to the best of our knowledge, does not contain any non-inclusive language. If you find something that concerns you, email [email protected].

Support / Contact

For any questions or to reach the CMSIS-NN team, please create a new issue in https://github.com/ARM-software/CMSIS-NN/issues

cmsis-nn's Issues

Compilation fail with arm-gcc-none-eabi 12.2.0

I using the arm gcc from official website: https://developer.arm.com/downloads/-/arm-gnu-toolchain-downloads

$ arm-none-eabi-gcc --version
arm-none-eabi-gcc (Arm GNU Toolchain 12.2.MPACBTI-Bet1 (Build arm-12-mpacbti.16)) 12.2.0
Copyright (C) 2022 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

When I try to compile with steps from: https://github.com/ARM-software/CMSIS-NN#building-cmsis-nn-as-a-library

I got the compilation error:

[ 60%] Building C object CMakeFiles/cmsis-nn.dir/Source/SoftmaxFunctions/arm_softmax_s8.c.obj
during RTL pass: combine
/CMSIS-NN/Source/SoftmaxFunctions/arm_softmax_s8.c: In function 'arm_exp_on_negative_values_mve_32x4':
/CMSIS-NN/Source/SoftmaxFunctions/arm_softmax_s8.c:74:1: internal compiler error: in trunc_int_for_mode, at explow.cc:59
   74 | }
      | ^
0x7f2a15836d8f __libc_start_call_main
        ../sysdeps/nptl/libc_start_call_main.h:58
0x7f2a15836e3f __libc_start_main_impl
        ../csu/libc-start.c:392
Please submit a full bug report, with preprocessed source (by using -freport-bug).
Please include the complete backtrace with any bug report.
See <https://bugs.linaro.org/> for instructions.
make[2]: *** [CMakeFiles/cmsis-nn.dir/build.make:580: CMakeFiles/cmsis-nn.dir/Source/SoftmaxFunctions/arm_softmax_s8.c.obj] Error 1
make[1]: *** [CMakeFiles/Makefile2:295: CMakeFiles/cmsis-nn.dir/all] Error 2
make: *** [Makefile:91: all] Error 2

Seems like the issue with mve support. I will need to use latest gcc as my target is also m85 that previous gcc doesn't support.

Is there some easy to get started guide?

I wanted to use this but don't know where to get started. I want to get aware with the work flow of how CMSIS NN work, with some easy to use example like some sine wave predictor which TensorFlow lite library comes with Arduino etc.
Is there getting's tarted guide for this?
Do I have to train the model on the device or I can convert the TensorFlow trained and quantized model with this? Can I run some simple NN example on STM32F103C8 blue pill board? no matter how small the example is. I just want to get use to with the work flow.
Thanks.

Potential bug in arm_depthwise_conv_wrapper_s8_get_buffer_size

We think there is a bug in arm_depthwise_conv_wrapper_s8_get_buffer_size for cases,
where arm_depthwise_conv_wrapper_s8 calls arm_depthwise_conv_3x3_s8.
arm_depthwise_conv_3x3_s8 has a buffer size of zero, but the arm_depthwise_conv_wrapper_s8_get_buffer_size function
does not handle the case
(filter_dims->w == 3) && (filter_dims->h == 3) && (dw_conv_params->padding.h <= 1) && (dw_conv_params->padding.w <= 1)) special.
It should return 0 then.
Please compare

CMSIS-NN/Source/ConvolutionFunctions/arm_depthwise_conv_wrapper_s8.c

Line 66 in ca5dc34

if ((filter_dims->w == 3) && (filter_dims->h == 3) && (dw_conv_params->padding.h <= 1) &&

and

CMSIS-NN/Source/ConvolutionFunctions/arm_depthwise_conv_wrapper_s8.c

Line 124 in ca5dc34

if (input_dims->c == output_dims->c && input_dims->n == 1 && dw_conv_params->dilation.w == 1 &&

@UlrikHjort @vdkhoi @bewagner

Unable to compile CMSIS-NN for cortex-M7

Hi,
As per readme.md file I could successfully use command "cmake .. -DCMAKE_TOOLCHAIN_FILE=/home/supratim/toolchains/ethos-u-core-platform/cmake/toolchain/arm-none-eabi-gcc.cmake -DTARGET_CPU=cortex-m7 -DCMSIS_PATH=/home/supratim/CMSIS-NN".

But when from the build folder, I executed 'make' command, the compilation fail with following error:
"In file included from /home/supratim/CMSIS-NN/Include/arm_nnfunctions.h:114,
from /home/supratim/CMSIS-NN/Source/ConvolutionFunctions/arm_convolve_s8.c:31:
/home/supratim/CMSIS-NN/Include/arm_nn_math_types.h:93:10: fatal error: cmsis_compiler.h: No such file or directory
93 | #include "cmsis_compiler.h"

I know that cmsis_compiler.;h exists in https://github.com/ARM-software/CMSIS_5/tree/develop/CMSIS/Core/Include, but this is a separate github repo and ideally one github repo should not have dependency on another one.

Could you please let me know if I am missing anything?

Thanks
Supratim

Requantize INT_32 to INT8 using arm_nn_requantize

Hi,
I am using this FC s8 CMSIS layer for a project. After we invoke a FC s8 kernel we have Int32 output that we need to dequantize-requantize (Int32->FP->Int8). For example if we have this FC layer in the image to dequantize-requantize the 32-bit output of the FC layer we need to apply this equation: Output_Int8 = (0.0039215669967234135*Weight_mult/0.14308570325374603) *Output_Int32 + 69. => Multiplier * Output_Int32 + shifter

CMSIS-NN has arm_nn_requantizefunction that takes as input Int32 Multiplier and shifter and returns (val * multiplier)/(2 ^ shift). Can you explain me how to map the above FP Multiplier and Int32 shifter into the requantize function?

Output diff for new lstm-operator

The new lstm-operator implementation is not bit exact to TFLM when running a network with lhs_offset = 0 using MVE.

'asm operand has impossible constraint' - GCC 11.3

GCC version : arm-none-eabi-gcc (Arm GNU Toolchain 11.3.Rel1) 11.3.1 20220712

Reproducer: arm-none-eabi-gcc -mcpu=cortex-m55 -Os -I../../Include/ -mfloat-abi=hard -mfpu=auto -S arm_nn_mat_mult_nt_t_s8.c

The issue is not seen in newer releases or other optimization levels.

Inconsistency of CMSIS-NN Quantization Method(Q-format) with ARM Documentation

Hello.

I am currently in the process of developing using the Q-Format (Qm.n) for quantization. However, upon reviewing the revision history, I noticed that starting from version 4.1.0, the q-format approach is no longer being followed. My current approach aligns with the methods outlined in the following ARM documentation links:

• Compute the layer shifts
• Choose a quantization scheme

While TensorFlow Lite for Microcontrollers employs Zero Point and Scale Factor for quantization, which necessitates additional memory and floating-point operations, it appears that Q-format based quantization would be more suitable for Cortex-M processors due to these constraints.

Could you kindly provide a clear explanation for the necessity of this change? The absence of discussion regarding its impact on speed and accuracy has left me somewhat perplexed. Any insight into the rationale behind this decision would be greatly appreciated, as it would aid in understanding the best practices for quantization within the context of TensorFlow Lite for Microcontrollers and CMSIS-NN.

Thank you for your time and consideration.

Unit tests fails to build with -O0

GCC cannot build all unit tests with -O0.

How to build on host

Hi,
I tried to build CMSIS-NN for a x86-64 host cpu and had some trouble to compile it using gcc 9.4.

Finally I ended with:

cmake  -D CMSIS_PATH="../../CMSIS_5" -DCMAKE_C_FLAGS:STRING="-D__GNUC_PYTHON__ -D__RESTRICT=__restrict" ..

Is this the recommended way to do it?
I had to specify __RESTRICT, otherwise I run into

/../cmsis/CMSIS-NN/Include/arm_nnsupportfunctions.h:967:62: error: expected ‘;’, ‘,’ or ‘)’ before ‘dst’
  967 | __STATIC_FORCEINLINE void arm_memcpy_q15(int16_t *__RESTRICT dst, const int16_t *__RESTRICT src, uint32_t block_size)

Best regards.

reserved identifier violation

I would like to point out that identifiers like “_ARM_NNFUNCTIONS_H” and “_UART_STDOUT_H_” do not fit to the expected naming convention of the C++ language standard.
Would you like to adjust your selection for unique names?

Create dynamic library for Cortex-M33

Hello,

Is it possible to compile a dynamic library for the Cortex-M33? I changed the CMakeLists.txt file as detailed below but I still get a static library.

cmake_minimum_required(VERSION 3.15.6)
set(CMAKE_POSITION_INDEPENDENT_CODE ON)
set(BUILD_SHARED_LIBS ON)
project(CMSISNN)
add_library(cmsis-nn SHARED)
target_compile_options(cmsis-nn PRIVATE -Ofast -fPIC)
target_include_directories(cmsis-nn PUBLIC "Include")
add_subdirectory(Source)
set_target_properties(cmsis-nn PROPERTIES
POSITION_INDEPENDENT_CODE ON
)

Does the MVEI version of CMSIS NN have implementations of q15 and q7?

What is the proper way to define `__CLZ`?

__CLZ is used in the softmax functions, however it is not defined for non-Arm compilers in CMSIS-NN.

Compare this to CMSIS-DSP, defines it by ultimately including dsp/none.h which contains an inline C function for it (and other intrinsics).

One can solve this by including dsp/none.h in the softmax headers, but that requires both installing CMSIS-DSP and hacking this library.

MVE Conv 1 x N does not handle all configurations

Some corner cases of conv 1_x_n falls into this condition:

CMSIS-NN/Source/ConvolutionFunctions/arm_convolve_1_x_n_s8.c

Line 95 in 1e0f44c

return ARM_CMSIS_NN_FAILURE;

This could then be handled by the generic conv function.

Support for N>1 in maxpool

One of the users of TVM via CMSIS-NN needs support of N>1 in maxpool. Here is the link: https://discuss.tvm.apache.org/t/cmsis-nn-qnn-max-pool2d-layout-on-cortex-m7-without-npu/12712/3. Would it be possible to add support for this?

What does CMSIS exactly refer to?

CMSIS: Common Microcontroller Software Interface Standard ?

Occur an ASM error when compiling arm_nn_mat_mul_core_4x_s8

Processor: Cortex-M55
Error message:
cmsis-nn/Source/NNSupportFunctions/arm_nn_mat_mul_core_4x_s8.c: In function 'arm_nn_mat_mul_core_4x_s8':
/(my project path)/cmsis-nn/Include/Internal/arm_nn_compiler.h:97:23: error: 'asm' operand has impossible constraints
97 | #define __ASM __asm
| ^~~~~
cmsis-nn/Source/NNSupportFunctions/arm_nn_mat_mul_core_4x_s8.c:84:9: note: in expansion of macro '__ASM'
84 | __ASM volatile(" .p2align 2 \n"
| ^~~~~

Why is this happening? And how to fix it?
Appreciate!

Doubts regarding some of the functions

(1) Can you please explain what exactly the line arm_nn_requantize() perform.
(2) Also, the functionality of the arm_nn_activation_s16 .
(3) How the sigmoid lookup table is created?
(4) Explain the working of vmaxq_s32(acc, vdupq_n_s32(NN_Q15_MIN)) which is present in the folder
NNSupportFunctions->arm_nn_vec_mat_mul_result_acc_s8.c
Can you please clear the above doubts as they will be very useful for me.

Documentation warning

There are a couple of documentation warnings.. If you scroll down to the end of https://github.com/ARM-software/CMSIS-NN/pull/78/files , you can see it..

No MVE support Conv2D int16

README claims to support Conv2D Int16 MVE but it is actually missing.
https://github.com/ARM-software/CMSIS-NN/blob/main/README.md#mve-extension

Vector of variable length used in arm_convolve_1x1_s8_fast

Compilation fails with [-Werror,-Wvla] and buffers of unknown size in stack shouldn't be allocated in stack as there is a memory already allocated in the scratch buffer for that.

Inconsistency of CMSIS-NN Quantization Method(Q-format) with ARM Documentation

Hello.

I am currently in the process of developing using the Q-Format (Qm.n) for quantization. However, upon reviewing the revision history, I noticed that starting from version 4.1.0, the q-format approach is no longer being followed. My current approach aligns with the methods outlined in the following ARM documentation links:

• Compute the layer shifts
• Choose a quantization scheme

While TensorFlow Lite for Microcontrollers employs Zero Point and Scale Factor for quantization, which necessitates additional memory and floating-point operations, it appears that Q-format based quantization would be more suitable for Cortex-M processors due to these constraints.

Could you kindly provide a clear explanation for the necessity of this change? The absence of discussion regarding its impact on speed and accuracy has left me somewhat perplexed. Any insight into the rationale behind this decision would be greatly appreciated, as it would aid in understanding the best practices for quantization within the context of TensorFlow Lite for Microcontrollers and CMSIS-NN.

Thank you for your time and consideration.

Confusing doc on `arm_nn_requantize`

The documentation of arm_nn_requantize is

/**
 * @brief           Requantize a given value.
 * @param[in]       val         Value to be requantized
 * @param[in]       multiplier  multiplier. Range {NN_Q31_MIN + 1, Q32_MAX}
 * @param[in]       shift       left or right shift for 'val * multiplier'
 *
 * @return          Returns (val * multiplier)/(2 ^ shift)
 *
 */

But the CMSIS_USE_SINGLE_ROUNDING path does not compute that, but some rounding of (val * multipler) / (2^(shift+31))

Besides, the left or right shift part of the doc is not very clear: how to specify the direction of the shift?
It seems that it is based on the sign of shift, with positive shifts being left shift and negative shifts being right shifts.

Is there a roadmap?

Hi, thank you for your work!

Is there a future operation roadmap in long/short term like adding RNN... transformer...?

How to implement a neural network by using CMSIS-NN？

Is there any tutorial? I am confused about the cmsis-nn transform of nn.lstm, nn.layernorm, nn.prelu etc。

How to use arm_lstm_unidirectional_s16_s8? Are there any tutorials？

Besides TFLM, are there any tutorials, especially the example codes, for us to learn how to use the arm_lstm_unidirectional_s16_s8 function? It is so complicated.

arm_elementwise_add_s8 does not work properly for a Residual Architecture

Hi, appreciate for your work!

I am trying to build a Residual-like NN module as the following:

import tensorflow.keras as keras
import keras.layers as KL

nnInput = keras.Input(
	shape=(inputH, inputW, inputCh))
x = KL.Convolution2D(
	1, 
	(3, 3), 
	strides=(2, 2), 
	padding='same', 
	name='conv1',
	bias_initializer='glorot_uniform',
	activation='relu')(nnInput)

shortcut = x 

x = KL.Conv2D(
	filters=num_filters, 
	kernel_size=1, 
	strides=stride, 
	name=name + "_1_conv",
	activation='relu')(x)

x = KL.Conv2D(
	filters=num_filters, 
	kernel_size=kernel_size, 
	padding="SAME", 
	name=name + "_2_conv",
	activation='relu')(x)
x = KL.Conv2D(
	filters=4 * num_filters, 
	kernel_size=1, 
	name=name + "_3_conv")(x)
x = KL.Add(
	name=name + "_add")([shortcut, x])

The calculation results of CMSIS-NN and tensorflow-lite before the elementwise add are all the same. I follow the testing case in the repo to build the arm_elementwise_add_s8 accordingly but it is still not working. I am wondering is the function proved to be compatible with tensorflow-lite or do I miss something?

Thanks in advance!

Move test platform to other FVPs

Hi,

I am trying to figure out how to use FVP as a test platform. I successfully build tests as the tutorial shows. But when I try to switch to Corstone SSE-310 FVP to test how my code performs on M85, it fails. It seems code in Corstone-300 is hardware-specific. How can I generate these codes for other FVPs?

Thanks!

missing arm_s8_to_s16_unordered_with_offset.c when I included the pack

I try to incorporate the CMSIS-NN into my project. However, I saw error message:

.\objects\myProject.axf: Error: L6218E: Undefined symbol arm_s8_to_s16_unordered_with_offset (referred from arm_convolve_s8.o).

I traced the function in the CMSIS-NN and found out the source code arm_s8_to_s16_unordered_with_offset.c is in the
directory C:\Users\XXX\AppData\Local\Arm\Packs\ARM\CMSIS-NN\4.1.0\Source\NNSupportFunctions. But the pack didn't
include the file.

Did the pack miss it accidentally? How can I add it back into the pack?

Best Regards,

/Tony

The mat_mul_kernel_s16 function needs __PKHTB & __PKHBT to re-order the value

Hi,
I found that at below line:

CMSIS-NN/Source/NNSupportFunctions/arm_nn_mat_mul_kernel_s16.c

Line 93 in d071e9f

ip_a0 = read_and_pad(ip_a0, &a01, &a02);

We use a read_and_pad to process the weights for the value expanding from q7_t to q15_t, and also a group: __PKHxx for
reording the value from (a0, a2, a1, a3) to (a0, a1, a2, a3).
My question is that why we add this two PKHxx operations, I think that We can still use the (a0, a2, a1, a3), if we process the
input with the same way (I found that the 1x1 conv2d has the similarity operation, without __PKHxx). So that we can save two-instructs and then save the inference time.

Regards,
Crist

Incorrect condition check for buffer size in DW convolve s16

The condition to use arm_depthwise_conv_fast_s16() reads as
if (dw_conv_params->ch_mult == 1 && dw_conv_params->dilation.w == 1 && dw_conv_params->dilation.h == 1 &&
filter_dims->w * filter_dims->h * input_dims->c < 512)

The 'filter_dims->w * filter_dims->h * input_dims->c' should be filter_dims->w * filter_dims->h as DW conv is a layer based op.

The consequence of this is that optimization is missed at times resulting in slower performance than what can be. This is not an accuracy error.

Include/Internal/arm_nn_compiler.h: undefined __ARM_FEATURE_MVE

Missing check that __ARM_FEATURE_MVE is defined before checking its value

Include/Internal/arm_nn_compiler.h:130

#if ((__ARM_FEATURE_MVE & 3) == 3) || (__ARM_FEATURE_MVE & 1)
    #include <arm_mve.h>
#endif

Update to:

#if defined(__ARM_FEATURE_MVE)
    #if ((__ARM_FEATURE_MVE & 3) == 3) || (__ARM_FEATURE_MVE & 1)
        #include <arm_mve.h>
    #endif
#endif

IAR compiler issues

There are some compile error for IAR

__STATIC_FORCEINLINE void arm_memcpy_s8(int8_t *__RESTRICT dst, const int8_t *__RESTRICT src, uint32_t block_size)
                                                             ^
"Include/arm_nnsupportfunctions.h",953  Error[Pe018]: 
          expected a ")"

  __STATIC_FORCEINLINE int32x4_t arm_doubling_high_mult_mve(const int32x4_t m1, const int32_t m2)
                       ^
"Include/arm_nnsupportfunctions.h",990  Error[Pe020]: 
          identifier "int32x4_t" is undefined

Possible undefined behavior for `arm_nn_requantize` when compiling with `CMSIS_NN_USE_SINGLE_ROUNDING`

In the CMSIS_NN_USE_SINGLE_ROUNDING ifdef branch, there is UB when shift >= 31 as total_shift-1 becomes negative.
This can make sense in some context when using a call to requantize to also left_align some quantized int8 value.

__STATIC_FORCEINLINE int32_t arm_nn_requantize(const int32_t val, const int32_t multiplier, const int32_t shift)
{
#ifdef CMSIS_NN_USE_SINGLE_ROUNDING
    const int64_t total_shift = 31 - shift;
    const int64_t new_val = val * (int64_t)multiplier;

    int32_t result = new_val >> (total_shift - 1); // <-- Here is the problematic line
    result = (result + 1) >> 1;

    return result;
#else
    return arm_nn_divide_by_power_of_two(arm_nn_doubling_high_mult_no_sat(val * (1 << LEFT_SHIFT(shift)), multiplier),
                                         RIGHT_SHIFT(shift));
#endif
}