DV network conversion tool
License: Apache License 2.0
This python script can parse Caffe and Keras CNN models and convert them to DV700 compatible formats.
Python with version >= 3.6 is required to run this script.
It also needs the following modules: numpy, protobuf, h5py, jinja2 and opencv-python. Run the following command to install them:
python -m pip install numpy protobuf h5py==2.7.1 opencv-python jinja2
or on Ubuntu 18.04:
sudo apt-get install python3-numpy python3-protobuf python3-h5py python3-opencv python3-jinja2
In email from steven:
Zong-san,
It seems that the commonly used buffer sizes for DV700 h/w are large
enough to support non-transposed first layer input in most cases.
As far as I understand, currently, non-transposed mode is supported by
h/w(risc-v program) but not by the weight-packing tool ... Would it
be possible to support non-transposed weight packing in the tool soon?
For Mercury project, the i/o requirements are quite challenging and we
need to make another specialized version of tinyIPU - it's too hard to
support transposed output for this, so we will need to use non-transposed flow.
And the follow up:
Zong-san,
As I understand it, it is implemented as a per-layer setting in the h/w config structure, using the top bit (0x8000) of input_circular_offset.
Now activation which follows element-wise add layer in ResNet50 exists as separate layer, but it can be merged with the configuration for element-wise add layer.
If not mistake, FPGA_Network.convert_network() seems to assume consecutive nodes are consecutive even in the traverse_list but currently it might not be satisfied if network with branches is given.
Example network is this one.
0 --> 1 ---> 3 ---> 4
\ \
--> 2 --> 5 --> 6 --> 7
traverse_list can become [0, 1, 3, 2, 4, 5, 6, 7].
Hi, all
I faced the below error on converting NASNetMobile by Keras application for DV.
INFO: normal_add_2_12 normal_add_2_12 Convolution 505 516 (4, 4, 176) 00009F80 00001600
INFO: normal_left3_12 normal_left3_12 Convolution 506 507 (5, 5, 176) 00002260 00002260
INFO: normal_add_3_12 normal_add_3_12 Convolution 507 516 (5, 5, 176) 0000B580 00002260
INFO: normal_left4_12 normal_left4_12 Convolution 508 510 (4, 4, 176) 00002260 00001600
INFO: normal_right4_12 normal_right4_12 Convolution 509 510 (4, 4, 176) 00003860 00001600
INFO: normal_add_4_12 normal_add_4_12 Convolution 510 516 (4, 4, 176) 0000D7E0 00001600
INFO: separable_conv_1_normal_left5_12 separable_conv_1_normal_left5_12 Convolution 511 512 (5, 5, 176) 00002260 00002260
INFO: separable_conv_1_normal_left5_12_point separable_conv_1_normal_left5_12_point Convolution 512 513 (5, 5, 176) 000044C0 00002260
INFO: separable_conv_2_normal_left5_12 separable_conv_2_normal_left5_12 Convolution 513 514 (5, 5, 176) 00002260 00002260
INFO: separable_conv_2_normal_left5_12_point separable_conv_2_normal_left5_12_point Convolution 514 515 (5, 5, 176) 000044C0 00002260
INFO: normal_add_5_12 normal_add_5_12 Convolution 515 516 (5, 5, 176) 0000EDE0 00002260
INFO: normal_concat_12 normal_concat_12 Concatenate 516 516 (4, 4, 1056) 00006720 0000A920*
INFO: allocated size:893216
Traceback (most recent call last):
File "tool/convertor.py", line 134, in <module>
output_gendoc, output_gengraph, graphviz_path)
File "/home/yuya.kobayashi/git_repo/dv-sdk/tool/cnn_convertor/fpga_layer.py", line 1247, in output_network
self.output_source(sf, network_name)
File "/home/yuya.kobayashi/git_repo/dv-sdk/tool/cnn_convertor/fpga_layer.py", line 1215, in output_source
gen_source_layer(of, name, n, layer, self.quantization)
File "/home/yuya.kobayashi/git_repo/dv-sdk/tool/cnn_convertor/fpga_layer.py", line 696, in gen_source_layer
gen_source_conv(of, name, n, layer, quantization)
File "/home/yuya.kobayashi/git_repo/dv-sdk/tool/cnn_convertor/fpga_layer.py", line 534, in gen_source_conv
' conf.output_mode = 1; // 0 = concat, 1 = eltwise add\n\n'.format(layer.layer_in[1].output_addr_offset))
IndexError: list index out of rangenasnetmobile.zip contains the following files.
Best regards,
Kobayashi
Currently dilation convolution by Keras is supported.
Changing parser is enough.
The current Caffe parser only handle squre kernel size and symmetric padding.
Write Unit Tests for checking the correctness of generated layer configuration, weights, C++ code.
Currently the parameter parsed from Keras network is in wrong order.
The Keras store parameters like kernel_size, stride, or pad in (height, width) order, but the tool parsed it in (width, height) order.
After further investigate NASNet in #4, it has some divergence paths where one path applies activation for the output and another does not.
Currently the tool automatically merges activation layer with convolution layer, so for this kinds of model it either will apply activation to both path or the activation layer will be lost.
It should be fixed so that the activation layers are treated as independent layer, and only merge with convolution layer when converting to FPGA configs.
I tried to convert network from Keras.
The model file DenseASPP_pretraining.h5 is available in here.
The graph image of the network outputed from Keras is network.jpg.
The log of DMP tool is tool_output.txt.
The image shows that the network's output shape is [None, 64 128, 19] but the converted network's output shape is bigger.
Hi,
I faced the error when converting pure Dense network.
The model:
model = Sequential() model.add(Dense(100, input_dim = 6, kernel_initializer = 'normal', activation = 'relu')) model.add(Dense(100, kernel_initializer = 'normal', activation = 'relu')) model.add(Dense(100, kernel_initializer = 'normal', activation = 'relu')) model.add(Dense(1, kernel_initializer = 'normal'))
The h5 is here:
https://drive.google.com/file/d/1mosKiCyzkmbGDT7-UaChOm7IEbyWNVmi/view?usp=sharing
Best regards
For example. class LayerLiveRange.output_concat_lr is assumed as only one concatenation node.
Convertion of
DenseASPP_0_epoch.h5.zip
failed with the below error message.
INFO: Start parsing. Network type:KERAS
INFO: Parsing Keras network.
Traceback (most recent call last):
File "../dv-sdk/tool/convertor.py", line 134, in <module>
custom_layer, dim_override)
File "/home/yuya.kobayashi/git_repo/dv-sdk/tool/cnn_convertor/cnn_parser.py", line 37, in parse_network
parser_keras.parse_keras_network(network, network_data)
File "/home/yuya.kobayashi/git_repo/dv-sdk/tool/cnn_convertor/parser_keras.py", line 601, in parse_keras_network
parse_keras_network2(network, netdef, netweight, need_flip)
File "/home/yuya.kobayashi/git_repo/dv-sdk/tool/cnn_convertor/parser_keras.py", line 444, in parse_keras_network2
if is_channel_first:
UnboundLocalError: local variable 'is_channel_first' referenced before assignment
This is because config->layers in the network definetion JSON does not contain layer directly but contain Sequential object that contains layers.
(NOT URGENT, because this problem is written in the SDK wiki.)
In current implementation, deeper network reuses input buffer so that input data is corrupted.
Instead of that, it is better to reuse buffer for middle layers.
Assume a concat node has concat node as the input node.
If the input concat node will be copy_concat, the successive concat node must be copy_concat in current impl.
Currently the tool calculate layer output dimensions for pooling layers using ceil() function. This comes from Caffe, and other frameworks should use floor() function.
Currently, Convertor tool only support for 2D data.
But in voice applications, data is 1D.
So the convertor need to support 1D input data (such as Conv1d and MaxPool1d and GlobalAveragePooling1D, ...)
The appliation API outputs reformated array.
I was trying to convert MobileNetV2 by Keras application to DV network as below.
$ python3 ./converter.py mobilenet.ini
Then I got Mobilenetv2Feature_gen.cpp but building the code failed due to gramatical error.
The below code is the part of the code. The 3rd line contains erroneous code.
conf.run[0].weight_buf.offs = 0;
conf.run[0].weight_fmt = 3; // Weight format (0 = random access blocks, 1 = compact stream, 3 = 8-bit qunatized stream)
conf.run[0].conv_pad = 0x-10001; // bits [7:0] = left padding, bits [15:8] = right padding, bits [23:16] = top padding, bits [31:24] = bottom padding
conf.run[0].conv_stride = 0x202; // bits [7:0] = X stride, bits [15:8] = Y stride
conf.run[0].conv_dilation = 0x0; // bits [7:0] = X dilation, bits [15:8] = Y dilationCould you fix this?
I attached mobilenet.zip including the following files.
Thank you,
Kobayashi
The run log is attached.
xc_caffe_vgg.log
Also the documents generated are a bit outdated.
change get_final_output() to accept index value for network outputs
How to determine is_input_hw_layout does not consider that Concatenation can be in hw_layout.
The HW have a upper limit of 16384 (or 32768 in newer version) input size. But if the layer is divided into several small chunks, it can support nearly unlimited length of input vectors.
currently SDK support only nearest-neighbor upsampling but for other upsampling tool does not show message about it.
Please change tool to show that.
Tool should ouput an error when the fully connected is larger than u_int16 range.
If the entire network is built using only convolution layers, then the input dimension don't need to be specified. For ex. in Keras the input shape will be some thing like (None, None, None, 3).
It is desired that the tool can also these models, by specifying the width and height use the width_override and height_override input options.
If not mistaken, the tool currently does not really use unified buffer (only GoogleNet / inception type of layer).
If 1 layer output can fit in the Unified buffer, and be used by the next layer, tool should actually configure DV to do so --> impact of performances and power consumption.
The problem network is attached.
The original paper of Batch Norm layer proposed that the operation should be applied per batch for fully connected layers, and be applied per 'global batch' for convolution layers.
(Global batch treats batch size as [b x h x w] if the output size is (b, h, w, c). So the mean is calculated per channel using all feature maps in the batch.)
Current tool's and Caffe's implementation follow this approach. But in Keras and Tensorflow, it is possible to apply the batch normalization along arbitrary axis (or more than one axes). In this case the tool can't convert the network correctly.
I am not sure if there will be networks use this feature in Keras (or Tensorflow). So I assume the priority of this feature support can be low.
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.