xilinx / qnn-mo-pynq Goto Github PK
View Code? Open in Web Editor NEWLicense: BSD 3-Clause "New" or "Revised" License
License: BSD 3-Clause "New" or "Revised" License
I have read dorefanet-imagenet-samples.ipynb,
In the second step, we loaded a trained classifier. What I want to ask is how to make a classifier. Is there a general template for reference? For example, in tensorflow, you can load it by reading ckpt. Model, if I want to do an image classification on PYNQ, how should I operate?
I compile this library for ZCU102 board (ZynqMP Ultrascale+) and get working app_hw.elf binary.
When trying to use library from Python3.6 getting error (python crash) with:
In console, when python crash/get back to shell:
"Bus error"
In system logs:
"
python3: unhandled alignment fault (7) at 0x7f78695080, esr 0x92000061, in libc-2.27.so[7f8bed6000+140000]
CPU: 0 PID: 5065 Comm: python3 Not tainted 4.14.0-xilinx-v2018.2 #1
Hardware name: ZynqMP ZCU102 Rev1.0 (DT)
"
hi,
i have a problem about the files of lib_sw_W1A3.so and lib_sw_W1A3.so. i want to know the function of these two documents. Thank you!
Hi,
I have a pynq board with latest (2.1) SD card image into it. I have downloaded this (https://github.com/Xilinx/QNN-MO-PYNQ) repository and trying to run "tiny-yolo-image-loop" jupyter nottebook. Sometimes it runs perfectly but many times kernel dies in-between.
Is there a way to debug or analyse the issue.
Hello,
Is it possible to change the parameter(s) in config.h file without affecting the inference result? As an example, I tried changing MAX_PE from 64 to 32 to generate a smaller accelerator (so that I could fit other IP blocks on the PYNQ). When I ran the Python test code, the inference could not accurately detect object in a given picture anymore. I thought modifying MAX_PE only affects the resource consumption and performance, yet I expected that the computational result would not change. Does changing configuration (in this case, MAX_PE) require re-training the network?
Hello,
I am trying to use Vivado HLS 2017.4 command prompt on Windows 10 to rebuild the script but when I followed the instructions, it gave the following error when entering the following line "./make-hw.sh W1A2 pynqz1-z2 h" .
C:\Users\Sayf\Desktop\QNN\QNN-MO-PYNQ-master\QNN-MO-PYNQ-master\qnn\src\network>./make-hw.sh W1A2 pynqz1-z2 h
'.' is not recognized as an internal or external command,
operable program or batch file.
Kind regards,
Sayf Khan
The provided ./make-hw.sh fails to fit the implemented design on the PYNQ 7z020.
Hi,
Could you please give us directions to train this to our own data set?
Or is it possible to train this network on original yolo implementation and covert the .weights files to the required format? Any script?
I was not able to run tiny-yolo-image-loop.ipynb out of the box, when trying to run the code in the box after "2. Launch Demo", I got this error
NameError Traceback (most recent call last)`
<ipython-input-3-282cc81fa050> in <module>()
59 print("class: {}\tprobability: {}".format(det[1], det[0]))
60 res = Image.open(file_name_out.value.decode() + ".png")
---> 61 display(res)
NameError: name 'display' is not defined
I was able to fix this with adding the following line to the imports
from IPython.display import display
hi~
If I want to download your model weights, how should I do it?
Hi, giuliogamba,
Really thanks for your QNN tutorial ,recently, i want to achieve real-time object detection by QNN with pynq, i changed the tiny-yolo-image-loop.ipynq code, capture the image through usb webcam in the loop because i want to achieve real-time detection in this way, but my frame rate is only 1/4 FPS, that means it takes 4 seconds to load an image, it is too slow, no real-time, so i want to ask how can i improve my frame rate? I am new to pynq domain, hope get some advise from you.
Hope to hearing from you soon, and thanks in advance!
Icarus.
Hello,
Can I get the training model of dorefanet-classifier? I want to train the plant classes.
Hi,
Very impressive work! Thank you for sharing this project to us.
I have some questions regarding the behaviour of "stride" parameter in the sliding window.
Is it true that assigning 0 to "stride" parameter will result in a striding of 1 during convolution?
If so, does this module support a striding of 2? And what value should we assign to this stride parameter?
Best,
awai54st
When I run tiny-yolo-image notebook on my pynq board I got this error.
.... /darknet/libdarknet.so: undefined symbol: forward_region_layer_pointer_nolayer
There is no function names forward_region_layer_pointer_nolayer, how to fix that?
I've created a custom dataset by following the original YOLO guidelines here:
https://github.com/AlexeyAB/darknet#how-to-train-to-detect-your-custom-objects
It is unclear to me how to train on the Tinier Yolo architecture to produce the necessary .npy files. Is this possible at all? If so what are some good resources to help me proceed?
Thanks!
Ran with no luck from PYNQ-Z2.
1.2
net = load_net(b"cfg/yolov2-tiny-voc.cfg", b'weight/yolov2-tiny-voc.weights',0)
Segmentation fault
1.3
net = load_net(b"cfg/yolov2.cfg", b'./yolov2.weights',0)
Segmentation fault
Hi! That's really excellent work!I have tried to detect object in your work tiny-yolo-image, however, there is nothing detected.
Would you please tell me the reason?
Is there a estimated time frame for zcu104 support?
Regards
Jack
When i use
sudo pip3 install git+https://github.com/Xilinx/QNN-MO-PYNQ.git
to install the QNN package to my board PYNQ Z2, the following error exists:
The directory '/home/xilinx/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executi.
The directory '/home/xilinx/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip .
Collecting git+https://github.com/Xilinx/QNN-MO-PYNQ.git
Cloning https://github.com/Xilinx/QNN-MO-PYNQ.git to /tmp/pip-cz7ywm9o-build
Installing collected packages: qnn-loopback
Running setup.py install for qnn-loopback ... error
Complete output from command /opt/python3.6/bin/python3.6 -u -c "import setuptools, tokenize;file='/tmp/pip-cz7ywm9o-build/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().repl:
Running pre installation script...
Building hardware library...
Cloning into '/tmp/pip-cz7ywm9o-build/qnn/src/library/rapidjson'...
Note: checking out 'tags/v1.1.0'.
You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.
If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:
git checkout -b <new-branch-name>
HEAD is now at f54b0e4... Merge pull request #714 from miloyip/version1.1.0
/usr/bin/ld: cannot find -lcma
collect2: error: ld returned 1 exit status
make[1]: *** [output/lib_hw.so] Error 1
make: *** [lib_hw] Error 2
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-cz7ywm9o-build/setup.py", line 56, in <module>
subprocess.check_output(["make", "-j2", "-C", my_path + "qnn/src/network/", "lib_hw"])
File "/opt/python3.6/lib/python3.6/subprocess.py", line 336, in check_output
**kwargs).stdout
File "/opt/python3.6/lib/python3.6/subprocess.py", line 418, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['make', '-j2', '-C', '/tmp/pip-cz7ywm9o-build/qnn/src/network/', 'lib_hw']' returned non-zero exit status 2.
----------------------------------------
Command "/opt/python3.6/bin/python3.6 -u -c "import setuptools, tokenize;file='/tmp/pip-cz7ywm9o-build/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.clos/
LUT 106%, does this matter? I am using 2017.4 on ubuntu.
Actually I also tried on 2018.3 on another machine and exceeds as well.
What might be the reason for these? Are there any additional settings I need to make?
Thank you!
Hello,
I don't understand why for the network W1A3, the output activation is 3-bit wide, the accumulated value (stored in the macRegisters) is compared with 5 (=3*2-1) thresholds read from the threshold memory. Can you explain me this case?
Many thanks!
This is a very powerful experiment.
Excuse me! in vivado hls , what should I do if I want to detect object for my model ?
I want to achieve a real-time face recognition system on PYNQ with a camera and HDMI output. Could I use this network to train my own parameters? Or could you please give some suggestion to me? Thank you very much!
Hi,
Currently, I want to train a new weights for the Tinier YOLO Topology. So I trained a network with the same topology as the network you provided in the example. I tried to match it with the work you done by using the BNN weight generator:
peCounts
and simdCounts
map have not been not public yet. Could you please provide me the peCounts
and simdCounts
map which you use for this topology? Because I try a lot of value and nothing can look alike like the original weights size.tinier-yolo-layers.json
and put the binparams to the folder binparam-tinier-yolo-nopool
is enough?i want to use another image for testbench, but i don't know how to get the bin file like verification.bin and input.bin, can anyone help me?
Hi,
Your framework is really good! It works great on my PYNQ board. Thanks for your job.
I am now trying to test the hardware utilization of the tiny YOLO QNN used in the examples on my FPGA.
I believe it should be done using Vivado.
Could you please provide a guide on doing this on Windows 10? I use Vivado 2018.3.
Thanks,
Aliaksei
Hey! Really thanks for the tutorial for QNN. I want to use my usb webcam to capture the image in real time and then use your model to identify it. Kidnly tell me where to change the code. I'm new to fpga domain.
Thanks.
My board is pynq-z2.
And, I tried to pip3 install git+https://github.com/Xilinx/QNN-MO-PYNQ.git,
but there was no any response after cloning.
Collecting git+https://github.com/Xilinx/QNN-MO-PYNQ.git
Cloning https://github.com/Xilinx/QNN-MO-PYNQ.git to /tmp/pip-bl5011_8-build
I had waited for about 20min.
By the way, I could ping github.com.
Hello,
import sys
import os, platform
import json
import numpy as np
import cv2
from ctypes import c_char_p # i add import
from matplotlib import pyplot as plt
from PIL import Image
from datetime import datetime
from qnn import TinierYolo
from qnn import utils
sys.path.append("/opt/darknet/python/")
from darknet import *
from tests.test_qnn import test_tinier_yolo # i add import
conv0_weights = np.load('/home/xilinx/jupyter_notebooks/QNN-MO-PYNQ/qnn/params/tinier-yolo-conv0-W.npy', encoding="latin1")
conv0_weights_correct = np.transpose(conv0_weights, axes=(3, 2, 1, 0))
conv8_weights = np.load('/home/xilinx/jupyter_notebooks/QNN-MO-PYNQ/qnn/params/tinier-yolo-conv8-W.npy', encoding="latin1")
conv8_weights_correct = np.transpose(conv8_weights, axes=(3, 2, 1, 0))
conv0_bias = np.load('/home/xilinx/jupyter_notebooks/QNN-MO-PYNQ/qnn/params/tinier-yolo-conv0-bias.npy', encoding="latin1")
conv0_bias_broadcast = np.broadcast_to(conv0_bias[:,np.newaxis], (net['conv1']['input'][0],net['conv1']['input'][1]net['conv1']['input'][1]))
conv8_bias = np.load('/home/xilinx/jupyter_notebooks/QNN-MO-PYNQ/qnn/params/tinier-yolo-conv8-bias.npy', encoding="latin1")
conv8_bias_broadcast = np.broadcast_to(conv8_bias[:,np.newaxis], (125,1313))
cfg = c_char_p("/home/xilinx/jupyter_notebooks/QNN-MO-PYNQ/qnn/params/tinier-yolo-bwn-3bit-relu-nomaxpool.cfg".encode())
NameError Traceback (most recent call last)
in ()
14 cfg = c_char_p("/home/xilinx/jupyter_notebooks/QNN-MO-PYNQ/qnn/params/tinier-yolo-bwn-3bit-relu-nomaxpool.cfg".encode())
15
---> 16 net_darknet = lib.parse_network_cfg(file_name_cfg)
How can I solve this problem? Thank you.
This is what the author wrote here:(in the project of tiny-yolo-image.ipynb)
Then he used this function (lib.draw_detection_python()) to draw the bounding-box,if I want to know the location of the box. What should I do?
Hi,
I am wondering if this Tinier YOLO design have the same flow with Tincy YOLO shown in Xilinx website?
https://forums.xilinx.com/t5/Xcell-Daily-Blog/Tincy-YOLO-a-real-time-low-latency-low-power-object-detection/ba-p/815840
Excuse me
The original tiny-yolo model filter number is 16-32-64-128-256-512-1024-1024
Why is the filter number used here 16-64-64-128-256-512-512?
And more accurate than the tiny-yolo model
OSError Traceback (most recent call last)
in ()
16 from qnn import utils
17 sys.path.append("/opt/darknet/python/")
---> 18 from darknet import *
19
20 from matplotlib import pyplot as plt
/opt/darknet/python/darknet.py in ()
37
38 #lib = CDLL("/home/pjreddie/documents/darknet/libdarknet.so", RTLD_GLOBAL)
---> 39 lib = CDLL("/opt/darknet/libdarknet.so", RTLD_GLOBAL)
40 lib.network_width.argtypes = [c_void_p]
41 lib.network_width.restype = c_int
/usr/lib/python3.6/ctypes/init.py in init(self, name, mode, handle, use_errno, use_last_error)
346
347 if handle is None:
--> 348 self._handle = _dlopen(self._name, mode)
349 else:
350 self._handle = handle
OSError: /opt/darknet/libdarknet.so: file too short
Can you help me? Thank you very much.
the supported platform ultra96 is v1 or v2? thanks
Hi, I am trying to rebuild this system for other platform, Could I change the device in make-hw.sh script to other device and redo the synthesis?
Hi, giuliogamba
After ran the tiny-yolo-image.iynb, only get the plotted image with bbox and a probs file.
How to get each boxes information like (meta.names[i], probs[j][i], (boxes[j].x, boxes[j].y, boxes[j].w, boxes[j].h) as /opt/darknet/python/darknet.py.
Thanks.
Hi
I use the pynq z2 with img v2.3. When I input the ''sudo pip3.6 install git+https://github.com/Xilinx/QNN-MO-PYNQ.git", the error is '/usr/bin/arm-linux-gnueabihf-ld : cannot find -lsds_lib'. Please help me, it troubled me for three days.
Thanks for your work .
I'm trying to open the HLS project of this work by
: vivado_hls -f hls-syn.tcl
but I don't know what exactly I should type for argv defined in the .tcl
such as config_hwsrcdir, config_proj_part, config_clkperiod ...
Could you please give me some guide ? I will be very greatful.
Sorry that I'm a beginer and Vivado or HLS.
In the "QNN-MO-PYNQ/qnn/src/network/sw/main.cpp"
line 66 to 68:
extern "C" { void initParameters(unsigned int const batch, unsigned int const threads); void initAccelerator(char const *networkJson, char const *layerJson);
The json file of network need to be assigned.
But is the format of this json file the same with general deeplearning framework like theano?
Hello,
Analyzing your provided Python code I came to :
Image loaded array axes (shape) (3,h,w) = [Color, Y, X] , it's ok
then going img_copy = np.swapaxes(img_copy, 0,2) with result [X,Y,C] <- Not Ok for most image operations
then in utils.conv_layer transpose(0, 3, 1, 2) on [1,X,Y,C] with result [1,C,X,Y] <- here it not ok for Darknet too..
And then you get elements for convolution like from correctly positioned image ...
After buffer prepare, same data (with incorrect convolution direction is passed to Accelerator) ..
Could you comment on that ??
Also I tried loading "dog.jpg" image whith OpenCV , it give me better result.
But OpenCV have different colour space BGR (not RGB) , did you use RGB for training ?
My board is PYNQ-Z2. When I implement the project, something wrong happens. How could I deal with it?
The follow is errors:
[Place 30-640] Place Check : This design requires more Slice LUTs cells than are available in the target device. This design requires 62315 of such cell types but only 53200 compatible sites are available in the target device. Please analyze your synthesis results and constraints to ensure the design is mapped to Xilinx primitives as expected. If so, please consider targeting a larger device. Please set tcl parameter "drc.disableLUTOverUtilError" to 1 to change this error to warning.
Hi giuliogamba,
I am sorry to bother you again, but i have some trouble.
I want my pynq to communicate with arduino because there is some data on pynq that needs to be sent to arduino, and i want to send this data through the pmod pin, but the .bit file used in this project doesnt seem to support pmod. I am a pynq beginner, and i dont know how to use vivado rebuild hardware, so i am frustrated. I have to implement pynq communicate with arduino as quickly as possible, so I would like to ask what is the method to realize the communication with arduino on the basis of the existing .bit files of this project? like spi? or uart? Can you give me some advise? Looking forward to your early reply. Thank you in advance.
Kind regards,
Icarus.
Hi,
I don't have the Internet connection on my PYNQ board, but I can access it locally.
Is there a way to install the framework without the Internet?
Thanks.
Aliaksei
Hi,
I want to send the coordinates of the identified object to my arduino, so I have to understand how the recognition box is drawn, which is done by the function "draw_detection_python", When I want to understand it through the help command, I cannot learn more about it. so could you please help me?
kind regards,
Icarus.
hi,
When i rebuild the HLS project,there are some errors: default template arguments for a function template are a C++11 extension [-Wc++11-extensions] and error: use of undeclared identifier 'constexpr'.
So i need your help. Thank you very much.
Hi, now i am working on zcu104, could you tell me that how to rebuild this for new platform ?
thank you very much ,now i can ran the PYNQ on my zcu104 board.next ,what should i do ?
Hey,
I am planning to implement QNN version of tiny yolo2. Because I want to implement tiny yolo2 on fpga and I am planning to use bnn for that. Then I got to know about this project. So I would like to have your advice and ideas about changing this to work for tiny yolo2 on pync.
It would be a great help if you could give me your ideas.
Thanks in advance !
Hi,
I ran ./make-hw.sh W1A2 pynq a
and got error bellow.
My version of vivado is 2018.2.
Can you help me?
Thanks
`Phase 1.2 IO Placement/ Clock Placement/ Build Placer Device
ERROR: [Place 30-640] Place Check : This design requires more Slice LUTs cells than are available in the target device. This design requires 60002 of such cell types but only 53200 compatible sites are available in the target device. Please analyze your synthesis results and constraints to ensure the design is mapped to Xilinx primitives as expected. If so, please consider targeting a larger device. Please set tcl parameter "drc.disableLUTOverUtilError" to 1 to change this error to warning.
ERROR: [Place 30-640] Place Check : This design requires more LUT as Memory cells than are available in the target device. This design requires 30417 of such cell types but only 17400 compatible sites are available in the target device. Please analyze your synthesis results and constraints to ensure the design is mapped to Xilinx primitives as expected. If so, please consider targeting a larger device. Please set tcl parameter "drc.disableLUTOverUtilError" to 1 to change this error to warning.
ERROR: [Place 30-640] Place Check : This design requires more LUT as Distributed RAM cells than are available in the target device. This design requires 29892 of such cell types but only 17400 compatible sites are available in the target device. Please analyze your synthesis results and constraints to ensure the design is mapped to Xilinx primitives as expected. If so, please consider targeting a larger device. Please set tcl parameter "drc.disableLUTOverUtilError" to 1 to change this error to warning.
ERROR: [Place 30-640] Place Check : This design requires more RAMS64E cells than are available in the target device. This design requires 19968 of such cell types but only 17400 compatible sites are available in the target device. Please analyze your synthesis results and constraints to ensure the design is mapped to Xilinx primitives as expected. If so, please consider targeting a larger device.
INFO: [Timing 38-35] Done setting XDC timing constraints.
Phase 1.2 IO Placement/ Clock Placement/ Build Placer Device | Checksum: 132c4f8d9
Time (s): cpu = 00:00:20 ; elapsed = 00:00:12 . Memory (MB): peak = 3554.590 ; gain = 0.000 ; free physical = 3301 ; free virtual = 25181
Phase 1 Placer Initialization | Checksum: 132c4f8d9
Time (s): cpu = 00:00:20 ; elapsed = 00:00:12 . Memory (MB): peak = 3554.590 ; gain = 0.000 ; free physical = 3300 ; free virtual = 25180
ERROR: [Place 30-99] Placer failed with error: 'Implementation Feasibility check failed, Please see the previously displayed individual error or warning messages for more details.'
Please review all ERROR, CRITICAL WARNING, and WARNING messages during placement to understand the cause for failure.
Ending Placer Task | Checksum: 132c4f8d9
Time (s): cpu = 00:00:20 ; elapsed = 00:00:12 . Memory (MB): peak = 3554.590 ; gain = 0.000 ; free physical = 3305 ; free virtual = 25185
INFO: [Common 17-83] Releasing license: Implementation
56 Infos, 21 Warnings, 0 Critical Warnings and 6 Errors encountered.
place_design failed
ERROR: [Common 17-69] Command failed: Placer could not place all instances
INFO: [Common 17-206] Exiting Vivado at Fri Aug 24 14:17:56 2018...
[Fri Aug 24 14:17:56 2018] impl_1 finished
wait_on_run: Time (s): cpu = 00:07:18 ; elapsed = 00:10:54 . Memory (MB): peak = 1732.375 ; gain = 0.000 ; free physical = 3657 ; free virtual = 25537
INFO: [Common 17-206] Exiting Vivado at Fri Aug 24 14:17:56 2018...
cp: cannot stat '/home/anhnt/QNN-MO-PYNQ/qnn/src/network/output/vivado/W1A2-pynq/W1A2-pynq.runs/impl_1/procsys_wrapper.bit': No such file or directory
cat: /home/anhnt/QNN-MO-PYNQ/qnn/src/network/output/vivado/W1A2-pynq/W1A2-pynq.runs/impl_1/procsys_wrapper_timing_summary_routed.rpt: No such file or directory
cat: /home/anhnt/QNN-MO-PYNQ/qnn/src/network/output/vivado/W1A2-pynq/W1A2-pynq.runs/impl_1/procsys_wrapper_utilization_placed.rpt: No such file or directory
cat: /home/anhnt/QNN-MO-PYNQ/qnn/src/network/output/vivado/W1A2-pynq/W1A2-pynq.runs/impl_1/procsys_wrapper_utilization_placed.rpt: No such file or directory
cat: /home/anhnt/QNN-MO-PYNQ/qnn/src/network/output/vivado/W1A2-pynq/W1A2-pynq.runs/impl_1/procsys_wrapper_utilization_placed.rpt: No such file or directory
Bitstream copied to /home/anhnt/QNN-MO-PYNQ/qnn/src/network/output/bitstream/W1A2-overlay-pynq.bit
Done!
`
When do you plan to release the customized darknet for training quantized YOLO?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.