Board: Alveo- U50DD
Environment details:
Machine Make and Model : x3650 M4
Operating System: Red Hat Enterprise Linux Server release 7.6 (Maipo)
Kernel version: 3.10.0-957.el7.x86_64
After make , make install and make install-mods , we attempted to add AXI-MM C2H and AXI-MM H2C Queues. However the queues did not get added with output as below .
dmactl qdma8b000 q add idx 0 mode mm dir h2c
Zero Qs
On checking dmactl dev list below was the output.
qdma8b000 0000:8b:00.0 max QP: 0, -~-
However the output should be:
qdma82000 0000:82:00.0 max QP: 512, 0~511
On debugging further below are the observations:
- root@ XXX.XX.XXX.XX: /sys/bus/pci/devices/0000:8b:00.0/qdma
$ cat qmax
0
This should have been automatically populated to 512.
We manually updated the same as 'echo 512 > qmax based on the article.
https://forums.xilinx.com/t5/PCIe-and-CPM/QDMA-dmactl-can-not-create-queue/td-p/1078463
After that we again attempted to add a queue. However that did not work as well as the other fs fles need to be populated as well.
- On further look we could see qdma error in dmesg as below:
"qdma:qdma_csr_read: Hardware Feature not supported"
Then taking a deep dive into qdma driver code 2019.1 version we could see the below code in qdma_access.c line number 1943.
if (dev_cap->st_en || dev_cap->mm_cmpt_en)
qdma_read_csr_values(dev_hndl, QDMA_OFFSET_C2H_TIMER_CNT, index,
count, glbl_tmr_cnt);
However when I look at the qdma_ip version 3.0 on Vivado 2019.2 I can see that theres no support for AXI-MM with completion infact that option is not available in the main menu.
Attempting to fix the issue I've modified the above check to
if (dev_cap->st_en || dev_cap->mm_en) in all the qdma code locally.
Seems to fix the issues we were facing and the H2C and C2H are up and running. Having said that the
dmactl qdma8b000 q list is not yet showing the C2H queue.
However need to know if this is the correct direction of the fix as the qdma-driver doesn't seem to be reflecting what the QDMA IP features. By that what I mean is if AXI-MM with completion is not supported by the QDMA IP 3.0 Vivado 2019.1 in hardware how can the code check for the mm_cmpt_en.
Below is the diff of the code fix ive applied locally. Please ignore the Makefile changes.
_$ git diff
diff --git a/QDMA/linux-kernel/Makefile b/QDMA/linux-kernel/Makefile
index 8ffc4c4..62cf6e3 100644
--- a/QDMA/linux-kernel/Makefile
+++ b/QDMA/linux-kernel/Makefile
@@ -21,6 +21,10 @@ grep = grep 2>/dev/null
ALL subdirectories
ALLSUBDIRS := drv
DRIVER_SRC_DIR := drv
+#DEBUG := 1
+#DEBUGFS := 1
+# DEBUG_THREADS := 1
+# ERR_DEBUG := 1
subdirectories to be build
SUBDIRS := $(ALLSUBDIRS)
diff --git a/QDMA/linux-kernel/libqdma/qdma_access/qdma_access.c b/QDMA/linux-kernel/libqdma/qdma_access/qdma_access.c
index cd7aee0..0fc29af 100644
--- a/QDMA/linux-kernel/libqdma/qdma_access/qdma_access.c
+++ b/QDMA/linux-kernel/libqdma/qdma_access/qdma_access.c
@@ -1805,7 +1805,7 @@ int qdma_set_default_global_csr(void *dev_hndl)
qdma_write_csr_values(dev_hndl, QDMA_OFFSET_GLBL_RNG_SZ, 0,
QDMA_NUM_RING_SIZES, rng_sz);
-
if (dev_cap->st_en || dev_cap->mm_cmpt_en) {
-
if (dev_cap->st_en || dev_cap->mm_en) {
/* Counter thresholds */
qdma_write_csr_values(dev_hndl, QDMA_OFFSET_C2H_CNT_TH, 0,
QDMA_NUM_C2H_COUNTERS, cnt_th);
@@ -1918,7 +1918,7 @@ int qdma_set_global_timer_count(void *dev_hndl, uint8_t index, uint8_t count,
qdma_get_device_attr(dev_hndl, &dev_cap);
-
if (dev_cap->st_en || dev_cap->mm_cmpt_en)
-
if (dev_cap->st_en || dev_cap->mm_en)
qdma_write_csr_values(dev_hndl, QDMA_OFFSET_C2H_TIMER_CNT,
index, count, glbl_tmr_cnt);
else
@@ -1940,7 +1940,7 @@ int qdma_get_global_timer_count(void *dev_hndl, uint8_t index, uint8_t count,
qdma_get_device_attr(dev_hndl, &dev_cap);
-
if (dev_cap->st_en || dev_cap->mm_cmpt_en)
-
if (dev_cap->st_en || dev_cap->mm_en)
qdma_read_csr_values(dev_hndl, QDMA_OFFSET_C2H_TIMER_CNT, index,
count, glbl_tmr_cnt);
else
@@ -1962,7 +1962,7 @@ int qdma_set_global_counter_threshold(void *dev_hndl, uint8_t index,
qdma_get_device_attr(dev_hndl, &dev_cap);
-
if (dev_cap->st_en || dev_cap->mm_cmpt_en)
-
if (dev_cap->st_en || dev_cap->mm_en)
qdma_write_csr_values(dev_hndl, QDMA_OFFSET_C2H_CNT_TH, index,
count, glbl_cnt_th);
else
@@ -1984,7 +1984,7 @@ int qdma_get_global_counter_threshold(void *dev_hndl, uint8_t index,
qdma_get_device_attr(dev_hndl, &dev_cap);
-
if (dev_cap->st_en || dev_cap->mm_cmpt_en)
-
if (dev_cap->st_en || dev_cap->mm_en)
qdma_read_csr_values(dev_hndl, QDMA_OFFSET_C2H_CNT_TH, index,_
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent