riscv-non-isa / riscv-arch-test Goto Github PK
View Code? Open in Web Editor NEWHome Page: https://jira.riscv.org/browse/RVG-141?src=confmacro
License: Apache License 2.0
Home Page: https://jira.riscv.org/browse/RVG-141?src=confmacro
License: Apache License 2.0
A few minor issues found in the initial release:
The top level README.md
has a reference to doc/README.md
which does not exist (should it be doc/README.adoc
instead?)
The setting of the RUN_TARGET
macro in the file riscv-target/riscvOVPsim/device/rv32i/Makefile.include
uses the path $(ROOTDIR)/riscv-ovpsim/bin/Linux64/riscvOVPsim.exe
for the executable, which does not work on a Windows host (the /Linux64/
needs be changed to Windows64
for Windows hosts).
If I run with RISCV_TARGET=spike
or RISCV_TARGET=Codasip-simulator
and the target executable is not on my path I do not get a useful error message.
Since there is a directory named riscv-test-suite/rv32im
I tried running make RISCV_TARGET=riscvOVPsim RISCV_DEVICE=rv32im
, but this does not work. This appears to be because the top level Makefile
hard codes ISA=rv32i
. However changing this still does not work because the directory riscv-test-suite/rv32im
does not have a Makefile or other files that are in riscv-test-suite/rv32i
. It seems this directory should either be completed or deleted.
Host: CentOS Linux 7
Prior to the most recent change:
commit 5a978cf
Author: Neel Gala [email protected]
a fresh clone would build and run with make without any configuration. It seems the most recent change regressed something (I haven't triaged it yet):
$ riscv64-unknown-elf-gcc -v
Using built-in specs.
COLLECT_GCC=riscv64-unknown-elf-gcc
COLLECT_LTO_WRAPPER=/home/tommy/install/20200211/libexec/gcc/riscv64-unknown-elf/7.2.0/lto-wrapper
Target: riscv64-unknown-elf
Configured with: /home/tommy/boom-template/riscv-tools/riscv-gnu-toolchain/build/../riscv-gcc/configure --target=riscv64-unknown-elf --prefix=/home/tommy/install/20200211 --disable-shared --disable-threads --enable-languages=c,c++ --with-system-zlib --enable-tls --with-newlib --with-sysroot=/home/tommy/install/20200211/riscv64-unknown-elf --with-native-system-header-dir=/include --disable-libmudflap --disable-libssp --disable-libquadmath --disable-libgomp --disable-nls --src=../../riscv-gcc --enable-checking=yes --disable-multilib --with-abi=lp64d --with-arch=rv64imafdc 'CFLAGS_FOR_TARGET=-Os -mcmodel=medlow'
Thread model: single
gcc version 7.2.0 (GCC)
$ make
for isa in rv32i rv32im rv32imc rv32Zicsr rv32Zifencei; do \
make -j8 --max-load=4 RISCV_TARGET=riscvOVPsim RISCV_TARGET_FLAGS="" RISCV_DEVICE=$isa RISCV_ISA=$isa variant; \
rc=$?; \
if [ $rc -ne 0 ]; then \
exit $rc; \
fi \
done
make[1]: Entering directory `/home/tommy/riscv-compliance'
make[1]: warning: -jN forced in submake: disabling jobserver mode.
make -j8 --max-load=4 \
RISCV_TARGET=riscvOVPsim \
RISCV_DEVICE=rv32i \
RISCV_PREFIX=riscv64-unknown-elf- \
run -C /home/tommy/riscv-compliance/riscv-test-suite/rv32i
make[2]: Entering directory `/home/tommy/riscv-compliance/riscv-test-suite/rv32i'
make[2]: warning: -jN forced in submake: disabling jobserver mode.
Compile /home/tommy/riscv-compliance/work/rv32i/I-ENDIANESS-01.elf
Compile /home/tommy/riscv-compliance/work/rv32i/I-RF_x0-01.elf
Compile /home/tommy/riscv-compliance/work/rv32i/I-RF_size-01.elf
Compile /home/tommy/riscv-compliance/work/rv32i/I-RF_width-01.elf
Compile /home/tommy/riscv-compliance/work/rv32i/I-MISALIGN_JMP-01.elf
Compile /home/tommy/riscv-compliance/work/rv32i/I-MISALIGN_LDST-01.elf
Compile /home/tommy/riscv-compliance/work/rv32i/I-DELAY_SLOTS-01.elf
Compile /home/tommy/riscv-compliance/work/rv32i/I-JAL-01.elf
src/I-MISALIGN_JMP-01.S: Assembler messages:
src/I-MISALIGN_JMP-01.S:282: Error: Instruction csrr requires absolute expression
src/I-MISALIGN_JMP-01.S:287: Error: Instruction csrr requires absolute expression
src/I-MISALIGN_LDST-01.S: Assembler messages:
src/I-MISALIGN_LDST-01.S:198: Error: Instruction csrr requires absolute expression
riscv64-unknown-elf-objdump: riscv64-unknown-elf-objdump: '/home/tommy/riscv-compliance/work/rv32i/I-MISALIGN_LDST-01.elf': No such file'/home/tommy/riscv-compliance/work/rv32i/I-MISALIGN_JMP-01.elf': No such file
Are GPRs ( x1- x31) reset by the CPU reset?
I noticed that the tests do not fail immediately if there is an error in $(TARGET_SIM)
.
So for example in Ibex the termination is done by ;
and not &&
.
I was just wondering if this a feature so all the other tests are also executed.
If that is the case maybe this can be improved by using &&
and ignore the error with -
in the calling Makefile so it is more obvious in the make
output.
Or maybe this is not the intended way. I am happy to create a PR but wanted to check before what the intentions here are.
I am curious if there is any intention to reorganize the test suites. Right now, this seems a bit superficial, but in the long run this may have an effect on this repo's maintainability. For example, the rv32si
and the rv32mi
suites both have separate subdirectories called rv64si
which appear not to be symlinked. This has so far caused at least one issue (#44). Given this,
rv64*
subdirectories not exist as their own suites? I.e., why is the rv64si
directory inside of rv32si
, for example, not its own test suite?rv32*
directories? Is the intention to eventually pull the rv64*
tests out into their own suites?Note: Beyond this superficial reorganization, some discussion of privileged/unprivileged test organization has occurred in #40.
Line 39 if illegal.S sets the SSIP bit. Implementations not supporting supervisor mode can choose to not have this bit, and thus such implementation will get stuck in the self-loop that follows.
Solution: user can provide the address or macros of the memory-mapped control registers responsible for setting MSIP as part of the compliance_test.h. These can be inserted instead of line 39 and expect the same effect then on.
The RV32I 'store byte' and 'store halfword' instruction tests expect that they will be able to store data at a negative offset from the test_1_res
label, which is located immediately after the RV_COMPLIANCE_DATA_BEGIN
label:
SB: https://github.com/riscv/riscv-compliance/blob/master/riscv-test-suite/rv32i/src/I-SB-01.S#L66
SH: https://github.com/riscv/riscv-compliance/blob/master/riscv-test-suite/rv32i/src/I-SH-01.S#L66
I believe that this can (and should) cause failures if the RV_COMPLIANCE_DATA_BEGIN
section is placed at the very beginning of RAM. If RAM starts at 0x20000000, these instructions attempt to store data in address 0x1FFFFFFD, which will likely fail or trigger a trap.
If I change the linker script to start the RAM address space at 0x20000004, the tests pass as expected. Should an extra word of data be added before the test_1_res
section if a negative offset will be used with that label?
Thanks.
I find that when I run the spike tests, not all of the 64-bit tests run. All that seems to run are the 64-bit tests having a "W" in the instruction name, that is, those that are not the namesake of a 32-bit test, such as those in riscv-test-suite/rv64i/src. In contrast, most of the 64-bit instructions that are the namesake of a 32-bit instruction do not seem to be tested.
I find the following directory that seems to contain 64-bit tests: riscv-test-suite/rv32ui/rv64ui . This does not seem to participate in the testing of spike. How come? I cannot find how to run these tests. How do I run all of the 64-bit tests through, say, spike?
Am I missing something obvious here? I could just find a way to hack this in, but it seems that there must be an official way to do this and I would rather use that if there is one.
Perhaps this is related to #30 , which seems to suggest that the 64-bit tests are not even finished?
Section 2.4 on processor configuration is unclear. There are two orthogonal issues. One is that a configurable implementation has to have its options specified before it can be used. The second is that the compliance test framework has to be told what features to test. The compliance test should only cover a subset of the configuration's supported features. This implementation configuration belongs in the target environment section, as it is part of selecting the target to be tested. This section 2.4 should be about the compliance test configuration, indicating how to instruct it to test a certain set of features.
if all other files in the same folder use I-xxx-01.S style then the file I-IO.S should also be named I-IO-01.S
Commenting out this line to enable more verbose test output, the I-SRL-01
test in the rv32i
suite fails an assertion:
Test Begin
# Test number 1
Assertion violation: file src/I-SRL-01.S, line 49: x3(0000007f) != 0xf0000000
Looking at the code for this test case:
TEST_RR_OP(srl, x3, x28, x13, 0xf0000000, 0x7ff, 0x4, x5, 12, x6) # Testcase 3
This expands to:
li x28, 0x7ff
li x13, 0x4
srl x3, x28, x13
sw x3, 12(x5)
RVTEST_IO_ASSERT_GPR_EQ(x6, x3, 0xf0000000)
This should shift the value 0x7ff right by 0x4 bits, which should yield the value 0x7f, which is what we see x3
contains in the assertion.
I discovered this while testing my own implementation, only to find out that OVPsim uncovers the same error under the same conditions. I think the expected value should be 0x7f
.
When commenting out lines in the test that fail, we see similar errors later:
Assertion violation: file src/I-SRL-01.S, line 60: x5(00000000) != 0x1000
Assertion violation: file src/I-SRL-01.S, line 61: x6(00000765) != 0x43210000
Assertion violation: file src/I-SRL-01.S, line 62: x7(00000000) != 0xfffffffe
Assertion violation: file src/I-SRL-01.S, line 74: x10(00000000) != 0x2468
Assertion violation: file src/I-SRL-01.S, line 75: x11(08000000) != 0x0
Assertion violation: file src/I-SRL-01.S, line 76: x12(00ffffed) != 0xcc000000
Assertion violation: file src/I-SRL-01.S, line 77: x13(00000001) != 0xfffffffe
Assertion violation: file src/I-SRL-01.S, line 78: x14(0000ffff) != 0xf8010000
Assertion violation: file src/I-SRL-01.S, line 92: x19(0000007f) != 0xf0000000
Assertion violation: file src/I-SRL-01.S, line 103: x21(00000000) != 0x1000
Assertion violation: file src/I-SRL-01.S, line 104: x22(00000765) != 0x43210000
Assertion violation: file src/I-SRL-01.S, line 105: x23(00000000) != 0xfffffffe
Assertion violation: file src/I-SRL-01.S, line 117: x26(00000000) != 0x2468
Assertion violation: file src/I-SRL-01.S, line 118: x27(08000000) != 0x0
Assertion violation: file src/I-SRL-01.S, line 119: x28(00ffffed) != 0xcc000000
Assertion violation: file src/I-SRL-01.S, line 120: x29(00000001) != 0xfffffffe
Assertion violation: file src/I-SRL-01.S, line 130: x30(0000ffff) != 0xf8010000
It seems the simulator is producing correct output in these cases, but the expected values in the test are wrong. This only triggers failures without the quiet macro defined, as otherwise, the register values are written to the result area, but not asserted to contain the expected values shown here.
The following macro from riscv-test-env/riscv_test_macros.h
#define TEST_AUIPC(inst, destreg, correctval, imm, swreg, offset, testreg) \
TEST_CASE(testreg, destreg, correctval, swreg, offset, \
1: \
inst destreg, imm; \
la swreg, 1b; \
sub destreg, destreg, swreg; \
)
clobbers swreg
to get an position independent result. But the macro TEST_CASE
assumes that swreg
points to the signature and therefore the result is written to somewhere in the code section, not the signature. Consequently, the reference signature of test rv32i/I-AUIPC-01
is unchanged from the initial all 1s and always correct, no matter if AUIPC
is correctly implemented or not.
I propose to add an additional argument to TEST_AUIPC
for a register that can be clobbered. I will submit a pull request soon.
Update:
The problem has been fixed in I-SW-01, but (to my knowledge) persists in I-SH-01 and I-SB-01.
Here is what i think is a bug in the I-SW-01 test.
Brief:
The test writes to an address below the .data section, and program code is overwritten.
More details:
Here is the first label that the test sets up within the .data section
# Input data section.
.data
# Output data section.
RV_COMPLIANCE_DATA_BEGIN
test_1_res:
.fill 5, 4, -1
In the beginning of the test, that address is saved to x5
# Addresses for test results
la x5, test_1_res
Shortly afterwards, it is used in a store operation
# Clear memory
sw x0, -0x4(x5)
This -0x4
gambles about not storing into the .text section.
At least for me, it overwrote my halt-simulation code, so the test never ended.
Why it might not be a problem:
One can set up linker scripts to place the data section at a safe distance.
Or one can set up the macros RV_COMPLIANCE_CODE_END and RV_COMPLIANCE_DATA_BEGIN to have some safety buffer.
Hi,
I executed breakpoint test on sweRV core (https://github.com/chipsalliance/Cores-SweRV) and it failed. After some digging and asking the community (chipsalliance/Cores-VeeR-EH1#33) I found out, that mstatus.mie must be set in order for success full execution. Therefore I simply added the following lines to the test:
# set mstatus.MIE=1 (enable M mode interrupt)
li t0, 8
csrrs zero, mstatus, t0
After that the test passes.
This behavior is described in Section 5.1, ‘Native M-Mode Triggers’ of the RISC-V External Debug Support specification.
Do you agree and would you add this to the test?
Best regards
cr
Since there is a "CHECK_XLEN" macro,
the first word(double word) of begin_signature will set to 1 if XLEN is 64.
If XLEN is 32, the first and second word of begin_signature are 0xffff_ffff.
I had some problems when porting the compliance tests to my RISC-V implementation. Finally, I understand the (in my opinion) weired interaction of macros. In this post I discribe the problems I faced and what I would change to simplify the structure and increase portability.
After porting RV_COMPLIANCE_HALT
, half of the tests passed and half failed. The failing tests are the old tests that were imported from riscv-tests
. The newer tests, explicitly written for riscv-compliance
all passed.
The signature of the failing tests all differ only in the first word: it should be 1 but it was -1. The reason for that is that in most ports RV_COMPLIANCE_HALT
maps to RVTEST_PASS
which in turn uses SWSIG
to write 1 to the first word of the signature. My implementation of RV_COMPLIANCE_HALT
did not include RVTEST_PASS
nor SWSIG
and therefore did not write the 1.
However, the reference signatures of the newer tests do not start with a 1, because the correct value of the first word matters. How is that implemented? The old and the new tests differ in an include file. While the old tests include aw_test_macros.h
, the newer tests include riscv_test_macros.h
or
test_macros.h
. The former defines SWSIG
to write to the signature, while the latter define SWSIG
as empty.
Consequently, including SWSIG
in my definition of RV_COMPLIANCE_HALT
solved the problem. However, I think that is a dirty solution because it breaks the independence between test suite, test environment and target. It adds a requirement to the implementation of RV_COMPLIANCE_HALT
that should at least be documented. Even better this dependency should be removed, which can be done by the following modification:
Remove SWSIG
from RVTEST_PASS
and move it to TEST_PASSFAIL
. That works, since the new tests are terminated by RV_COMPLIANCE_HALT
(without. SWSIG
), while the old test are terminated by TEST_PASSFAIL
(with SWSIG
), which in turn calls RV_COMPLIANCE_HALT
. The empty SWSIG
definitions in riscv_test_macros.h
and test_macros.h
can be removed.
When testing this modification, I found another (related) bug: TEST_PASSFAIL
writes 1 to the first word, no matter if the test passed or failed. However, the definitions of RVTEST_PASS
and RVTEST_FAIL
imply that in the case of an error, the failing test number should be written. Therefore I propose to copy the code from these two macros to TEST_PASSFAIL
(the labels are more important than the bne
):
#define TEST_PASSFAIL \
bne x0, TESTNUM, pass; \
fail: \
sll TESTNUM, TESTNUM, 1; \
or TESTNUM, TESTNUM, 1; \
j 1f; \
pass: \
li TESTNUM, 1; \
1: \
RVTEST_SYNC; \
SWSIG (0, TESTNUM); \
RV_COMPLIANCE_HALT \
I tested the modifications in my local repo. So, when the modifications are accepted, I can provide a pull request.
first line of I-IO.S reads:
? this is wrong/misleading
Line 145 and 146 in the riscv-test-env/p/riscv_test.h should be optional and not mandatory.
The mplementations not supporting Supervisor mode will not have satp register and thus would trap again. Probably the macro RVTEST_ENABLE_SUPERVISOR can be used for this.
Similarly, setting/reading PMP should be optional based on user defined inputs - not sure if there is a macro to do this.
In the doc/ directory when I type "make" I get this:
Version number of title and document history do not match
This happens in both release version 0.1 and in the current master HEAD at commit 13e1b57
I think this is a similar issue to fix for ma_fetch, but in rv32si
.
When I run the test with target spike
with --isa=rv32ic
, the signature I get is:
00000001
ffffffff
00000002
00000003
00000004
00000005
00000006
00000007
ffffffff
.
.
.
Which does not match up with the signature produced with --isa=rv32i
, which matches the current reference signature.
Hi
Our team is considering to use Syntacore SCR-1. Syntacore provided a pre-built toolchain and the version is 7.1.0. However, we compiled a newer version 8.1.0. Recently, we ran the RISC-V compliance test but we got 8 test cases failed which are:
compliance_I-LB-01
compliance_I-LBU-01
compliance_I-LH-01
compliance_I-LHU-01
compliance_I-LW-01
compliance_I-SB-01
compliance_I-SH-01
compliance_I-SW-01
After tracing the cause, we located the toolchain generated some strange assembly codes for the following pseudo-instruction:
la x10, test_A4_data + 2048
8.1.0
55c: 00000f97 auipc t6,0x0
560: 5a4f8f93 addi t6,t6,1444 # b00 <begin_regstate>
564: 000fa183 lw gp,0(t6)
568: 004fa203 lw tp,4(t6)
56c: 008fa283 lw t0,8(t6)
570: b8418513 addi a0,gp,-1148 # 12cc <_end+0x36c>
The correct address of x10 (a0) should be 0x12cc, however, the above codes make x10 to 0xFFFFFB84. This value causes ASSERT to the core and makes the test failed. So, I did 2 trial tests:
The results are:
7.1.0
570: 00001517 auipc a0,0x1
574: d7c50513 addi a0,a0,-644 # 12ec <_end+0x38c>
la x10, test_A4_data
addi x10, x10, 1024
addi x10, x10, 1024
570: 00000517 auipc a0,0x0
574: 55c50513 addi a0,a0,1372 # acc <test_A4_data>
578: 40050513 addi a0,a0,1024
57c: 40050513 addi a0,a0,1024
The value of x10 (a0) is 0x12cc which is correct. And, all the test cases are passed as well.
=============================================================
a. May I ask if the newer version of RISC-V GNC toolchain support this syntax "la rd, symbol + offset"??
b. Why the old version of the toolchain can generate correct assemble codes but not in newer version? Can you help to verify this issue for me?
Thank you very much!
Jason
If a target simulator (defined with TARGET_SIM
) fails to execute, this error should be reported clearly.
A reason for an error could be that the path to the simulator is incorrect, or an argument causes an error.
Currently each test is executed without reporting the failure of execution.
The signature comparison should fail, if no previous results are present in the output folder.
Not reporting the actual error can cause some misunderstanding to the actual problem.
@allenjbaum as discussed in #70 and #80
In running the regression on Spike, I've come across a mismatch in the Spike execution. In particular for the I-FENCE.I-01 test at line 67 (reference below), Spike executes:
(0x001101b3) instead of (0x00000137)
It is something related on how Spike implements the FETCH instruction?
3 months back I downloaded and have been running compliance tests, things were fine.
Updated test have LOCAL_IO_PUTC(a0); in the test case causing failure.
What is the expectation here from the test case? What is testing here by this in all test cases? Please comment on this.
FN_WriteStr:
mv t0, a0;
10000:
lbu a0, (t0);
addi t0, t0, 1;
beq a0, zero, 10000f;
**LOCAL_IO_PUTC(a0);**
j 10000b;
10000:
ret;
From the RISC-V specification, Load the data into X0 register, it can be illegal instruction. you can change the code into below format
# Addresses for test data and results
la x21, test_C_data
la x22, test_C_res
# Test
lbu x1, 0(x21)
# Store results
sw x1, 0(x22)
RVTEST_IO_ASSERT_GPR_EQ(x1, 0x00000000)
Same like I-LBU-01 test
From line 280, the beginning of the trap handler has:
_trap_handler:
# increment return address
csrr x30, mbadaddr
addi x30, x30, -2
csrw mepc, x30
The privileged spec states, "The mtval
register replaces the mbadaddr
register in the previous specification."
It also states, "When a hardware breakpoint is triggered, or an instruction-fetch, load, or store address-misaligned, access, or page-fault exception occurs, mtval
is written with the faulting e�ffective address... For other exceptions, mtval
is set to zero..."
So within the trap handler, mtval
is used in place of mbadaddr
, but the behavior of the two do not match up, since the spec does not actually define mtval
behavior during misaligned jumps.
What I'm seeing is mtvec
(i.e. mbadaddr
in the test) being set to 0x0
as the spec says, so then mepc
gets set to some large invalid address, which causes mret
to return back into the trap handler, and then gets stuck in a loop.
Line 51 tries to switch off the compressed support before proceeding with the tests.
Implementations with either of the following specs:
will fail to execute the test completely.
Possible solutions:
re-arrange instructions and trap-handler such that an implementation with compressed support traps with illegal instruction, updates signature and jumps back to next test. Similar to what is done in ma_fetch.
Have compile time options to generate a different elf for different target devices.
The target environment section should explain that software is likely a component of the target environment. I would like to use the phrase "target test execution environment". The "execution environment" for a RISC-V system is something that is actually defined in Section 1.2 of the user ISA manual. It would be good to build on that here.
The following rv32i test: I-MISALIGN_JMP-01.S and rv32mi test: ma_fetch.S will not produce a signature which matches the reference signatures if the implementation supports compressed by default. I found this while running the tests on SHAKTI's e-class core.
Now this sort-of breaks the make-flow that is present. I see two options:
I prefer the second option. Avoids users from hacking the make-flow.
Following macros in test_macros.h are used for testing compressed instructions but do not use it directly:
TEST_CLWSP, TEST_CSWSP, TEST_CADDI16SP, TEST_CADDI4SPN.
I-MISALIGN_LDST-01 assumes misaligned data access will trap. Misaligned data access traps are NOT required by the ISA spec v2.2 nor the privileged spec; it is entirely reasonable for the misaligned access to be handled in hardware.
The test should be written to succeed if either the misaligned access completes correctly with no trap, or traps with the appropriate values in the CSRs.
This issue came up when I was testing a core I designed for the RISC-V soft CPU core design contest, which requires that the core pass all 55 RV32I tests. My core implements misaligned data access in hardware, but I had to add extra stuff to my core to implement the misaligned data access traps to pass the test.
I just wanted to raise awareness that the RISC-V compliance suite currently seems not respect coarser alignment restrictions on the BASE field for mtvec
of a given target implementation.
The minimum alignment mandated by the RISC-V Privileged Spec v.1.11 is 4-byte but it explicitly allows for coarser restrictions:
If mtvec is writable, the set of values the register may hold can vary by implementation. The value in the BASE field must always be aligned on a 4-byte boundary, and the MODE setting may impose additional alignment constraints on the value in the BASE field.
An implementation may have different alignment constraints for different modes. In particular, MODE=Vectored may have stricter alignment constraints than MODE=Direct.
Allowing coarser alignments in Vectored mode enables vectoring to be implemented without a hardware adder circuit.
Ibex requires mtvec
to be 256-byte aligned. Since the RISC-V compliance suite currently requires a finer alignment, the I-EBREAK
and I-ECALL
tests keep failing.
We are not sure what your view on this is, but we thought this should maybe be tracked here. For further information, please also have a look here lowRISC/ibex#428, lowRISC/ibex#99, lowRISC/ibex#100.
Hi,
I am creating my own RISCV core based on the RISCV spec. I am running my core against the RISCV compliance test suite and I hit a problem with the I-MISALIGN_LDST-01 test under the rv32i test suite.
It seems the test has a trap handler that does nothing but to store the mbadaddress lower 2 bits and the mcause csr to the memory location 0x2020 onwards.
Also the test will store the test data in the 0x2020 onwards. From the reference output it shows:
91a1b1c1
91a1b1c1
91a1b1c1
91a1b1c1
00000001
00000004
etc
I believe the above corresponds to the test:
lw x4, 0(x3)
sw x4, 0(x2)
lw x4, 1(x3)
sw x4, 4(x2)
lw x4, 2(x3)
sw x4, 8(x2)
lw x4, 3(x3)
sw x4, 12(x2)
I can understand the part about 00000001, 00000004 onwards, but I don't get it why the 1st 4 words of the reference output is 4 times 91a1b1c1.
Because a misaligned load should cause a synchronous exception, I would expect that the misaligned 'sw' instruction is not going to be committed. Hence, shouldn't the output be the original test result filler of 0xFFFFFFFF? i.e.
91a1b1c1 // first store has no misalignment, so the store is committed
ffffffff // 2nd to 4th stores causes misalignment exception, so the instruction is not committed
ffffffff
ffffffff
Does the spec says that the instruction causing the exception should be committed first before jumping to the trap handler?
Thanks,
SC
We have a handful of linker scripts, in riscv-test-env/{p,pm,pt,v}/link.ld
. These impose certain views on section naming and memory maps. These really are target platform specific details, and so should be configured on a per platform basis. Otherwise we'll be adding a new version for every platform to be tested.
The ma_fetch test has been written in a way that implementations with and without "C" support should generate the same signature. The aim is to avoid
j fail
at multiple places.
However, currently the signatures generated by implementation with and without C support are different. This is because the trap handler sets the return address in mepc to point to
SWSIG(?, TESTNUM)
instead of
li TESTNUM, ?
This merge request fixes that and also updates the signature. Has been verified with e-class, riscvOVPsim and spike.
Related issue: #5
https://github.com/riscv/riscv-compliance/blob/d8e99ee55d5fa3460c0a2f0898dd917a616763fb/riscv-test-suite/rv32imc/src/C-SWSP.S#L86
The test performs a write to offset 252, which is outside of section test_data (32 words).
in release 0.1, file riscv-test-suite/rv64i/Makefrag, variable rv64i_sc_tests contains "SRAIW" twice; this is still the case in commit 13e1b57
Makefile:49: target 'SRAIW.elf' given more than once in the same rule
In I-MISALIGN-JMP-01, mcause is written into the signature (e.g. second value of signature). The provided signature file assumes that mcause is zero (instruction address misaligned). While this makes sense, the misaligned address also triggers another synchronous 'illegal instruction' exception in my implementation because some nonsense instruction is read from that misaligned address. According to RISC-V Privileged Architectures V20190608, Table 3.7 on page 38, the 'illegal instruction' exception has a higher priority than the 'instruction address misaligned' exception and is therefore reported, leading to a failing test.
From the TestFormatSpec:
Tests shall not depend on tool specific features. For example test shall avoid usage of internal GCC macros (e.g. __risc_xlen), specific syntax (char 'a' instead of 'a) or simulator features (e.g. tohost) etc.
The environment files, such as p/riscv_test.h and linkers all use tohost
. I am curious if, in the future, this will be changed for compatibility with targets that do not use tohost
and fromhost
, or if there's some other workaround for this.
An example of a test currently affected by tohost
functionality is scall.S.
The macros are target dependent and are implemented in compliance_io.h, e.g. RVTEST_IO_ASSERT_GPR_EQ. Some registers are said to be reserved and are used by the tests.
False errors get reported by macros because the checked value is a side effect of the macro implementation. As an example rv32i/src/I-ADD-01.S line 84: the code asserts x5 is 0, but the signature checks it should be 1.
The macros are pretty useful because they basically run in many environments. The asserted value should check the real value, and not a side effect of the checking macro. If a macro is using reserved registers, then it should check it and not execute it if a conflict occurs. It is likely possible to use target specific macros that can use fixed memory locations to swap registers and thus avoid solve reserved register issues.
Last comment: by not implementing the macros, it is possible to skip it. If macros are choosen, it is however not possible to skip saving the signature to memory. It would probably save memory and accelerate tests if it was possible to disable signature saving when using self-checking macros.
The Atomic suite in this repository is labeled 'ua'. Are there any machine mode only Atomic extension compliance suite tests?
When a trigger doesn't exist, the test goes to the pass
label. The result is that the signature is not written to after the trigger check, and the target will fail make verify
. This doesn't seem like the ideal action, since it's valid for a target to have no trigger.
For context, I'm trying to add rocket chip's Default[RV32]Config
as a target, and it only has one trigger.
https://github.com/riscv/riscv-compliance/blob/master/riscv-test-suite/rv32imc/src/C.LWSP.S#L86
test_data + 252 > .word 31
It is currently passing simulators tests because they have their memory cleared before starting the test (probably).
Looking at this section of the test leaves me confused. According my understanding, we load the symbol's address of test_E2_res
into x28
, subtract 4 from it and compare it against the constant 0x8000403C
. This test fails for me since my symbol's address test_E2_res
is not anywhere near that value.
Here the excerpt from I-SB-01.S
:
RVTEST_IO_WRITE_STR("# Test part E2 - test war hazard (address register)\n");
# Address for test results
la x28, test_E2_res
# Clear memory
sw x0, 0(x28)
# Test
li x27, 0x89ABCDEF
sb x27, 0(x28)
addi x28, x28, -4
RVTEST_IO_ASSERT_GPR_EQ(x27, 0x89ABCDEF)
RVTEST_IO_ASSERT_GPR_EQ(x28, 0x8000403C) # what is this assertion here for?
I am making a new target of my RISC-V platform, and I encounter these compiler error in the test suite:
src/I-RF_x0-01.S:57: Error: illegal operands `xori x0,x0,0xFFFFF803'
src/I-JALR-01.S:291: Error: illegal operands `jalr x0,x3,0xFFFFFFFF'
And so on. I used Zephyr SDK 0.10.1, which provides GCC 8.3.0.
I believe that GCC is unhappy with the "sign extension" done by hand, as I could remove these errors by using corresponding negative numbers.
The Clang/LLVM compiler is stricter than GCC in complying with the RISC-V assembler specification. If I used the latest Clang/LLVM compiler to compile the benchmarks, I get a failure. Extracting an individual test shows this:
riscv32-unknown-elf-clang --target=riscv32-unknown-elf -march=rv32i -c
-o I-ENDIANESS-01.o -I ../../../riscv-test-env -I
../../../riscv-test-env/p -I ../../../riscv-target/ri5cy I-ENDIANESS-01.S
../../../riscv-target/ri5cy/compliance_io.h:187:12: error: operand must
be a bare symbol name
la t3, 0x10000000; sw a0, (0)(t3);;
This instruction sequence comes from the expansion of one of the standard header macros.
The error is that the RISC-V standard requires the second operand to the la
pseudo-instruction to be a symbol, not a constant. It just happens that GCC accepts a constant - arguably a bug in GCC, certainly an unofficial extension.
It seems to me that a compliance test suite should strictly follow the standard!
(this would also seem to indicate that no one is using Clang/LLVM with the compliance tests, which would certainly seem to be a good sanity check).
srli.S test included in rv32ui set doesn't have it's 'reference_output' file and also there's no error message during signature comparison.
Tests c-jalt and c-jr store value from link register directly into signature. Signature should not be dependant on program location in memory.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.