riscv-software-src/riscv-pk

RISC-V Proxy Kernel

RISC-V Proxy Kernel and Boot Loader

About

The RISC-V Proxy Kernel, pk, is a lightweight application execution environment that can host statically-linked RISC-V ELF binaries. It is designed to support tethered RISC-V implementations with limited I/O capability and thus handles I/O-related system calls by proxying them to a host computer.

This package also contains the Berkeley Boot Loader, bbl, which is a supervisor execution environment for tethered RISC-V systems. It is designed to host the RISC-V Linux port.

Build Steps

We assume that the RISCV environment variable is set to the RISC-V tools install path, and that the riscv-gnu-toolchain package is installed. Please note that building the binaries directly inside the source directory is not supported; you need to use a separate build directory.

$ mkdir build
$ cd build
$ ../configure --prefix=$RISCV --host=riscv64-unknown-elf
$ make
$ make install

Alternatively, the GNU/Linux toolchain may be used to build this package, by setting --host=riscv64-unknown-linux-gnu.

By default, 64-bit (RV64) versions of pk and bbl are built. To built 32-bit (RV32) versions, supply a --with-arch=rv32i flag to the configure command.

The install step installs 64-bit build products into a directory matching your host (e.g. $RISCV/riscv64-unknown-elf). 32-bit versions are installed into a directory matching a 32-bit version of your host (e.g. $RISCV/riscv32-unknown-elf).

OpenBSD Build Steps

Install the riscv-gnu-toolchain, and follow generic build steps.

# pkg_add riscv-elf-binutils riscv-elf-gcc riscv-elf-newlib

riscv-pk's Issues

undefined reference to `__stack_chk_guard'

Failed with Linking process.

[...]
riscv64-linux-gnu-gcc -Wl,--build-id=none -nostartfiles -nostdlib -static -o pk pk.o -L. -lpk -lmachine -lsoftfloat -lutil -lgcc -T ../pk/pk.lds
pk.o: In function rest_of_boot_loader': pk.c:(.text+0x0): undefined reference to __stack_chk_guard'
./libpk.a(file.o): In function file_pwrite': file.c:(.text+0x39c): undefined reference to __stack_chk_guard'
./libpk.a(file.o): In function .L0 ': file.c:(.text+0x406): undefined reference to __stack_chk_fail'
./libpk.a(elf.o): In function load_elf': elf.c:(.text+0x0): undefined reference to __stack_chk_guard'
elf.c:(.text+0x1e8): undefined reference to __stack_chk_guard' elf.c:(.text+0x216): undefined reference to __stack_chk_fail'
./libpk.a(console.o): In function .L0 ': console.c:(.text+0x4): undefined reference to __stack_chk_guard'
console.c:(.text+0x50): undefined reference to __stack_chk_fail' ./libpk.a(console.o): In function .L7':
console.c:(.text+0x5c): undefined reference to __stack_chk_guard' ./libpk.a(console.o): In function .L0 ':
console.c:(.text+0x96): undefined reference to __stack_chk_fail' ./libpk.a(console.o): In function .L22':
console.c:(.text+0x150): undefined reference to __stack_chk_guard' ./libpk.a(syscall.o): In function sys_chdir':
syscall.c:(.text+0x750): undefined reference to __stack_chk_guard' ./libpk.a(syscall.o): In function sys_ftruncate':
syscall.c:(.text+0x7da): undefined reference to __stack_chk_fail' syscall.c:(.text+0x7e6): undefined reference to __stack_chk_guard'
./libpk.a(syscall.o): In function .L168': syscall.c:(.text+0x898): undefined reference to __stack_chk_fail'
./libmachine.a(mtrap.o): In function .L47': mtrap.c:(.text+0x16c): undefined reference to __stack_chk_guard'
./libmachine.a(mtrap.o): In function .L0 ': mtrap.c:(.text+0x18e): undefined reference to __vsnprintf_chk'
./libmachine.a(mtrap.o): In function .L55': mtrap.c:(.text+0x1bc): undefined reference to __stack_chk_fail'
./libmachine.a(mtrap.o): In function .L53': mtrap.c:(.text+0x1c8): undefined reference to __stack_chk_guard'
./libmachine.a(mtrap.o): In function .L0 ': mtrap.c:(.text+0x202): undefined reference to __stack_chk_fail'
./libmachine.a(htif.o): In function .L0 ': htif.c:(.text+0x262): undefined reference to __stack_chk_guard'
htif.c:(.text+0x2aa): undefined reference to __stack_chk_fail' ./libmachine.a(uart.o): In function uart_getchar':
uart.c:(.text+0xde): undefined reference to __stack_chk_guard' ./libmachine.a(uart.o): In function .L0 ':
uart.c:(.text+0x124): undefined reference to __stack_chk_fail' ./libmachine.a(uart16550.o): In function uart16550_getchar':
uart16550.c:(.text+0x106): undefined reference to __stack_chk_guard' ./libmachine.a(uart16550.o): In function .L0 ':
uart16550.c:(.text+0x14c): undefined reference to __stack_chk_fail' ./libmachine.a(finisher.o): In function finisher_exit':
finisher.c:(.text+0xce): undefined reference to __stack_chk_guard' ./libmachine.a(finisher.o): In function .L0 ':
finisher.c:(.text+0x114): undefined reference to __stack_chk_fail' ./libmachine.a(fdt.o): In function .L144':
fdt.c:(.text+0x93e): undefined reference to __stack_chk_guard' ./libmachine.a(fdt.o): In function .L162':
fdt.c:(.text+0xbb2): undefined reference to __stack_chk_fail' ./libmachine.a(fdt.o): In function .L0 ':
fdt.c:(.text+0x124e): undefined reference to __stack_chk_guard' fdt.c:(.text+0x12d2): undefined reference to __stack_chk_fail'
fdt.c:(.text+0x12de): undefined reference to __stack_chk_guard' fdt.c:(.text+0x1370): undefined reference to __stack_chk_fail'
fdt.c:(.text+0x137c): undefined reference to __stack_chk_guard' ./libmachine.a(fdt.o): In function .L292':
fdt.c:(.text+0x13f0): undefined reference to __stack_chk_fail' ./libmachine.a(fdt.o): In function .L0 ':
fdt.c:(.text+0x13fc): undefined reference to __stack_chk_guard' fdt.c:(.text+0x1446): undefined reference to __stack_chk_fail'
fdt.c:(.text+0x1452): undefined reference to __stack_chk_guard' ./libmachine.a(fdt.o): In function query_clint':
fdt.c:(.text+0x149c): undefined reference to __stack_chk_fail' fdt.c:(.text+0x14a8): undefined reference to __stack_chk_guard'
fdt.c:(.text+0x14f8): undefined reference to __stack_chk_fail' fdt.c:(.text+0x1504): undefined reference to __stack_chk_guard'
./libmachine.a(fdt.o): In function .L0 ': fdt.c:(.text+0x1552): undefined reference to __stack_chk_fail'
./libmachine.a(misaligned_ldst.o): In function misaligned_load_trap': misaligned_ldst.c:(.text+0x0): undefined reference to __stack_chk_guard'
misaligned_ldst.c:(.text+0x1fa): undefined reference to __stack_chk_fail' misaligned_ldst.c:(.text+0x202): undefined reference to __stack_chk_guard'
./libmachine.a(misaligned_ldst.o): In function .L1�8': misaligned_ldst.c:(.text+0x3f2): undefined reference to __stack_chk_fail'
./libsoftfloat.a(f32_to_f64.o): In function .L0 ': f32_to_f64.c:(.text+0x4): undefined reference to __stack_chk_guard'
./libsoftfloat.a(f32_to_f64.o): In function .L15': f32_to_f64.c:(.text+0xa0): undefined reference to __stack_chk_fail'
./libsoftfloat.a(f64_to_f32.o): In function .L0 ': f64_to_f32.c:(.text+0x4): undefined reference to __stack_chk_guard'
./libsoftfloat.a(f64_to_f32.o): In function .L5': f64_to_f32.c:(.text+0xb0): undefined reference to __stack_chk_fail'
./libutil.a(snprintf.o): In function .L0 ': snprintf.c:(.text+0x274): undefined reference to __stack_chk_guard'
snprintf.c:(.text+0x2aa): undefined reference to `__stack_chk_fail'
collect2: error: ld returned 1 exit status
Makefile:302: recipe for target 'pk' failed
make: *** [pk] Error 1

Cannot run an up-to-date Linux kernel in Spike or Qemu

Hi,

(this problem should probably be tagged "help" or something, cannot find a way to achieve this before submitting).

I have problems running the examples below, it was working with a version from November. I guess that this is because I am doing something wrong, or because the code is in flux in preparation for the next supervisor spec version.

I updated the repo and recompiled without making a copy of the binary before, so I don't have a working set-up now for my tests. I tried to go back to the repo state by the start of November and several other points between then and now, but there are always errors compiling or it doesn't work (hangs without doing anything). If you can tell me a way to get this working, perhaps by going back to a different repo commit, it would be very helpful.

Now onto the problem... When I compile with this line:

../configure --prefix=/usr/local/riscv64-unknown-linux-gnu --host=riscv64-unknown-linux-gnu

and then run with Spike, it complains:

$ spike +disk=root.bin bbl vmlinux
[logo]
       INSTRUCTION SETS WANT TO BE FREE
This is bbl's dummy_payload.  To boot a real kernel, reconfigure
bbl with the flag --with-payload=PATH, then rebuild bbl.

When I compile with the command line below (what I think to be the correct option for payload, I didn't find any concrete example), with a vmlinux image compiled on the same day:

../configure --prefix=/usr/local/riscv64-unknown-linux-gnu --host=riscv64-unknown-linux-gnu --with-payload=/archive/qemu-images/vmlinux

... then Spike just starts and hangs after the logo and "motto".

When I go back to mid November (and several other dates around that time), I always have this compilation problem:

make
riscv64-unknown-linux-gnu-gcc -MMD -MP -Wall -Werror -D__NO_INLINE__ -mcmodel=medany -O2 -std=gnu99 -Wno-unused -Wno-attributes -fno-delete-null-pointer-checks  -I. -I../pk -I../softfloat -c ../pk/mtrap.c
riscv64-unknown-linux-gnu-gcc -MMD -MP -Wall -Werror -D__NO_INLINE__ -mcmodel=medany -O2 -std=gnu99 -Wno-unused -Wno-attributes -fno-delete-null-pointer-checks  -I. -I../pk -I../softfloat -c ../pk/minit.c
../pk/mtrap.h: Assembler messages:
../pk/mtrap.h:208: Error: Instruction csrr requires absolute expression
../pk/mtrap.h:208: Error: Instruction csrr requires absolute expression
Makefile:299: recipe for target 'minit.o' failed

Cheers.

building bbl fails

I ran the following commands:
../configure --prefix=$RISCV --host=riscv64-unknown-linux-gnu --with-payload=../../riscv-linux/vmlinux
make

The build fails with the following error:
riscv64-unknown-linux-gnu-gcc -nostartfiles -nostdlib -static -o pk pk.o -L. -lpk -lmachine -lsoftfloat -lutil -lgcc -T ../pk/pk.lds
pk.o: In function .L16': pk.c:(.text+0x23c): undefined reference to __riscv_flush_icache'
collect2: error: ld returned 1 exit status
Makefile:302: recipe for target 'pk' failed
make: *** [pk] Error 1

32 bit skipe+pk

From what I understand from #35 , there is no clean way to compile code for RV32I and simulate it together with pk in spike, and this problem will not be fixed for a while.

I have found a workaround, however it lacks generality. Perhaps someone has ideas for improvement:

Given some code that one wants to simulate for RV32I in a file called lupi.c:

int lupi(register int x, register int y) {
   register int s;
   s = x+y;
   return s;
}

Use this command to compile it and obtain RV32I assembly code in file lupi.s:

riscv64-unknown-elf-gcc -m32 -march=RV32I -O1 -S lupi.c 

Create a file lupi_main.c that acts as interface between lupi and pk:

#include <stdio.h>
int lupi(int, int);
int main(int argc,char *argv[]) {
   int x,y,s;
   x = argv[1][0]-'0';       // 1st letter of 1st argument converted to a value between 0 e 9
   y = argv[2][0]-'0';       // 1st letter of 2nd argument
   s =  lupi(x,y);
   printf("Hello, result is %d.\n", s);
   return s;
}

Compile, assemble and link:

riscv64-unknown-elf-gcc lupi_main.c lupi.s -o lupi

Simulate:

spike pk lupi 7 2

trap_illegal_instruction and trap_instruction_access_fault when running program

I have compiled linux for RISCV and some programs for RISCV. When running these programs on the linux kernel through spike, I get the trap_illegal_instruction and trap_instruction_access_fault, and the programs stop working. I am running spike with -p2 processors.

The instructions that result in trap_illegal_instruction are:
csrr
fsd
fld

When the second core, core 1, starts running, it runs for a bit, then does the mret instruction, and immediately after this it gets stuck on the trap_instruction_access_fault exception. The full error is:
core 1: exception trap_instruction_access_fault, epc 0x0000000000000000
core 1: badaddr 0x0000000000000000

It continues like this until I CTRL+C.
Any ideas how to fix this? I would like to get some memory traces from a shared memory program, to be able to test a cache coherence protocol I am designing for a computer with a RISCV core.

spike pk hello

I installed 32-bit tool chain. (riscv32-elf-unknown-gcc)

When i check for spike pk hello, following error came..
$riscv32-unknown-elf-gcc -o hello hello.c
$spike pk hello
terminate called after throwing an instance of 'std::runtime_error'
what(): could not open pk

pk is in $RISCV/riscv32-unknown-elf/bin

spike pk hello failed

I follow the guideline in this page to check out riscv-pk and install. The install is success. I can see pk as below directory

ywang@ubuntu:~/bruce/my_riscv_rv64/riscv-tools/riscv-pk/build$ ls /home/ywang/bruce/my_riscv_rv64/myinstall_rv64/riscv64-unknown-elf/riscv64-unknown-elf/bin/
bbl dummy_payload pk

($RISCV is set to ~/bruce/my_riscv_rv64)

I've also got spike installed under ~/bruce/my_riscv_rv64/myinstall_rv64/bin/

When I try to run a binary code as suggested by riscv-tools github page, I got below error, could you tell me how to fix it?

ywang@ubuntu:/bruce/my_riscv_rv64$ mkdir junk
ywang@ubuntu:/bruce/my_riscv_rv64$ cd junk
ywang@ubuntu:/bruce/my_riscv_rv64/junk$ echo -e '#include <stdio.h>\n int main(void) { printf("Hello world!\n"); return 0; }' > hello.c
ywang@ubuntu:/bruce/my_riscv_rv64/junk$ riscv64-unknown-elf-gcc -o hello hello.c
ywang@ubuntu:~/bruce/my_riscv_rv64/junk$ spike pk hello
terminate called after throwing an instance of 'std::runtime_error'
what(): could not open pk

By the way, please ignore strikethrough mark in above. I just don't know why it appear and how to remove it

make clean removes pk dir

If you use
../configure --prefix=$RISCV --host=riscv64-unknown-elf

and then try to use
make clean

then make simply removes pk directory. I believe you meant to remove pk/pk and not parent directory

BBL stuck in loop when booting without FPU

Trying to run bbl on a rocket core sans FPU. I manually added the "-msoft-float" flag in the Makefile and compiled. BBL starts up and prints the logo, but then hangs after that.

Inspecting the instruction stream, bbl seems to be hanging while acquiring the spinlock in frontend syscall.

https://github.com/riscv/riscv-pk/blob/master/pk/frontend.c#L30

Since there was only a single core, the lock shouldn't have been taken at that point. My guess is that frontend_syscall gets interrupted before it releases the lock. The interrupt handler then calls frontend_syscall itself (presumably to send a console message) and thus hangs while trying to take the lock.

Accessing physical memory in a program

How to translate physical address to virtual address? I want to run a program on spike which reads from physical memory.

bbl always access "Invalid htif register address"

when I build this project to boot the 32-bit linux kernel with bbl, it always print out that “Invalid htif register address" and exit.
Here is my configuration of riscv-pk:

$ cd riscv-pk
$ mkdir build
$ cd build
$ ../configure --prefix=$RISCV --host=riscv32-unknown-linux-gnu --enable-32bit --with-payload=/home/shore/linux-4.6.2/vmlinux
$ make
$ make install

And when I run the bbl with payload of RV32 linux kernel on qemu

$ cd ../riscv-qemu
$ ./riscv32-softmmu/qemu-system-riscv32 -nographic -kernel /home/shore/riscv/riscv32-unknown-elf/bin/bbl

I wonder how to fix this error to boot the linux kernel. Anyone can help?

spike pk hello. could not open pk

Hello,

I know this has been asked so many times and I still can't solve it following other responses.
I cloned my sources from the rocket-chip git.

The toolchain built was successful (adding zlib).

When I run spike pk hello it yields:

terminate called after throwing an instance of 'std::runtime_error'
  what():  could not open pk

So, I need guidance and instructions for setting up the environment variables (I'm new to linux). Also, I've noticed that building from the LowRISC repo worked for me as opposed to building from the rocket-chip repo. in LowRISC spike pk hello works fine after I source the set_riscv_env file. In rocket-chip there isn't such file, and I don't know how I can set up the environment myself.

Also, when I configured the riscv-pk/build directory I ran ../configure --prefix=$RISCV/ --host=riscv64-unknown-elf. Then ../configure --prefix=$RISCV/riscv64-unknown-elf/bin/pk --host=riscv64-unknown-elf. Both didn't work.

Does anyone know how I can fix this ?

Thank you :)

How to boot from an initramfs ramdisk

I have built linux kernel from priv-1.9 branch and it seems +disk root.bin is no longer works , how to boot from an initramfs ramdisk ?

build fails on arch linux

Hello, I am on x86_64 GNU/Linux arch linux.

I did../configure --host=riscv64-unknown-elf but

$ make
gcc -MMD -MP -Wall -Werror -D__NO_INLINE__ -mcmodel=medany -O2 -std=gnu99 -Wno-unused -Wno-attributes -fno-delete-null-pointer-checks  -DBBL_PAYLOAD=\"bbl_payload\" -DBBL_LOGO_FILE=\"bbl_logo_file\" -I. -I../pk -I../bbl -I../softfloat -I../dummy_payload -I../machine -I../util -c ../pk/file.c
gcc: error: unrecognized argument in option ‘-mcmodel=medany’
gcc: note: valid arguments to ‘-mcmodel=’ are: 32 kernel large medium small; did you mean ‘medium’?
make: *** [Makefile:302: file.o] Error 1

If I try the gcc command with -mcmodel=medany deleted it give this error:

../pk/file.c: Assembler messages:
../pk/file.c:28: Error: no such instruction: `fence'

`spike pk hello` does not terminate (32bit)

On macOS, I have built and installed the 32bit version of pk and of the riscv-gnu-toolchain.
I manage to compile programs using riscv32-unknown-elf-gcc, but when running spike -l path/to/pk32 hello, the script never stops, and if I halt it manually I get exception trap_instruction_access_fault.

I've looked for the problem online, and my impression is that the 32bit version of the toolchain is not really that stable. I haven't found a solution for my issue.
Am I doing something wrong?

How to enable RoCC in BBL

I know the way to enable RoCC in pk, but when I boot linux with BBL. It seems that the RoCC is not enable.
Could anybody know how to enable RoCC in BBL?

Error: Instruction csrr requires absolute expression

Trying to Compile FreeRTOS for riscV

Error:

CC ../../Source/portable/GCC/RISCV/port.c

../../Source/portable/GCC/RISCV/port.c: Assembler messages:
../../Source/portable/GCC/RISCV/port.c:121: Error: Instruction csrr requires absolute expression
../../Source/portable/GCC/RISCV/port.c:123: Error: Instruction csrw requires absolute expression
../../Source/portable/GCC/RISCV/port.c:130: Error: Instruction csrr requires absolute expression
../../Source/portable/GCC/RISCV/port.c:132: Error: Instruction csrw requires absolute expression
Makefile:123: recipe for target '../../Source/portable/GCC/RISCV/port.o' failed
make: *** [../../Source/portable/GCC/RISCV/port.o] Error 1

i know that some registers like mtime,mtimecmo, mtohost, are invalid in this source code of FreeRTOS

how to add that support in this Source?

Compile error, "Instruction csrw requires absolute expression"

Compiling with the latest toolchain built by riscv-tools results in:

riscv64-unknown-elf-gcc -MMD -MP -Wall -Werror -D__NO_INLINE__ -mcmodel=medany -O2 -std=gnu99 -Wno-unused -Wno-attributes -fno-delete-null-pointer-checks  -DBBL_PAYLOAD=\"dummy_payload\" -I. -I../pk -I../bbl -I../softfloat -I../dummy_payload -I../machine -I../util -c ../machine/minit.c
../machine/minit.c: Assembler messages:
../machine/minit.c:29: Error: Instruction csrw requires absolute expression
../machine/minit.c:30: Error: Instruction csrw requires absolute expression
../machine/mtrap.h:24: Error: Instruction csrr requires absolute expression
../machine/minit.c:47: Error: Instruction csrw requires absolute expression
../machine/minit.c:48: Error: Instruction csrw requires absolute expression
../machine/minit.c:49: Error: Instruction csrr requires absolute expression
../machine/minit.c:50: Error: Instruction csrr requires absolute expression
../machine/minit.c:150: Error: unrecognized opcode 'mret'
make: *** [Makefile:300: minit.o] Error 1

-mcmodel=medany problem when compiling

Hello! I have been trying to set up the RISC V tools, I got riscv-gnu-toolchain built and installed to my prefix but then when I try to build this riscv-pk I get an error:

$ make
gcc -MMD -MP -Wall -Werror -D__NO_INLINE__ -mcmodel=medany -O2 -std=gnu99 -Wno-unused -Wno-attributes -fno-delete-null-pointer-checks  -DBBL_PAYLOAD=\"dummy_payload\" -I. -I../pk -I../bbl -I../softfloat -I../dummy_payload -I../machine -I../util -c ../pk/file.c
gcc: error: unrecognized argument in option ‘-mcmodel=medany’
gcc: note: valid arguments to ‘-mcmodel=’ are: 32 kernel large medium small
make: *** [Makefile:300: file.o] Error 1

Thanks for any input on solving this.

hello world is broken: misaligned instruction access exception

Currently, building riscv-tools and running ``spike pk hello" throws a misaligned instruction access exception. I'm not sure which tool is responsible for the error, but I believe it's the proxy kernel.

The hello world:

echo -e 'int main(void) {return 0; }' > hello.c
riscv-gcc -o hello hello.c

The output:

celio@a5:/scratch/celio$spike /home/eecs/celio/install/riscv/riscv-elf/bin/pk hello 
z  0000000000000000 ra 0000000000000000 sp 0000000000000000 gp 0000000000000000
tp 0000000000000000 t0 0000000000000000 t1 0000000000000017 t2 0000000000003514
s0 0000000000000000 s1 000000007f800000 a0 0000000000000000 a1 0000000000000032
a2 ffffffffffffffff a3 0000000080000000 a4 000000007f800000 a5 00000000000037c0
a6 0000000000000017 a7 0000000000003514 s2 0000000000000000 s3 0000000000000000
s4 0000000000000000 s5 0000000000000000 s6 0000000000000000 s7 0000000000000000
s8 0000000000000000 s9 000000000000d028 sA 0000000000800000 sB 0000000000800000
t3 00000000ff800000 t4 0000000000000000 t5 000000007f800000 t6 0000000000003f3c
sr 0000000000000000 pc 00000000ff800000 va 0000000000000000 insn       00000000
Misaligned instruction access!

It looks like the PC address 0x00000000ff800000 corresponds to the page table address.

problem compiling: 'GIGAPAGE_SIZE' undeclared (first use in this function)

This happens with master. Is it broken, or am I doing something wrong?

[...]
a - snprintf.o
a - string.o
riscv64-unknown-elf-ranlib libutil.a
riscv64-unknown-elf-gcc -MMD -MP -Wall -Werror -D__NO_INLINE__ -mcmodel=medany -O2 -std=gnu99 -Wno-unused -Wno-attributes -fno-delete-null-pointer-checks  -DBBL_PAYLOAD=\"dummy_payload\" -I. -I../pk -I../bbl -I../softfloat -I../dummy_payload -I../machine -I../util -c ../pk/pk.c
riscv64-unknown-elf-gcc -nostartfiles -nostdlib -static  -o pk pk.o -L.  -lpk  -lmachine  -lsoftfloat  -lutil -lgcc -T ../pk/pk.lds
riscv64-unknown-elf-gcc -MMD -MP -Wall -Werror -D__NO_INLINE__ -mcmodel=medany -O2 -std=gnu99 -Wno-unused -Wno-attributes -fno-delete-null-pointer-checks  -DBBL_PAYLOAD=\"dummy_payload\" -I. -I../pk -I../bbl -I../softfloat -I../dummy_payload -I../machine -I../util -c ../bbl/bbl.c
../bbl/bbl.c: In function 'supervisor_vm_init':
../bbl/bbl.c:25:58: error: 'GIGAPAGE_SIZE' undeclared (first use in this function)
   size_t num_middle_pts = (-info.first_user_vaddr - 1) / GIGAPAGE_SIZE + 1;
                                                          ^~~~~~~~~~~~~
../bbl/bbl.c:25:58: note: each undeclared identifier is reported only once for each function it appears in
Makefile:300: recipe for target 'bbl.o' failed
make: *** [bbl.o] Error 1

In the Proxy Kernel, gettimeofday always returns 0

I'm building a simple program with riscv-gcc and running with the proxy kernel. I'm using the proxy kernel to obtain some system libraries such as time.h.

Here's an example usage:

struct timeval tp;
struct timezone tzp;
gettimeofday(&tp,&tzp);
printf("time = %f \n", (double)tp.tv_sec + (double)tp.tv_usec*1.e-6); 

This always prints 0.0000, even when called in a long loop. Strange...

Any ideas what might be going wrong?

I can provide code and give more details if anyone wants to try to reproduce it.

Thanks,
Forrest

Makefile and sources have errors

I cannot build riscv-pk. error:
gcc: error: unrecognized argument in option ‘-mcmodel=medany’

when changed to 'medium' or 'kernel' getting an error:
../pk/file.c: Assembler messages:
../pk/file.c:28: Error: no such instruction: `fence'

Calculation of Enable Offsets for PLIC

Hi

I am struggling to understand the logic of the address calculation for the PLIC setup
in machine/fdt.c we have the following defines

#define HART_BASE	0x200000
#define HART_SIZE	0x1000
#define ENABLE_BASE	0x2000
#define ENABLE_SIZE	0x80

These defines are used later on to define the Machine and Supervisor offsets for the HARTS.
if HART-0 has Machine and Supervisor mode, I would expect these registers to appear at (referenced to HART_BASE)

0x202000 - 0x20201F (Hart-0 Mode-M Interrupt Enables 0-1023)
0x202020 - 0x20203F (Hart-0 Mode-S Interrupt Enables 0-1023)

However, in the address calculations performed using the current implementation this actually appears as

0x202000 - 0x20201F (Hart-0 Mode-M Interrupt Enables 0-1023)
0x202080 - 0x20209F (Hart-0 Mode-S Interrupt Enables 0-1023)

this is because the address calculation uses index to multiply the ENABLE size of 0x80

I would have thought the enable size of 0x80 is for the entire set of HART enables for all modes, ie Machine, Supervisor (and futureproof Hypervisor, User)
The existing address calculation seems to make the layout look as follows

0x202000 - 0x20201F (Hart-0 Mode-M Interrupt Enables 0-1023)
0x202080 - 0x20209F (Hart-1 Mode-M Interrupt Enables 0-1023)

Have I got this completely wrong ?
I cannot understand how this is working at all, it appears that this is defining two HARTS, both with only M-Mode

Thx
Lee

Exception related problem

I tried to read the code about the exception. Found some problems.

Some exceptions will enter the M mode first, and will return to the S mode through the exceptions return after the M mode cannot be processed.
ex:

Instruction access fault
Load access fault
Store/AMO access fault
Illegal instruction
misaligned load
misaligned store

Some code is missing in the S-mode exception handling code (pk/handlers.c).

Instruction access fault
Load access fault
Store/AMO access fault
misaligned load

At least needs to be report error

error: Cannot find simulator for target

While run the RISCV-PK we are getting the errors like
Command: ../configure --prefix=/PATH/RISCV/riscv64-unknown-elf --host=riscv64-unknown-elf

checking build system type... x86_64-unknown-linux-gnu
checking host system type... riscv64-unknown-elf
checking for riscv64-unknown-elf-gcc... no
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking for riscv64-unknown-elf-g++... no
checking for riscv64-unknown-elf-c++... no
checking for riscv64-unknown-elf-gpp... no
checking for riscv64-unknown-elf-aCC... no
checking for riscv64-unknown-elf-CC... no
checking for riscv64-unknown-elf-cxx... no
checking for riscv64-unknown-elf-cc++... no
checking for riscv64-unknown-elf-cl.exe... no
checking for riscv64-unknown-elf-FCC... no
checking for riscv64-unknown-elf-KCC... no
checking for riscv64-unknown-elf-RCC... no
checking for riscv64-unknown-elf-xlC_r... no
checking for riscv64-unknown-elf-xlC... no
checking for g++... g++
checking whether we are using the GNU C++ compiler... yes
checking whether g++ accepts -g... yes
checking for riscv64-unknown-elf-ar... no
checking for ar... ar
checking for riscv64-unknown-elf-ranlib... no
checking for ranlib... ranlib
checking for a BSD-compatible install... /usr/bin/install -c
checking for riscv64-unknown-elf-spike... no
checking for spike... no
configure: error: Cannot find simulator for target

#error single-float only is not supported

I got this error while build pk using riscv32-unkonwn-elf with config --with-arch=rv32imafc ?
"

Removing existing riscv-pk/build directory
Configuring project riscv-pk
Building project riscv-pk
../machine/fp_asm.S:6:3: error: #error single-float only is not supported

error single-float only is not supported

^~~~~
make: *** [fp_asm.o] Error 1
make: *** Waiting for unfinished jobs....
"

Infinite trap_instruction_access_fault loop while calling sbi_hart_id during Linux startup

While trying to get Linux 4.6.2 running in spike, I am getting a series of trap_instruction_access_faults that appear to be looping. In case I am doing something wrong, here is the sequence of commands that I used to clone and build riscv-tools and riscv-linux: https://pastebin.com/F0g5p8ZX. I am on the latest master for both repos (14bbe5a and 9079be6, respectively).

The first fault occurs when calling sbi_hart_id at riscv-linux/arch/riscv/kernel/head.S:28.

core   0: 0x0000000080200018 (0x800000e7) jalr    ra, zero, -2048
: 
core   0: exception trap_instruction_access_fault, epc 0xfffffffffffff800
core   0:           badaddr 0xfffffffffffff800

Stepping past the fault results in the pc getting sent to the start of trap_vector (riscv-pk/machine/mentry.S:35), which jumps to .Lhandle_trap_in_machine_mode, and results in a series of nested calls: pmp_trap → redirect_trap → __redirect_trap → restore_mscratch. At mentry.S:185, restore_mscratch does an mret, which results in a second fault:

core   0: 0x00000000800001a4 (0x30200073) mret
: 
core   0: exception trap_instruction_access_fault, epc 0x0000000000000000
core   0:           badaddr 0x0000000000000000

The pc gets sent back to trap_vector again, and goes through the same series of jumps and calls, ending up back at mentry.S:185. The mret faults again with the same address, which seems like an infinite loop:

core   0: 0x00000000800001a4 (0x30200073) mret
: 
core   0: exception trap_instruction_access_fault, epc 0x0000000000000000
core   0:           badaddr 0x0000000000000000

Double-Checked Lock in __vmr_alloc in mmap.c

I was looking in mmap.c and noticed the following code:

static vmr_t* vmrs;

static uintptr_t __page_alloc()
{
  kassert(next_free_page != free_pages);
  uintptr_t addr = first_free_page + RISCV_PGSIZE * next_free_page++;
  memset((void*)addr, 0, RISCV_PGSIZE);
  return addr;
}

static vmr_t* __vmr_alloc(uintptr_t addr, size_t length, file_t* file,
                          size_t offset, unsigned refcnt, int prot)
{
  if (!vmrs) {
    spinlock_lock(&vm_lock);
      if (!vmrs)
        vmrs = (vmr_t*)__page_alloc();
    spinlock_unlock(&vm_lock);
  }
 /* ... */

The above is a straightforward example of the common double-checked-lock anti-pattern. In essence, the outer unlocked check of vmrs is used to check to see whether vmrs has been initialized. Unfortunately, vmrs not being NULL is no guarantee it has been fully initialized; if __page_alloc is inlined into __vmr_alloc, it is possible that it assigns vmrs a non-null value before the memset of the pointed-to memory completes. This can cause problems later if vmrs is used by another thread before fully initialized; for instance, if the other thread notes that vmrs is not null before the memset completes, it may be fooled into believing that a garbage value for an element's refct is genuine.

This can be fixed by simply deleting the check of vmrs outside of the lock.

I have created tool chain with --with-arch=rv64ima option and I have cross compiled linux 4

I have created tool chain with --with-arch=rv64ima option and I have cross compiled linux 4.12. When I am trying to booting linux from spike with "spike bbl vmlinux--isa=rv64ima" option but I am getting error

Error:

billa@billa:~/Pictures/latest/riscv-tools/riscv-pk/build$ spike bbl vmlinux --isa=rv64ima
../machine/mtrap.c:18: machine mode: unhandlable trap 4 @ 0x0000000080003054
Power off

Please help me I am struck with this error

Thanks in advance

Correct configure command for 32-bit pk

This is just a minor issue, but I've been trying to get spike working with 32-bit binaries and have found that supplying --enable-32bit here causes build failures (there's no -m32 option any more), but the following configure command does work (assuming you have a riscv32-unknown-elf toolchain built):

../configure --prefix=$RISCV --host=riscv32-unknown-elf

riscv-isa-sim can then be configured as follows:

../configure --prefix=$RISCV --with-fesvr=$RISCV --with-isa=rv32ima

If you could update the README.md for this repo (and maybe also riscv-isa-sim), I think that will help people.

config.guess/config.sub should be updated

When trying to compile riscv-pk on a Fedora/RISC-V machine ...

checking build system type... ../scripts/config.guess: unable to guess system type

This script, last modified 2008-01-23, has failed to recognize
the operating system you are using. It is advised that you
download the most up to date version of the config scripts from

  http://git.savannah.gnu.org/gitweb/?p=config.git;a=blob_plain;f=config.guess;hb=HEAD
and
  http://git.savannah.gnu.org/gitweb/?p=config.git;a=blob_plain;f=config.sub;hb=HEAD

Please update them from the suggested sources!

Link libfdt in bbl ?

Hi,
Can we add libfdt in bbl? If we need to add new device tree nodes, that would be bit complex by doing in hand. It has already inbuilt functions for adding/modifying/removing DT sub nodes. We can leverage inbuilt libfdt functions.
Is there any specific reason not to include libfdt in bbl ? Is it size or license ?
libfdt is dual licensed (BSD & GPL.)

One of the possible use cases:
I wanted to add "cpu-map" node to the device tree to setup the cpu topologies correctly.
It would be much simpler with libfdt functions. I believe there will be more use cases in future where we need to add/modify DT nodes as well.

Here are some more details
http://lists.infradead.org/pipermail/linux-riscv/2018-May/000646.html

@aswaterman @terpstra @palmer-dabbelt

Regards,
Atish

hart_filter_done() trashes DTB when changing status to "masked"

The hart_filter_done() function in machine/fdt.c changes the value of the "status" property in the cpu node to "masked" for harts that it determines should be filtered (e.g. the E51 core in a U54-MC). However the method used will trash the DTB if the string being replaced is not the same length as the new string ("masked").

You should be able to see this by adding calls to fdt_print() before and after hart_filter_done() is called and running with a device tree file with the status for a cpu set to "disabled", for example.

The workaround is to set the status in the device tree to a string the same length as "masked".

spike pk test

spike pk hello
warning: only got 2422210560 bytes of target mem (wanted 4294967296)
terminate called after throwing an instance of 'std::runtime_error'
what(): could not open pk

How to solve this issue

Regarding ABI

When I ran "riscv64-unknown-elf-gcc -m32 check.c"

It gave response..
/home/suseela/Desktop/riscv/lib/gcc/riscv64-unknown-elf/5.3.0/../../../../riscv64-unknown-elf/bin/ld: /home/suseela/Desktop/riscv/lib/gcc/riscv64-unknown-elf/5.3.0/../../../../riscv64-unknown-elf/lib/crt0.o: ABI is incompatible with that of the selected emulation
;
;
;
;
/home/suseela/Desktop/riscv/lib/gcc/riscv64-unknown-elf/5.3.0/../../../../riscv64-unknown-elf/bin/ld: /home/suseela/Desktop/riscv/lib/gcc/riscv64-unknown-elf/5.3.0/../../../../riscv64-unknown-elf/lib/crt0.o: file class ELFCLASS64 incompatible with ELFCLASS32
/home/suseela/Desktop/riscv/lib/gcc/riscv64-unknown-elf/5.3.0/../../../../riscv64-unknown-elf/bin/ld: final link failed: File in wrong format
collect2: error: ld returned 1 exit status

Is there any issue with version? How to solve this issue?

In-tree builds are broken

If you do try to do an in-tree build, make errors out with

riscv64-unknown-elf/bin/ld: cannot open output file pk: Is a directory

This is because ‘pk’ is in fact a directory in the source repository.

README.md already instructs users to do an out-of-tree build, but it should be easy to make in-tree builds work as well.

configure --enable-32bit uses non-existent -m32 compiler flag

Trying to build 32-bit bbl with a multilib toolchain so tried the --enable-32bit option:

riscv64-unknown-elf-gcc: error: unrecognized command line option '-m32'
Makefile:302: recipe for target 'file.o' failed
make: *** [file.o] Error 1

This works:

CC='riscv64-unknown-elf-gcc -march=rv32imac -mabi=ilp32' \
    .../configure --host=riscv64-unknown-elf

I tried CFLAGS and LDFLAGS but they don't get passed through.

It showed up a separate 32-bit compile bug due to my recent change for firmware only builds (can't cast from uin64_t to void* on riscv32).

$ make
riscv64-unknown-elf-gcc -march=rv32imac -mabi=ilp32 -MMD -MP -Wall -Werror -D__NO_INLINE__ -mcmodel=medany -O2 -std=gnu99 -Wno-unused -Wno-attributes -fno-delete-null-pointer-checks -fno-PIE  -DBBL_PAYLOAD=\"bbl_payload\" -DBBL_LOGO_FILE=\"bbl_logo_file\" -I. -I../src/pk/pk -I../src/pk/bbl -I../src/pk/softfloat -I../src/pk/dummy_payload -I../src/pk/machine -I../src/pk/util -c ../src/pk/machine/fdt.c
../src/pk/machine/fdt.c: In function 'chosen_prop':
../src/pk/machine/fdt.c:583:26: error: cast to pointer from integer of different size [-Werror=int-to-pointer-cast]
     scan->kernel_start = (void*)val;
                          ^
../src/pk/machine/fdt.c:586:24: error: cast to pointer from integer of different size [-Werror=int-to-pointer-cast]
     scan->kernel_end = (void*)val;
                        ^

Creating BBL with vmlinux for FPGA

Hi,
I am trying to build a bbl (from riscv-tools release branch) with my custom built vmlinux payload (using the --with-payload flag). However, when I run the bbl on the FPGA, the RISCV logo comes up, but the kernel does not boot.

I have tested the vmlinux on the freedom-u-sdk spike, and it boots.
I have tried using the bbl from the freedom-u-sdk on the FPGA and it doesn't work, even the logo does not show up.

Is the bbl board specific?
FYI, I have tried on both Zybo and Zedboard.

Thanks,
Asmit

Incorrect result from single-precision FP emulation code

See riscv-software-src/riscv-isa-sim#147 for earlier discussion.

$ cat t.c
#include <stdlib.h>

float s2f(int s) {
  return s;
}

int main() {
  if (s2f(0x80000000) != -2147483648.0)
    abort();
  return 0;
}

$ riscv32-unknown-elf-gcc -O0 -march=rv32imaf -mabi=ilp32f t.c
$ spike --isa=rv32ima pk a.out; echo $?
1

The emulated fcvt+fmv pair results in the incorrect value being generated in the a0 register in the s2f function body. Specifically, a0 contains 0xdf800000 rather than the expected 0xcf000000. It's possible there's a spike emulation bug or a GCC bug, but the pk soft float code seems the most likely culprit at this time.

For reference, this is the assembly of the test program:

	.file	"t.c"
	.option nopic
	.text
	.align	2
	.globl	s2f
	.type	s2f, @function
s2f:
	add	sp,sp,-32
	sw	s0,28(sp)
	add	s0,sp,32
	sw	a0,-20(s0)
	lw	a5,-20(s0)
	fcvt.s.w	fa5,a5
	fmv.x.s	a0,fa5
	lw	s0,28(sp)
	add	sp,sp,32
	jr	ra
	.size	s2f, .-s2f
	.align	2
	.globl	main
	.type	main, @function
main:
	add	sp,sp,-16
	sw	ra,12(sp)
	sw	s0,8(sp)
	add	s0,sp,16
	li	a0,-2147483648
	call	s2f
	fmv.s.x	fa4,a0
	lui	a5,%hi(.LC0)
	flw	fa5,%lo(.LC0)(a5)
	feq.s	a5,fa4,fa5
	bnez	a5,.L4
	call	abort
.L4:
	li	a5,0
	mv	a0,a5
	lw	ra,12(sp)
	lw	s0,8(sp)
	add	sp,sp,16
	jr	ra
	.size	main, .-main
	.section	.rodata
	.align	2
.LC0:
	.word	3472883712
	.ident	"GCC: (GNU) 8.0.0 20171025 (experimental)"

Tool versions:

• riscv-fesvr matching the version pinned in the riscv-tools repo (f683e01)
• riscv-isa-sim matching the version pinned in the riscv-tools repo (3a4e893), configured with --with-isa=rv32imaf
• riscv-pk matching the version pinned in the riscv-tools repo (2dcae92). Built with a GCC defaulting to -march=rv32ima.

sbi_num_harts not getting the correct number of CPUs on FPGA

When booting Linux on a dual-core rocket instance on the FPGA, sbi_num_harts still returns 1. How is the num_harts variable set? From what I can tell, it seems to be parsing the command line to get the number.

riscv-pk is broken if GCC adds build-id notes

The Fedora GCC is configured to add build-id notes. Unfortunately the riscv-pk linker script puts the build-id section at address 0x8000_0000 which breaks booting.

There are various ways to fix this. A simple way is to modify the bbl linker script so that it puts the build-id note into the bss. This patch was by Stef O'Rear:

diff --git a/bbl/bbl.lds b/bbl/bbl.lds
index 2fd0d7c..59aeed1 100644
--- a/bbl/bbl.lds
+++ b/bbl/bbl.lds
@@ -94,6 +94,7 @@ SECTIONS
     *(.bss.*)
     *(.sbss*)
     *(.gnu.linkonce.b.*)
+    *(.note.gnu.build-id)
     *(COMMON)
   }

But there are other ways too. You could add -Wl,--build-id=none to the linker command. Or there is likely to be some way to suppress build-ids in the linker script.

Rebuild BBL

I tried to run VMLINUX using spike this is the message :

This is bbl's dummy_payload. To boot a real kernel, reconfigure bbl with the flag --with-payload=PATH, then rebuild bbl.

How to reconfigure bbl ?

thank you

C compiler cannot create executables

Commamd: CC=riscv64-unknown-elf-gcc ../configure --prefix=INSTALL_PATH/RISCV/riscv64-unknown-elf --host=riscv64-unknown-elf

checking build system type... x86_64-unknown-linux-gnu
checking host system type... riscv64-unknown-elf
checking for riscv64-unknown-elf-gcc... riscv64-unknown-elf-gcc
checking whether the C compiler works... no
configure: error: in /Downloads/RISCV/Toolchain/riscv-pk/build':
configure: error: C compiler cannot create executables

segfault on binary with large static initialization data

I have a small matrix multiply benchmark that previously used to run with spike pk. I installed the latest riscv-tools from github, but now it segfaults:

$ spike pk mm
z  0000000000000000 ra 000000000001022c sp 000000007ffc51a0 gp 0000000000039cc8
tp 0000000000000000 t0 0003c5230004131b t1 00000000800052b0 t2 0003a662000420a4
s0 000000007ffe2670 s1 0000000000009c40 a0 000000007ffec2b0 a1 000000007ffe2670
a2 000000007ffec2b0 a3 0000000000000000 a4 0000000000000024 a5 00000000800052b0
a6 000000007ffcef80 a7 000000008001e2b0 s2 fffffffffffcf000 s3 0000000000000000
s4 0000000000000000 s5 0000000000000000 s6 0000000000000000 s7 0000000000000000
s8 0000000000000000 s9 0000000000000000 sA 0000000000000000 sB 0000000000000000
t3 0000000000019000 t4 000000007ffe28f0 t5 00000000800052b0 t6 0003e6ec00033547
pc 000000000001046c va 00000000800052b0 insn       ffffffff sr 8000000000006000
User load segfault @ 0x00000000800052b0

The matrix data is defined in the code as a static initialization of three 160x160 matrices (const int a[160][160] = {...}; etc). I reduced the data size to 100x100 and then it runs correctly. My predecessors used to compile and run in 32bit mode whereas I installed the default 64bit tools, so that probably increased the space used by the static data.

Increasing the memory space available to spike using the -m flag has no effect.

Is this a bug or am I running into some size limit imposed on the input binary by pk?

move all input comment sections out of output data section

I'm trying to build bbl and pk with the LLD-based toolchain.
LLD complains it's unable to merge incompatible section flags.

linux/bin/ld: error: incompatible section flags for .data
:(.comment): 0x30
output section .data: 0x13

Is it better to isolate comment section in bbl/bbl.lds and pk/pk.lds?

In FP emulation mode, writes to FCSR should write when the FPU is disabled

Michael and I were just scrolling through the code and noticed. Note that I didn't actually try this, but I'm pretty sure it doesn't match the RISC-V spec: https://github.com/riscv/riscv-pk/blob/66701f82f88d08d3700d8b0bc5d5306abfd0044f/machine/emulation.c#L198

Support clone system call in riscv-pk

Hi,

I plan to support clone() system call to create and map threads to cores in riscv-pk. Basically I want pk to make a copy of current thread context that calls clone(), make a new thread context and start the new thread on a new core. Can anyone give me some directions for implementing this system call in pk? Thank you very much!

Tuan

building 32-bit pk failed

I'm sorry about it but I found that it was my mistake. Could you please close this issue?
Thank you!

Config string code looks like it was lost in a bad push?

b2b484e is now showing on zero branches, and https://github.com/riscv/riscv-pk/blob/master/machine/sbi.h no longer documents any way to access the config string. It could be that the config string is deliberately no longer exposed to S-mode but I think it more plausible that the repository is broken.