nuclei-software / nuclei-linux-sdk Goto Github PK

View Code? Open in Web Editor NEW

38.0 7.0 10.0 2.89 MB

Nuclei RISC-V Linux Software Development Kit

License: Apache License 2.0

Makefile 42.96% Batchfile 1.99% C 16.52% Shell 2.10% Python 36.43%

riscv linux embedded sdk nuclei riscv64 buildroot

nuclei-linux-sdk's Introduction

Nuclei Linux SDK

Please check about each branch feature to learn which branch you should choose.

Tested Configurations

Docker

If you want to use it in docker image, please follow steps below:

Not tested for upload freeloader, need USB connection.

See How to evaluate Nuclei Linux SDK in docker

Ubuntu 20.04 x86_64 host

Status: Working
Build dependencies
- packages: see apt.txt
- python pip packages: pipreq.txt
Get prebuilt qemu and openocd 2023.10 from Nuclei Development Tools
Setup qemu and openocd and add it into PATH
mtools version >= 4.0.24

Build Instructions

Install Dependencies

Install the software dependencies required by this SDK using command:

sudo xargs apt-get install -y < .github/apt.txt
sudo pip3 install -r .github/pipreq.txt

Install Nuclei Tools

Download prebuilt 64bit openocd tool and qemu from Nuclei Development Tools, and extract it into your PC, and then setup PATH using this command:

>= 2023.10 release is required. You can install Nuclei Studio, which contains prebuilt openocd/qemu

# Make sure you changed /path/to/openocd/bin and /path/to/qemu/bin to the real path of your PC
export PATH=/path/to/openocd/bin:/path/to/qemu/bin:$PATH
# Check path is set correctly
which openocd qemu-system-riscv64

Fix nuclei riscv gdb run issue

You may meet with this issue: error while loading shared libraries: libgmp.so.3: cannot open shared object file: No such file or directory

# see issue https://github.com/Nuclei-Software/nuclei-linux-sdk/issues/5
sudo ln -s /lib/x86_64-linux-gnu/libgmp.so /lib/x86_64-linux-gnu/libgmp.so.3

If you met other strange issues not documented in this doc, please check Linux SDK Issues, if the existing issues not address your problem, please create a new issue

Clone Repo

Gitee Mirror not longer work, since linux mirror repo is blocked by gitee, see #10 (comment)

Checkout this repository using git.

Change the below dev_nuclei_5.10_v2 to your desired branch.

If you have good network access to github, you can clone this repo using command git clone -b dev_nuclei_5.10_v2 https://github.com/Nuclei-Software/nuclei-linux-sdk
Otherwise, you can try methods provided #10
If https is not stable, you can try ssh, please search about git clone ssh/https difference

Then you will need to checkout all of the linked submodules using:

cd nuclei-linux-sdk
# the following command might fail due to network connection issue
# you can clone less code with --depth=1
# you can also try some github mirror tech, search in baidu/google
# if still not working, you can try our prepared source code(maybe out of date), see https://github.com/Nuclei-Software/nuclei-linux-sdk/issues/4
git submodule update --recursive --init
# if you want to clone less source code to speed up or make clone stable, please add extra --depth=1
# --depth=1 is used to do shallow clone, see https://git-scm.com/docs/git-submodule#Documentation/git-submodule.txt---depth
# git submodule update --init --depth=1

To make sure you have checked out clean source code, you need to run git status command, and get expected output as below:

On branch dev_nuclei_5.10_v2
Your branch is up to date with 'origin/dev_nuclei_5.10_v2'.

nothing to commit, working tree clean

If you have trouble in get clean working tree, you can try command git submodule update --recursive --init --depth=1 again, you might need to retry several times depending on your network access speed.
If you still have issues, please check FAQ sections at the bottom of this README.md

This will take some time and require around 2GB of disk space. Some modules may fail because certain dependencies don't have the best git hosting. The only solution is to wait and try again later (or ask someone for a copy of that source repository).

Update source code

Update source code if there are new commits in this repo.

Assume currently you are in dev_nuclei_5.10_v2 branch, and the working tree is clean.

Then you run the following command to update this repo:

# Pull lastest source code and rebase your local commits onto it
git pull --rebase origin dev_nuclei_5.10_v2
# Update git submodules
git submodule update
# Check workspace status to see whether it is clean
git status

Switch branch

See #10

Show Help

You can run make help to show help message about how to use this Nuclei Linux SDK.

But if you want to change and adapt for your SoC, you need to understand the build system in Makefile.

Nuclei Linux SDK Integration

Here are the version numbers of sub projects used in Nuclei Linux SDK.

Linux 5.10.y
Uboot v2021.01
OpenSBI v0.9
Buildroot 2020.11.2

Our changes to support Nuclei Eval SoC are adapted based on above version.

Modify Build Configuration

You can choose different core configuration by modify the CORE ?= ux900fd line in Makefile.

We support four configurations for CORE, choose the right core according to your configuration:

ux600 or ux900: rv64imac RISC-V CORE configuration without FPU.
ux600fd or ux900fd: rv64imafdc RISC-V CORE configuration with FPU.
u900 or u900fd: rv32imac/rv32imafdc RISC-V CORE configuration.

You can choose different SoC by modify SOC ?= evalsoc line in Makefile.

demosoc: Deprecated, the demostration SoC from nuclei.
evalsoc: The next generation of the demosoc, we call it evalsoc, when your cpu has iregion feature, please use this one
you can add your SoC support by adding configuration in conf/$SOC folder refer to conf/evalsoc

You can check the dts difference for evalsoc and demosoc, for more details, need to check the Nuclei RISC-V CPU ISA spec.

Now evalsoc default cpu/peripheral frequency change from 100MHz to 50MHz from 2023.06

Now evalsoc ddr base address changed from 0xA0000000 to 0x80000000 from 2023.06, so previous bitstream will not work

You can choose different boot mode by modify the BOOT_MODE ?= sd line in Makefile.

sd: boot from flash + sdcard, extra SDCard is required(kernel, rootfs, dtb placed in it)
flash: boot from flash only, flash will contain images placed in sdcard of sd boot mode, at least 8M flash is required, current onboard mcu-flash of DDR200T is only 4M, so this feature is not ready for it.

Please modify the Makefile to your correct core configuration before build any source code.

For each SoC, in conf/$SOC/, it contains a build.mk you can specify qemu, timer/cpu/peripheral hz.

TIMER_HZ: implementation dependent, you can change timer frequency to different value to overwrite the one in dts.
CPU_HZ: implementation dependent, you can change cpu frequency to different value to overwrite the one in dts.
PERIPH_HZ: implementation dependent, you can change peripheral frequency to different value to overwrite the one in dts.
SIMULATION: implementation dependent, if SIMULATION=1, only the peripherals can be simulated in rtl will be present in dts, for demosoc/evalsoc, only uart will be present, qspi will not.

TIMER_HZ/CPU_HZ/PERIPH_HZ are all implementation dependent, it required your SoC dts implement this feature, currently demosoc/evalsoc all support this.

For each SoC, in conf/$SOC, it also contains a freeloader.mk, it is used to configure freeloader feature to set cpu configuration when bring up, such as configure cache, tlb, smp feature, for details, please refer to freeloader source code.

Deprecated: If you want to compile and run using simulator xl-spike, please check steps mentioned in Booting Linux on Nuclei xl-spike
If you want to compile and run using FPGA evaluation board, please check steps mentioned in Booting Linux on Nuclei FPGA Evaluation Board

Booting Linux on Nuclei xl-spike

Note: xl_spike tool should be installed and added into PATH in advance. Contact with our sales via email [email protected] to get xl_spike tools.

This feature is deprecated now, please use Nuclei Qemu.

Run on xl_spike

If you have run make bootimages command before, please make sure you run make presim to prepare build environment for running linux in simulation.

When toolchain steps are finished, then, you can build buildroot, linux and opensbi, and run opensbi with linux payload on xlspike by running make sim.

Here is sample output running in xl_spike:

Log is not up to date, and may not working.

xl_spike --isa=rv64imac /home/hqfang/workspace/software/nuclei-linux-sdk/work/demosoc/opensbi/platform/nuclei/demosoc/firmware/fw_payload.elf
rv64 file
call xl_spike_t construct function
warning: tohost and fromhost symbols not in ELF; can't communicate with target
UART: 
UART: OpenSBI v0.9
UART:    ____                    _____ ____ _____
UART:   / __ \                  / ____|  _ \_   _|
UART:  | |  | |_ __   ___ _ __ | (___ | |_) || |
UART:  | |  | | '_ \ / _ \ '_ \ \___ \|  _ < | |
UART:  | |__| | |_) |  __/ | | |____) | |_) || |_
UART:   \____/| .__/ \___|_| |_|_____/|____/_____|
UART:         | |
UART:         |_|
UART: 
UART: Platform Name             : Nuclei Demo SoC
UART: Platform Features         : timer,mfdeleg
UART: Platform HART Count       : 1
UART: Firmware Base             : 0xa0000000
UART: Firmware Size             : 84 KB
UART: Runtime SBI Version       : 0.2
UART: 
UART: Domain0 Name              : root
UART: Domain0 Boot HART         : 0
UART: Domain0 HARTs             : 0*
UART: Domain0 Region00          : 0x00000000a0000000-0x00000000a001ffff ()
UART: Domain0 Region01          : 0x0000000000000000-0xffffffffffffffff (R,W,X)
UART: Domain0 Next Address      : 0x00000000a0400000
UART: Domain0 Next Arg1         : 0x00000000a8000000
UART: Domain0 Next Mode         : S-mode
UART: Domain0 SysReset          : yes
UART: 
UART: Boot HART ID              : 0
UART: Boot HART Domain          : root
UART: Boot HART ISA             : rv64imacsu
UART: Boot HART Features        : scounteren,mcounteren,time
UART: Boot HART PMP Count       : 16
UART: Boot HART PMP Granularity : 4
UART: Boot HART PMP Address Bits: 54
UART: Boot HART MHPM Count      : 0
UART: Boot HART MHPM Count      : 0
UART: Boot HART MIDELEG         : 0x0000000000000222
UART: Boot HART MEDELEG         : 0x000000000000b109
UART: [    0.000000] Linux version 5.10.0+ (hqfang@softserver) (riscv-nuclei-linux-gnu-gcc (GCC) 9.2.0, GNU ld (GNU Binutils) 2.32) #1 Fri Mar 19 14:47:22 CST 2021
UART: [    0.000000] OF: fdt: Ignoring memory range 0xa0000000 - 0xa0400000
UART: [    0.000000] earlycon: sbi0 at I/O port 0x0 (options '')
UART: [    0.000000] printk: bootconsole [sbi0] enabled
UART: [    0.000000] efi: UEFI not found.
UART: [    0.000000] Zone ranges:
UART: [    0.000000]   DMA32    [mem 0x00000000a0400000-0x00000000afffffff]
UART: [    0.000000]   Normal   empty
UART: [    0.000000] Movable zone start for each node
UART: [    0.000000] Early memory node ranges
UART: [    0.000000]   node   0: [mem 0x00000000a0400000-0x00000000afffffff]
UART: [    0.000000] Initmem setup node 0 [mem 0x00000000a0400000-0x00000000afffffff]
UART: [    0.000000] software IO TLB: mapped [mem 0x00000000abc8b000-0x00000000afc8b000] (64MB)
UART: [    0.000000] SBI specification v0.2 detected
UART: [    0.000000] SBI implementation ID=0x1 Version=0x9
UART: [    0.000000] SBI v0.2 TIME extension detected
UART: [    0.000000] SBI v0.2 IPI extension detected
UART: [    0.000000] SBI v0.2 RFENCE extension detected
UART: [    0.000000] riscv: ISA extensions acim
UART: [    0.000000] riscv: ELF capabilities acim
UART: [    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 63630
UART: [    0.000000] Kernel command line: earlycon=sbi
UART: [    0.000000] Dentry cache hash table entries: 32768 (order: 6, 262144 bytes, linear)
UART: [    0.000000] Inode-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)
UART: [    0.000000] Sorting __ex_table...
UART: [    0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off
UART: [    0.000000] Memory: 172900K/258048K available (2696K kernel code, 2811K rwdata, 2048K rodata, 128K init, 280K bss, 85148K reserved, 0K cma-reserved)
UART: [    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
UART: [    0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
UART: [    0.000000] riscv-intc: 64 local interrupts mapped
UART: [    0.000000] plic: interrupt-controller@8000000: mapped 53 interrupts with 1 handlers for 2 contexts.
UART: [    0.000000] random: get_random_bytes called from 0xffffffe000002910 with crng_init=0
UART: [    0.000000] riscv_timer_init_dt: Registering clocksource cpuid [0] hartid [0]
UART: [    0.000000] clocksource: riscv_clocksource: mask: 0xffffffffffffffff max_cycles: 0x179dd7f66, max_idle_ns: 28210892933900 ns
UART: [    0.001760] sched_clock: 64 bits at 100kHz, resolution 10000ns, wraps every 35184372085000ns
UART: [    0.022800] printk: console [hvc0] enabled
UART: [    0.022800] printk: console [hvc0] enabled
UART: [    0.041290] printk: bootconsole [sbi0] disabled
UART: [    0.041290] printk: bootconsole [sbi0] disabled
UART: [    0.065580] Calibrating delay loop (skipped), value calculated using timer frequency.. 0.20 BogoMIPS (lpj=1000)
UART: [    0.089740] pid_max: default: 32768 minimum: 301
UART: [    0.132740] Mount-cache hash table entries: 512 (order: 0, 4096 bytes, linear)
UART: [    0.152020] Mountpoint-cache hash table entries: 512 (order: 0, 4096 bytes, linear)
UART: [    0.324900] EFI services will not be available.
UART: [    0.359550] devtmpfs: initialized
UART: [    0.519290] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
UART: [    0.543440] futex hash table entries: 256 (order: 0, 6144 bytes, linear)
UART: [    0.566050] pinctrl core: initialized pinctrl subsystem
UART: [    0.620770] NET: Registered protocol family 16
UART: [    1.452200] clocksource: Switched to clocksource riscv_clocksource
UART: [    1.624680] NET: Registered protocol family 2
UART: [    1.795650] tcp_listen_portaddr_hash hash table entries: 256 (order: 0, 4096 bytes, linear)
UART: [    1.818280] TCP established hash table entries: 2048 (order: 2, 16384 bytes, linear)
UART: [    1.849690] TCP bind hash table entries: 2048 (order: 2, 16384 bytes, linear)
UART: [    1.877280] TCP: Hash tables configured (established 2048 bind 2048)
UART: [    1.904640] UDP hash table entries: 256 (order: 1, 8192 bytes, linear)
UART: [    1.925370] UDP-Lite hash table entries: 256 (order: 1, 8192 bytes, linear)
UART: [    1.970120] NET: Registered protocol family 1
UART: [    2.043020] RPC: Registered named UNIX socket transport module.
UART: [    2.057540] RPC: Registered udp transport module.
UART: [    2.069730] RPC: Registered tcp transport module.
UART: [    2.082040] RPC: Registered tcp NFSv4.1 backchannel transport module.
UART: [    9.297610] workingset: timestamp_bits=62 max_order=16 bucket_order=0
UART: [   10.637960] io scheduler mq-deadline registered
UART: [   10.649850] io scheduler kyber registered
UART: [   18.585530] brd: module loaded
UART: [   19.555270] loop: module loaded
UART: [   19.697090] NET: Registered protocol family 17
UART: [   19.780090] Freeing unused kernel memory: 128K
UART: [   19.799080] Run /init as init process
UART: Starting syslogd: OK
UART: Starting klogd: OK
UART: Running sysctl: OK
UART: Starting mdev... OK
UART: modprobe: can't change directory to '/lib/modules': No such file or directory
UART: Saving random seed: [  130.621540] random: dd: uninitialized urandom read (512 bytes read)
UART: OK
UART: 
UART: Welcome to Nuclei System Techology
nucleisys login: root
root
UART: Password: nuclei

UART: # cat /proc/cpuinfo
cat /proc/cpuinfo
UART: processor	: 0
UART: hart		: 0
UART: isa		: rv64imac
UART: mmu		: sv39
UART: 
UART: # uname -a
uname -a
UART: Linux nucleisys 5.10.0+ #1 Fri Mar 19 14:47:22 CST 2021 riscv64 GNU/Linux
UART: # ls /
ls /
UART: bin      init     linuxrc  opt      run      tmp
UART: dev      lib      media    proc     sbin     usr
UART: etc      lib64    mnt      root     sys      var
UART: #

If you want to remove the login, and directly enter to bash, please check Known issues and FAQ.

Booting Linux on Nuclei QEMU

From 2023.06, this branch will no longer work with Nuclei QEMU 2022.12 release, please take a try with Nuclei Qemu 2023.10 release.

Note: qemu-system-riscv64 tool should be installed and added into PATH in advance.

When the required changes has been done, then you can run make run_qemu to run riscv linux on Nuclei QEMU, here are the sample output.

You can check latest output in github action https://github.com/Nuclei-Software/nuclei-linux-sdk/actions/workflows/build.yml?query=branch%3Adev_nuclei_5.10_v2

The output may be out of date, take care.

Run on qemu for simulation
qemu-system-riscv64 -M nuclei_evalsoc,download=flashxip -smp 8 -m 2G -cpu nuclei-ux900fd,ext= -bios /Local/hqfang/workspace/software/nuclei-linux-sdk/work/evalsoc/freeloader/freeloader.elf -nographic -drive file=/Local/hqfang/workspace/software/nuclei-linux-sdk/work/evalsoc/disk.img,if=sd,format=raw

OpenSBI v0.9
   ____                    _____ ____ _____
  / __ \                  / ____|  _ \_   _|
 | |  | |_ __   ___ _ __ | (___ | |_) || |
 | |  | | '_ \ / _ \ '_ \ \___ \|  _ < | |
 | |__| | |_) |  __/ | | |____) | |_) || |_
  \____/| .__/ \___|_| |_|_____/|____/_____|
        | |
        |_|

Platform Name             : Nuclei Evaluation SoC
Platform Features         : timer,mfdeleg
Platform HART Count       : 8
Firmware Base             : 0x80000000
Firmware Size             : 156 KB
Runtime SBI Version       : 0.2

Domain0 Name              : root
Domain0 Boot HART         : 6
Domain0 HARTs             : 0*,1*,2*,3*,4*,5*,6*,7*
Domain0 Region00          : 0x0000000080000000-0x000000008003ffff ()
Domain0 Region01          : 0x0000000000000000-0xffffffffffffffff (R,W,X)
Domain0 Next Address      : 0x0000000080200000
Domain0 Next Arg1         : 0x0000000088000000
Domain0 Next Mode         : S-mode
Domain0 SysReset          : yes

Boot HART ID              : 6
Boot HART Domain          : root
Boot HART ISA             : rv64imafdcsu
Boot HART Features        : scounteren,mcounteren,time
Boot HART PMP Count       : 16
Boot HART PMP Granularity : 4
Boot HART PMP Address Bits: 54
Boot HART MHPM Count      : 0
Boot HART MHPM Count      : 0
Boot HART MIDELEG         : 0x0000000000000222
Boot HART MEDELEG         : 0x000000000000b109


U-Boot 2021.01-00021-g7e7c388fc6 (Nov 16 2023 - 16:06:13 +0800)

CPU:   rv64imafdc
Model: nuclei,evalsoc
DRAM:  2 GiB
Board: Initialized
MMC:   Nuclei SPI version 0x0
spi@10034000:mmc@0: 0
In:    serial@10013000
Out:   serial@10013000
Err:   serial@10013000
Net:   No ethernet found.
Hit any key to stop autoboot:  0
switch to partitions #0, OK
mmc0 is current device
Scanning mmc 0:1...
Found U-Boot script /boot.scr
725 bytes read in 22 ms (31.3 KiB/s)
## Executing script at 80200000
Boot images located in .
Loading kernel: ./uImage.lz4
4035022 bytes read in 9962 ms (395.5 KiB/s)
Loading ramdisk: ./uInitrd.lz4
6260962 bytes read in 15372 ms (397.5 KiB/s)
Loading dtb: ./kernel.dtb
4677 bytes read in 36 ms (126 KiB/s)
Starts booting from SD
## Booting kernel from Legacy Image at 83000000 ...
   Image Name:   Linux
   Image Type:   RISC-V Linux Kernel Image (lz4 compressed)
   Data Size:    4034958 Bytes = 3.8 MiB
   Load Address: 80400000
   Entry Point:  80400000
   Verifying Checksum ... OK
## Loading init Ramdisk from Legacy Image at 88300000 ...
   Image Name:   Initrd
   Image Type:   RISC-V Linux RAMDisk Image (lz4 compressed)
   Data Size:    6260898 Bytes = 6 MiB
   Load Address: 00000000
   Entry Point:  00000000
   Verifying Checksum ... OK
## Flattened Device Tree blob at 88000000
   Booting using the fdt blob at 0x88000000
   Uncompressing Kernel Image
   Using Device Tree in place at 0000000088000000, end 0000000088004244

Starting kernel ...

[    0.000000] Linux version 5.10.196+ ([email protected]) (riscv-nuclei-linux-gnu-gcc (GCC) 10.2.0, GNU ld (GNU Binutils) 2.36.1) #1 SMP Thu Nov 16 15:59:29 CST 2023
[    0.000000] OF: fdt: Ignoring memory range 0x80000000 - 0x80400000
[    0.000000] Machine model: nuclei,evalsoc
[    0.000000] earlycon: sbi0 at I/O port 0x0 (options '')
[    0.000000] printk: bootconsole [sbi0] enabled
[    0.000000] efi: UEFI not found.
[    0.000000] Initial ramdisk at: 0x(____ptrval____) (6262784 bytes)
[    0.000000] Zone ranges:
[    0.000000]   DMA32    [mem 0x0000000080400000-0x00000000fdffffff]
[    0.000000]   Normal   empty
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000080400000-0x00000000fdffffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000080400000-0x00000000fdffffff]
[    0.000000] software IO TLB: mapped [mem 0x00000000f8475000-0x00000000fc475000] (64MB)
[    0.000000] SBI specification v0.2 detected
[    0.000000] SBI implementation ID=0x1 Version=0x9
[    0.000000] SBI v0.2 TIME extension detected
[    0.000000] SBI v0.2 IPI extension detected
[    0.000000] SBI v0.2 RFENCE extension detected
[    0.000000] SBI v0.2 HSM extension detected
[    0.000000] riscv: ISA extensions acdfim
[    0.000000] riscv: ELF capabilities acdfim
[    0.000000] percpu: Embedded 16 pages/cpu s25432 r8192 d31912 u65536
[    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 508030
[    0.000000] Kernel command line: earlycon=sbi console=ttyNUC0
[    0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)
[    0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)
[    0.000000] Sorting __ex_table...
[    0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off
[    0.000000] Memory: 1943116K/2060288K available (4737K kernel code, 4114K rwdata, 2048K rodata, 192K init, 333K bss, 117172K reserved, 0K cma-reserved)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=8, Nodes=1
[    0.000000] rcu: Hierarchical RCU implementation.
[    0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 10 jiffies.
[    0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
[    0.000000] riscv-intc: 64 local interrupts mapped
[    0.000000] plic: interrupt-controller@1c000000: mapped 53 interrupts with 8 handlers for 16 contexts.
[    0.000000] riscv_timer_init_dt: Registering clocksource cpuid [0] hartid [6]
[    0.000000] clocksource: riscv_clocksource: mask: 0xffffffffffffffff max_cycles: 0x1ef4687b1, max_idle_ns: 112843571739654 ns
[    0.000152] sched_clock: 64 bits at 32kHz, resolution 30517ns, wraps every 70368744171142ns
[    0.005554] Calibrating delay loop (skipped), value calculated using timer frequency.. 0.06 BogoMIPS (lpj=327)
[    0.006164] pid_max: default: 32768 minimum: 301
[    0.007659] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)
[    0.008056] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)
[    0.035339] rcu: Hierarchical SRCU implementation.
[    0.037139] EFI services will not be available.
[    0.038848] smp: Bringing up secondary CPUs ...
[    0.054565] smp: Brought up 1 node, 8 CPUs
[    0.068145] devtmpfs: initialized
[    0.077972] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
[    0.079010] futex hash table entries: 2048 (order: 5, 131072 bytes, linear)
[    0.080230] pinctrl core: initialized pinctrl subsystem
[    0.082885] NET: Registered protocol family 16
[    0.135375] clocksource: Switched to clocksource riscv_clocksource
[    0.145080] NET: Registered protocol family 2
[    0.148315] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)
[    0.159118] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)
[    0.159729] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)
[    0.160308] TCP bind hash table entries: 16384 (order: 6, 262144 bytes, linear)
[    0.160827] TCP: Hash tables configured (established 16384 bind 16384)
[    0.162322] UDP hash table entries: 1024 (order: 3, 32768 bytes, linear)
[    0.162841] UDP-Lite hash table entries: 1024 (order: 3, 32768 bytes, linear)
[    0.165191] NET: Registered protocol family 1
[    0.170776] RPC: Registered named UNIX socket transport module.
[    0.171112] RPC: Registered udp transport module.
[    0.171325] RPC: Registered tcp transport module.
[    0.171569] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    0.175933] Trying to unpack rootfs image as initramfs...
[    0.458953] Freeing initrd memory: 6108K
[    0.464111] workingset: timestamp_bits=62 max_order=19 bucket_order=0
[    0.481048] jffs2: version 2.2. (NAND) © 2001-2006 Red Hat, Inc.
[    0.483306] JFS: nTxBlock = 8192, nTxLock = 65536
[    0.492706] jitterentropy: Initialization failed with host not compliant with requirements: 2
[    0.493286] NET: Registered protocol family 38
[    0.493896] io scheduler mq-deadline registered
[    0.494293] io scheduler kyber registered
[    0.617645] 10013000.serial: ttyNUC0 at MMIO 0x10013000 (irq = 1, base_baud = 3125000) is a Nuclei UART/USART
[    0.621459] printk: console [ttyNUC0] enabled
[    0.621459] printk: console [ttyNUC0] enabled
[    0.623016] printk: bootconsole [sbi0] disabled
[    0.623016] printk: bootconsole [sbi0] disabled
[    0.660919] brd: module loaded
[    0.670166] loop: module loaded
[    0.672973] nuclei_spi 10014000.spi: mapped; irq=2, cs=1
[    0.688415] spi-nor spi0.0: is25wp256 (32768 Kbytes)
[    0.794097] ftl_cs: FTL header not found.
[    0.800598] nuclei_spi 10034000.spi: mapped; irq=3, cs=1
[    0.848205] mmc_spi spi1.0: SD/MMC host mmc0, no DMA, no WP, no poweroff, cd polling
[    0.851928] ipip: IPv4 and MPLS over IPv4 tunneling driver
[    0.857940] NET: Registered protocol family 10
[    0.871582] Segment Routing with IPv6
[    0.872680] sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
[    0.880401] NET: Registered protocol family 17
[    0.918304] Freeing unused kernel memory: 192K
[    0.924316] mmc0: host does not support reading read-only switch, assuming write-enable
[    0.925354] mmc0: new SD card on SPI
[    0.928375] mmcblk0: mmc0:0000 QEMU! 1.00 GiB
[    0.958953] Run /init as init process
[    0.961059]  mmcblk0: p1
Starting syslogd: OK
Starting klogd: OK
Running sysctl: OK
Starting mdev... OK
modprobe: can't change directory to '/lib/modules': No such file or directory
Saving random seed: [    4.833526] random: dd: uninitialized urandom read (32 bytes read)
OK

Welcome to Nuclei System Technology
nucleisys login: root
Password:
# cat /proc/cpuinfo
processor       : 0
hart            : 6
isa             : rv64imafdc
mmu             : sv39

processor       : 1
hart            : 0
isa             : rv64imafdc
mmu             : sv39

processor       : 2
hart            : 1
isa             : rv64imafdc
mmu             : sv39

processor       : 3
hart            : 2
isa             : rv64imafdc
mmu             : sv39

processor       : 4
hart            : 3
isa             : rv64imafdc
mmu             : sv39

processor       : 5
hart            : 4
isa             : rv64imafdc
mmu             : sv39

processor       : 6
hart            : 5
isa             : rv64imafdc
mmu             : sv39

processor       : 7
hart            : 7
isa             : rv64imafdc
mmu             : sv39

# uname -a
Linux nucleisys 5.10.196+ #1 SMP Thu Nov 16 15:59:29 CST 2023 riscv64 GNU/Linux
# mount /dev/mmcblk0p1 /mnt/
# ls /mnt/
boot.scr     kernel.dtb   uImage.lz4   uInitrd.lz4
# free -m
              total        used        free      shared  buff/cache   available
Mem:           1904          27        1856          14          21        1850
Swap:             0           0           0

Booting Linux on Nuclei FPGA Evaluation Board

Get Nuclei Eval SoC FPGA Bitstream from Nuclei

Demo SoC is deprecated, please use Eval SoC bitstream from Nuclei.

Contact with our sales via email [email protected] to get FPGA bitstream for Nuclei Eval SoC and get guidance about how to program FPGA bitstream in the board.

Nuclei Eval SoC can be configured using Nuclei RISC-V Linux Capable Core such as UX600 and U900/UX900, To learn about Nuclei RISC-V Linux Capable Core, please check:

Nuclei FPGA Evaluation Board, DDR200T/KU060/VCU118 are correct hardwares to run linux on it, click Nuclei FPGA Evaluation Board to learn about more.

Apply changes for your SoC

Before compiling this source code, please make sure you have done the following changes.

Now we have two version of SoC for customer to evaluate our RISC-V CPU IP, if the bitstream you get from us has the iregion feature, you should use evalsoc, otherwise choose demosoc(deprecated).

If there is double float fpu and isa is rv64 in the bitstream supported, you should choose ux600fd or ux900fd.

Default cpu/periph freq and timer freq are 16MHz and 32768Hz for demosoc.
Default cpu/periph freq and timer freq are 50Mhz and 32768Hz for evalsoc.

If the bitstream you get not matching above settings, please change co-reponsibing conf/<SOC>/build.mk's TIMER_HZ/CPU_HZ/PERIPH_HZ.

If you don't change this build.mk you can also change the dts files conf/<SOC>/*.dts to match the correct frequency.

For example, you have get a bitstream which is our ux900 series cpu ip, with double float fpu, and cpu frequency is 100MHz.

You should change SOC to evalsoc, CORE to ux900fd in Makefile.

And change CPU_HZ in conf/<SOC>/build.mk or CPUCLK_FREQ in the nuclei_rv64imafdc.dts and nuclei_rv64imac.dts in conf/$SOC/(conf/evalsoc for this case).

Build Freeloader

Make sure you have network access to outside world, buildroot will download required packages in build steps.

freeloader is a first stage bootloader which contains opensbi, uboot and dtb binaries, when bootup, it will enable I/D cache and load opensbi, uboot and dtb from onboard norflash to DDR, and then goto entry of opensbi.

To build freeloader, you just need to run make freeloader

Upload Freeloader to FPGA Evaluation Board

If you have connected your board to your Linux development environment, and setuped JTAG drivers, then you can run make upload_freeloader to upload the work/$SOC/freeloader/freeloader.elf to your board.

You can also use riscv-nuclei-elf-gdb and openocd to download this program by yourself, for simple steps, please see Known issues and FAQs.

Build SDCard Boot Images

If BOOT_MODE is set to flash, then no need to prepare the boot images, just program the freeloader.elf to on board flash, but it required at least 8M flash.

If you have run make sim command before, please make sure you run make preboot to prepare build environment for generate boot images.

If the freeloader is flashed to the board, then you can prepare the SDCard boot materials, you can run make bootimages to generate the boot images to work/$SOC/boot, and an zip file called work/$SOC/boot.zip , you can extract this boot.zip to your SDCard or copy all the files located in work/$SOC/boot, make sure the files need to be put right in the root of SDCard, then you can insert this SDCard to your SDCard slot(J57) beside the TFT LCD.

The contents of work/$SOC/boot or work/$SOC/boot.zip are as below:

kernel.dtb : optional, device tree binary file, this dtb is optional now, since freeloader.elf already contains dtb, we can use dtb inside freeloader.elf, and if you want to use a different device tree for linux kernel, you can change this dtb, and place it in sdcard, otherwise this dtb is not a required file for sdcard.
boot.scr : required, boot script used by uboot, generated from ./conf/evalsoc/uboot.cmd
uImage.lz4 : required, lz4 archived kernel image
uInitrd.lz4 : required, lz4 archived rootfs image

SDCard is recommended to use SDHC format. SDCard need to be formatted to FAT32 format, with only 1 partition.

Run Linux

When all above is done, you can reset the power on board, then opensbi will boot uboot, and uboot will automatically load linux image and initramfs from SDCard and boot linux if everything is prepared correctly.

If you met with issues, please check the Known issues and FAQ.

The linux login user name and password is root and nuclei.

Sample output in UART @ 115200bps, Data 8bit, Parity None, Stop Bits 1bit, No Flow Control.

Flow control must be disabled in UART terminal.

UART baudrate changed from 57600bps to 115200bps, due to evaluation SoC frequency by default changed from 8MHz to 16MHz or 50MHz, and now uart can work correctly on 115200bps.

The output may be out of date, take care.

OpenSBI v0.9
   ____                    _____ ____ _____
  / __ \                  / ____|  _ \_   _|
 | |  | |_ __   ___ _ __ | (___ | |_) || |
 | |  | | '_ \ / _ \ '_ \ \___ \|  _ < | |
 | |__| | |_) |  __/ | | |____) | |_) || |_
  \____/| .__/ \___|_| |_|_____/|____/_____|
        | |
        |_|

Platform Name             : Nuclei Evaluation SoC
Platform Features         : timer,mfdeleg
Platform HART Count       : 8
Firmware Base             : 0x80000000
Firmware Size             : 156 KB
Runtime SBI Version       : 0.2

Domain0 Name              : root
Domain0 Boot HART         : 0
Domain0 HARTs             : 0*,1*,2*,3*,4*,5*,6*,7*
Domain0 Region00          : 0x0000000080000000-0x000000008003ffff ()
Domain0 Region01          : 0x0000000000000000-0xffffffffffffffff (R,W,X)
Domain0 Next Address      : 0x0000000080200000
Domain0 Next Arg1         : 0x0000000088000000
Domain0 Next Mode         : S-mode
Domain0 SysReset          : yes

Boot HART ID              : 0
Boot HART Domain          : root
Boot HART ISA             : rv64imafdcbsu
Boot HART Features        : scounteren,mcounteren,time
Boot HART PMP Count       : 16
Boot HART PMP Granularity : 4096
Boot HART PMP Address Bits: 30
Boot HART MHPM Count      : 0
Boot HART MHPM Count      : 0
Boot HART MIDELEG         : 0x0000000000000222
Boot HART MEDELEG         : 0x000000000000b109


U-Boot 2021.01-00021-g7e7c388fc6 (Jun 09 2023 - 17:01:18 +0800)

CPU:   rv64imafdc
Model: nuclei,evalsoc
DRAM:  2 GiB
Board: Initialized
MMC:   Nuclei SPI version 0xee010102
spi@10034000:mmc@0: 0
In:    serial@10013000
Out:   serial@10013000
Err:   serial@10013000
Net:   No ethernet found.
Hit any key to stop autoboot:  0
switch to partitions #0, OK
mmc0 is current device
Scanning mmc 0:1...
Found U-Boot script /boot.scr
725 bytes read in 334 ms (2 KiB/s)
## Executing script at 80200000
Boot images located in .
Loading kernel: ./uImage.lz4
4030405 bytes read in 19703 ms (199.2 KiB/s)
Loading ramdisk: ./uInitrd.lz4
6261647 bytes read in 30264 ms (201.2 KiB/s)
./kernel.dtb not found, ignore it
Starts booting from SD
## Booting kernel from Legacy Image at 83000000 ...
   Image Name:   Linux
   Image Type:   RISC-V Linux Kernel Image (lz4 compressed)
   Data Size:    4034958 Bytes = 3.8 MiB
   Load Address: 80400000
   Entry Point:  80400000
   Verifying Checksum ... OK
## Loading init Ramdisk from Legacy Image at 88300000 ...
   Image Name:   Initrd
   Image Type:   RISC-V Linux RAMDisk Image (lz4 compressed)
   Data Size:    6260898 Bytes = 6 MiB
   Load Address: 00000000
   Entry Point:  00000000
   Verifying Checksum ... OK
## Flattened Device Tree blob at 88000000
   Booting using the fdt blob at 0x88000000
   Uncompressing Kernel Image
   Using Device Tree in place at 0000000088000000, end 0000000088004244

Starting kernel ...

[    0.000000] Linux version 5.10.181+ ([email protected]) (riscv-nuclei-linux-gnu-gcc (GCC) 10.2.0, GNU ld (GNU Binutils) 2.36.1) #1 SMP Fri Jun 9 17:03:39 CST 2023
[    0.000000] OF: fdt: Ignoring memory range 0x80000000 - 0x80400000
[    0.000000] Machine model: nuclei,evalsoc
[    0.000000] earlycon: sbi0 at I/O port 0x0 (options '')
[    0.000000] printk: bootconsole [sbi0] enabled
[    0.000000] efi: UEFI not found.
[    0.000000] Initial ramdisk at: 0x(____ptrval____) (6262784 bytes)
[    0.000000] Zone ranges:
[    0.000000]   DMA32    [mem 0x0000000080400000-0x00000000fdffffff]
[    0.000000]   Normal   empty
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000080400000-0x00000000fdffffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000080400000-0x00000000fdffffff]
[    0.000000] software IO TLB: mapped [mem 0x00000000f8475000-0x00000000fc475000] (64MB)
[    0.000000] SBI specification v0.2 detected
[    0.000000] SBI implementation ID=0x1 Version=0x9
[    0.000000] SBI v0.2 TIME extension detected
[    0.000000] SBI v0.2 IPI extension detected
[    0.000000] SBI v0.2 RFENCE extension detected
[    0.000000] SBI v0.2 HSM extension detected
[    0.000000] riscv: ISA extensions acdfim
[    0.000000] riscv: ELF capabilities acdfim
[    0.000000] percpu: Embedded 16 pages/cpu s25432 r8192 d31912 u65536
[    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 508030
[    0.000000] Kernel command line: earlycon=sbi console=ttyNUC0
[    0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)
[    0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)
[    0.000000] Sorting __ex_table...
[    0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off
[    0.000000] Memory: 1943116K/2060288K available (4731K kernel code, 4122K rwdata, 2048K rodata, 192K init, 333K bss, 117172K reserved, 0K cma-reserved)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=8, Nodes=1
[    0.000000] rcu: Hierarchical RCU implementation.
[    0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 10 jiffies.
[    0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
[    0.000000] riscv-intc: 64 local interrupts mapped
[    0.000000] plic: interrupt-controller@1c000000: mapped 53 interrupts with 8 handlers for 16 contexts.
[    0.000000] riscv_timer_init_dt: Registering clocksource cpuid [0] hartid [0]
[    0.000000] clocksource: riscv_clocksource: mask: 0xffffffffffffffff max_cycles: 0x1ef4687b1, max_idle_ns: 112843571739654 ns
[    0.000091] sched_clock: 64 bits at 32kHz, resolution 30517ns, wraps every 70368744171142ns
[    0.010131] Calibrating delay loop (skipped), value calculated using timer frequency.. 0.06 BogoMIPS (lpj=327)
[    0.019897] pid_max: default: 32768 minimum: 301
[    0.027313] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)
[    0.035186] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)
[    0.068054] rcu: Hierarchical SRCU implementation.
[    0.075897] EFI services will not be available.
[    0.089385] smp: Bringing up secondary CPUs ...
[    0.187866] smp: Brought up 1 node, 8 CPUs
[    0.202362] devtmpfs: initialized
[    0.227691] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
[    0.237274] futex hash table entries: 2048 (order: 5, 131072 bytes, linear)
[    0.248870] pinctrl core: initialized pinctrl subsystem
[    0.264373] NET: Registered protocol family 16
[    0.407867] clocksource: Switched to clocksource riscv_clocksource
[    0.429260] NET: Registered protocol family 2
[    0.441894] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)
[    0.493499] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)
[    0.502899] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)
[    0.515624] TCP bind hash table entries: 16384 (order: 6, 262144 bytes, linear)
[    0.532165] TCP: Hash tables configured (established 16384 bind 16384)
[    0.542694] UDP hash table entries: 1024 (order: 3, 32768 bytes, linear)
[    0.550628] UDP-Lite hash table entries: 1024 (order: 3, 32768 bytes, linear)
[    0.561370] NET: Registered protocol family 1
[    0.574340] RPC: Registered named UNIX socket transport module.
[    0.579925] RPC: Registered udp transport module.
[    0.584472] RPC: Registered tcp transport module.
[    0.589416] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    0.598327] Trying to unpack rootfs image as initramfs...
[    4.828521] Freeing initrd memory: 6108K
[    4.844085] workingset: timestamp_bits=62 max_order=19 bucket_order=0
[    4.932403] jffs2: version 2.2. (NAND) © 2001-2006 Red Hat, Inc.
[    4.944885] JFS: nTxBlock = 8192, nTxLock = 65536
[    6.455963] NET: Registered protocol family 38
[    6.460235] io scheduler mq-deadline registered
[    6.464508] io scheduler kyber registered
[    7.040069] 10013000.serial: ttyNUC0 at MMIO 0x10013000 (irq = 1, base_baud = 3125000) is a Nuclei UART/USART
[    7.049835] printk: console [ttyNUC0] enabled
[    7.049835] printk: console [ttyNUC0] enabled
[    7.058288] printk: bootconsole [sbi0] disabled
[    7.058288] printk: bootconsole [sbi0] disabled
[    7.178802] brd: module loaded
[    7.289184] loop: module loaded
[    7.295959] nuclei_spi 10014000.spi: mapped; irq=2, cs=4
[    7.308898] spi-nor spi0.0: w25q128 (16384 Kbytes)
[    8.300964] ftl_cs: FTL header not found.
[    8.315277] nuclei_spi 10034000.spi: mapped; irq=4, cs=4
[    8.368408] mmc_spi spi1.0: SD/MMC host mmc0, no DMA, no WP, no poweroff, cd polling
[    8.380371] ipip: IPv4 and MPLS over IPv4 tunneling driver
[    8.398406] NET: Registered protocol family 10
[    8.417083] Segment Routing with IPv6
[    8.421417] sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
[    8.434722] NET: Registered protocol family 17
[    8.457489] Freeing unused kernel memory: 192K
[    8.490570] Run /init as init process
[    8.549285] mmc0: host does not support reading read-only switch, assuming write-enable
[    8.556854] mmc0: new SDHC card on SPI
[    8.575378] mmcblk0: mmc0:0000 NCard 29.1 GiB
[    8.628082]  mmcblk0: p1
Starting syslogd: OK
Starting klogd: OK
Running sysctl: OK
Starting mdev... OK
modprobe: can't change directory to '/lib/modules': No such file or directory
Saving random seed: [   15.487396] random: dd: uninitialized urandom read (32 bytes read)
OK

Welcome to Nuclei System Technology
nucleisys login:

Application Development

Quick help

You can run make help to show quick help message about how to use this linux sdk.

For detailed usage about components like buildroot, linux kernel, opensbi or uboot, please

check Reference.

Customize buildroot packages

You can customize buildroot packages to add or remove package in buildroot using command:

make buildroot_initramfs-menuconfig

The new configuration for evalsoc will be saved to conf/evalsoc folder, for when a full rebuild of buildroot is necessary, please check this link.

conf/evalsoc/buildroot_initramfs_rv64imac_config: The buildroot configuration for RISC-V ISA/ARCH is rv64imac, such as ux600 and ux900
conf/evalsoc/buildroot_initramfs_rv64imafdc_config: The buildroot configuration for for RISC-V ISA/ARCH is rv64imafdc, such as ux600fd and ux900fd
conf/evalsoc/buildroot_initramfs_rv32imac_config: The buildroot configuration for RISC-V ISA/ARCH is rv32imac, such as u900
conf/evalsoc/buildroot_initramfs_rv32imafdc_config: The buildroot configuration for for RISC-V ISA/ARCH is rv32imafdc, such as u900fd

By default, we add many packages in buildroot default configuration, you can remove the packages you dont need in configuration to generate smaller rootfs, a full rebuild of SDK is required for removing buildroot package.

Customize kernel configuration

You can customize linux kernel configuration using command make linux-menuconfig, the new configuration will be saved to conf folder

conf/evalsoc/linux_rv64imac_defconfig: The linux kernel configuration for RISC-V rv64imac ARCH.
conf/evalsoc/linux_rv64imafdc_defconfig: The linux kernel configuration for RISC-V rv64imafdc ARCH.
conf/evalsoc/linux_rv32imac_defconfig: The linux kernel configuration for RISC-V rv32imac ARCH.
conf/evalsoc/linux_rv32imafdc_defconfig: The linux kernel configuration for RISC-V rv32imafdc ARCH.
conf/evalsoc/nuclei_rv64imac.dts: Device tree for RISC-V rv64imac ARCH used in hardware
conf/evalsoc/nuclei_rv64imafdc.dts: Device tree for RISC-V rv64imafdc ARCH used in hardware
conf/evalsoc/nuclei_rv32imac.dts: Device tree for RISC-V rv32imac ARCH used in hardware
conf/evalsoc/nuclei_rv32imafdc.dts: Device tree for RISC-V rv32imafdc ARCH used in hardware

xlspike dts are only used internally

conf/evalsoc/nuclei_rv64imac_sim.dts: Device tree for RISC-V rv64imac ARCH used in xlspike simulation
conf/evalsoc/nuclei_rv64imafdc_sim.dts: Device tree for RISC-V rv64imafdc ARCH used in xlspike simulation

Customize uboot configuration

You can customize linux kernel configuration using command make uboot-menuconfig, the new configuration will be saved to conf folder

conf/evalsoc/uboot_rv64imac_flash_config: uboot configuration for RISC-V rv64imac ARCH, flash boot mode
conf/evalsoc/uboot_rv64imafdc_flash_config: uboot configuration for RISC-V rv64imafdc ARCH, flash boot mode
conf/evalsoc/uboot_rv64imac_sd_config: uboot configuration for RISC-V rv64imac ARCH, flash boot mode
conf/evalsoc/uboot_rv64imafdc_sd_config: uboot configuration for RISC-V rv64imafdc ARCH, sd boot mode
conf/evalsoc/uboot_rv32imac_flash_config: uboot configuration for RISC-V rv32imac ARCH, flash boot mode
conf/evalsoc/uboot_rv32imafdc_flash_config: uboot configuration for RISC-V rv32imafdc ARCH, flash boot mode
conf/evalsoc/uboot_rv32imac_sd_config: uboot configuration for RISC-V rv32imac ARCH, flash boot mode
conf/evalsoc/uboot_rv32imafdc_sd_config: uboot configuration for RISC-V rv32imafdc ARCH, sd boot mode

Remove generated boot images

You can remove generated boot images using command make cleanboot.

Prebuilt applications with RootFS

If you want to do application development in Linux with FPGA Evaluation board, please follow these steps.

Currently, SDCard is not working in Linux, so if you want to put your own application, and run it in linux, you have to add your application into rootfs and rebuild it, and use the newly generated boot images, and put it into SDCard.

For example, I would like to compile new dhrystone application and run it in linux.

Make sure you have built boot images, using make bootimages
Copy the old dhrystone source code from work/$SOC/buildroot_initramfs/build/dhrystone-2 to work/$SOC/buildroot_initramfs/build/dhrystone-3

cd to work/$SOC/buildroot_initramfs/build/dhrystone-3, and modify Makefile as below:

# Make sure you use the compiler in this path below
CC = ../../host/bin/riscv-nuclei-linux-gnu-gcc
CPPFLAGS += -DNO_PROTOTYPES=1 -DHZ=100
# Customized optimization options
CFLAGS +=  -O2 -flto -funroll-all-loops -finline-limit=600 \
         -ftree-dominator-opts -fno-if-conversion2 -fselective-scheduling \
         -fno-code-hoisting -fno-common -funroll-loops -finline-functions \
         -falign-functions=4 -falign-jumps=4 -falign-loops=4
LDLIBS += -lm

all: dhrystone

dhrystone: dhry_1.o dhry_2.o
   $(CC) $(CFLAGS) $(LDFLAGS) -o $@ $^ $(LDLIBS)

clean:
   rm -f *.o dhrystone

.PHONY: all clean

Run make clean all to rebuild this dhrystone
Copy generated dhrystone to previous generated buildroot_initramfs_sysroot folder, using command cp dhrystone ../../../buildroot_initramfs_sysroot/usr/bin/dhrystone_opt
cd to Nuclei linux SDK root, and run make preboot to clean previously generated boot images.
Generate new boot images with dhrystone_opt application added using command make bootimages
Download the generated work/$SOC/boot.zip and extract it right under the SDCard root.
If you have already flashed freeloader using openocd, then just insert the SDCard, and reboot the board, when board is power on, and linux kernel is up, you can run the application dhrystone_opt in linux shell.

Put prebuilt applications into SDCard

In the lastest commits since 079414d, sdcard can be initialized successfully during kernel boot.

Sample console output of kernel init message for sdcard ready.

[  124.646575] mmcblk0: mmc0:0000 SA08G 7.21 GiB
[  125.242645]  mmcblk0: p1

Note: Currently the sdcard driver is using polling mode as temporary workaround.

When you have login the linux system, you can run command below to check whether sdcard is initialized successfully.

# ls -l /dev/mmc*
brw-rw----    1 root     root      179,   0 Jan  1 00:04 /dev/mmcblk0
brw-rw----    1 root     root      179,   1 Jan  1 00:04 /dev/mmcblk0p1

If there are /dev/mmcblk0p1 devices, then you can mount sdcard in mnt directory using command:

# Mount /dev/mmcblk0p1 into /mnt
mount -t vfat /dev/mmcblk0p1 /mnt
# Check whether sdcard is mounted successfully
ls -l /mnt

If you want to put your prebuilt applications into SDCard, you need to unmount the sdcard first using umount /mnt, and then eject the sdcard from the tf slot, and then insert the sdcard to sdcard reader and connect to your PC, and copy your prebuilt applications into SDCard.

When your applications are placed into the sdcard correctly, then you can insert your card into tf slot, and mount it into /mnt directory.

For example, if you have an application called coremark, then you can directly run it using /mnt/coremark.

Port to your target

demosoc is deprecated, please take evalsoc as example.

For our current development evalsoc, we used the following resources:

RV64IMAC or RV64IMAFDC Core, with 16 PMP entries
DDR RAM: 0x80000000 - 0x100000000, 2GB, DDR RAM is seperated to place opensbi, uboot, kernel, rootfs, dtb binaries.
I/D Cache enabled
UART @ 0x10013000
PLIC @ 0x1C000000
QSPI @ 0x10034000, which connect to SDCard, SDCard will be used when boot from SDCard
QSPI @ 0x10014000, which connect to XIP SPIFlash 4M, memory mapped started at 0x20000000.

SPIFlash is used to place freeloader, which contains opensbi, uboot, dtb, and optional kernel and rootfs when flash-only boot is performed. Flash-only boot will required at least 8M flash.

To basically port this SDK to match your target, you can make a copy of conf/evalsoc such as conf/nsoc:

freeloader.mk: change the variable defined in this mk to match your design
- DDR_BASE, FLASH_BASE and FLASH_SIZE are used to set DDR base address and Flash base address and size used by freeloader.
- If you want to use SMP linux, you need to set ENABLE_SMP and ENABLE_L2 to 1
- If you only have 1 core, please make sure ENABLE_SMP and ENABLE_L2 is 0
- If you will not using amp mode, please set AMP_START_CORE to max hart id, for example, if you have four core, change it to 4.
- CACHE_CTRL and TLB_CTRL is used to control L1/L2 cache control CSR mcache_ctl and TLB CTRL csr mtlb_ctl
- SPFL1DCTRL1, SPFL1DCTRL2 and MERGL1DCTRL are used to control L1 DCache Prefetch and Write Streaming or Merge Control registers spfl1dctrl1, spfl1dctrl2 and mergel1dctrl
build.mk:
- Change UIMAGE_AE_CMD to match your DDR base, used by Makefile to generate rootfs for uboot.
- If you have qemu support, you can change your qemu machine options QEMU_MACHINE_OPTS to match your qemu machine.
- If you are using AMP, CORE1_APP_BIN, CORE2_APP_BIN, CORE3_APP_BIN, CORE4_APP_BIN, CORE5_APP_BIN, CORE6_APP_BIN and CORE7_APP_BIN need to be configured, CORE1-CORE7 each memory is default 4MB(configured by AMPFW_SIZE) and application base address is default offset 0x7E000000(configured by AMPFW_START_OFFSET) at DDR base, you can refer to #18 for how to use AMP demo.
  
  Here each core memory is changed from 8M to 4M, due to only 32MB is reserved for amp binaries, and now we support 8 cores.
  - CORE1_APP_BIN start offset is DDR_BASE + 0x7E000000, such as $(confdir)/amp/c1.bin
  - CORE2_APP_BIN start offset is DDR_BASE + 0x7E000000 + 4M, such as $(confdir)/amp/c2.bin
  - CORE3_APP_BIN start offset is DDR_BASE + 0x7E000000 + 4M*2, such as $(confdir)/amp/c3.bin
  - CORE4_APP_BIN start offset is DDR_BASE + 0x7E000000 + 4M*3, such as $(confdir)/amp/c4.bin
  - CORE5_APP_BIN start offset is DDR_BASE + 0x7E000000 + 4M*4, such as $(confdir)/amp/c5.bin
  - CORE6_APP_BIN start offset is DDR_BASE + 0x7E000000 + 4M*5, such as $(confdir)/amp/c6.bin
  - CORE7_APP_BIN start offset is DDR_BASE + 0x7E000000 + 4M*6, such as $(confdir)/amp/c7.bin
- TIMER_HZ, CPU_HZ, PERIPH_HZ are used by *.dts files to generate correct timer, cpu, peripheral clock hz, if you directly set it in dts, not need for this variables.
opensbi/: Change the opensbi support code for your soc, all the files need to be modified.
nuclei_rv32imac.dts, nuclei_rv32imafdc.dts, nuclei_rv64imac.dts, nuclei_rv64imafdc.dts and openocd.cfg: Change these files to match your SoC design.
- Select the right dts which match your cpu isa, for example, if you are using rv64imafdc, please use nuclei_rv64imafdc.dts
- External interrupts connected to plic interrupt number started from 1, 0 is reserved. For example, in evalsoc, interrupt id of UART0 is 32, then plic interrupt number is 33, and if elic also present, the eclic interrupt number will be 32+19=51
- If you want to boot linux using hvc console(console via sbi console, useful when uart driver in linux is not ready), you can change bootargs to make console=/dev/hvc0, then it will use sbi console to print message
- If you UART driver in linux is ready, then you can change the console to your real uart device name.
uboot.cmd: Change to match your memory map.
uboot_rv64imac_sd_config, uboot_rv64imac_flash_config, uboot_rv64imafdc_sd_config and uboot_rv64imafdc_flash_config:
uboot_rv32imac_sd_config, uboot_rv32imac_flash_config, uboot_rv32imafdc_sd_config and uboot_rv32imafdc_flash_config: change CONFIG_SYS_TEXT_BASE and CONFIG_BOOTCOMMAND to match your uboot system text address and boot command address.
If you have a lot of changes in uboot or linux, please directly change code in it.

In evalsoc support, spmp bypass is controlled by code in conf/evalsoc/opensbi/platform.c for opensbi v0.9, conf/evalsoc/opensbi/evalsoc.c for opensbi >= v1.2

From commit 6507c68 on, the spmp will be bypassed(done in code as below) when tee is present(checked mcfg_info csr). If you have enabled TEE feature(sPMP module included), you need to configure spmp csr registers as this commit https://github.com/Nuclei-Software/opensbi/commit/1d28050d01b93b6afe590487324b663c65a2c429 . Then you will be able to boot up linux kernel, otherwise the init process will fail.

Known issues and FAQs

Please also track issues located in https://github.com/Nuclei-Software/nuclei-linux-sdk/issues

Clone source code from github or gitee failed with issue the remote end hung up unexpectedly.

see #10 for up to date answer.

This Nuclei Linux SDK repo is a very big repo with many submodules, just simple clone is not enough, you always need to do submodule init and update, sometimes due to connection issue or http clone not stable issue, please switch to use ssh protocol used in git clone, see similar issue posted in stackoverflow, see https://stackoverflow.com/questions/6842687/the-remote-end-hung-up-unexpectedly-while-git-cloning

Or you can try shallow submodule update:
```
## clone repo from github
git clone https://github.com/Nuclei-Software/nuclei-linux-sdk.git
## if github is not working please use gitee, no longer working, deprecated now, see https://github.com/Nuclei-Software/nuclei-linux-sdk/issues/10#issuecomment-1728920670
# git clone https://gitee.com/Nuclei-Software/nuclei-linux-sdk.git
## if https is not ok, please switch to ssh method, but you need to follow guidance in github or gitee
## github: https://docs.github.com/cn/authentication/connecting-to-github-with-ssh/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent
## gitee: https://gitee.com/help/articles/4181
cd nuclei-linux-sdk
git submodule update --init --depth=1
## please check sample output here: https://github.com/Nuclei-Software/nuclei-linux-sdk/wiki/Sample-Outputs#sample-output-for-git-clone
```
Or you just want to get latest source code without any git histories, you can also open our repo's github action link in github https://github.com/Nuclei-Software/nuclei-linux-sdk/actions , make sure your github is logged in, and then click on any successful action of your desired branch, and find the nuclei_linux_sdk_source in the Artifacts at the bottom of page, and click the nuclei_linux_sdk_source and download it.

For Nuclei Demo SoC, if you run simulation using xl_spike, it can run to login prompt, but when you login, it will show timeout issue, this is caused by xl_spike timer is not standard type, but the boot images for FPGA evaluation board can boot successfully and works well.

If you want to execute using xl_spike without the login, you can edit the work/$SOC/buildroot_initramfs_sysroot/etc/inittab file(started from # now run any rc scripts) as below, and save it:

# now run any rc scripts
#::sysinit:/etc/init.d/rcS

# Put a getty on the serial port
#console::respawn:/sbin/getty -L  console 0 vt100 # GENERIC_SERIAL
::respawn:-/bin/sh

And then type make presim and make sim to run linux in xl_spike, you will be able to get following output in final:

## a lot of boot message are reduced here ##
UART: [  246.974853] sdhci: Secure Digital Host Controller Interface driver
UART: [  247.057922] sdhci: Copyright(c) Pierre Ossman
UART: [  247.447723] NET: Registered protocol family 17
UART: [  266.205169] Freeing unused kernel memory: 36052K
UART: [  266.314117] Run /init as init process
UART: # ls
ls
UART: bin      init     linuxrc  opt      run      tmp
UART: dev      lib      media    proc     sbin     usr
UART: etc      lib64    mnt      root     sys      var
UART: # cat /proc/cpuinfo
cat /proc/cpuinfo
UART: processor	: 0
UART: hart		: 0
UART: isa		: rv64imac
UART: mmu		: sv39
UART:

For some SDCard format, it might not be supported, please check your SDCard is SDHC format.
If you can't boot with the sdcard boot images, you can run the following commands in uboot to check whether sdcard is recognized.
1. Type mmcinfo to check whether sdcard is recognized? If no output, please re-insert the sdcard, and try this command again, if still not working, please confirm that the MCS is correct or not?
```
Hit any key to stop autoboot:  0
=> mmcinfo
Device: spi@10034000:mmc@0
Manufacturer ID: 2
OEM: 544d
Name: SA08G
Bus Speed: 20000000
Mode: MMC legacy
Rd Block Len: 512
SD version 2.0
High Capacity: Yes
Capacity: 7.2 GiB
Bus Width: 1-bit
Erase Group Size: 512 Bytes
```
2. If SDCard is recognized correctly, please type fatls mmc 0, and check whether the following files are listed as below, if you can get the following files in your sdcard, please reformat your sdcard to Fat32 format, and copy the generated files in work/$SOC/boot/ to the root of sdcard, and re-insert the sdcard to SD slot, and retry from step 1.
  
  Note: Please make sure your SDCard is safely injected in your OS, and SDCard is formated to Fat32.
```
=> fatls mmc 0
    2594   kernel.dtb   # device tree binary file
     345   boot.scr     # boot script used by uboot, generated from ./conf/<SOC>/uboot.cmd
 3052821   uImage.lz4   # lz4 archived kernel image
 19155960  uInitrd.lz4  # lz4 archived rootfs image

 4 file(s), 0 dir(s)
```
3. If the above steps are all correct, then you can run boot command to boot linux, or type commands located in ./conf/evalsoc/uboot.cmd for evalsoc.
The linux kernel and rootfs size is too big, is there any way to reduce it to speed up boot speed?

If you are familiar with linux and buildroot configuration files, you can directly modify the configuration files located in conf folder.

If you modified this files directly and want to take effects, you need to make clean first, and regenerate boot images.

You can also try make buildroot_initramfs-menuconfig to get a terminal menuconfig to configure the buildroot packages.

You can also try make linux-menuconfig to get a menuconfig to configure the linux kernel.

You can also try make uboot-menuconfig to get a menuconfig to configure the uboot.
Other possible ways to reduce generated rootfs image size.

If you are familiar with the generated rootfs files located in work/<SOC>/buildroot_initramfs_sysroot, you can manually remove the files you think it is not used, and type make cleanboot and then make bootimages, you can check the size information generated by the command.
The best way to learn this project is taking a look at the Makefile of this project to learn about what is really done in each make target.
Download work/$SOC/freeloader/freeloader.elf using Nuclei SDK.

If you don't want to build the nuclei sdk, you can also download the boot images generated by github action.

For example, for dev_nuclei_next branch, you can find the previous built artifacts in prebuilt artifacts.

Then you can extra the downloaded bootimages_ux600.zip and extract freeloader/freeloader.elf to your disk, such as D:/freeloader.elf.

Make sure you have followed steps to setup nuclei sdk development environment, then you can follow steps below to download this D:/freeloader.elf.

see https://github.com/Nuclei-Software/nuclei-linux-sdk/wiki/Program-freeloader

Reference

nuclei-linux-sdk's People

Contributors

Stargazers

Watchers

Forkers

yuyongep mfkiwl ambercatxu kdjia linuxgiteee cryindance flzxsqc200 liuruyi-repo xiaozisoft zengxin129949

nuclei-linux-sdk's Issues

Using Linux Perf Tool For Nuclei RISC-V Processor

Linux Perf Tool is an outstanding tool that serves the significant purpose of measuring CPU performance and offers valuable assistance to developers in optimizing software.

Linux Kernel For RISC-V improved perf support for RISC-V using SBI PMU extension and Sscofpmf extension, which are both supported by Nuclei RISC-V Processor now.

SBI PMU extension is an interface for supervisor-mode to configure and use the RISC-V hardware performance counters with assistance from the machine-mode (or hypervisor-mode)
Sscofpmf risc-v extension is used to add an counter overflow interrupt and Privilege Mode counter Filtering

By default, now in dev_nuclei_6.1_v2/dev_nuclei_6.6_v2/dev_nuclei_6.9_v2 branches, we enabled CONFIG_PERF_EVENTS=y for Linux Kernel, which means by PMU support is enabled in Linux Kernel, if your CPU is configured with PMU, you will be able to see following output during boot up, see fd3dc31 commit for changes we made to support PMU.

We tested Linux Kernel 6.9 with PMU.

# In OpenSBI
Boot HART ISA Extensions  : sscofpmf,time,sstc
... ...
Boot HART MHPM Count      : 4
# In Linux Kernel
[   17.066711] riscv-pmu-sbi: SBI PMU extension is available
[   17.071960] riscv-pmu-sbi: 16 firmware and 6 hardware counters

But when CONFIG_PERF_EVENTS=y enabled, you will not be able to read cycle/instret/time csr, you need to execute echo 2 > /proc/sys/kernel/perf_user_access to allow user mode access these counters, and now the cycle/instret counter by default is not counted, it is controlled by PMU now, you need to use linux perf tool to enable it.

Here is a sample usage for coremark program ported by us using perf tool.

# echo 2 > /proc/sys/kernel/perf_user_access
# perf stat -e cycles -e instructions -e cache-misses -e branches -e branch-misses coremark_1core
Start to run CoreMark
2K performance run parameters for coremark.
CoreMark Size    : 666
Total ticks      : 14788
Total time (secs): 14.788000
Iterations/Sec   : 270.489586
Iterations       : 4000
Compiler version : GCC10.2.0
Compiler flags   : -Ofast -mbranch-cost=1 -mstrict-align -funroll-all-loops -finline-limit=1000 -ftree-dominator-opts -fselective-scheduling -funroll-loops -finline-functions -falign-functions=4 -falign-jumps=4 -falign-loops=4 -fipa-pta -fno-code-hoisting -fno-common -fno-if-conversion -fno-if-conversion2 -fno-tree-loop-distribute-patterns -fno-tree-vectorize -fno-tree-loop-ivcanon -fno-tree-vrp -fgcse-las --param=max-loop-header-insns=4 --param loop-max-datarefs-for-datadeps=0 --param=unroll-jam-min-percent=0 --param=max-goto-duplication-insns=0 -DMULTITHREAD=1 -DUSE_PTHREAD  -lrt -lpthread -march=rv64imafdc -mabi=lp64d
Memory location  : Please put data memory location here
                        (e.g. code in flash, data on heap etc)
seedcrc          : 0xe9f5
[0]crclist       : 0xe714
[0]crcmatrix     : 0x1fd7
[0]crcstate      : 0x8e3a
[0]crcfinal      : 0x65c5
Correct operation validated. See README.md for run and reporting rules.
CoreMark 1.0 : 270.489586 / GCC10.2.0 -Ofast -mbranch-cost=1 -mstrict-align -funroll-all-loops -finline-limit=1000 -ftree-dominator-opts -fselective-scheduling -funroll-loops -finline-functions -falign-functions=4 -falign-jumps=4 -falign-loops=4 -fipa-pta -fno-code-hoisting -fno-common -fno-if-conversion -fno-if-conversion2 -fno-tree-loop-distribute-patterns -fno-tree-vectorize -fno-tree-loop-ivcanon -fno-tree-vrp -fgcse-las --param=max-loop-header-insns=4 --param loop-max-datarefs-for-datadeps=0 --param=unroll-jam-min-percent=0 --param=max-goto-duplication-insns=0 -DMULTITHREAD=1 -DUSE_PTHREAD  -lrt -lpthread -march=rv64imafdc -mabi=lp64d / Heap
Begin_Cycle 9223372037062820250, End_Cycle 9223372037805837188, User_Cycle 743016938 cycles
Begin_Instret 9223372037119765118, End_Instret 9223372038073768277, User_Instret 954003159 Instrets
CoreMark/MHz calc via cycle: 5.383457 CoreMark/MHz
IPC:  1.284

 Performance counter stats for 'coremark_1core':

         951534830      cycles                                                  
        1219293100      instructions                     #    1.28  insn per cycle
             13957      cache-misses                                            
         264065366      branches                                                
           6903336      branch-misses                    #    2.61% of all branches

      18.942352295 seconds time elapsed

      18.884026000 seconds user
       0.039923000 seconds sys

About Linux Perf Tool, you need to cross build it using Linux Kernel, you will need at least to cross build zlib, elfutils and libtraceevent and then build linux perf tool

zlib 1.3.1: github.com/madler/zlib.git
elfutils 0.191: https://sourceware.org/git/elfutils.git
libtraceevent 1.8.3: https://git.kernel.org/pub/scm/libs/libtrace/libtraceevent.git

We provide sample script to cross build perf tool for linux kernel 6.9 for rv64imafdc:

https://github.com/Nuclei-Software/linux/blob/dev_nuclei_6.9.y-perf/tools/perf/pre_perf.sh : build perf required library
https://github.com/Nuclei-Software/linux/blob/dev_nuclei_6.9.y-perf/tools/perf/do_perf.sh: build perf tool itself

Tools directories must be like this:

├── linux
├── elfutils
├── libtraceevent
└── zlib

Download prebuilt Nuclei RISC-V Toolchain ( Linux/Glibc )(gcc13) from https://nucleisys.com/download.php

# Make sure above directory include source code are setup
# Make sure toolchain PATH is already setup
$ which riscv64-unknown-linux-gnu-gcc
/path/to/your/gcc/bin/riscv64-unknown-linux-gnu-gcc
$ cd linux/tools/perf
$ ls *_perf.sh
do_perf.sh  pre_perf.sh
# Build perf required third party libraries
./pre_perf.sh riscv64-unknown-linux-gnu rv64imafdc lp64d install
# Build perf tools
./do_perf.sh riscv64-unknown-linux-gnu rv64imafdc lp64d install

# Then you will be able to see perf tools in `install` folder
$ ls install/
bin/  env.sh  etc/  include/  lib/  libexec/  share/

# WARNING: Below is in a riscv linux environment, not x86 linux environment
# You just need to copy these install folder in your riscv linux environment and execute following command
# you will be able to use perf in your riscv linux
source /path/to/install/env.sh
perf --version

AMP Demo about Linux + FreeRTOS

Let us take Nuclei U900FD(rv32imafdc) 2 SMP Core as example, CPU 0 run Linux Kernel 5.10, and CPU 1 run FreeRTOS demo.

Similar changes could be maded on UX900FD Core and 6.1/6.6/6.9 Linux Kernel.

CPU required both PLIC and ECLIC present

Both Linux and FreeRTOS runs on DDR memory:

Core 0 Linux use: 0x80000000 -> 0xFE000000
Core 1 FreeRTOS use 4M started from 0xFE000000 to 0xFE040000

Let us clone source code and switch to dev_nuclei_5.10_v2 branch, following steps #10
Modify conf/evalsoc/build.mk and conf/evalsoc/freeloader.mk as below:

diff --git a/conf/evalsoc/build.mk b/conf/evalsoc/build.mk
index c18315c..220ec62 100644
--- a/conf/evalsoc/build.mk
+++ b/conf/evalsoc/build.mk
@@ -4,7 +4,7 @@ QEMU_MACHINE_OPTS := -M nuclei_evalsoc,download=flashxip -smp 8 -m 2G
 # initramfs pre command before generate initrd ramfs
 INITRAMFS_PRECMD := bash $(confdir)/preramfs.sh $(confdir) $(buildroot_initramfs_sysroot) copyfiles.txt
 # eg. $(confdir)/amp/cx.bin
-CORE1_APP_BIN :=
+CORE1_APP_BIN := $(confdir)/amp/amp_c1.bin
 CORE2_APP_BIN :=
 CORE3_APP_BIN :=
 CORE4_APP_BIN :=
diff --git a/conf/evalsoc/freeloader.mk b/conf/evalsoc/freeloader.mk
index 31e3d3b..7ac9644 100644
--- a/conf/evalsoc/freeloader.mk
+++ b/conf/evalsoc/freeloader.mk
@@ -12,4 +12,4 @@ ENABLE_SMP ?= 1
 ENABLE_L2 ?= 1
 AMPFW_START_OFFSET ?= 0x7E000000
 AMPFW_SIZE ?= 0x400000
-AMP_START_CORE ?= 8
+AMP_START_CORE ?= 1

Clone Nuclei SDK 0.5.0 master branch from https://github.com/Nuclei-Software/nuclei-sdk
Following steps in https://doc.nucleisys.com/nuclei_sdk/quickstart.html to setup Nuclei SDK build environment
Download prepared ampdemo.zip and unzip it to /path/to/nuclei-sdk/application/freertos/

Now in Nuclei SDK folder

cd /path/to/nuclei-sdk/application/freertos/ampdemo
# make sure Makefile is in this ampdemo folder
# build this application for U900FD, and generate binary
make CORE=u900fd clean
make CORE=u900fd bin
# sample binary as below
$ ll -h freertos_demo.bin
-rwxr-xr-x 1 hqfang hqfang 130K Nov 16 18:03 freertos_demo.bin
# copy this binary to Nuclei Linux SDK's folder as  conf/evalsoc/amp/amp_c1.bin
mkdir -p /path/to/nuclei-linux-sdk/conf/evalsoc/amp/
cp -f freertos_demo.bin /path/to/nuclei-linux-sdk/conf/evalsoc/amp/amp_c1.bin

Build Linux SDK for U900FD

Now in Nuclei Linux SDK folder, make sure you are using a clean and up to date dev_nuclei_5.10_v2 branch

cd /path/to/nuclei-linux-sdk
# build and generate freeloader and boot images for sdcard
# make sure step 1 changes have been made, and step 5 freertos binary has been copied
make CORE=u900fd freeloader bootimages

Get a 2 Core U900FD FPGA bitstream from Nuclei AE and evaluate it on hardware, this is not possible to run on Nuclei QEMU.

Following steps in https://github.com/Nuclei-Software/nuclei-linux-sdk/tree/dev_nuclei_5.10_v2#upload-freeloader-to-fpga-evaluation-board to upload freeloader and place boot images into SDCard.

Here is sample output, you can see freertos and linux both output in the same uart:

linux_5.10_freertos_ampdemo.log

Our prebuilt images could be found here:

U900FD Freeloader with FreeRTOS demo: freeloader_u900fd_amp_demo_20231116.zip
U900FD Prebuilt boot images: boot_u900fd.zip

Linux Bringup suggestion when using Nuclei RISC-V CPU

Nuclei RISC-V CPU IP user should following this guidance to port your SoC to enable quickly linux bringup and then you can port linux to your own linux sdk platform.

Nuclei Linux SDK is a really simple Linux SDK platform for customer to quickly bring up their SoC with Linux.

If you find there is something wrong with our document, please create a issue, and let us improve it.

Note

Make sure you have confirmed that your L1/L2 cache integration is correct and testing passed, please cowork with your team member to make sure it is ok. For software, you can disable L1 and L2 cache in freeloader to see whether cache is not ok which affect normal boot.
If you are using SMP core, please make sure single core version works by generate a single core version bitstream, and then port a smp version.
If you are using SMP core, you also need to update openocd configuration file, if you want to debug it, please contact with our AE for help about how to update this configuration file.
Please take care clint0 in dts should be nuclei timer base address + 0x1000, please check Nuclei ISA Spec

Nuclei Linux SDK is developed for Nuclei evaluation SoC which is very simple, with SPI XIP Flash, UART, and SPI SDCard and DDR memory enabled evaluation SoC, for details about this SoC, please see our Nuclei_Eval_SoC_Intro.pdf
Our evaluation SoC don't have a bootrom to do some basic SOC initialization and load image from flash or other storages to memory, so we create a software called freeloader to handle this.
If your DDR memory need to be initialized, you can do this in freeloader code, just like fsl91030m's freeloader does in https://github.com/Nuclei-Software/freeloader/blob/8f27d8dc7d92ba13b4f6f62039dc09abbc5731cd/freeloader.S#L94-L176, this kind of asm code is generated from c code, and just disassemble it, and copy and adapted to freeloader code
Nuclei Linux SDK has different branches to evaluate different Linux version and different features, see #2 , so you should choose a correct branch you want to use, and following the guidance in that branch
For example, for dev_nuclei_5.10_v2 branch, you should check its README.md in https://github.com/Nuclei-Software/nuclei-linux-sdk/blob/dev_nuclei_5.10_v2/README.md, and it has a Port to your target section to tell you how to port your SoC to this linux SDK and help you to bringup.
We strongly requested you to bring up your SoC using our Linux SDK, it will be easy to debug what is not acting as expected, if the bring up is done, then you can port it to your own SDK environment, it will be much easier.
When you are expecting to use some higher level feature such as OpTEE, please make sure normal linux is already brought up, and then you can follow the guidance in OpTEE feature branch.

NOTE: You may need to do two versions of bitstream, first version for normal boot, just configure cpu without any secure feature, and do a bitstream, when comes to optee related porting, then you can configure cpu with secure feature, and do a bitstream and port linux to it.
About how to debug linux, uboot, opensbi, please check https://github.com/Nuclei-Software/nuclei-linux-sdk/wiki/Debug-Linuxsdk-using-NucleiStudio
Regarding the frequently asked questions, please check #12

Thanks

qemu-system-riscv64: error while loading shared libraries: libpixman-1.so.0: cannot open shared object file: No such file or directory

Using Nuclei QEMU 2022.08 met above issue, check with ldd.

nuclei@8dd23cfd9e26:~$ ldd ./prebuilt/qemu/bin/qemu-system-riscv64
        linux-vdso.so.1 (0x00007fffaff18000)
        libpixman-1.so.0 => not found
        libgio-2.0.so.0 => not found
        libgobject-2.0.so.0 => not found
        libglib-2.0.so.0 => not found
        libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f1703565000)
        libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007f170355e000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f170340f000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f17033f4000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f17033d1000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f17031df000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f17045f2000)

Fixed by install these packages in ubuntu 20.04 LTS: sudo apt install libglib2.0-dev libfdt-dev libpixman-1-dev zlib1g-dev

sudo apt install libglib2.0-dev libfdt-dev libpixman-1-dev zlib1g-dev
./prebuilt/qemu/bin/qemu-system-riscv64 --version
QEMU emulator version 6.0.50 (v6.0.0-1792-g35bce38-dirty)
Copyright (c) 2003-2021 Fabrice Bellard and the QEMU Project developers
Nuclei QEMU 2022.08

Add "depth=1" when git clone or git module update to save time.

Hi maintainner,
i met a error message when git module update, may be we can add "depth=1" when update module to resolve this error and sava a lot of time, i'm not sure if take any side effect, please advise, thanks a lot.

Use gdb to download linux images and debug

This document guides customers how to download the image to the fpga board through jtag to run, to avoid burning flash every time debugging, especially the image that has just been transplanted, which may have more problems. Improve debugging efficiency.

This is mainly for BOOT_MODE=flash mode, to avoid you have to download freeloader.elf to flash for a long time due to flash programing is time costing for large size freeloader.elf

software：Nuclei Linux SDK dev_nuclei_5.10_v2 branch #2
hardware：ux900 bitfile + vcu118 or ku060

Here we just take one linux sdk branch and our evaluation board as example, you can take reference for your own development board.

You should always follow this guidance to port linux to your SoC, see #21

Compile SDK

Assume you don't have a SPI can access to SDCard.

Assume you are using a RV64 CPU.

If you are using a RV32 CPU, need to change ux to u for below CORE settings

If you are not using evalsoc, please set SOC=, change <yoursoc> to the real soc name of yours
You have to choose BOOT_MODE=flash, pass it in each make, or directly update it in Makefile
CORE parameter should be matched with your bitfile, if your bitfile is not support double float, you should set CORE=ux900, otherwise CORE=ux900fd
Then modify the Makefile's BOOT_MODE, CORE and SOC according to above settings
according to your hardware to set CPU frequency CPU_HZ

more detail info about compile sdk, please refer to Nuclei Linux SDK README

Before you build SDK, make sure you have not build SDK before with a different CORE and BOOT_MODE settings

Here I am using evalsoc, so the work folder is work/evalsoc

Here I assume you have modified the Makefile as request above.

IMPORTANT, if you need to change CORE/SOC/BOOT_MODE, please rename work folder to other name such as work_20240308, and build it again.

BOOT_MODE=flash is always required for this case.

make CPU_HZ=50000000 freeloader -j16

After compiling, you will see the output image in the nuclei-linux-sdk/work/evalsoc/freeloader/ directory:

nuclei-linux-sdk$ ls work/evalsoc/freeloader/
fdt.dtb         freeloader.dasm  freeloader.map  kernel.bin  opensbi.bin          u-boot.bin
freeloader.bin  freeloader.elf   initrd.bin      memory.lds

Maybe the first time you are not be able to bring up linux for your SoC using the freeloader.elf, you can try to use gdb to help to speed up your program speed via directly load opensbi/uboot/linux/dtb to ddr memory and can debug linux.

config gdb download script

See https://sourceware.org/gdb/documentation/ for gdb usage.

You can configure gdb download script based on your soc hardware.
the gdb script reset hardware before download image using monitor reset halt
restore command can fill image into ddr, if you don't want to download some image, you can add # to the beginning to the line to comment it.
- eg. If you just want to download opensbi, uboot and dtb, you can comment out kernel and rootfs image
- dtb is always required.
generally focusing on below a few parameters:
- ddr_base: config your soc ddr base address
- thread_num: config your smp core numbers，core should be found by openocd, if openocd config file only connect 1 core, you should set thread_num to 1
- Please check the TODO items in the script file

Note

If you want to use smp cores, please update your openocd configuration file to support smp, contact with our AE for help in updating this smp openocd configuration file.

set $opensbi_base=$ddr_base + 0x0
set $uboot_base=$ddr_base + 0x200000
set $fdt_base=$ddr_base + 0x8000000
set $kernel_base=$ddr_base + 0x3000000
set $rootfs_base=$ddr_base + 0x8300000

each image load address should be same with nuclei-linux-sdk/freeloader/freeloader.S

if your change freeloader.S image load address, you should change gdb script accordingly.

download images to ddr

place your gdb download script to nuclei-linux-sdk/work/evalsoc/freeloader/ directory

nuclei-linux-sdk/work/evalsoc/freeloader$ ls
fdt.dtb         freeloader.dasm  freeloader.map  kernel.bin           memory.lds   u-boot.bin
freeloader.bin  freeloader.elf   initrd.bin      load_image.gdb_init  opensbi.bin

startup gdb

NOTE: Each time, you have made changes to code, and want to reboot, you need to rebuild images, and download it again, and debug it.

Note

If you are still debugging it, we strongly recommend you execute the command list in the gdb script line by line for single core version(no need to have the while loop in it) to help you debug which command is failing.

First startup openocd, then execute gdb from nuclei-linux-sdk/work/evalsoc/freeloader/ directory to connect openocd.

load_image.gdb_init is modified based on gdb download script of previous steps

nuclei-linux-sdk/work/evalsoc/freeloader/$ riscv64-unknown-linux-gnu-gdb
GNU gdb (GDB) 13.2.90.20230712-git
Copyright (C) 2023 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "--host=x86_64-pc-linux-gnu --target=riscv64-unknown-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word".
# 192.168.40.52:3333 is my port, please change to your correct port
(gdb) target remote 192.168.40.52:3333
Remote debugging using 192.168.40.52:3333
warning: No executable has been specified and target does not support
determining executable automatically.  Try using the "file" command.
warning: multi-threaded target stopped without sending a thread-id, using first non-exited thread
0xffffffff8000334e in ?? ()
(gdb) source load_image.gdb_init
JTAG tap: riscv.cpu tap/device found: 0x16000a6d (mfg: 0x536 (Nuclei System Technology Co Ltd), part: 0x6000, ver: 0x1)
init hardware....
[Switching to thread 1 (Thread 1)]
#0  0x0000000020000000 in ?? ()
[Switching to thread 2 (Thread 2)]
#0  0x0000000020000000 in ?? ()
[Switching to thread 1 (Thread 1)]
#0  0x0000000020000000 in ?? ()
load image ....
Restoring binary file opensbi.bin into memory (0x80000000 to 0x80041e98)
Restoring binary file u-boot.bin into memory (0x80200000 to 0x802678a0)
Restoring binary file fdt.dtb into memory (0x88000000 to 0x88001377)
Restoring binary file kernel.bin into memory (0x83000000 to 0x8342a885)
Restoring binary file initrd.bin into memory (0x88300000 to 0x88a71926)
start run ....
(gdb)

Above load_image.gdb_init script will do continue in the end of the script, you can remove it, if you want to execute it in gdb by yourself.

If you want to debug in IDE, you can refer to https://github.com/Nuclei-Software/nuclei-linux-sdk/wiki/Debug-Linuxsdk-using-NucleiStudio

If you are now able to execute in gdb, and successfully bring up linux with several tries, then you can rebuild freeloader and program the freeloader.elf, you may be able to bring up now.

Uboot SPL Support in Nuclei Linux SDK

We have introduced a new branch called dev_nuclei_6.6_v2_spl to provide support for uboot spl.

Note

Currently only CORE=ux900fd BOOT_MODE=sd is supported, other configurations are not yet supported.
Nuclei CPU need to configured with L2 cache or your SoC have a SRAM can be used to run uboot spl
AMP is not supported in this branch.

With new introduced uboot spl, now boot process changed:

freeloader -> opensbi + uboot -> kernel -> freeloader -> uboot-spl -> opensbi + uboot -> kernel

The freeloader now assumed ddr is not yet initialized, so we have to copy uboot-spl from a fixed address in xip spiflash to a shared memory(here we use L2 cache CLM feature to provide a Cluster Local Memory), and then go to uboot spl, and let uboot spl to initialize ddr memory, and in uboot spl, you can do far more things than what you can do in freeloader.

And then uboot spl will initialize ddr memory, and load opensbi + uboot + dtb .composed uboot FIT image from fixed address in xip spiflash, and got to opensbi.

And in opensbi, it will disable L2 cache CLM feature, and enable L2 cache, and then go to Uboot, and uboot will load kernel and rootfs image, and boot linux kernel.

how to do git clone clean source code 如何Clone下载干净的代码

How to do git clone clean source code

If your github access is not good, please download the full existing git repo archive from nuclei-linux-sdk-2024.09.02.tar.xz

Github访问很慢就直接下载我们提前准备好的完整仓库,不经常更新,需要自己手动fetch更新 nuclei-linux-sdk-2024.09.02.tar.xz

Old 2024.05.30 version can be found here https://drive.weixin.qq.com/s?k=ABcAKgdSAFcojrVCJQ

tar -xJvf nuclei-linux-sdk-2024.09.02.tar.xz

This repo is a big repo with many submodule repos, a full clone required about 10G space, and rely on a stable network connection to github.com, if you don't have good network connection, please try other ways list in the bottom

Basic clone source code step can be found in https://github.com/Nuclei-Software/nuclei-linux-sdk?tab=readme-ov-file#clone-repo , please read it carefully including the comment in it.

NOTE When you clone source code, you can add --depth 1 to speed up, see https://cloud.tencent.com/developer/article/2212263

If you are switch to a new branch, remember to sync and update submodule source code, and check git status to make sure source code is clean.

git switch require git version >= 2.23, see https://git-scm.com/docs/git-switch

For example, here I switched to feature/fsl branch, successful submodule update like this:

If you get status message like this, it means the submodule repo is not updated

WARNING: No longer available now: When you do submodule update, you get error message like this, please check github connection or switch to gitee mirror of nuclei linux sdk https://gitee.com/Nuclei-Software/nuclei-linux-sdk

gitee mirror project https://gitee.com/Nuclei-Software/linux is blocked now! No longer accessable.

Other ways to get source code

NOTICE: We don't provide any tech support for using github mirror website!!!

You can also take a try with github mirror website https://githubfast.com/, for this repo https://githubfast.com/Nuclei-Software/nuclei-linux-sdk
You can also seach github mirror website, and follow steps provided by them, such as https://github.com/RC1844/FastGithub, https://gitclone.com/
You can also try our docker image, see #8
You can also try our prepared nuclei linux sdk source code here: #4

Build fail in ubuntu 22.04 c-stack.c:55:26: error: missing binary operator before token "("

如下图所示

make[5]: Entering directory '/work/work/evalsoc/buildroot_initramfs/build/host-m4-1.4.18/lib'
  CC       binary-io.o
  CC       gl_avltree_oset.o
  CC       c-ctype.o
  CC       c-stack.o
  CC       c-strcasecmp.o
  CC       cloexec.o
  CC       c-strncasecmp.o
  CC       clean-temp.o
  CC       closeout.o
  CC       close-stream.o
  CC       closein.o
  CC       basename.o
  CC       dirname.o
  CC       dirname-lgpl.o
  CC       basename-lgpl.o
  CC       stripslash.o
  CC       execute.o
  CC       exitfail.o
In file included from /usr/include/signal.h:328,
                 from ./signal.h:52,
                 from c-stack.c:49:
c-stack.c:55:26: error: missing binary operator before token "("
   55 | #elif HAVE_LIBSIGSEGV && SIGSTKSZ < 16384
      |                          ^~~~~~~~
  CC       fd-safer-flag.o
  CC       fatal-signal.o
  CC       fd-hook.o
  CC       dup-safer-flag.o
make[5]: *** [Makefile:1915: c-stack.o] Error 1
make[5]: *** Waiting for unfinished jobs....
  CC       filenamecat.o

Using prebuilt linuxsdk docker image

We have prepared a linuxsdk docker image located in https://hub.docker.com/repository/docker/nucleisoftware/linuxsdk, you can use the docker image there, if you don't want to step the environment by yourself.

If you don't want to use docker, please following #4 to download code

The following branches are enabled with docker support:
Available tags: https://hub.docker.com/repository/docker/nucleisoftware/linuxsdk/tags

dev_nuclei_next: nucleisoftware/linuxsdk:ci , later on it will be nucleisoftware/linuxsdk:dev_nuclei_next

dev_nuclei_5.10/dev_nuclei_5.10_v2/dev_nuclei_6.1/dev_nuclei_6.1_v2/dev_nuclei_6.6_v2: docker tag is the branch name, eg. for branch dev_nuclei_5.10, the docker image is nucleisoftware/linuxsdk:dev_nuclei_5.10

NOTE For evaluating optee, see #13 , this docker image is uploaded by @fanghuaqi manually, not ci generated.

Recommended docker tags: dev_nuclei_5.10_v2, dev_nuclei_6.1_v2, dev_nuclei_6.6_v2

NOTE: not work for upload freeloader, you need to copy the freeloader to out side of docker, and program to flash using openocd and gdb by yourself, see https://github.com/Nuclei-Software/nuclei-linux-sdk/wiki/Program-freeloader

Sample steps:

Make sure your docker is installed in your linux environment, see https://docs.docker.com/engine/install/ubuntu/
Make sure you have good connection to docker hub, otherwise use mirror of docker hub, search it by yourself
Use the docker image of linux sdk

# In Host
## if using docker hub mirror from docker.mirrors.sjtug.sjtu.edu.cn , you can run like this
# docker run -it docker.mirrors.sjtug.sjtu.edu.cn/nucleisoftware/linuxsdk:dev_nuclei_5.10_v2
# Here below just pull docker from official docker hub
docker run -it nucleisoftware/linuxsdk:dev_nuclei_5.10_v2
# In docker now, user/pwd: nuclei/riscv123
cd nuclei-linux-sdk
# build project just follow https://github.com/Nuclei-Software/nuclei-linux-sdk#show-help
# this linux sdk repo cloned in this docker image is not full clone
# if you want to update source code or checkout different branch, do it as you like

如何下载最新的源码(无git) How to download latest source code without git

Clone source code using git

If your github access is not good, please download the full existing git repo archive from nuclei-linux-sdk-2024.09.02.tar.xz

tar -xJvf nuclei-linux-sdk-2024.09.02.tar.xz

If you want to clone with git, follow steps here: https://github.com/Nuclei-Software/nuclei-linux-sdk#clone-repo
If you want to try docker/podman with source code and required tools installed, check #8

Download without git

Click https://github.com/Nuclei-Software/nuclei-linux-sdk/actions/workflows/build.yml
Find the latest successful github action workflow for dev_nuclei_next branch, if you are using dev_nuclei_next

We recommended to use dev_nuclei_5.10_v2 for linux 5.10
- At the time this issue is written(2023.12.22), the latest successful workflow action for dev_nuclei_5.10_v2 is https://github.com/Nuclei-Software/nuclei-linux-sdk/actions/runs/6977662176
- To find the other latest build for different branch such as dev_nuclei_6.6_v2, please click https://github.com/Nuclei-Software/nuclei-linux-sdk/actions/workflows/build.yml?query=branch%3Adev_nuclei_6.6_v2 , you can copy this link and change dev_nuclei_6.6_v2 to other branch you want, and paste the changed link to your browser to find latest successful build.
- Then select build artifacts as you like, such as the source code of this sdk, to use it see #4 (comment).

Using GDBREMOTE to do remote debug and upload freeloader

See https://github.com/Nuclei-Software/nuclei-linux-sdk/blob/dev_nuclei_next/Makefile#L172-L179.

If you are do linux sdk development in remote machine, and your board is connect to your local PC, and you want to upload or debug freeloader to your local PC, you can do the following steps:

Assume your are using SOC=evalsoc CORE=ux900fd

run openocd -f conf/evalsoc/openocd.cfg in your local pc, make sure bindto 0.0.0.0 in openocd.cfg is uncommented
get your local pc ip, such as 192.168.43.120
In remote pc, you can do following command to upload or debug freeloader
a) upload freeloader: make SOC=evalsoc CORE=ux900fd GDBREMOTE="192.168.43.120:3333" upload_freeloader
b) debug freeloader: make SOC=evalsoc CORE=ux900fd GDBREMOTE="192.168.43.120:3333" debug_freeloader

buildroot_initramfs/host/bin/riscv-nuclei-linux-gnu-gdb: error while loading shared libraries: libgmp.so.3: cannot open shared object file: No such file or directory

先按照文档按照了需要的软件，参见README.md

通过如下命令可以查看缺少的库

然后执行如下命令

$ ls -l /lib/x86_64-linux-gnu/libgmp.so*
lrwxrwxrwx 1 root root     16 Feb 24  2020 /lib/x86_64-linux-gnu/libgmp.so -> libgmp.so.10.4.0
lrwxrwxrwx 1 root root     16 Feb 24  2020 /lib/x86_64-linux-gnu/libgmp.so.10 -> libgmp.so.10.4.0
-rw-r--r-- 1 root root 534880 Feb 24  2020 /lib/x86_64-linux-gnu/libgmp.so.10.4.0
# 手动创建一个软链接到新版本的libgmp.so上，需要root权限
$ sudo ln -s /lib/x86_64-linux-gnu/libgmp.so /lib/x86_64-linux-gnu/libgmp.so.3

如下是正常链接的结果

$ ldd work/evalsoc/buildroot_initramfs/host/bin/riscv-nuclei-linux-gnu-gdb
        linux-vdso.so.1 (0x00007ffc69f18000)
        libtinfo.so.5 => /lib/x86_64-linux-gnu/libtinfo.so.5 (0x00007ff4bc539000)
        libncursesw.so.5 => /lib/x86_64-linux-gnu/libncursesw.so.5 (0x00007ff4bc507000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007ff4bc501000)
        libexpat.so.1 => /lib/x86_64-linux-gnu/libexpat.so.1 (0x00007ff4bc4d3000)
        libgmp.so.3 => /lib/x86_64-linux-gnu/libgmp.so.3 (0x00007ff4bc44f000)
        libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007ff4bc26b000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007ff4bc11c000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007ff4bc101000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007ff4bc0de000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007ff4bbeec000)
        /lib64/ld-linux-x86-64.so.2 (0x00007ff4bc583000)

opensbi boot traped with a lot of sbi_trap_error

Issue Description and Solution

As image below described:

You can see mcause = 0x7, mtval = 0x08000000

mcause check riscv privilege manual is Store/AMO access fault,

mtval means whether the store access fault happened, which is 0x08000000, it is the previous Nuclei DemoSoC using cpu without iregion feature(iregion feature reallocate plic/timer base address).

nuclei cpu in demosoc without iregion: plic0: interrupt-controller@8000000
nuclei cpu in evalsoc with iregion: plic0: interrupt-controller@1c000000

See https://github.com/Nuclei-Software/nuclei-linux-sdk#modify-build-configuration about evalsoc/demosoc.

So the answer is change the SOC from demosoc to evalsoc.

For example, you want to test ux900fd, then build with the following command.

make SOC=evalsoc CORE=ux900fd freeloader
make SOC=evalsoc CORE=ux900fd bootimages

Known issues

currently evalsoc can't run in qemu, if you really want to run in qemu, please change the evalsoc dts you are using like this.

Caused by plic/timer base address changed.

If you changed this, opensbi will not run on your hardware, please remember change it back.

index b04d0a2..947d56c 100644
--- a/conf/evalsoc/nuclei_rv64imafdc.dts
+++ b/conf/evalsoc/nuclei_rv64imafdc.dts
@@ -171,7 +171,7 @@
     clock-output-names = "hfclk";
   };

-  plic0: interrupt-controller@1c000000 {
+  plic0: interrupt-controller@8000000 {
     #interrupt-cells = <1>;
     compatible = "riscv,plic0";
     interrupt-controller;
@@ -185,13 +185,13 @@
            &cpu5_intc 11 &cpu5_intc 9
            &cpu6_intc 11 &cpu6_intc 9
            &cpu7_intc 11 &cpu7_intc 9>;
-    reg = <0x0 0x1c000000 0x0 0x4000000>;
+    reg = <0x0 0x8000000 0x0 0x4000000>;
   };

-  clint0: clint@18031000 {
+  clint0: clint@2001000 {
     #interrupt-cells = <1>;
     compatible = "riscv,clint0";
-    reg = <0x0 0x18031000 0x0 0xC000>;
+    reg = <0x0 0x2001000 0x0 0xC000>;
     interrupts-extended =
           <&cpu0_intc 3 &cpu0_intc 7
            &cpu1_intc 3 &cpu1_intc 7

OP-TEE build does not continue after OpenSBI

Hello,

I am currently working on a research project that involves OP-TEE on RISC-V. Your project would serve as a great starting point, however I cannot get it to run.

I have used your docker image: sudo docker run -it nucleisoftware/linuxsdk:optee

And then followed the advice here: https://hub.docker.com/r/nucleisoftware/linuxsdk

# cd linux sdk workspace
cd nuclei-linux-sdk
# see commit log of last one
git log -1
# fetch latest commits if needs
git fetch
# pull latest changes if needs
git pull 
# do submodule updates 
git submodule update
# build project just follow https://github.com/Nuclei-Software/nuclei-linux-sdk#show-help in each branch
# run qemu for SOC=evalsoc CORE=ux900fd
make run_qemu

Unfortunately, the boot process never continues after OpenSBI:

[...]
/home/nuclei/nuclei-linux-sdk/work/demosoc/buildroot_initramfs/host/bin/riscv-nuclei-linux-gnu-objcopy /home/nuclei/nuclei-linux-sdk/work/demosoc/freeloader/freeloader.elf -O binary /home/nuclei/nuclei-linux-sdk/work/demosoc/freeloader/freeloader.bin
/home/nuclei/nuclei-linux-sdk/work/demosoc/buildroot_initramfs/host/bin/riscv-nuclei-linux-gnu-objdump -d /home/nuclei/nuclei-linux-sdk/work/demosoc/freeloader/freeloader.elf > /home/nuclei/nuclei-linux-sdk/work/demosoc/freeloader/freeloader.dasm
make[1]: Leaving directory '/home/nuclei/nuclei-linux-sdk/freeloader'
Run on qemu for simulation
qemu-system-riscv64 -M nuclei_u,download=flashxip -smp 8 -m 256M -cpu nuclei-ux600,ext= -bios /home/nuclei/nuclei-linux-sdk/work/demosoc/freeloader/freeloader.elf -nographic -drive file=/home/nuclei/nuclei-linux-sdk/work/demosoc/disk.img,if=sd,format=raw

OpenSBI v0.9
   ____                    _____ ____ _____
  / __ \                  / ____|  _ \_   _|
 | |  | |_ __   ___ _ __ | (___ | |_) || |
 | |  | | '_ \ / _ \ '_ \ \___ \|  _ < | |
 | |__| | |_) |  __/ | | |____) | |_) || |_
  \____/| .__/ \___|_| |_|_____/|____/_____|
        | |
        |_|

Platform Name             : Nuclei Demo SoC
Platform Features         : timer,mfdeleg
Platform HART Count       : 8
Firmware Base             : 0xa0000000
Firmware Size             : 184 KB
Runtime SBI Version       : 0.2

Domain0 Name              : root
Domain0 Boot HART         : 0
Domain0 HARTs             : 0*,1*,2*,3*,4*,5*,6*,7*
Domain0 Region00          : 0x00000000a0000000-0x00000000a003ffff ()
Domain0 Region01          : 0x0000000000000000-0xffffffffffffffff (R,W,X)
Domain0 Next Address      : 0x00000000a0200000
Domain0 Next Arg1         : 0x00000000a8000000
Domain0 Next Mode         : S-mode
Domain0 SysReset          : yes

Boot HART ID              : 0
Boot HART Domain          : root
Boot HART ISA             : rv64imafdcsu
Boot HART Features        : scounteren,mcounteren,time
Boot HART PMP Count       : 16
Boot HART PMP Granularity : 4
Boot HART PMP Address Bits: 54
Boot HART MHPM Count      : 0
Boot HART MHPM Count      : 0
Boot HART MIDELEG         : 0x0000000000000222
Boot HART MEDELEG         : 0x000000000000b109
~ stop ~

Since I believe I have followed every step as indicated, could it be that the docker image is no longer compatible with the latest commits? Or any other idea that could point me in the right direction? Thanks!

How to create dma coherent memory on Nuclei Linux Platform

The RISC-V Instruction Set Manual Privileged Architecture have added the Svpbmt Standard Extension for sv39,sv48,sv57, more details info refer to riscv standard doc. According to the doc, Svpbmt extension is only used for rv64.

Warning

Please make sure you follow the correct steps list below, if it is not working as expected, when you are looking for help from us, please let us know what is the difference you made compared to the doc below, otherwise it will be hard for you and us to investigate the root cause.

Nuclei 900 series cpu core above v2.8.0 support riscv svpbmt extension and cmo extension.

core with svpbmt extension

when cpu core support svpbmt extension, we can use this extension to create dma coherent memory area.

list patch based on nuclei sdk dev_nuclei_6.6_v2 branch.

Note

Tested on dev_nuclei_6.6_v2 branch, for other branches, please find similiar way by yourself.

dts:

If SOC have no hardware dma coherent, dma-noncoherent dts properties need to be added.

dma-noncoherent;

add zicbom and svpbmt to isa, 64 is core cache line size(unit:byte), you can change it according to your hardware.

each core should attach with svpbmt to isa, and cbo block size.

riscv,isa = "rv64imafdc_zicbom_svpbmt"
riscv,cbom-block-size =<64>;

In theory,zicbom isa is not necessary,but from the perspective of kernel configuration dependencies and user scenarios, it is necessary to enable zicbom configuration.

dts_svpbmt.patch

kernel config:

CONFIG_RISCV_ISA_ZICBOM=y
CONFIG_RISCV_ISA_SVPBMT=y

toolchain:

At least this version of the toolchain can meet the requirements for selecting the CONFIG_RISCV_ISA_ZICBOM configuration.
nuclei gcc10 toolchain cannot meet gcc version requirements, while nuclei gcc13 toolchain can meet.

core without svpbmt extension

When cpu core have no svpbmt extension, we can first reserve a non-cachable memory area that meets the system requirements based on Nuclei mattri csr and use it in conjunction with the kernel global DMA pool to adapt kernel dma_alloc_coherent API.

list patch based on nuclei linux sdk dev_nuclei_6.6_v2 branch.

Note

Tested on dev_nuclei_6.6_v2 branch, for other branches, please find similiar way by yourself.

opensbi:

we can use Nuclei mattri csr to reserve a non-cacheable memory area, please refer to Nuclei_RISC-V_ISA_Spec.pdf about mattri csr usage.

example:

reserve base:0xf0000000, size:0x100000 non-cachable area

static int nuclei_evalsoc_final_init(bool cold_boot, const struct fdt_match *match)) {
    ....

    #define mattri1_base 0x7f5
    #define mattri1_mask 0x7f6
    csr_write(mattri1_mask, 0xfff00000);
    csr_write(mattri1_base, 0xf0000005);

    return 0;
}

note:

mattri mask value: ~(size-1), area base address should be aligned area size.

Generally there are 5 sets of mattri CSR. If mattri CSR is used in multiple places, please allocate mattri CSR reasonably.

dts:

If SOC have no hardware dma coherent, dma-noncoherent dts properties need to be added.

dma-noncoherent;

riscv,isa = "rv64imafdc_zicbom"
riscv,cbom-block-size =<64>;

if no cmo extension, Nuclei ccm extension is an alternative solution，Nuclei ccm commit refer to https://github.com/Nuclei-Software/nuclei-linux-sdk/commits/feature/5.10_ccm , Nuclei ccm doc refer to Nuclei_CCM_Mechanism.pdf

create reserved memory base:0xf0000000, size:0x100000 in dts node.

  reserved-memory {
    #address-cells = <2>;
    #size-cells = <2>;
    ranges;
    dam_coherent@f0000000 {
        compatible = "shared-dma-pool";
        reg = <0 0xf0000000 0 0x100000>;
        no-map;
        linux,dma-default;
    };
  };

above dts config is for rv64，rv32 can refer this to adapt.

rv32 reserve memory node

  reserved-memory {
    #address-cells = <1>;
    #size-cells = <1>;
    ranges;
    dam_coherent@f0000000 {
        compatible = "shared-dma-pool";
        reg = <0xf0000000 0x100000>;
        no-map;
        linux,dma-default;
    };
  };

dts_no_svpbmt.patch

kernel config:

CONFIG_DMA_DECLARE_COHERENT=y
CONFIG_DMA_GLOBAL_POOL=y

select CONFIG_DMA_GLOBAL_POOL to default y in kernel/dma/Kconfig

 config DMA_GLOBAL_POOL
    select DMA_DECLARE_COHERENT
    default y
    bool

Common Questions 常见问题

Repo Branch Usage

About branch naming and its support features, see #2

How to use this repo

About how to download source code, see below topics
- #4
- #10
About how to run it using docker/podman environment, see #8
About how to bring up, see #21
About how to debug linux, see https://github.com/Nuclei-Software/nuclei-linux-sdk/wiki/Debug-Linuxsdk-using-NucleiStudio
About how to use amp feature, see #18
About how to use optee feature, see #17 #16 #20
About how to use dma_alloc_coherent API in Nuclei RISC-V Linux, see #22
About #23

About toolchain

Common issues or bugs met

You can also check the open or closed issues here https://github.com/Nuclei-Software/nuclei-linux-sdk/issues?q=is%3Aissue+

Quick steps to evaluate optee solution on nuclei qemu

Recently, we have open source our OpTEE hardware and software solution in this repo, see OP-TEE/optee_os#6173, and now we also implement required hardware feature in Nuclei QEMU, so you can take a try with it.

latest source code of nuclei linux sdk is required.

General steps

Here are the basic steps to try it:

# pull latest source code of feature/optee_5.10 branch
git clone -b feature/optee_5.10 https://github.com/Nuclei-Software/nuclei-linux-sdk
cd nuclei-linux-sdk
# --depth=3 is used to do shallow clone, do less source code download
git submodule update --init --depth=3
# make sure all required source code are checked out
git submodule
# refer the doc here: https://github.com/Nuclei-Software/nuclei-linux-sdk/blob/feature/optee_5.10/optee/README_en.md
# download a special version of nuclei qemu
wget -q https://nucleisys.com/upload/files/toochain/qemu/nuclei_qemu-2023.07.19-linux.zip
unzip nuclei_qemu-2023.07.19-linux.zip
# you can also evaluate it using Nuclei Qemu 2023.10 downloaded from https://www.nucleisys.com/download.php#tools
export PATH=$(pwd)/linux_qemu/bin:$PATH
# install extra required tools following steps in https://github.com/Nuclei-Software/nuclei-linux-sdk/tree/feature/optee_5.10#ubuntu-2004-x86_64-host
# install extra packages for optee, for details check https://optee.readthedocs.io/en/latest/building/prerequisites.html
pip3 install -U pyelftools cryptography
# check whether qemu required share library is installed
ldd `which qemu-system-riscv64`
# if not, please install extra required packages to let it work
# here are sample packages for ubuntu 20.04
# sudo apt install -yq python3 python3-pip unzip libz-dev libssl-dev libglib2.0-dev libfdt-dev libpixman-1-dev zlib1g-dev libncursesw5-dev libncursesw5 libgmp-dev libmpfr-dev libmpc-dev libpng-dev libasound2-dev
qemu-system-riscv64 --version
# build images
make SOC=evalsoc CORE=ux900fd BOOT_MODE=sd freeloader
make SOC=evalsoc CORE=ux900fd BOOT_MODE=sd bootimages
# run and evaluate it on nuclei qemu
make SOC=evalsoc CORE=ux900fd BOOT_MODE=sd run_qemu

About how to check optee tests and examples, please check https://github.com/Nuclei-Software/nuclei-linux-sdk/blob/feature/optee_5.10/optee/README_en.md#start-and-running-log

Sample git submodule output as below(until 2023.07.19):

nuclei@8bfe7bdac196:~/nuclei-linux-sdk$ git status -uno
On branch feature/optee_5.10
Your branch is up to date with 'origin/feature/optee_5.10'.

nothing to commit (use -u to show untracked files)
nuclei@8bfe7bdac196:~/nuclei-linux-sdk$ git log --oneline -1
a8a4391 (HEAD -> feature/optee_5.10, origin/feature/optee_5.10) bump optee_os update max tee core to 8.
nuclei@8bfe7bdac196:~/nuclei-linux-sdk$ git submodule
 16931968ddf5225f92f93d0db2eaa1eafcd63c44 buildroot (2020.11.2-2-g16931968dd)
 c0c88a724998493932bb29cb64d7de88b02ac0cd freeloader (remotes/origin/feature/optee_5.10)
 9b9a37c4ec83cd46e2dc7c4fbadabd7cb017614a linux (v5.10-13-g9b9a37c4ec83)
 3deace78edea403815df51db6999e8a7127e89da opensbi (v0.9-17-g3deace7)
 0cbbcf206b0452a3aa6ad521ea61d907720ad553 optee/optee_benchmark (remotes/origin/nuclei_dev)
 1ad692a6868226429a438f9192a380de9fc4552b optee/optee_client (3.18.0-1-g1ad692a)
 cca34e82cc80dae0986707386da0c8c92eeac5f6 optee/optee_examples (remotes/origin/nuclei/3.18_dev)
 34e68e2b1b27e5222aaee4d9956aa2e53bd98808 optee/optee_os (3.18.0-103-g34e68e2b1)
 29eaa54b5adda4a096298ed2313eb852dc758e0d optee/optee_test (3.18.0-3-g29eaa54)
 b9162c6c8f30098b09bdf79aa2b40204deed7bfd u-boot (v2021.01-19-gb9162c6c8f)

Update

2023.08.11

Regarding facing issue of get source code or building linux sdk, we provide prebuilt qemu images, which can be executed directly using qemu, you can download this prebuilt qemu optee images, and directly try it.

unzip qemu_optee_images_20230811.zip
# set qemu path before execute it, see above steps
bash run_qemu.sh

Full run log see optee_5.10_full_runlog.txt

We also provide a docker image docker.io/nucleisoftware/linuxsdk:optee, in this docker, the source code is checkout to feature/optee_5.10 branch and requried software are installed, sudo password is riscv123.

docker run -it docker.io/nucleisoftware/linuxsdk:optee
# in docker now
cd nuclei-linux-sdk
# direct run in qemu
make SOC=evalsoc CORE=ux900fd BOOT_MODE=sd run_qemu

About Linux Toolchain 关于Linux工具链

In Nuclei Linux SDK, if you have chosen your branch #2, and build linux sdk for your selected SOC and CORE, and then you will be able to get a installed toolchain in work directory.

For example, if you are building for SOC=evalsoc CORE=ux900fd using branch dev_nuclei_6.1_v2, you should be able to find the toolchain in work/evalsoc/buildroot_initramfs/host/bin/.

For some configuration of Linux SDK, we are using buildroot built toolchain instead of prebuilt toolchain, you need to check with the buildroot menuconfig via make SOC=evalsoc CORE=ux900fd buildroot_initramfs-menuconfig, see https://github.com/Nuclei-Software/nuclei-linux-sdk#customize-buildroot-packages

关于各分支功能介绍 About each branch feature

请关注最新的issue评论，这里的文字说明可能不是最新版本，以具体代码为准

请注意 本issue与其他issue会经常更新，请点击上面的 edited 来获知最新的修改的变更

Note

点击这里还可以看到更多 Nuclei Linux SDK使用的各种技巧和场景的文章，持续不断更新完善中。

中文介绍

截止到2023.07号起更新的代码，分支特性说明如下：

主要支持如下SoC:

demosoc: 比较早版本的评估用的SoC，不带有CPU IREGION特性，已经弃用。
evalsoc: 目前最新的评估用的SoC，带CPU IREGION特性，演进了两个版本
- evalsoc v1: DDR Base为0xA0000000, 默认提供的FPGA Soft CPU Core Frequency 通常是 100MHz 或者 50Mhz
- evalsoc v2: DDR Base为 0x80000000, 默认提供的FPGA Soft CPU Core Frequency 通常是 50Mhz, 2023.07引入作为evalsoc v1的替代。
- 最新的 900 评估bitstream都是采用的evalsoc v2

分支说明

请注意：切换分支或者切换启动模式(sd/flash), 需要清理 work目录，删掉或者重命名，且切换分支的时候，需要同时更新submodule的代码

请注意: 如果你打算在你自己的SOC上bringup Linux环境，请先阅读我们准备好的bringup建议 #21

目前活跃开发的分支为 dev_nuclei_5.10_v2, dev_nuclei_6.1_v2 和 dev_nuclei_6.6_v2: 主要针对最新的900 CPU的FPGA evalsoc进行适配和优化。

提醒: 目前 dev_nuclei_6.6_v2 分支使用的是开发中的预编译好的gcc13 glibc工具链，其他分支暂时使用的都是预编译好的gcc10 glibc工具链或者buildroot生成的工具链，工具链的信息记录在 buildroot_initramfs_*_config文件中。

提醒: 计划使用 dev_nuclei_next / dev_nuclei_5.10分支的，建议直接使用 dev_nuclei_5.10_v2 分支

提醒: 关于默认的CPU/Peripheral/Timer频率以选中的SoC中对应的dts(如nuclei_rv64imafdc.dts)和build.mk配置为准。

提醒: 请根据你当前拿到的bitstream的CPU版本以及CPU主频来选择分支，切换到对应分支，并确保代码是最新的，然后检查并修改对应的dts或者build.mk来满足你的bitstream的要求，再进行代码的构建。

文档做了更新的分支如下:

dev_nuclei_next/dev_nuclei_5.10/dev_nuclei_6.1/dev_nuclei_6.1_v2/dev_nuclei_5.10_v2
其他特定分支，文档可能没有及时更新

以下是各个分支的情况说明

dev_nuclei_next: 默认分支，支持SD卡启动和Flash启动, 请切换到 dev_nuclei_5.10_v2 分支。
- 支持demosoc 和 evalsoc v1，基于Kernel 5.10, 支持RV64
- 从flash启动，并借助sd卡启动linux，freeloader整合了opensbi+uboot，sd卡加载kernel+rootfs+dtb
- 从flash直接启动，不需要依赖sd卡，freeloader整合了opensbi+uboot+kernel+rootfs+dtb（对spiflash大小有要求>=8M），也可以在uboot阶段打断启动后从SD卡启动（运行 run distro_bootcmd）
feature/fsl: 用于评估FSL91030M 开发板，使用参见 https://github.com/Nuclei-Software/nuclei-linux-sdk/tree/feature/fsl/conf/fsl91030m
dev_nuclei_keystone: 主要用于评估Keystone TEE相关的方案，基于 Kernel 5.10, 支持RV64
dev_nuclei_penglai: 主要用于评估Penglai TEE相关的方案，基于 Kernel 5.10, 支持RV64
dev_nuclei_5.10 & dev_nuclei_6.1: 分别用于评估最新的5.10 或者 6.1 LTS版本内核，支持 evalsoc v1版本, 支持RV64
dev_nuclei_5.10_v2 & dev_nuclei_6.1_v2: 分别用于评估最新的5.10 或者 6.1 LTS版本内核，支持 evalsoc v2版本, 支持RV64 & RV32
dev_nuclei_6.6_v2: 用于评估6.6版本的内核，采用我们最新的预编译的gcc13 glibc 2.38的工具链(开发中)，支持evalsoc v2, 支持rv32 & rv64, 支持 Svnapot/Svpbmt/Zbb/Zicbom/Zicboz/Vector 特性，demosoc支持已经被删除了
dev_nuclei_6.9_v2: 用于评估6.9版本的内核，采用我们最新的预编译的gcc13 glibc 2.38的工具链(开发中)，支持evalsoc v2, 支持rv32 & rv64, 支持 Svnapot/Svpbmt/Zbb/Zicbom/Zicboz/Vector 特性，demosoc支持已经被删除了
dev_nuclei_6.6_v2_customsoc: 基于6.6内核增加对Nuclei SoC IP Driver的支持和测试（持续增加中），测试使用本地参考平台，支持位于 conf/customsoc, 修改的内核位于 dev_nuclei_6.6.y分支
dev_nuclei_6.6_v2_android: 基于Android Linux Kernel 6.6版本针对 CORE=ux900fd 进行的移植和适配，用于测试AOSP的Bringup，详细使用参见该分支下的 conf/evalsoc/README.md
feature/optee_5.10: 主要用于评估OpTEE 相关的方案，基于Kernel 5.10, , 支持RV64, 参见 OP-TEE/optee_os#6173 和 #13
feature/optee_5.10_wg: 主要用于评估OpTEE 相关的方案，基于Kernel 5.10, , 支持RV64, 同时需要CPU硬件支持更多安全特性
dev_nuclei_6.4_v2_rt: 基于Linux 6.4内核 + RT Patches用于评估实时内核
feature/5.10_ccm: 基于 5.10内核，增加了Linux CCM的cache 管理的适配，代码适配，可以boot，但是需要带DMA的设备来测试这个
dev_nuclei_5.10_v2_64baddr: 基于 5.10内核，用于测试ddr基地址大于 32位，uboot加载镜像采用FIT格式来解决之前方式只能加载32位地址空间内的镜像问题，这个分支只适配了 ux900fd - sd 模式
dev_nuclei_5.10_v2_fit: 基于 5.10内核，用于测试ddr基地址大于 32位，uboot加载镜像采用FIT格式来解决之前方式只能加载32位地址空间内的镜像问题
dev_nuclei_6.1_v2_fit: 基于 6.1内核，用于测试ddr基地址大于 32位，uboot加载镜像采用FIT格式来解决之前方式只能加载32位地址空间内的镜像问题
dev_nuclei_6.6_v2_fit: 基于 6.6内核，用于测试ddr基地址大于 32位，uboot加载镜像采用FIT格式来解决之前方式只能加载32位地址空间内的镜像问题

Build error optee_os/core/tee.bin Unknown e_machine "243" when try to build optee option

I follow https://github.com/Nuclei-Software/nuclei-linux-sdk/tree/feature/optee_5.10_wg/optee#optee-%E8%AF%B4%E6%98%8E to build linux SDK with optee_5.10_wg branch, and get the following error after run make SOC=evalsoc CORE=ux900fd BOOT_MODE=sd freeloader command:

 GEN     /mnt/fileroot2/sichun.qin/code/github/nuclei-linux-sdk/work/evalsoc/optee/optee_os/core/tee.bin
Unknown e_machine "243"
core/arch/riscv/kernel/link.mk:127: recipe for target '/mnt/fileroot2/sichun.qin/code/github/nuclei-linux-sdk/work/evalsoc/optee/optee_os/core/tee.bin' failed

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.