Add neorv32 to LiteX as soft-cpu option

[stnolting]

Add neorv32 to LiteX as soft-cpu option

[enjoy-digital]

As discussed in #264, I just experimented with NEORV32 and LiteX. Thanks to NEORV32 very clean code and documentation, it only took a couple of hours to get a first version working in simulation and on hardware:

$ litex_sim --cpu-type=neorv32

        __   _ __      _  __
       / /  (_) /____ | |/_/
      / /__/ / __/ -_)>  <
     /____/_/\__/\__/_/|_|
   Build your hardware, easily!

 (c) Copyright 2012-2022 Enjoy-Digital
 (c) Copyright 2007-2015 M-Labs

 BIOS built on Feb 14 2022 16:46:59
 BIOS CRC passed (0a0e47a6)

 Migen git sha1: ac70301
 LiteX git sha1: 0d218306

--=============== SoC ==================--
CPU:		NEORV32 @ 1MHz
BUS:		WISHBONE 32-bit @ 4GiB
CSR:		32-bit data
ROM:		128KiB
SRAM:		8KiB


--============== Boot ==================--
Booting from serial...
Press Q or ESC to abort boot completely.
sL5DdSMmkekro
Timeout
No boot medium found

--============= Console ================--

litex> 

$ python3 -m litex_boards.targets.digilent_nexys_video --cpu-type=neorv32 --uart-name=usb_fifo --build --load
        __   _ __      _  __
       / /  (_) /____ | |/_/
      / /__/ / __/ -_)>  <
     /____/_/\__/\__/_/|_|
   Build your hardware, easily!

 (c) Copyright 2012-2022 Enjoy-Digital
 (c) Copyright 2007-2015 M-Labs

 BIOS built on Feb 14 2022 16:10:24
 BIOS CRC passed (83edf3c3)

 Migen git sha1: ac70301
 LiteX git sha1: 0d218306

--=============== SoC ==================--
CPU:		NEORV32 @ 100MHz
BUS:		WISHBONE 32-bit @ 4GiB
CSR:		32-bit data
ROM:		128KiB
SRAM:		8KiB
L2:		8KiB
SDRAM:		524288KiB 16-bit @ 800MT/s (CL-7 CWL-5)

--========== Initialization ============--
Initializing SDRAM @0x40000000...
Switching SDRAM to software control.
Read leveling:
  m0, b00: |00000000000000000000000000000000| delays: -
  m0, b01: |11000000000000000000000000000000| delays: 01+-01
  m0, b02: |00011111111111111000000000000000| delays: 10+-07
  m0, b03: |00000000000000000000111111111111| delays: 26+-06
  m0, b04: |00000000000000000000000000000000| delays: -
  m0, b05: |00000000000000000000000000000000| delays: -
  m0, b06: |00000000000000000000000000000000| delays: -
  m0, b07: |00000000000000000000000000000000| delays: -
  best: m0, b02 delays: 10+-07
  m1, b00: |00000000000000000000000000000000| delays: -
  m1, b01: |10000000000000000000000000000000| delays: 00+-00
  m1, b02: |00111111111111111000000000000000| delays: 09+-07
  m1, b03: |00000000000000000001111111111111| delays: 25+-06
  m1, b04: |00000000000000000000000000000000| delays: -
  m1, b05: |00000000000000000000000000000000| delays: -
  m1, b06: |00000000000000000000000000000000| delays: -
  m1, b07: |00000000000000000000000000000000| delays: -
  best: m1, b02 delays: 09+-07
Switching SDRAM to hardware control.
Memtest at 0x40000000 (2.0MiB)...
  Write: 0x40000000-0x40200000 2.0MiB     
   Read: 0x40000000-0x40200000 2.0MiB     
Memtest OK
Memspeed at 0x40000000 (Sequential, 2.0MiB)...
  Write speed: 9.3MiB/s
   Read speed: 13.2MiB/s

--============== Boot ==================--
Booting from serial...
Press Q or ESC to abort boot completely.
sL5DdSMmkekro
Timeout
No boot medium found

--============= Console ================--

litex> 

The integration test is available here: https://github.com/enjoy-digital/litex_neorv32_test and works both in simulation and on hardware through verilog conversion with GHDL-Yosys-Plugin. (Haven't checked yet if resources were optimal yet, but the fact it work is already a good thing!).

This is only a first test and we could now improve integration by exposing different variants of NEORV32, add interrupts and add direct VHDL use for toolchains supporting it.

This should also be useful to #266 since NEORV32 is first converted to verilog here and to #269 since CFU Playground also uses LiteX for the infrastructure and since the CFU interface is standardized (at least will be if not already), things should now then be very similar to use with VexRiscv.

[umarcor]

Actually, I wanted to give Litex a try after #110. In #110, the filesets are defined in Python, which makes it easier to import them and use them in a Litex script (along with the litex-boards features).

At the moment, filesets, constraints and tools are all defined through makefiles, which would imply duplicating that in the Litex script. Actually, constraints and tools are part of Litex I think, so it would mostly be about the filesets. However, since we have ~6 different places where filesets are defined already, I'd like to avoid increasing the complexity.

[umarcor]

BTW, this is related to #105, since litedram is a Litex project. So, the easiest path for testing the DDR on the OrangeCrab, Arty, etc. is probably supporting NEORV32 in Litex first.

[enjoy-digital]

@stnolting: It was really just a very quick test to get a first version running, we should indeed discuss which features should be supported/integrated. There is also some overlap between features in NEORV32 and LiteX and that's why I only reused the neorv32_cpu for now.

The overlap is not an issue and as discussed previously, I would be really happy to help you use the full NEORV32 processor with LiteX or export some LiteX components that could be useful for reuse on NEORV32 as we are already doing with project like Microwatt.

But for now I'm just trying to make the integration very similar to the other CPUs and offer the same capabilities (ie allow a LiteX user to select NEORV32 CPU by just adding --cpu-type=neorv32 to their LiteX targets instead of --cpu-type=vexriscv/serv/picorv32/etc...)

Since neorv32_cpu is the "equivalent" of the other CPUs, this feels more natural to use this instead of the processor but as you suggest, we can have a specific wrapper (in LiteX) that would optionally re-integrate some of the components:

• I/D Cache.
• Wishbone interface.
• Machine Timer.
• On-Chip Debugger.

And we could also define different variants of the CPU that would allow users to simplify use different version of NEORV32 with --cpu-variant=xxyy command.

BTW: If you want to test this without installing LiteX, please just tell me which FPGA board you have, if it's already supported in LiteX-Boards, I'll try to build the design for it and share it :)

[enjoy-digital]

Antmicro has created a tester around LiteX that can be used to compare different CPU:

• https://antmicro.com/blog/2020/09/automatic-cpu-testing-with-embench/
• https://antmicro.github.io/embench-tester/

The results from NEORV32 could also probably be added to it in the future.

Other users have also been using LiteX to compare results of different CPUs running the same application (for example @BrunoLevy in https://github.com/BrunoLevy/learn-fpga/tree/master/LiteX).

So the framework can indeed be used for this (and also provide you a simple way to test your CPU on the already supported boards).

For the peripherals integrated around the CPU, we have some common ones and depending the CPU we also integrate specific ones (ex Debugger/RISC-V Machine Timer on VexRiscv). The specific peripherals can be directly added in the CPU wrapper, so we could also do this for NEORV32.

The variants seem similar to your templates: https://github.com/enjoy-digital/litex/wiki/CPUs#soft-cpu-variants The idea is to provide different configurations that can be easily reused without knowing the RISC-V specificities.

Here is the bitstream for the Digilent Arty A7-35: digilent_arty_neorv32_2022_02_17.zip

[mithro]

There is some WIP documentation about this at https://github.com/enjoy-digital/litex/wiki/Add-A-New-CPU

[stnolting]

That would be really great! 👍 🚀
But right now I am not familiar with the Litex build system at all. Thanks for the documentation hints - I will take a closer look 😉

[stnolting]

@enjoy-digital

Holy cricket! This is amazing!!! 🎉 🚀

The integration test is available here: https://github.com/enjoy-digital/litex_neorv32_test and works both in simulation and on hardware

I need to check that out!

This is only a first test and we could now improve integration by exposing different variants of NEORV32, add interrupts and add direct VHDL use for toolchains supporting it.

This is a CPU-only implementation, right?

Maybe the NEORV32 processor setup is contradicting the Litex concept, but what do you think about adding certain modules of the processor like the RISC-V machine timer and the on-chip debugger (making both optional of course)?

[stnolting]

but for now I'm just trying to make the integration very similar to the other CPUs and offer the same capabilities (ie allow a LiteX user to select NEORV32 CPU by just adding --cpu-type=neorv32 to their LiteX targets instead of --cpu-type=vexriscv/serv/picorv32/etc...)

Since neorv32_cpu is the "equivalent" of the other CPUs

I agree! 👍

Seems like there are several RISC-V cores supported by LiteX. Is there some kind of comparative overview of those cores? I think it would be interesting to see the differences to be able to choose the right one for a certain application (features like ISA extensions, max frequency, throughput, size, ...).

as you suggest, we can have a specific wrapper (in LiteX) that would optionally re-integrate some of the components:

I need to check the other Litex single-core RISC-V setups again. How do they handle things like on-chip debugger and machine timer? Are they in the "CPU complex" or somewhere outside coupled via the Litex infrastructure?

And we could also define different variants of the CPU that would allow users to simplify use different version of NEORV32 with --cpu-variant=xxyy command.

That sounds good! But how does that "variant" configuration work? Does it select one out of many pre-configured CPU systems (just like the NEORV32 processor templates) or is there some kind of "parsing" that enables certain options (for example by setting specific core generics)?

BTW: If you want to test this without installing LiteX, please just tell me which FPGA board you have, if it's already supported in LiteX-Boards, I'll try to build the design for it and share it :)

That would be very nice! I am still struggling with the installation/handling of LiteX on my machine. It is not because of your framework, but because oft Windows and WSL... 😉

I have a Digilent Arty A7-35 FPGA board here - I think that is this litex_boards/platforms/digilent_arty.py, right?

(By the way, I am still reading through your documentation... So some of my questions might be quite naive 😅)

[enjoy-digital]

BTW: I was prototyping NEORV32 integration in litex_neorv32_test but since there is now an initial version working, I integrated it directly in LiteX with enjoy-digital/litex@38a047b. This then give access to NEORV32 to allow boards from LiteX-Boards and will simplify further improvements.

[stnolting]

Thank you very much for the references! And sorry for the delay...

Right now I am trying to solve some NEORV32 issues to make the CPU as RISC-V spec. compatible as possible so it can be integrated into LiteX as flawlessly as possible.

The specific peripherals can be directly added in the CPU wrapper, so we could also do this for NEORV32.

I have already made a list with features/modules that might be suitable for the CPU wrapper. Maybe I should start an additional issue for that... 😉

However, there are still some questions regarding some LiteX details:

• If the CPU wrapper includes memory-mapped modules the according memory address will not be available for LiteX SoC modules (like memory). How do you specify the global address map? Is this something general or is it a specific thing for each CPU (variant)?
• What about "tightly coupled memories"? The NEORV32 processor uses internal memories for data and instructions. Is this something one could support for a CPU wrapper or this is something that should be outside of the CPU complex?
• Do the CPU wrappers need to have separate buses for instruction and data access? I understand that this is a performance-relevant design decision. However, NEORV32 would require only a single port (with internal cache and maybe even tightly coupled memories this would not be a big performance downside).
• I think it would be a good idea to add the on-chip debugger modules and maybe even the RISC-V machine timer to the NEORV32 CPU wrapper. However, I am not sure about all the IO devices (UARTs, GPIO, SPI, ...). LiteX already provides those standard interfaces and - from a software point of view - I think it would be easier to use the LiteX modules. What do you think about this?

The variants seem similar to your templates: https://github.com/enjoy-digital/litex/wiki/CPUs#soft-cpu-variants The idea is to provide different configurations that can be easily reused without knowing the RISC-V specificities.

I read across the CPU variants. Providing lite, standard and full (maybe even minimal) NEORV32/RISC-V configuration should be no problem - this is just a matter of defining the different generics.

Here is the bitstream for the Digilent Arty A7-35: digilent_arty_neorv32_2022_02_17.zip

Thank you very much! It works like a charm! 😄 👍

[enjoy-digital]

Hi @stnolting,

If the CPU wrapper includes memory-mapped modules the according memory address will not be available for LiteX SoC modules (like memory). How do you specify the global address map? Is this something general or is it a specific thing for each CPU (variant)?

This is flexible and it's possible to enforce the mapping from the CPU (since some CPU can introduce restrictions).The SoC builder will then adapt itself to this specified mapping. It's also possible to integrated specific peripherals along with the CPU with the optional add_soc_components method of CPUs.

What about "tightly coupled memories"? The NEORV32 processor uses internal memories for data and instructions. Is this something one could support for a CPU wrapper or this is something that should be outside of the CPU complex?

The L1 Instruction/Data caches are considered part of the CPU complex. With the current NEORV32 integration, the caches aren't integrated but it would be good to have a specific wrapper integrating them to improve performance.

Do the CPU wrappers need to have separate buses for instruction and data access? I understand that this is a performance-relevant design decision. However, NEORV32 would require only a single port (with internal cache and maybe even tightly coupled memories this would not be a big performance downside).

This is flexible, the CPU can have arbitrary number of buses connected to the main bus of the LiteX SoC and arbitrary number of direct memory buses (directly connected to the main RAM of the SoC). For now, the NEORV32 integration uses 2 separate Instruction/Data buses, but we could only use one if this is regrouped in a single bus after the caches.

I think it would be a good idea to add the on-chip debugger modules and maybe even the RISC-V machine timer to the NEORV32 CPU wrapper. However, I am not sure about all the IO devices (UARTs, GPIO, SPI, ...). LiteX already provides those standard interfaces and - from a software point of view - I think it would be easier to use the LiteX modules. What do you think about this?

The on-chip debugger and RISC-V machine timer are not yet provided by LiteX, so it would indeed make sense to integrate it in the NEORV32 wrapper. This would be very similar to what we are doing for VexRiscv. The other peripherals are provided by LiteX, so wouldn't be required in a first time, at least for the CPU integration in LiteX.

I read across the CPU variants. Providing lite, standard and full (maybe even minimal) NEORV32/RISC-V configuration should be no problem - this is just a matter of defining the different generics.

Good, that's indeed what I had in mind since NEORV32 is already highly configurable!