freechipsproject / chisel-testers Goto Github PK
View Code? Open in Web Editor NEWProvides various testers for chisel users
License: Apache License 2.0
Provides various testers for chisel users
License: Apache License 2.0
Supposedly you can specify a file location, but seems like the VCD is just dumped into the test_run_dir/YourProjectHere directory...
I thought it was via testCmd = Seq(), but it's not doing anything for me???
PeekPokeTester catches TestApplicationException, returning failure if the exit code is non-zero. Some simulations (notably vcs) do not set a reliable exit code, but in any case, we should signal up-level that the simulation is no longer running.
Asked on stackoverflow.
Is there a way to specify options to VCS when using the PeekPokeTester? In particular, I would like to: 1) Enable System Verilog for black box code 2) Control the macro defines to disable the random number-based initialization. (I want to see initialization issues as Xs.)
I'd propose that there be two new options to chisel-testers, one to specify additional command line arguments, and another to expand the subcommands that are passed to the C compiler.
This is what we have right now
class MyDUT extends Module {
... instantiate MyDevice here ...
}
class MyDUTTester(val c: MyDUT) extends PeekPokeTester(c) {
expect(c.io.blah, someNumber)
...
}
class MyDeviceSpec extends ChiselFlatSpec {
"MyDevice" should "blah" in {
Driver(() => new MyDUT) {
c => new MyDUTTester(c)
} should be (true)
}
}
The amount of repetition there is pretty insane, and even worse is that the Tester logic is its own class, but is dependent on the parameterization that is specified externally in the Spec. Individual expect failures also aren't registered by the test framework (which can automatically generate nice error messages at the invocation site, I think including a stack trace); instead the test framework only tells your test failed somewhere. (yes, the default verbose failures messages helps, but that's a workaround at best, and for whatever reason isn't showing up on Verilator tests)
I think ideally we would have something like:
class MyDeviceSpec extends ChiselPeekPokeSpec {
"MyDevice" should "blah" in {
val t, c = createTester(new MyDUT) // essentially creates a test fixture, return an emulator link and the elaborated module for IO access
t.expect(c.io.blah, someNumber)
...
}
}
Much more compact, less repetition, less boilerplate, and most importantly makes writing tests fun for everyone!
Thoughts?
previous options parsing must be integrated with the current stuff
Right now, when I try to change the FirrtlExecutionOptions at the tester level to include the ReplSeqMem pass, nothing meaningful happens. Once the black box annotations are in place, it might make sense to allow custom transforms that would've previously broken simulation to be passed down... These days, I'm kind of ok using sbt test to do generation + tests in one go, so needing to enact a separate chisel Driver (edit: ok doesn't need to be in main) makes me sad...
In general, ExecutionsManagers are used in a really weird way. It seems bad if higher level stuff can't see changes made at the lower level... and It's also bad that options you set at the higher levels might not get propagated down without any warning...
The standard extension used for firrtl files is .fir.
There are situation when the directions of io are not specified that the tester does not detect any of the inputs. I think the tester should throw consider throwing an exception when no inputs are detected.
Alternatively, the tester should assume that unspecfied directions are outputs, which seems to be the current assumption within chisel3 and firrtl
When I run sbt test
, I got unresolved dependencies:
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: UNRESOLVED DEPENDENCIES ::
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: edu.berkeley.cs#chisel3_2.11;3.0: not found
[warn] :: edu.berkeley.cs#firrtl_2.11;0.1-SNAPSHOT: not found
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
When I run testers as a submodule, I got this:
/scratch/dgkim/chisel-tests/testers/build.sbt:38: error: value publishTo is not a member of scala.xml.NodeBuffer
possible cause: maybe a semicolon is missing before `value publishTo'?
publishTo <<= version { v: String =>
^
#32 is supposedly fixed, but in the latest revision (50224d0) this error seems to have come up again, and instead of using a test_run_dir it just puts all the generated files in the root of the repo:
# ls
build.sbt build.sh c.py log.txt project README.md scalastyle-config.xml scalastyle-test-config.xml src target
# sbt "test-only utils.test.DebouncerTester"
[...first test runs successfully...]
verilator --cc Debouncer.v --assert -Wno-fatal -Wno-WIDTH -Wno-STMTDLY --trace -O0 --top-module Debouncer +define+TOP_TYPE=VDebouncer +define+PRINTF_COND=!Debouncer.reset +define+STOP_COND=!Debouncer.reset -CFLAGS -Wno-undefined-bool-conversion -O0 -DTOP_TYPE=VDebouncer -include VDebouncer.h -Mdir . --exe Debouncer-harness.cpp
make: Entering directory `/home/cc/chisel-template'
g++ -I. -MMD -I/home/cc/verilator/include -I/home/cc/verilator/include/vltstd -DVL_PRINTF=printf -DVM_TRACE=1 -DVM_COVERAGE=0 -Wno-char-subscripts -Wno-parentheses-equality -Wno-sign-compare -Wno-uninitialized -Wno-unused-but-set-variable -Wno-unused-parameter -Wno-unused-variable -Wno-undefined-bool-conversion -O0 -DTOP_TYPE=VDebouncer -include VDebouncer.h -c -o Debouncer-harness.o Debouncer-harness.cpp
Debouncer-harness.cpp: In member function ‘void Debouncer_api_t::init_sim_data()’:
Debouncer-harness.cpp:23:61: error: ‘class VDebouncer’ has no member named ‘io_in_1’
sim_data.inputs.push_back(new VerilatorCData(&(dut->io_in_1)));
^
Debouncer-harness.cpp:24:61: error: ‘class VDebouncer’ has no member named ‘io_in_2’
sim_data.inputs.push_back(new VerilatorCData(&(dut->io_in_2)));
^
Debouncer-harness.cpp:25:61: error: ‘class VDebouncer’ has no member named ‘io_in_3’
sim_data.inputs.push_back(new VerilatorCData(&(dut->io_in_3)));
^
Debouncer-harness.cpp:26:61: error: ‘class VDebouncer’ has no member named ‘io_in_4’
sim_data.inputs.push_back(new VerilatorCData(&(dut->io_in_4)));
^
# ls
build.sbt Debouncer.v scalastyle-test-config.xml VDebouncer__ALLcls.cpp VDebouncer_classes.mk VDebouncer__Syms.cpp veri_api.h
build.sh Debouncer.vcd sim_api.h VDebouncer__ALLcls.d VDebouncer.cpp VDebouncer__Syms.h verilated.d
c.py log.txt src VDebouncer__ALLcls.o VDebouncer_Debouncer.cpp VDebouncer__Trace.cpp verilated.o
Debouncer-harness.cpp project target VDebouncer__ALLsup.cpp VDebouncer_Debouncer.h VDebouncer__Trace__Slow.cpp verilated_vcd_c.d
Debouncer-harness.d README.md VDebouncer VDebouncer__ALLsup.d VDebouncer.h VDebouncer__ver.d verilated_vcd_c.o
Debouncer-harness.o scalastyle-config.xml VDebouncer__ALL.a VDebouncer__ALLsup.o VDebouncer.mk VDebouncer__verFiles.dat
Here is the relevant portion of the test code:
class DebouncerTester extends ChiselFlatSpec {
behavior of "Debouncer"
def get_options(backend:String): TesterOptionsManager = {
val options = new TesterOptionsManager{
interpreterOptions = interpreterOptions.copy(setVerbose = false)
testerOptions = testerOptions.copy(backendName = backend)
}
options
}
val backendNames = Array[String]("verilator")
for ( backendName <- backendNames ) {
it should s"do a basic 6 count (with ${backendName})" in {
Driver.execute(() => new Debouncer(2, samples=6, percent_stable=5f/6), get_options(backendName)) { c =>
new DebouncerBasic6Count(c)
} should be(true)
}
it should s"stay high with n=1 (with ${backendName})" in {
Driver.execute(() => new Debouncer(1, samples=10), get_options(backendName)) { c =>
new DebouncerStayHigh(c)
} should be(true)
}
it should s"stay high with n=100 (with ${backendName})" in {
Driver.execute(() => new Debouncer(100, samples=10), get_options(backendName)) { c =>
new DebouncerStayHigh(c)
} should be(true)
}
}
}
Currently, chisel-testers uses implementation details to interrogate the DUT graph. We need to provide a suitable API in chisel3.
@jcmartin @donggyukim anyone else -- have you guys run PeekPokeTester w/ the VCS backend? I'm getting a
../vpi.h: In member function 'virtual void vpi_api_t::tick()':
../vpi.h:26: error: 'template<class T> class sim_api_t' used without template parameters
error. Have no idea what's going on. Kind of desperate, since it looks like Verilator has a horrible bug that makes testing my design next to impossible.
Make it clean. Should be set in one consistent way
I haven't had time to produce a small example but this will force me to do so
In the default apply(), TesterOptionsManager is hardcoded in the function and passed to the execute call.
def apply[T <: Module](
dutGen: () => T,
backendType: String = "firrtl",
verbose: Boolean = false,
testerSeed: Long = System.currentTimeMillis())(
testerGen: T => PeekPokeTester[T]): Boolean = {
val optionsManager = new TesterOptionsManager {
testerOptions = testerOptions.copy(backendName = backendType, isVerbose = verbose, testerSeed = testerSeed)
}
execute(dutGen, optionsManager)(testerGen)
}
Now, this is the call to excecute. Observe the use of the DynamicVariable construct and call to optionsManagerVar.withValue(..)
private val optionsManagerVar = new DynamicVariable[Option[TesterOptionsManager]](None)
def optionsManager = optionsManagerVar.value.getOrElse(new TesterOptionsManager)
// [..]
def execute[T <: Module](
dutGenerator: () => T,
optionsManager: TesterOptionsManager
)
(
testerGen: T => PeekPokeTester[T]
): Boolean = {
optionsManagerVar.withValue(Some(optionsManager)) {
if(optionsManager.topName.isEmpty) {
if(optionsManager.targetDirName == ".") {
optionsManager.setTargetDirName("test_run_dir")
}
val genClassName = testerGen.getClass.getName
val testerName = genClassName.split("""\$\$""").headOption.getOrElse("") + genClassName.hashCode.abs
optionsManager.setTargetDirName(s"${optionsManager.targetDirName}/$testerName")
}
val testerOptions = optionsManager.testerOptions
In my test case, I see the passed-in value of optionsManager gets squashed. Here is a code snippet.
val optionsManager = new TesterOptionsManager
optionsManager.setTargetDirName("not_test_run_dir")
optionsManager.setTopName("ChiselTop")
optionsManager.interpreterOptions = new InterpreterOptions(writeVCD=true)
optionsManager.commonOptions = new CommonOptions()
Driver.execute(() => new ChiselTopWrapper(p), optionsManager) { c => new ChiselTopTester(c) }
My questions is: How to use the DyanmicVariable interface? Is this intentional or an omission. I don't see this strategy/pattern used in rocket-core or chisel3.
The code below fails to compile in VCS but works with Verilator and firrtl-interpreter.
package bug
import org.scalatest.{ Matchers, FlatSpec}
import chisel3._
import chisel3.iotesters._
class VCSBug extends Module {
val io = IO(new Bundle)
}
class Tester(c:VCSBug) extends PeekPokeTester(c) {
step(1)
}
class VCSTest extends FlatSpec with Matchers {
behavior of "VCSBug"
it should "work" in {
chisel3.iotesters.Driver( () => new VCSBug, "vcs") { c =>
new Tester( c)
} should be ( true)
}
}
class VerilatorTest extends FlatSpec with Matchers {
behavior of "VCSBug"
it should "work" in {
chisel3.iotesters.Driver( () => new VCSBug, "verilator") { c =>
new Tester( c)
} should be ( true)
}
}
class FirrtlTest extends FlatSpec with Matchers {
behavior of "VCSBug"
it should "work" in {
chisel3.iotesters.Driver( () => new VCSBug, "firrtl") { c =>
new Tester( c)
} should be ( true)
}
}
The trailing comma after the reset binding in the harness is the issue:
module test;
reg clock = 1;
reg reset = 1;
always #`CLOCK_PERIOD clock = ~clock;
reg vcdon = 0;
reg [1023:0] vcdfile = 0;
reg [1023:0] vpdfile = 0;
/*** DUT instantiation ***/
VCSBug VCSBug(
.clock(clock),
.reset(reset),
);
Potential things to refactor:
.
, but this should be a upstream (firrtl) fix where those variables are defined.I am testing a hardware that has streaming IO. From vcd waveform, it looks like that "poke" always happens at negedge. Is there a way to poke at the posedge of the clock?
The code seems to just ignore the second argument to assert.
I am trying to get the IntelliJ debugger working with Chisel and Chisel-Testers. The issue I am facing is a ClassNotFoundException for: org.jetbrains.plugins.scala.testingSupport.scalaTest.ScalaTestRunner
I root caused (I believe) to a dependency on newer scalatest version by IDEA Scala plugin and chisel-testers.
Uprev chisel-testers to latest mater and scalatest version to 3.0.1 results in compile error. Output for sbt compile
follows:
[error] /home/edc/chisel_plus/chisel-testers/src/main/scala/chisel3/iotesters/ChiselSpec.scala:32: overriding value generatorDrivenConfig in trait Configuration of type ChiselPropSpec.this.PropertyCheckConfiguration;
[error] value generatorDrivenConfig has incompatible type
[error] implicit override val generatorDrivenConfig =
[error] ^
[error] one error found
Once time permits, I'll continue to workaround, but until then, can developers take a look?
Verilator fails on lines like this in generated verilog:
$fwrite(32'h80000002,"output test event %d testing mem_out.bits = %d, should be %d\n",_T_12,device_under_test_io_mem_out_bits,_GEN_7);
when device_under_test_io_mem_out_bits width is larger than 64 bits with this message:
%Error: Foo.v:1116: Unsupported: $fwrite of dec format of > 64 bit results (use hex format instead)
This is due to %d formatting used in OrderedDecoupledHWIOTester.scala to generate fwrite messages in verilog. In my local version I changed those to %x and it worked.
Here's the diff, but I am not sure these are the only lines that need fixes.
--- a/src/main/scala/chisel3/iotesters/OrderedDecoupledHWIOTester.scala
+++ b/src/main/scala/chisel3/iotesters/OrderedDecoupledHWIOTester.scala
@@ -315,11 +315,11 @@ abstract class OrderedDecoupledHWIOTester extends HWIOTester {
when(controlling_port.ready && controlling_port.valid) {
ports_referenced_for_this_controlling_port.foreach { port =>
printf(s"output test event %d testing ${name(port)} = %d, should be %d\n",
printf(s"output test event %d testing ${name(port)} = %x, should be %x\n",
event_counter.value, port.asInstanceOf[UInt], port_vector_events(port)(counter_for_this_decoupled.value)
)
when(port.asInstanceOf[UInt] != port_vector_events(port)(counter_for_this_decoupled.value)) {
printf(s"Error: event %d ${name(port)} was %d should be %d\n",
printf(s"Error: event %d ${name(port)} was %x should be %x\n",
event_counter.value, port.asUInt, port_vector_events(port)(counter_for_this_decoupled.value))
assert(false.B)
stop()
Hey @edwardcwang, you closed #60, but I observe the issue is there there on master branch.
In the thread, you referenced this commit by @chick that removed the call to chisel3.Driver.createTempDirectory in file Driver.scala:
ucb-bar@ed1a1d4
If the call to createTempDirectory is the workaround, it has not been restored.
The issue seen in #60 is trivial to reproduce. Make repeated calls to your tester configured for verilator backend.
class ArbiterPeekPokeTester[T <: Data](c: Arbiter[T]) extends PeekPokeTester(c) {
(1 to 100) foreach (x => step(1))
}
it should "elaborate" in {
for (backend <- "verilator firrtl".split(" ")) {
for (n <- List(2, 4, 8, 16, 32)) {
Driver(() => new Arbiter[UInt](n = n), backendType = backend) { c =>
new ArbiterPeekPokeTester[UInt](c)
}
//[..]
I've tried to do some parameterized design in chisel3.
I've defined two tests which use testbenches and implemention of the same module but with different parameters. It seems that the implementation in the second test is not elaborated (caused the implementation with different/wrong parameters to be used.)
Here is a complete file that will expose this issue.
package bug
import chisel3._
import chisel3.iotesters._
import org.scalatest.GivenWhenThen
import scala.runtime.ScalaRunTime.stringOf
class CarryChainIfc(N:Int,width:Int) extends Module {
val io = new Bundle {
val a = Vec( N, UInt(INPUT,width=width))
val co = Vec( N, UInt(OUTPUT,width=width))
}
}
class ORChain(N:Int) extends CarryChainIfc(N,1) {
require( io.a.length == io.co.length)
require( io.a.length == N)
io.co(0) := io.a(0)
for( ((a,ci),co)<-io.a.toList.tail.zip( io.co).zip( io.co.tail)) {
co := a | ci
}
}
abstract class Action
case class ActionInputArray(val nm:String, val s:Array[BigInt]) extends Action
case class ActionOutputArray(val nm:String, val s:Array[BigInt]) extends Action
case class ActionDelay(val v:Int) extends Action
class CarryChainTests[T <: CarryChainIfc]( ActList:List[Action], c: T) extends AdvTester(c,verbose=true)
{
for (a <- ActList) {
a match {
case ActionInputArray("a",s) => {
require( s.length == c.io.a.length)
for ( (f,a) <- c.io.a.zip( s)) {
reg_poke( f, a)
}
}
case ActionOutputArray("co",s) => {
require( s.length == c.io.co.length)
for ( (f,a) <- c.io.co.zip( s)) {
expect( f, a)
}
}
case ActionDelay(v) => takesteps(1)()
}
}
}
class CarryChainTester[ +T <: CarryChainIfc]( val dut: () => T) extends ChiselFlatSpec with GivenWhenThen {
// val backend = "firrtl"
val backend = "verilator"
val action_buffer= new collection.mutable.ListBuffer[Action]()
def InputArray( nm:String, tag:String, s:Array[BigInt]):String = {
action_buffer += new ActionInputArray(nm,s)
s"InputArray $nm $tag ${stringOf(s)}"
}
def OutputArray( nm:String, tag:String, s:Array[BigInt]):String = {
action_buffer += new ActionOutputArray(nm,s)
s"OutputArray $nm $tag ${stringOf(s)}"
}
def Delay(v:Int):String = {
action_buffer += new ActionDelay(v)
s"After $v cycle" + (if ( v != 1) "s" else "")
}
def init():Unit = action_buffer.clear
def run():Boolean = {
chisel3.iotesters.Driver(dut,backend) {
(component) =>
new CarryChainTests( action_buffer.toList,component)
}
}
}
class ORChainTester(val N:Int) extends CarryChainTester( ()=>new ORChain(N)) {
def v(bin: String): Array[BigInt] = {
assert( N === bin.length)
bin.toList.map( "01".indexOf(_)).map( BigInt(_)).reverse.toArray
}
}
class ORChainTest2 extends ORChainTester(2) {
s"ORChain of size ${N}" should s"work with various inputs" in {
init()
Given("a ORChain module")
for ( (a,co) <- List( (v("00"),v("00")),
(v("01"),v("11")),
(v("10"),v("10")),
(v("11"),v("11")))) {
When(InputArray("a","is",a))
And(Delay(1))
Then(OutputArray("co","is",co))
}
run() should be (true)
}
}
class ORChainTest4 extends ORChainTester(4) {
s"ORChain of size ${N}" should s"work with various inputs" in {
init()
Given("a ORChain module")
for ( (a,co) <- List( (v("0000"),v("0000")),
(v("0001"),v("1111")),
(v("0010"),v("1110")),
(v("0100"),v("1100")),
(v("1000"),v("1000")))) {
When(InputArray("a","is",a))
And(Delay(1))
Then(OutputArray("co","is",co))
}
run() should be (true)
}
}
I get this LOG file with a compiler error when the second module is compiled (it then continues to run the wrong test.)
smburns@smburns-VirtualBox:~/bug-parameterized-dut$ sbt test
[info] Set current project to SCL PPT Testing Experiments (in build file:/home/smburns/bug-parameterized-dut/)
[info] [0.004] Elaborating design...
[info] [0.287] Done elaborating.
verilator --cc ORChain.v --assert -Wno-fatal -Wno-WIDTH -Wno-STMTDLY --trace -O2 --top-module ORChain +define+TOP_TYPE=VORChain +define+PRINTF_COND=!ORChain.reset +define+STOP_COND=!ORChain.reset -CFLAGS -Wno-undefined-bool-conversion -O2 -DTOP_TYPE=VORChain -include VORChain.h -Mdir test_run_dir/bug.ORChain --exe ORChain-harness.cpp
make: Entering directory `/home/smburns/bug-parameterized-dut/test_run_dir/bug.ORChain'
g++ -I. -MMD -I/usr/local/share/verilator/include -I/usr/local/share/verilator/include/vltstd -DVL_PRINTF=printf -DVM_TRACE=1 -DVM_COVERAGE=0 -Wno-char-subscripts -Wno-parentheses-equality -Wno-sign-compare -Wno-uninitialized -Wno-unused-but-set-variable -Wno-unused-parameter -Wno-unused-variable -Wno-undefined-bool-conversion -O2 -DTOP_TYPE=VORChain -include VORChain.h -c -o ORChain-harness.o ORChain-harness.cpp
g++ -I. -MMD -I/usr/local/share/verilator/include -I/usr/local/share/verilator/include/vltstd -DVL_PRINTF=printf -DVM_TRACE=1 -DVM_COVERAGE=0 -Wno-char-subscripts -Wno-parentheses-equality -Wno-sign-compare -Wno-uninitialized -Wno-unused-but-set-variable -Wno-unused-parameter -Wno-unused-variable -Wno-undefined-bool-conversion -O2 -DTOP_TYPE=VORChain -include VORChain.h -c -o verilated.o /usr/local/share/verilator/include/verilated.cpp
g++ -I. -MMD -I/usr/local/share/verilator/include -I/usr/local/share/verilator/include/vltstd -DVL_PRINTF=printf -DVM_TRACE=1 -DVM_COVERAGE=0 -Wno-char-subscripts -Wno-parentheses-equality -Wno-sign-compare -Wno-uninitialized -Wno-unused-but-set-variable -Wno-unused-parameter -Wno-unused-variable -Wno-undefined-bool-conversion -O2 -DTOP_TYPE=VORChain -include VORChain.h -c -o verilated_vcd_c.o /usr/local/share/verilator/include/verilated_vcd_c.cpp
/usr/bin/perl /usr/local/share/verilator/bin/verilator_includer -DVL_INCLUDE_OPT=include VORChain.cpp > VORChain__ALLcls.cpp
/usr/bin/perl /usr/local/share/verilator/bin/verilator_includer -DVL_INCLUDE_OPT=include VORChain__Trace.cpp VORChain__Syms.cpp VORChain__Trace__Slow.cpp > VORChain__ALLsup.cpp
g++ -I. -MMD -I/usr/local/share/verilator/include -I/usr/local/share/verilator/include/vltstd -DVL_PRINTF=printf -DVM_TRACE=1 -DVM_COVERAGE=0 -Wno-char-subscripts -Wno-parentheses-equality -Wno-sign-compare -Wno-uninitialized -Wno-unused-but-set-variable -Wno-unused-parameter -Wno-unused-variable -Wno-undefined-bool-conversion -O2 -DTOP_TYPE=VORChain -include VORChain.h -c -o VORChain__ALLcls.o VORChain__ALLcls.cpp
g++ -I. -MMD -I/usr/local/share/verilator/include -I/usr/local/share/verilator/include/vltstd -DVL_PRINTF=printf -DVM_TRACE=1 -DVM_COVERAGE=0 -Wno-char-subscripts -Wno-parentheses-equality -Wno-sign-compare -Wno-uninitialized -Wno-unused-but-set-variable -Wno-unused-parameter -Wno-unused-variable -Wno-undefined-bool-conversion -O2 -DTOP_TYPE=VORChain -include VORChain.h -c -o VORChain__ALLsup.o VORChain__ALLsup.cpp
Archiving VORChain__ALL.a ...
ar r VORChain__ALL.a VORChain__ALLcls.o VORChain__ALLsup.o
ar: creating VORChain__ALL.a
ranlib VORChain__ALL.a
g++ ORChain-harness.o verilated.o verilated_vcd_c.o VORChain__ALL.a -o VORChain -lm -lstdc++ 2>&1 | c++filt
make: Leaving directory `/home/smburns/bug-parameterized-dut/test_run_dir/bug.ORChain'
sim start on smburns-VirtualBox at Fri Sep 16 15:03:52 2016
inChannelName: 00009370.in
outChannelName: 00009370.out
cmdChannelName: 00009370.cmd
STARTING test_run_dir/bug.ORChain/VORChain
SEED 1474063432674
STEP 0 -> 1
POKE ORChain.io_a_1 <- 0x0
POKE ORChain.io_a_0 <- 0x0
EXPECT ORChain.io_co_0 -> 0x0 == 0x0 PASS
EXPECT ORChain.io_co_1 -> 0x0 == 0x0 PASS
STEP 1 -> 2
POKE ORChain.io_a_1 <- 0x0
POKE ORChain.io_a_0 <- 0x1
EXPECT ORChain.io_co_0 -> 0x1 == 0x1 PASS
EXPECT ORChain.io_co_1 -> 0x1 == 0x1 PASS
STEP 2 -> 3
POKE ORChain.io_a_1 <- 0x1
POKE ORChain.io_a_0 <- 0x0
EXPECT ORChain.io_co_0 -> 0x0 == 0x0 PASS
EXPECT ORChain.io_co_1 -> 0x1 == 0x1 PASS
STEP 3 -> 4
POKE ORChain.io_a_1 <- 0x1
POKE ORChain.io_a_0 <- 0x1
EXPECT ORChain.io_co_0 -> 0x1 == 0x1 PASS
EXPECT ORChain.io_co_1 -> 0x1 == 0x1 PASS
Enabling waves..
RAN 4 CYCLES PASSED
[info] ORChainTest2:
[info] ORChain of size 2
[info] - should work with various inputs
[info] + Given a ORChain module
[info] + When InputArray a is Array(0, 0)
[info] + And After 1 cycle
[info] + Then OutputArray co is Array(0, 0)
[info] + When InputArray a is Array(1, 0)
[info] + And After 1 cycle
[info] + Then OutputArray co is Array(1, 1)
[info] + When InputArray a is Array(0, 1)
[info] + And After 1 cycle
[info] + Then OutputArray co is Array(0, 1)
[info] + When InputArray a is Array(1, 1)
[info] + And After 1 cycle
[info] + Then OutputArray co is Array(1, 1)
[info] [0.000] Elaborating design...
[info] [0.017] Done elaborating.
verilator --cc ORChain.v --assert -Wno-fatal -Wno-WIDTH -Wno-STMTDLY --trace -O2 --top-module ORChain +define+TOP_TYPE=VORChain +define+PRINTF_COND=!ORChain.reset +define+STOP_COND=!ORChain.reset -CFLAGS -Wno-undefined-bool-conversion -O2 -DTOP_TYPE=VORChain -include VORChain.h -Mdir test_run_dir/bug.ORChain --exe ORChain-harness.cpp
make: Entering directory `/home/smburns/bug-parameterized-dut/test_run_dir/bug.ORChain'
g++ -I. -MMD -I/usr/local/share/verilator/include -I/usr/local/share/verilator/include/vltstd -DVL_PRINTF=printf -DVM_TRACE=1 -DVM_COVERAGE=0 -Wno-char-subscripts -Wno-parentheses-equality -Wno-sign-compare -Wno-uninitialized -Wno-unused-but-set-variable -Wno-unused-parameter -Wno-unused-variable -Wno-undefined-bool-conversion -O2 -DTOP_TYPE=VORChain -include VORChain.h -c -o ORChain-harness.o ORChain-harness.cpp
ORChain-harness.cpp: In member function ‘void ORChain_api_t::init_sim_data()’:
ORChain-harness.cpp:24:61: error: ‘class VORChain’ has no member named ‘io_a_2’
sim_data.inputs.push_back(new VerilatorCData(&(dut->io_a_2)));
^
ORChain-harness.cpp:25:61: error: ‘class VORChain’ has no member named ‘io_a_3’
sim_data.inputs.push_back(new VerilatorCData(&(dut->io_a_3)));
^
ORChain-harness.cpp:28:62: error: ‘class VORChain’ has no member named ‘io_co_2’
sim_data.outputs.push_back(new VerilatorCData(&(dut->io_co_2)));
^
ORChain-harness.cpp:29:62: error: ‘class VORChain’ has no member named ‘io_co_3’
sim_data.outputs.push_back(new VerilatorCData(&(dut->io_co_3)));
^
At global scope:
cc1plus: warning: unrecognized command line option "-Wno-undefined-bool-conversion" [enabled by default]
cc1plus: warning: unrecognized command line option "-Wno-parentheses-equality" [enabled by default]
make: *** [ORChain-harness.o] Error 1
make: Leaving directory `/home/smburns/bug-parameterized-dut/test_run_dir/bug.ORChain'
sim start on smburns-VirtualBox at Fri Sep 16 15:03:54 2016
inChannelName: 00009388.in
outChannelName: 00009388.out
cmdChannelName: 00009388.cmd
STARTING test_run_dir/bug.ORChain/VORChain
SEED 1474063434297
STEP 0 -> 1
POKE ORChain.io_a_1 <- 0x0
POKE ORChain.io_a_0 <- 0x0
POKE ORChain.io_a_3 <- 0x0
POKE ORChain.io_a_2 <- 0x0
EXPECT ORChain.io_co_0 -> 0x0 == 0x0 PASS
EXPECT ORChain.io_co_1 -> 0x0 == 0x0 PASS
EXPECT ORChain.io_co_2 -> 0x0 == 0x0 PASS
EXPECT ORChain.io_co_3 -> 0x0 == 0x0 PASS
STEP 1 -> 2
POKE ORChain.io_a_1 <- 0x0
POKE ORChain.io_a_0 <- 0x1
POKE ORChain.io_a_3 <- 0x0
POKE ORChain.io_a_2 <- 0x0
EXPECT ORChain.io_co_0 -> 0x1 == 0x1 PASS
EXPECT ORChain.io_co_1 -> 0x1 == 0x1 PASS
EXPECT ORChain.io_co_2 -> 0x0 == 0x1 FAIL
EXPECT ORChain.io_co_3 -> 0x0 == 0x1 FAIL
STEP 2 -> 3
POKE ORChain.io_a_1 <- 0x1
POKE ORChain.io_a_0 <- 0x0
POKE ORChain.io_a_3 <- 0x0
POKE ORChain.io_a_2 <- 0x0
EXPECT ORChain.io_co_0 -> 0x0 == 0x0 PASS
EXPECT ORChain.io_co_1 -> 0x1 == 0x1 PASS
EXPECT ORChain.io_co_2 -> 0x0 == 0x1 FAIL
EXPECT ORChain.io_co_3 -> 0x0 == 0x1 FAIL
STEP 3 -> 4
POKE ORChain.io_a_1 <- 0x0
POKE ORChain.io_a_0 <- 0x0
POKE ORChain.io_a_3 <- 0x0
POKE ORChain.io_a_2 <- 0x1
EXPECT ORChain.io_co_0 -> 0x0 == 0x0 PASS
EXPECT ORChain.io_co_1 -> 0x0 == 0x0 PASS
EXPECT ORChain.io_co_2 -> 0x0 == 0x1 FAIL
EXPECT ORChain.io_co_3 -> 0x0 == 0x1 FAIL
STEP 4 -> 5
POKE ORChain.io_a_1 <- 0x0
POKE ORChain.io_a_0 <- 0x0
POKE ORChain.io_a_3 <- 0x1
POKE ORChain.io_a_2 <- 0x0
EXPECT ORChain.io_co_0 -> 0x0 == 0x0 PASS
EXPECT ORChain.io_co_1 -> 0x0 == 0x0 PASS
EXPECT ORChain.io_co_2 -> 0x0 == 0x0 PASS
EXPECT ORChain.io_co_3 -> 0x0 == 0x1 FAIL
Enabling waves..
RAN 5 CYCLES FAILED FIRST AT CYCLE 2
[info] ORChainTest4:
[info] ORChain of size 4
[info] - should work with various inputs *** FAILED ***
[info] false was not true (bug.scala:116)
[info] + Given a ORChain module
[info] + When InputArray a is Array(0, 0, 0, 0)
[info] + And After 1 cycle
[info] + Then OutputArray co is Array(0, 0, 0, 0)
[info] + When InputArray a is Array(1, 0, 0, 0)
[info] + And After 1 cycle
[info] + Then OutputArray co is Array(1, 1, 1, 1)
[info] + When InputArray a is Array(0, 1, 0, 0)
[info] + And After 1 cycle
[info] + Then OutputArray co is Array(0, 1, 1, 1)
[info] + When InputArray a is Array(0, 0, 1, 0)
[info] + And After 1 cycle
[info] + Then OutputArray co is Array(0, 0, 1, 1)
[info] + When InputArray a is Array(0, 0, 0, 1)
[info] + And After 1 cycle
[info] + Then OutputArray co is Array(0, 0, 0, 1)
[info] ScalaCheck
[info] Passed: Total 0, Failed 0, Errors 0, Passed 0
[info] ScalaTest
[info] Run completed in 11 seconds, 89 milliseconds.
[info] Total number of tests run: 2
[info] Suites: completed 2, aborted 0
[info] Tests: succeeded 1, failed 1, canceled 0, ignored 0, pending 0
[info] *** 1 TEST FAILED ***
[error] Failed: Total 2, Failed 1, Errors 0, Passed 1
[error] Failed tests:
[error] bug.ORChainTest4
[error] (test:test) sbt.TestsFailedException: Tests unsuccessful
[error] Total time: 14 s, completed Sep 16, 2016 3:03:55 PM
chisel-testers revision d4f29ce
firrtl revision chipsalliance/firrtl@e571ef8
After running sbt publish-local on firrtl, chisel3, firrtl-interpreter in that order, trying to build chisel-testers with "sbt publish-local" results in the following error:
[info] Set current project to Chisel.iotesters (in build file:/nscratch/edwardw/chisel-testers/)
[info] Compiling 19 Scala sources to /nscratch/edwardw/chisel-testers/target/scala-2.11/classes...
[info] Main Scala API documentation to /nscratch/edwardw/chisel-testers/target/scala-2.11/api...
[info] :: delivering :: edu.berkeley.cs#chisel-iotesters_2.11;1.2-SNAPSHOT :: 1.2-SNAPSHOT :: integration :: Fri Mar 10 02:54:52 PST 2017
[info] delivering ivy file to /nscratch/edwardw/chisel-testers/target/scala-2.11/ivy-1.2-SNAPSHOT.xml
[info] Wrote /nscratch/edwardw/chisel-testers/target/scala-2.11/chisel-iotesters_2.11-1.2-SNAPSHOT.pom
[error] /nscratch/edwardw/chisel-testers/src/main/scala/chisel3/iotesters/ChiselMain.scala:112: class FirrtlEmitter is abstract; cannot be instantiated
[error] (new firrtl.FirrtlEmitter).emit(firrtl.CircuitState(chirrtl, firrtl.ChirrtlForm), writer)
[error] ^
[error] /nscratch/edwardw/chisel-testers/src/main/scala/chisel3/iotesters/ChiselMain.scala:112: class FirrtlEmitter is abstract; cannot be instantiated
[error] (new firrtl.FirrtlEmitter).emit(firrtl.CircuitState(chirrtl, firrtl.ChirrtlForm), writer)
[error] ^
[error] one error found
Consider automatically or with some sort of api, re-application of tests after asserting reset.
If there are multiple ways of applying reset, try them.
Make this the default behavior, but provide means to turn it off if necessary
class LinkNode extends Bundle {
val next = UInt(64.W)
val count = UInt(32.W)
override def cloneType = (new LinkNode).asInstanceOf[this.type]
}
See code below for which simulation log reports:
In Bla 96
[info] [0.001] In Foo 32
class Foo (bla: Bla) extends PeekPokeTester(bla) {
println(s"In Foo ${(new LinkNode).getWidth}")
}
class Bla extends Module {
val io = IO (new Bundle())
println(s"In Bla ${(new LinkNode).getWidth}")
}
class SorterAccTester extends ChiselFlatSpec {
"FooBla" should "simulate" in {
chisel3.iotesters.Driver(() => new Bla,"firrtl"){ c=>
new Foo(c)
}should be(true)
}
}
I have something like [again, how to setup options would be nice]:
val options = new TesterOptionsManager {
testerOptions = TesterOptions(
isVerbose = true,
backendName = "verilator",
// TODO: Figure out what isGenVerilog really does
isGenVerilog = true)
}
dsptools.Driver.execute(() => new CordicStage(p, 0 , 1), options) { c =>
new CordicTester(c)
} should be (true)
It's definitely using Verilator instead of firrtl-interpreter, but it's not printing out my peeks/pokes/expects. For dspPeek,dspPoke,dspExpect, it prints out dspExpect but not dspP*.
Ok, at least with DSPTester, _verbose is definitely still false...
Edit 2: _verbose comes from iotesters.Driver's (private) optionsManagerVar, which is never set.......
Is there any problem with Chisel.testers._ ?
chipsalliance/chisel@2e6444c causes:
[error] .../ucb-bar/chisel-testers/src/main/scala/chisel3/iotesters/OrderedDecoupledHWIOTester.scala:394: type mismatch;
[error] found : chisel3.core.Record
[error] required: chisel3.Bundle
[error] (which expands to) chisel3.core.Bundle
[error] io_info = new IOAccessor(device_under_test.io)
[error] ^
[error] .../ucb-bar/chisel-testers/src/main/scala/chisel3/iotesters/SteppedHWIOTester.scala:172: type mismatch;
[error] found : chisel3.core.Record
[error] required: chisel3.Bundle
[error] (which expands to) chisel3.core.Bundle
[error] io_info = new IOAccessor(device_under_test.io)
EDIT: Ok, apparently it's not just this change... very confused.
Problem seems to be caused by this line in VerilatorBackend (and maybe an equivalent in VCS??):
optionsManager.chiselOptions = optionsManager.chiselOptions.copy(
runFirrtlCompiler = false)
What does this do??? If I remove this line, my tests pass.
Original e-mail I sent to @chick:
I pulled in a bunch of changes and tried running sbt test in dsptools, but now I'm getting a really weird bug that I can't for the life of me figure out.
Basically, all of the branches, etc. I'm using are at @ https://github.com/ucb-art/Chisel3DSPDependencies
In my case, when I run sbt-test, I get 2 errors. 1 on ParameterizedAdder (which should fail b/c I removed Saturating) and 2 on my TBSpec. Jenny, who's also using this repo, gets something like 4-5 failures...
I think it has something to do with concurrency issues? But I just can't pinpoint what's going on.
If I run sbt "test-only SimpleTB.SimpleTBSpec"
It fails on two tests:
and
This is super weird b/c it only fails on gen types (stuff I pass in as type parameters), and only when gen = some FixedPoint. Even weirder is the fact that the test after the last mentioned one doesn't fail, even though I still use gen = fixed. In that case, the only difference is that I've removed DspReal from the module. And weirder? Is that when I comment out all of the other tests except these two and rerun, things pass...
Of course, this is only an issue with Verilator... but this is driving me crazy. Any idea what's up?
We're trying to place runtime asserts in mocked implementation code. The following idea works using firrtl-interpreter and SteppedHWIOTester, but not with the PeekPokeTester due to a missing symbol.
g++ myRisc-harness.o verilated.o verilated_vcd_c.o VmyRisc__ALL.a -o VmyRisc -lm -lstdc++ 2>&1 | c++filt
VmyRisc__ALL.a(VmyRisc__ALLcls.o): In function `VmyRisc::_sequent__TOP__3(VmyRisc__Syms*)':
VmyRisc__ALLcls.cpp:(.text+0x1ca): undefined reference to `sc_time_stamp()'
collect2: error: ld returned 1 exit status
Here is the source code and the complete log file.
package myRisc
import chisel3._
class mockRegFileOut() extends Module{
val io = new Bundle {
val addr = Bits(INPUT, 8)
val DataO = Bits(OUTPUT, 32)
}
assert(io.addr === UInt(0),"ASSERT:Read at the wrong Register")
when(io.addr === UInt(0)){
io.DataO := UInt(128)
}
}
class myRisc extends Module{
val io = new Bundle {
val wrData = Bits(INPUT, 32)
val valid = Bool(OUTPUT)
val out = Bits(OUTPUT, 32)
}
val inst = io.wrData
//instantiate mock
val mock = Module(new mockRegFileOut).io
mock.addr := UInt(16)//inst(15, 8)
// Comment the next line to see a mocking failure
mock.addr := inst(15, 8)
val rci = inst(23,16)
when (rci === UInt(255)){
io.valid := Bool(true)
io.out := mock.DataO
}.otherwise{
io.valid := Bool(false)
}
}
import org.scalatest.{ Matchers, FlatSpec, GivenWhenThen}
import chisel3.iotesters._
class myRiscPeekPokeTester( val c:myRisc) extends PeekPokeTester(c) {
poke( c.io.wrData, (0<<24)+(255<<16))
step(1)
expect( c.io.valid, 1)
expect( c.io.out, 128)
}
class myRiscPeekPokeTest extends FlatSpec with Matchers {
behavior of "myRiscPeekPokeTest"
it should s"work" in {
chisel3.iotesters.Driver((() => new myRisc),"verilator") { c => new myRiscPeekPokeTester(c) } should be (true)
}
}
The output of "sbt test":
> test
[info] [0.002] Elaborating design...
[info] [0.073] Done elaborating.
verilator --cc myRisc.v --assert -Wno-fatal -Wno-WIDTH -Wno-STMTDLY --trace -O2 --top-module myRisc +define+TOP_TYPE=VmyRisc +define+PRINTF_COND=!myRisc.reset +define+STOP_COND=!myRisc.reset -CFLAGS -Wno-undefined-bool-conversion -O2 -DTOP_TYPE=VmyRisc -include VmyRisc.h -Mdir test_run_dir/myRisc.myRisc --exe myRisc-harness.cpp
make: Entering directory `/nfs/site/disks/scl.work.59/ppt/smburns/tdd_using_chisel-testing_flows/bugs/bug-sc_time_stamp/test_run_dir/myRisc.myRisc'
g++ -I. -MMD -I/nfs/site/disks/scl.work.59/ppt/smburns/chisel/chisel3/verilator_install/share/verilator/include -I/nfs/site/disks/scl.work.59/ppt/smburns/chisel/chisel3/verilator_install/share/verilator/include/vltstd -DVL_PRINTF=printf -DVM_TRACE=1 -DVM_COVERAGE=0 -Wno-char-subscripts -Wno-parentheses-equality -Wno-sign-compare -Wno-uninitialized -Wno-unused-but-set-variable -Wno-unused-parameter -Wno-unused-variable -Wno-undefined-bool-conversion -O2 -DTOP_TYPE=VmyRisc -include VmyRisc.h -c -o myRisc-harness.o myRisc-harness.cpp
g++ -I. -MMD -I/nfs/site/disks/scl.work.59/ppt/smburns/chisel/chisel3/verilator_install/share/verilator/include -I/nfs/site/disks/scl.work.59/ppt/smburns/chisel/chisel3/verilator_install/share/verilator/include/vltstd -DVL_PRINTF=printf -DVM_TRACE=1 -DVM_COVERAGE=0 -Wno-char-subscripts -Wno-parentheses-equality -Wno-sign-compare -Wno-uninitialized -Wno-unused-but-set-variable -Wno-unused-parameter -Wno-unused-variable -Wno-undefined-bool-conversion -O2 -DTOP_TYPE=VmyRisc -include VmyRisc.h -c -o verilated.o /nfs/site/disks/scl.work.59/ppt/smburns/chisel/chisel3/verilator_install/share/verilator/include/verilated.cpp
g++ -I. -MMD -I/nfs/site/disks/scl.work.59/ppt/smburns/chisel/chisel3/verilator_install/share/verilator/include -I/nfs/site/disks/scl.work.59/ppt/smburns/chisel/chisel3/verilator_install/share/verilator/include/vltstd -DVL_PRINTF=printf -DVM_TRACE=1 -DVM_COVERAGE=0 -Wno-char-subscripts -Wno-parentheses-equality -Wno-sign-compare -Wno-uninitialized -Wno-unused-but-set-variable -Wno-unused-parameter -Wno-unused-variable -Wno-undefined-bool-conversion -O2 -DTOP_TYPE=VmyRisc -include VmyRisc.h -c -o verilated_vcd_c.o /nfs/site/disks/scl.work.59/ppt/smburns/chisel/chisel3/verilator_install/share/verilator/include/verilated_vcd_c.cpp
/usr/intel/bin/perl /nfs/site/disks/scl.work.59/ppt/smburns/chisel/chisel3/verilator_install/share/verilator/bin/verilator_includer -DVL_INCLUDE_OPT=include VmyRisc.cpp > VmyRisc__ALLcls.cpp
/usr/intel/bin/perl /nfs/site/disks/scl.work.59/ppt/smburns/chisel/chisel3/verilator_install/share/verilator/bin/verilator_includer -DVL_INCLUDE_OPT=include VmyRisc__Trace.cpp VmyRisc__Syms.cpp VmyRisc__Trace__Slow.cpp > VmyRisc__ALLsup.cpp
g++ -I. -MMD -I/nfs/site/disks/scl.work.59/ppt/smburns/chisel/chisel3/verilator_install/share/verilator/include -I/nfs/site/disks/scl.work.59/ppt/smburns/chisel/chisel3/verilator_install/share/verilator/include/vltstd -DVL_PRINTF=printf -DVM_TRACE=1 -DVM_COVERAGE=0 -Wno-char-subscripts -Wno-parentheses-equality -Wno-sign-compare -Wno-uninitialized -Wno-unused-but-set-variable -Wno-unused-parameter -Wno-unused-variable -Wno-undefined-bool-conversion -O2 -DTOP_TYPE=VmyRisc -include VmyRisc.h -c -o VmyRisc__ALLsup.o VmyRisc__ALLsup.cpp
g++ -I. -MMD -I/nfs/site/disks/scl.work.59/ppt/smburns/chisel/chisel3/verilator_install/share/verilator/include -I/nfs/site/disks/scl.work.59/ppt/smburns/chisel/chisel3/verilator_install/share/verilator/include/vltstd -DVL_PRINTF=printf -DVM_TRACE=1 -DVM_COVERAGE=0 -Wno-char-subscripts -Wno-parentheses-equality -Wno-sign-compare -Wno-uninitialized -Wno-unused-but-set-variable -Wno-unused-parameter -Wno-unused-variable -Wno-undefined-bool-conversion -O2 -DTOP_TYPE=VmyRisc -include VmyRisc.h -c -o VmyRisc__ALLcls.o VmyRisc__ALLcls.cpp
Archiving VmyRisc__ALL.a ...
ar r VmyRisc__ALL.a VmyRisc__ALLcls.o VmyRisc__ALLsup.o
ranlib VmyRisc__ALL.a
g++ myRisc-harness.o verilated.o verilated_vcd_c.o VmyRisc__ALL.a -o VmyRisc -lm -lstdc++ 2>&1 | c++filt
VmyRisc__ALL.a(VmyRisc__ALLcls.o): In function `VmyRisc::_sequent__TOP__3(VmyRisc__Syms*)':
VmyRisc__ALLcls.cpp:(.text+0x1ca): undefined reference to `sc_time_stamp()'
collect2: error: ld returned 1 exit status
make: Leaving directory `/nfs/site/disks/scl.work.59/ppt/smburns/tdd_using_chisel-testing_flows/bugs/bug-sc_time_stamp/test_run_dir/myRisc.myRisc'
[info] myRiscPeekPokeTest:
[info] myRiscPeekPokeTest
[info] - should work *** FAILED ***
[info] java.lang.IllegalArgumentException: requirement failed: test_run_dir/myRisc.myRisc/VmyRisc doesn't exists
[info] at scala.Predef$.require(Predef.scala:219)
[info] at chisel3.iotesters.TesterProcess$.apply(PeekPokeTesterUtils.scala:112)
[info] at chisel3.iotesters.SimApiInterface.<init>(SimApiInterface.scala:278)
[info] at chisel3.iotesters.VerilatorBackend.<init>(VerilatorBackend.scala:290)
[info] at chisel3.iotesters.setupVerilatorBackend$.apply(VerilatorBackend.scala:282)
[info] at chisel3.iotesters.Driver$.apply(Driver.scala:22)
[info] at myRisc.myRiscPeekPokeTest$$anonfun$4.apply$mcV$sp(t.scala:62)
[info] at myRisc.myRiscPeekPokeTest$$anonfun$4.apply(t.scala:62)
[info] at myRisc.myRiscPeekPokeTest$$anonfun$4.apply(t.scala:62)
[info] at org.scalatest.Transformer$$anonfun$apply$1.apply$mcV$sp(Transformer.scala:22)
[info] ...
[info] ScalaCheck
[info] Passed: Total 0, Failed 0, Errors 0, Passed 0
[info] ScalaTest
[info] Run completed in 3 seconds, 1 milliseconds.
[info] Total number of tests run: 1
[info] Suites: completed 1, aborted 0
[info] Tests: succeeded 0, failed 1, canceled 0, ignored 0, pending 0
[info] *** 1 TEST FAILED ***
[error] Failed: Total 1, Failed 1, Errors 0, Passed 0
[error] Failed tests:
[error] myRisc.myRiscPeekPokeTest
[error] (test:test) sbt.TestsFailedException: Tests unsuccessful
[error] Total time: 3 s, completed Sep 21, 2016 11:50:06 AM
The current tester API for aggregates needs lots of work.
Proposal:
Find a way to support Bundle and Vector Literals.
To fix or eliminate
Current poking of Vecs assigns values from an seq: IndexedSeq[BigInt] into the Vec in reverse order, i.e
for a Vec sized 10
vec(0) <= seq(9)
...
vec(9) <= seq(0)
Yet peeking from that Vec does not reverse the elements, peek and poke are asymmetric.
seq(0) <= vec(0), etc.
When poking bundles, the order seems to be correct, i.e. the first element of the seq goes into the first element of the bundle. The peek of a Bundle returns and Map of strings to BigInts, another asymmetry between peek and poke.
Users may have worked around this but it seems worth breaking the current API to fix it.
This all gets worse when Vecs and Bundles are nested.
Currently the users should write their own synthetic peek and poke routines and not use the existing API. Aggregate literals would seem to be the most elegant
There's no signed conversion, so everything comes out as unsigned BigInts, I believe [tho I'm testing w/ something that extends PeekPokeTester, but I believe this is the current behavior]. For Dsp checks, this kind of matters. I can either do the signed conversion here [to be consistent with Chisel2] or within my own DspTester, but doing it here is somewhat of an API change for people. Does anyone have any opinions? If I do it at a higher level, I feel like I'm just adding patch upon patch to things... but I also need some of these changes ASAP.
@ducky64 @chick @donggyukim /whoever else might be in charge of testers...
I have some PeekPokeTester
s for rocket-chip LazyModuleImp
s. Chisel has refactored what Module
points to and added a whole hierarchy. I don't really understand what all the different modules are meant to be, but evidently LazyModuleImp
is not a Module
(I think it is an ImplicitModule
?) so I can't use PeekPokeTester
s on it anymore.
I think the fix for this is simple- wherever we have a T <: Module
should be replaced with T <: BaseModule
or UserModule
or whatever the appropriate Module
type is. @chick, do you know what the appropriate Module
class is?
The various backends share quite a bit of common code. It would be good to refactor this before they diverge.
Previously, in Chisel 2, it was very easy to peek/poke Vecs that were contained within a Bundle. However, in Chisel 3, that doesn't seem to be the case. In seems in this version of Peek/Poke, it expects that the entities in a bundle inherit only from the Data class, not Aggregate classes. Therefore, if I had a Bundle with a Vec as a sub-entity, I would not be able to poke or peek into that Bundle properly.
This is problem because in our project: our parameterized interface uses Vecs within Bundles on our inputs and outputs. This is preventing us from upgrading our project from Chisel 2 to Chisel 3.
I noticed some mention of this issue in #77, but there doesn't seem to have been any update to that.
Also, there was a student (@dt27182) who worked on a fix for this within the chisel-tester framework. It seems all of the work is contained within this branch.
I'd be more than happy to upstream the code and put in a pull request...
If, in certain conditions, it's not allowed, I wouldn't mind having Verilator fail... but otherwise, even if an inout port (of a blackbox) is actually just the sume of two inputs, I can't actually probe that it's right. Basically, I want to sanity check that connections are made correctly, and I can't do that in simulation.
behavior of "simple module io + lits"
it should "read lits with gen = sint properly" in {
dsptools.Driver.execute(() => new SimpleModule(p.genShortS, p.genLongS, includeR = true, p), optionsPass) { c =>
new LitTester(c)
} should be (true)
}
it should "read lits with gen = fixed properly" in {
dsptools.Driver.execute(() => new SimpleModule(p.genShortF, p.genLongF, includeR = true, p), optionsPass) { c =>
new LitTester(c)
} should be (true)
}
When FIRRTL interpreter is used, both tests pass. When Verilator is used, the second test starts to run, but then fails with
Enabling waves..
chisel3.iotesters.TestApplicationException: test application exit - exit code 139
at chisel3.iotesters.SimApiInterface.throwExceptionIfDead(SimApiInterface.scala:91)
at chisel3.iotesters.SimApiInterface.chisel3$iotesters$SimApiInterface$$mwhile(SimApiInterface.scala:100)
at chisel3.iotesters.SimApiInterface.update(SimApiInterface.scala:194)
at chisel3.iotesters.SimApiInterface.peek(SimApiInterface.scala:281)
at chisel3.iotesters.VerilatorBackend.peek(VerilatorBackend.scala:336)
at dsptools.VerboseDspTester.peek(VerboseDspTester.scala:100)
If I comment out and run individual tests by themselves, things are fine... So there's something wrong with running Verilator twice? i.e. it shouldn't throw an exception when the test application "dies," because the first one actually did finish?
Full test code @ https://github.com/ucb-art/Chisel3DSPExample/blob/master/src/test/scala/UsefulExamples/SimpleTBwGenTypeOption.scala
It's hard for me to figure out what should actually be done in this case, since throwExceptionIfDead(exitValue) was put there for some reason???
@chick any ideas?
Apparently Verilator calls GNU Make at some point which doesn't work with paths with spaces in the path...
@huangchiye
I just cloned chisel-testers and tried to run sbt
and got the following errors:
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: UNRESOLVED DEPENDENCIES ::
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: edu.berkeley.cs#chisel3_2.11;3.1-SNAPSHOT: not found
[warn] :: edu.berkeley.cs#firrtl-interpreter_2.11;1.1-SNAPSHOT: not found
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn]
[warn] Note: Unresolved dependencies path:
[warn] edu.berkeley.cs:chisel3_2.11:3.1-SNAPSHOT (/scratch/celio/chisel-testers/build.sbt#L15-18)
[warn] +- edu.berkeley.cs:chisel-iotesters_2.11:1.2-SNAPSHOT
[warn] edu.berkeley.cs:firrtl-interpreter_2.11:1.1-SNAPSHOT (/scratch/celio/chisel-testers/build.sbt#L15-18)
[warn] +- edu.berkeley.cs:chisel-iotesters_2.11:1.2-SNAPSHOT
[trace] Stack trace suppressed: run last *:update for the full output.
[error] (*:update) sbt.ResolveException: unresolved dependency: edu.berkeley.cs#chisel3_2.11;3.1-SNAPSHOT: not found
[error] unresolved dependency: edu.berkeley.cs#firrtl-interpreter_2.11;1.1-SNAPSHOT: not found
Relatedly, I'm trying to follow (https://github.com/ucb-bar/chisel-testers/wiki/Using%20the%20PeekPokeTester) but the > test-only examples.GCDSpec
throws an invalid input
error.
I've declared a memory via val mem = Mem(sz, UInt(64.W))
, but peekAt
just returns random values, even when using pokeAt
on the same addresses a few cycles earlier. Is this bug, or am I missing something here? (I'm aware that peek and poke now only work on values in the top-level IO
; but since Mem
s cannot be declared in the Bundle, how can they be accessed efficiently in simulation?)
The basic issue is that I need to do synthesis post-simulation. The old way of doing this was to generate the harness from chisel, use VCS to compile it with the post-synthesis verilog and standard cells library, and use the resulting executable (usually simv) back with the chisel infrastructure.
There are two issues, one major and one minor.
Minor: There is no way to generate the harness in a known path directory, and you have to use one of the backends to get it to work even when you do not want to run vcs or verilator.
Major: Test-command is unusable.
The new option to "firrtl" isn't enabled in the testing flow.
This code still flags a combinational loop. (Does work from "firrtl" (command line) and the chisel3.Driver methods.)
package bug
import org.scalatest.{ Matchers, FlatSpec, GivenWhenThen}
import chisel3._
import chisel3.util._
import chisel3.iotesters._
class HasCycle extends Module {
val io = IO( new Bundle {
val a = Input(Bool())
val o = Output(Bool())
})
val b = Wire(Bool())
b := b&&io.a
io.o := b
}
class HasCycleTester( c:HasCycle) extends PeekPokeTester(c) {
poke( c.io.a, 0)
step(1)
}
class HasCycleTest extends FlatSpec with Matchers {
behavior of "HasCycle"
it should "work" in {
chisel3.iotesters.Driver.execute( Array( "--no-check-comb-loops", "--backend-name", "verilator"), () => new HasCycle) { c =>
new HasCycleTester( c)
} should be ( true)
}
}
Here is the error:
[info] HasCycleTest:
[info] HasCycle
[info] [0.003] Elaborating design...
[info] [0.149] Done elaborating.
[info] - should work *** FAILED ***
[info] firrtl.transforms.CheckCombLoops$CombLoopException: : [module HasCycle] Combinational loop detected:
[info] HasCycle._T_5
[info] HasCycle.b
[info] HasCycle._T_5
[info] at firrtl.transforms.CheckCombLoops$$anonfun$run$1$$anonfun$apply$8.apply(CheckCombLoops.scala:216)
[info] at firrtl.transforms.CheckCombLoops$$anonfun$run$1$$anonfun$apply$8.apply(CheckCombLoops.scala:211)
[info] at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
[info] at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
[info] at firrtl.transforms.CheckCombLoops$$anonfun$run$1.apply(CheckCombLoops.scala:211)
[info] at firrtl.transforms.CheckCombLoops$$anonfun$run$1.apply(CheckCombLoops.scala:204)
[info] at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
[info] at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
[info] at firrtl.transforms.CheckCombLoops.run(CheckCombLoops.scala:204)
[info] at firrtl.transforms.CheckCombLoops.execute(CheckCombLoops.scala:231)
[info] ...
[info] ScalaTest
Example DUT:
package example
import chisel3._
import chisel3.util._
class ExampleModule extends Module {
val io = IO(new Bundle {
val in = Flipped(Decoupled(Bits(16.W)))
val out = Decoupled(Bits(16.W))
})
val delay_value = 5.U
val busy = Reg(init = false.B)
val in_reg = Reg(init=0.U(16.W))
io.in.ready := !busy
when(io.in.valid && !busy) {
in_reg := io.in.bits
busy := true.B
}
val wait_counter = Reg(init=0.U(16.W))
when(io.in.valid && !busy) {
wait_counter := 0.U
}
when(busy) {
when(wait_counter === delay_value) {
io.out.bits := in_reg
}.otherwise {
io.out.bits := 0.U
wait_counter := wait_counter + 1.U
}
}
io.out.valid := (io.out.bits === in_reg) && (wait_counter === delay_value) && busy
printf("From printf -- in: ready %d valid %d value %d -- out: ready %d valid %d value %d\n",
io.in.ready, io.in.valid, io.in.bits,
io.out.ready, io.out.valid, io.out.bits)
}
Tests:
package example.test
import chisel3._
import chisel3.util._
import chisel3.iotesters._
import org.scalatest.{Matchers, FlatSpec}
import example._
class ExampleTester(dut: ExampleModule) extends AdvTester(dut){
wire_poke( dut.io.in.valid, 1)
wire_poke( dut.io.in.bits, 20)
val inreadySignal:BigInt = peek(dut.io.in.ready)
println(s"From peek in.ready: $inreadySignal")
step(1)
}
class ExampleTestFirrtl extends ChiselFlatSpec{
"Example" should "show decoupled port controls correctly" in {
chisel3.iotesters.Driver(() => new ExampleModule,"firrtl"){ c =>
new ExampleTester(c)
}should be(true)
}
}
class ExampleTestVerilator extends ChiselFlatSpec{
"Example" should "show decoupled port controls correctly" in {
chisel3.iotesters.Driver(() => new ExampleModule,"verilator"){ c =>
new ExampleTester(c)
}should be(true)
}
}
when running the test with "Verilator" backend, we get:
From peek in.ready: 1
From printf -- in: ready 1 valid 1 value 20 -- out: ready 0 valid 0 value 0
However, when running the test with "Firrtl" backend, we get:
From peek in.ready: 1
From printf -- in: ready 0 valid 1 value 20 -- out: ready 1 valid 0 value 34543
The printf result of in.ready is incorrectly with "Firrtl", and there's a discrepancy between Firrtl and Verilator for the same test.
For now, the backends for verilator and vcs do not use the latest Driver.execute api methods.
This should be fixed.
I'm trying to have FIRRTL automatically generate a scan chain when a "scan" bundle is included in my design.
Scan bundle = list of signals I want to have custom values for.
But since the added scan signals (scan clk, enable, in, out) are not part of the original Chisel code, there's no way I can test it via the usual peek, poke mechanisms... any idea if there might be some workaround?
To interface to an existing test-system, I would like to launch a test from the command line hundreds of times by passing different test-input vector files on the command line.
E.g. given the input data in the "test.txt" file, I'd like to execute a test given:
$ sbt "test:runMain testmain test.txt"
The code below skips Verilator compilation(which is slow), but still performs design elaboration:
Driver.run(() => new system(),
"Vsystem")(c =>
new system_Tests(c, args(0)))
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.