reticulatedpines/qemu-eos

Fork of official qemu github repo, with ML qemu-eos patches applied

qemu-eos's Issues

(/bin/sh^M: bad interpreter) Script Folder, Line ending in DOS Windows (CRLF) will confuse many UNIX bash interpreters

According to this message at here

e.g. with the scripts/git-submodule.sh used during building will have this weird error

joelwindows7@JOEL-ROG-GL503GE:~/sauce/simplified/qemu-eos-build$ ./scripts/git-submodule.sh
-bash: ./scripts/git-submodule.sh: /bin/sh^M: bad interpreter: No such file or directory

which basically the bash (like in my WSL) confused and thinks you had /bin/sh^M at the first line, while actually not.

This is due to a fact I just figured out that the line ending format is in DOS (CRLF) which happens to confuse various Bash interpreters including mine.

Now what I did to fix this problem is to convert every file possible inside the qemu-eos/script from DOS Windows (CRLF) to UNIX line ended (LF). So I went to that script folder & do this command:

for FILER in ./*; do chmod +x $FILER; dos2unix $FILER;

Basically, for each file, add executable right & also convert the line format from CRLF to LF. Forget the former since permission won't uploads into git. the latter does.

I suggest that should also convert your script and maybe others to such line ending format & push them all here so others will not encounter such error. No idea if others don't have this error, but just in case, it's recommended to do so anyways.

See this answer from Ask Ubuntu on the dos2unix utility.

Using locking for stdout / stderr with multiple workers

Console output from multiple processes is interleaved and messy. Volume is low, use locks to prevent this, e.g. https://docs.python.org/3/library/multiprocessing.html#synchronization-between-processes

Qemu-eos backtrace.c duplicates code

Old process copied in src/backtrace.c and backtrace.h. New repo has had these copied in, but now they may get out of sync. A better solution should be found.

Long term, we may want to use a third party library to replace all uses of backtrace.c, either as a submodule or a forked repo we maintain, e.g: http://www.mcternan.me.uk/ArmStackUnwinding/

Cams that fail pre-test checks don't get shown in summary

Need some way to include very early failures in the summary. The summary currently only checks Cam objects, but early failures prevent these getting created. This also means we report overall test suite status as Pass if the only failures are pre-test ones, which is obviously bad.

qemu-eos logging mangles uart log

Both emulated uart and our additional qemu logging goes to stderr. This is unbuffered. It is fairly common for some messages, e.g. DMA logging, to occur while Canon is logging. This causes interleaved output that makes comparing the results of a test run difficult.

Ideally we would be able to know the difference between our logging and Canon output, while also being able to reconstruct order of events. We could log each to separate files, but that makes reconstruction of order hard. We could wrap our output in magic prefix/suffix (a little ugly). We could use some kind of event store (DB, json?) - line by line for uart, fprintf based for our additional logging?

debugmsg skips most messages in Qemu 4

QEMU_EOS_DEBUGMSG doesn't work as well in Qemu 4. This is what -d debugmsg uses. I believe this is because we use tb_exec_cb() to look for and log DryosDebugMsg() calls. But one TB can be many instructions, we get tb->pc and hope it matches DryosDebugMsg, and it doesn't in most cases (some will print, presumably when they're the first insn in a TB, but this seems rare).

If you use a gdb script to put a break on DryosDebugMsg, that does nothing except continue, then tb_exec_cb matches every time with tb->pc. Here I'm guessing inserting the breakpoint triggers changes to TBs (it will write into code region and Qemu will have to notice this somehow). So this is an effective workaround but requires gdb.

From Qemu 4, there's a plugin architecture that can do a callback at instruction level, not just TB level - this is what we should be doing. But it's not well documented. I spent a few hours on it. Can't find any docs on how to actually build a plugin.

We want to be able to do instruction level hooking in Qemu 4.

Log qemu output into test_output dir

If a LogTest fails, there is currently no way to inspect why. There's some output to console but you can't verify this because the log isn't saved. We should save this.

Update location for romcpy etc output

Old scripting assumes run scripts are used from within the combined build and src dir, and assumes a named dir exists per cam at top-level.

We now use split build and src dirs (cleaner, and the recommended Qemu build process). We have per cam dirs under cam_config so the top-level is not as crowded.

-d romcpy and similar output files according to the old assumptions. Where should these go under the new scheme? If we use run_qemu.py from the source dir, output into that dir or lower will be git managed, so we'd have to have a good .gitignore which might be impractical. If we output outside of git managed dirs (rom dir?), we need to be clear where the output is, and would like it to work neatly with the consumers of the output files.

Building with vte triggers eos_uart_can_rx(opaque) assert

If configure pulls in libvte, trying to access serial is okay for reads, but attempting to send data triggers a crash on the second keypress:

hw/eos/../eos/eos.c:1596: eos_uart_rx: Assertion `eos_uart_can_rx(opaque)' failed

Workaround is to configure with --disable-vte (but you lose scrollback and nice resolution terminal...)

Why does framebufferUpdateRequest(incremental=1) cause expectScreen() to fail?

framebufferUpdateRequest(incremental=1), used internally by expectScreen(), always fails. There's a cryptic note by alex about this in the old vnc patch, which claims it was intermittent then.

I use _fbReplacer() to work around this without requiring upstream changes, but it's kind of ugly. Can we understand this problem so we can get upstream to fix it, or find a real fix our side? Might be a qemu interaction, if we could get a clean repro on vanilla qemu with some standard OS, that would probably be enough to open a useful issue with vncdotool.

See 43bd315

Image comparison should be done on image data, not file data

When we test emulation by comparing screenshot capture to expected image file, we currently compare file hashes. This has been reliable so far, but could be annoying if e.g. vncdotool starts using a different PNG header. We'd fail even though the images looked identical.

It would be better to compare the image data, this is conceptually correct and should be more robust.

Current code is this in tests.py:

            for k in key_sequence:
                capture_filename = q.key_press(k)
                capture_filepath = os.path.join(self.output_dir, capture_filename)
                expected_hash = 0
                with open(capture_filepath, "rb") as f:
                    test_hash = hashlib.md5(f.read()).hexdigest()

-d tasks only tracks cpu0

Old code assumes only one cpu, eos_tasks_log_exec() logs task switches via eos_state->model->current_task_addr, which is only for cpu0.

Really this current_task_addr should be part of the cpu, not the model, or we need multiple current_task_addr fields for the model.

print_task_switch() should output which cpu it's logging (adding this but never seeing cpu1 is how I found this bug).

eos_tasks_log_exec() should be moved to the instruction monitoring plugin, qemu 4 doesn't provide accurate per instruction logging anymore so we may be missing events - and the plugin is faster. In practice task switch logging seems to work okay, maybe because large jumps tend to generate new TBs.

Incomplete GIC / SGI handling

We have a naive software implementation of GIC / SGI handling, see eos_handle_intengine_gic().

Known bugs:

• doesn't handle nested interrupts
• sometimes cpu0 will intend to signal cpu1 but win the race and handle its own interrupt (and vice versa)

Currently we create an interrupt with cpu_interrupt(cpu_id, CPU_INTERRUPT_HARD), this presumably is visible to both cores (although you do have to specify a cpu id...) and maybe real SGI aren't? I assume the actual interrupt is some inter-core signal.

Possibly related, we don't store the cpu_id in the interrupt value we send via "iar". This is wrong according to ARM manual on GICC_IAR and GICC_EOIR. Unsure if this is used to route the interrupt or not, maybe it is on real hardware - we're not emulating this.