larsbrinkhoff/lbforth

Self-hosting metacompiled Forth, bootstrapping from a few lines of C; targets Linux, Windows, ARM, RISC-V, 68000, PDP-11, asm.js.

lbforth's Issues

search-path entries are truncated to NAME_LEN

lbForth$ strace -e open ./b-forth
[...]
s" /this/is/a/really/really/long/path" searched
 ok
include foo.fth
open("/this/is/a/realfoo.fth", O_RDONLY) = -1 ENOENT (No such file or directory)

Clean up C target metacompiler.

Windows write-file is wrong

Check why this isn't caught by tests.

Building in 32-bit Cywgin results in "Undefined: file-source"

Building make TARGET=x86 OS=windows in Cygwin32 seemingly works ok, but the resulting Forth is a one-line file with the text Undefined: file-source.

Also see #28, reported by @vigilancetech-com.

Rearrange header.

Exceptions don't work well with the SEARCH wordset

When an exception is thrown, one wordlist is dropped from the search order.

Test case:

include search.fth
also forth order \ "forth forth only"
abort
order \ "forth only"

Clean up asm.js target.fth

The asm.js target.fth has two definitions for header,, which is probably a mistake.

Also, docol, etc compile a zero cell before the code field proper. I'm not sure that's by design or not. Is that the place for the "codeid", or the does field? I think the latter. But then the zero could probably move to header, like in the C target.

@pipcet, do you have any comment on this?

Z80 CP/M target

Add a target for Z80 running CP/M.

Use @jhallen's cpm emulator.

Windows READ-FILE should return 0 0 on EOF

When ReadFile returns error, check GetLastError for ERROR_HANDLE_EOF.

Then change KEY to call ABORT instead of BYE on errors.

Finish shell script and batch file targets.

Use them for bootstrapping.

Tangential: syscall emulators

List of emulators which include emulation for system calls:

Emulator	Processor	System
QEMU	Many	Linux, BSD
Wine	Same as host	Windows
WSL	Same as host	Linux
tosemu	68000	TOS (GEMDOS etc)
os9exec	68000	OS-9
Apout	PDP-11	Unix V1, V5, V6, V7. 2BSD
cpm	Z80	CP/M
flexemu	6809	FLEX

Fedora Core 25 x86 64 bits target

I thought it would be interesting to try out lbForth. Target is Fedora Core 25 x86 64 bits and here are my notes:

• gnu/stubs-32.h is missing (it‘s not installed by default on 64 bits FC25), but it seems to work by simply removing -m32 in targets/c/forth.mk (see compiler log below).
• Make warns about “circular kernel.c <- b-forth dependency dropped” (see compiler log below).
• lbForth prints “File not found” but it would be helpful if the failed file path is printed too. Strace reveals the details so it was relatively easy to figure out the cause.
• I used s" /some/dir/..." searched to make lbForth find source files of interest. Not sure if this is correct but it did seem to work.
• The search path seems to be limited to 23 characters which caused some problems with mangled file paths.
• require subdirectories seems to be lost, so that foo in require foo/bar.fth is ignored?
• The code I tried makes use of Forth locals to some extent, and it seems src/locals.fth doesn’t implement it so I didn‘t get further after the error message Undefined: {.

make -ftargets/c/bootstrap.mk
make[1]: Entering directory '/usr/local/src/lbForth'
make[1]: Circular kernel.c <- b-forth dependency dropped.
gcc -m32 -O2 -fomit-frame-pointer -fno-unit-at-a-time -Itargets/c targets/c/params.c -o params
In file included from /usr/include/features.h:392:0,
                 from /usr/include/string.h:25,
                 from targets/c/params.c:1:
/usr/include/gnu/stubs.h:7:27: fatal error: gnu/stubs-32.h: No such file or directory
 # include <gnu/stubs-32.h>
                           ^
compilation terminated.
targets/c/forth.mk:25: recipe for target 'params' failed
make[1]: *** [params] Error 1
make[1]: Leaving directory '/usr/local/src/lbForth'
Makefile:27: recipe for target 'b-forth' failed
make: *** [b-forth] Error 2

ARM target

Parallel build doesn't work

A fresh build with "make -j" fails.

Xtensa target

ESP8266 and ESP32 are everywhere. Need that Xtensa target.

Throw out DO-LOOP

Remove (LOOP) from nucleus. Replace uses of DO-LOOP with FOR-NEXT in the kernel. Provide DO-LOOP in a compatability layer.

https://www.reddit.com/r/Forth/comments/6b9lsz/alternatives_to_loop/

(number) strips digits from undefined symbols

When entering "2dummy", the error message is "Undefined: dummy". It's even worse when base is 36, as all letters and digits will be stripped from the error message:

36 base !
 ok
abcdef-ghij
Undefined: -ghij

COLD and WARM crash with search-order wordset

To reproduce:

include search.fth
cold \ or warm

This is probably because CONTEXT isn't reset in WARM, and the ONLY wordlist is still active.

min vs umin for size

Why do you use min but not umin in the code like the following:

: name, ( a u -- ) #name min dup c, tuck move, offset, ;

Here min protects against too long u, but not from negative u (or very long positive ;).

Pass Hayes' CORE tests.

Then COREEXT, TOOLS, STRING, EXCEPTION, SEARCHORDER, DOUBLE, etc.

PDP-10 target

Get up and running in ITS.

It's a word-adressed machine. But there are also special "byte pointers". Consider what to do about C@ and C! etc.

More Forth2012 words.

SYNONYM, S", ...

Use save-image to write a new executable

Use the save-image to write a new executable with everything pre-loaded.

• It will make Forth start faster.
• It will test the save-image library.
• There will be an incentive to make save-image work in more targets.

Port asm.js target to more engines

I have no idea how the various JS engines interface to the operating system. Or how to run this in a browser.

C64 6502 target

PRG file format:
https://hkn.eecs.berkeley.edu/~mcmartin/ophis/manual/x51.html

Help targeting macOS

I'd like to make a port to macOS, but it's not really viable to do that without access to a macOS machine.

If I could get ssh access to a machine for a month or so, that'd be really helpful.

Finish PDP-1 target.

More helpful message when including missing file

Including a file which doesn't exist just prints "File not found".

overload of EXECUTE in cygwin/clisp causes failure to metacompile

$ make
make -ftargets/c/bootstrap.mk
make[1]: Entering directory '/home/kevin/lbForth'
make[1]: Circular kernel.c <- b-forth dependency dropped.
gcc -m32 -O2 -fomit-frame-pointer -fno-unit-at-a-time -Itargets/c targets/c/params.c -o params
targets/c/run.sh ./params -lisp > params.lisp
cp targets/c/jump.fth jump.fth
cp targets/ctc.fth threading.fth
cp targets/c/target.fth target.fth
echo ": sysdir s" src/" ;" >> target.fth
./lisp/lisp.sh '(load "lisp/meta.lisp") (compile-forth "targets/c/nucleus.fth" "src/kernel.fth")'
;; Loading file lisp/meta.lisp ...
;; Loading file params.lisp ...
;; Loaded file params.lisp
WARNING: DEFUN/DEFMACRO(EXECUTE): # is locked
Ignore the lock and proceed
WARNING: DEFUN/DEFMACRO: redefining function EXECUTE in /home/kevin/lbForth/lisp/meta.lisp, was defined in C
;; Loading file lisp/words.lisp ...
;; Loaded file lisp/words.lisp
;; Loaded file lisp/meta.lisp
gcc -m32 -O2 -fomit-frame-pointer -fno-unit-at-a-time -Itargets/c -c -o kernel.o kernel.c
gcc -m32 kernel.o -o b-forth
rm -f kernel.c kernel.o params.lisp jump.fth threading.fth target.fth
make[1]: Leaving directory '/home/kevin/lbForth'
cp b-forth forth
rm -f *-stamp
touch x86-windows-stamp
cat targets/x86/params.fth targets/x86/windows/params.fth > params.fth
cp targets/x86/jump.fth jump.fth
cp targets/itc.fth threading.fth
cat targets/x86/target.fth targets/x86/windows/target.fth > target.fth
echo ": sysdir s" /usr/local/share/lbForth/" ;" >> target.fth
echo include targets/x86/meta.fth | targets/c/run.sh ./forth | tail -n+3 > x86-forth
mv image x86-forth
mv: cannot stat 'image': No such file or directory
make: [targets/x86/forth.mk:6: x86-forth] Error 1 (ignored)
chmod a+x x86-forth
rm -f forth.exe
cp x86-forth forth
0;~/lbForth

INCLUDE-FILE doesn't restore the input buffer

E.g.

include foo.fth .( this part of the input may be clobbered by foo.fth )

Clean up quotations and move to libraries.

JavaScript target

Metacompiler overhaul

The metacompiler is due for an overhaul.

• The current implementation is closely tied to building the kernel.

  • The kernel-specific code should be split off.
  • The compiler should be made a generic cross compiler.

• Everything is written to a single memory space.

  • Many microcontrollers use a Harvard architecture and need separate code and data spaces.
  • Some applications may want to have separate read-only and read-write data spaces.
  • The word headers can be written to a separate area.

• Building and saving the image is constrained by the single memory space.

  • Must write everything sequentially from start to end.
  • It would be better to be able to write many separate sections and assemble them together in the end.

Ensure x86 performance at least on par with Gforth

See e.g. https://groups.google.com/d/msg/comp.lang.forth/YVjtNQwSIH0/YHduzDG4c9EJ

Some may believe that a threaded-code engine written in assembly language would have led to better performance than that exhibited by Gforth. However, looking at http://www.complang.tuwien.ac.at/forth/performance.html, Gforth 0.4.9 performed faster than any of the threaded-code Forth systems written in assembly language (Win32Forth, NT Forth, and eforth) on all of the benchmarks on a 486DX2/66:

The interpretive systems written in assembly language (except eforth opt) are, surprisingly, slower than Gforth. One important reason for the disappointing performance of these systems is probably that they are not written optimally for the 486 (e.g., they use the lods instruction).

Assembler overhaul

There's a lot of copy-pasting between the different assemblers. Much of the assembler implementation should go into a generic library, and target-specific parts should built on top of that.

Undefined behaviour in C target

@pipcet reported a case of undefined behaviour in the C target. This may affect SP@ and SP!.

Accomodate asm.js changes in metacompiler

The asm.js target added some [IF] sections to the metacompiler. See #33 and f832c94. These should be fixed to use code from some target file instead.

The target also abuses cold.fth to insert special code for dumping the image. This doesn't hurt the overall structure of the metacompiler, but there is room for improvement.

LICENSE?

you know the drill

Windows test fails

make check fails in Windows. There are two failing tests:

• test-exe
• test-save-image

Separate XT from NT.

Make XT point directly at code field.

Create STC x86 target?

With inlining?

Retire Lisp metacompiler.

Go completely self-hosting by ditching the Lisp metacompiler.

Make backtrace more useful by searching all vocabularies.

Also SEE.

calling windows system calls

Is there currently a high level way to call system calls?

I look in io.fth and see you setting up the system calls but you only do them from assembly below.

For example, in FreeForth there is the word:

lib:` ( @ # <name> -- ; a n -- funEntry ) loads the dynamic-link-library
  with filename starting at address "@" for "#" bytes, and defines a word
  which will lookup in this library for the function matching the name
  starting at address "a" for "n" bytes, and if found will return its entry,
  for use with "#call" or "fun:".
  If the library is not found, or if later a function is not found,
  an exception is thrown with corresponding error message.
  : lib:` :` #lib lit` #fun ' call, ;` ;
  see also: lib: #call fun: libc k32  ff.boot

and:

#call ( argN ... arg1 N funEntry -- funResult ) calls the library function
  with entry at "funEntry", which consumes "N" arguments from DATAstack and
  returns a single result on top of DATAstack. The N arguments must be pushed
  on DATAstack in the same order as in C, i.e. C-prototype (see Linux man or
  Windows win32.hlp) rightmost argument first, leftmost last.

Emacs bytecode target

The assembler is done.

RISC-V unconditional jumps

Should use jal.

bench.fth may be broken

Reported in larsbrinkhoff/forth-mode#69

License for commercial use

@vigilancetech-com has expressed interest in using lbForth to build commercial software.

Switching search order between interpretation and compilation

The search order is different in interpretation state and compilation state. When interpreting, only forth is searched, but when compiling there's an implicit also compiler-words. This is currently done in a buggy and nongeneral way.

The solution might also consider metacompilation. Inside the metacompiler in interpretation state, the search order is host-interpreter, meta-interpreter, forth. In compilation state, the search order is host-compiler, meta-interpreter, forth.

When metacompiling in interpretation state, the search order is: meta-interpreter, forth. When metacompiling in compilation state, the search order is: meta-compiler, then target dictionary.

Add UM/MOD and friends.