thomasdickey/byacc-snapshots

snapshots of releases and interim labels for "byacc" https://invisible-island.net/personal/git-exports.html

byacc-snapshots's Issues

bug: byacc 20190617 emits wrong line directives

hello, thanks for your work maintaining this beautiful project.

here's a testcase: https://0x0.st/zxQL.y

under some circumstances, yacc emits
#line XXX y.tab.c instead of #line XXX parser.y which then confuses tools
using that information, such as gdb.

Feedback and patch for byacc under Windows commandline

Thank you Thomas Dickey for a piece of beautiful code.

I first compiled byacc a couple of years ago for Interix, where it compiled right out of the box and could replace the incompatible and cumbersome bison. Somewhat of a sensation to me at the time. Now Interix is no more and I'm trying to get a couple of tools working for the Windows commandline.

When it comes to byacc, it appears the biggest difference between the various "unix" environments in Windows (Interix, Cygwin etc) and Windows commandline, is the presence of the TMPDIR environment variable. In my experience a proper path for temporary files is needed by byacc under Windows commandline, otherwise there's a risk of empty files. The TEMP environment variable is always defined in Windows and can easily replace the TMPDIR environment variable.

I also made a quick fix for the noreturn attribute.

byacc-msvcrt.patch.txt

Some additional definitions needed for MSVCRT (from my Interix tree):

/* POSIX const for MSVCRT access */
enum _access_const
{
    F_OK            = 00,   /* Test for existence of file */
/*  X_OK            = 01,*/ /* Test for execute (search) permission */
    W_OK            = 02,   /* Test for write permission */
    R_OK            = 04,   /* Test for read permission */
    RW_OK           = 06    /* Test for read and write permission */
};
typedef int32_t mode_t;

and for completeness my config.h settings

#ifndef HAVE_FCNTL_H
#define HAVE_FCNTL_H 1
#endif
#ifndef HAVE_VSNPRINTF
#define HAVE_VSNPRINTF 1
#endif

/* Define to maximum table size (default: 32500) */
#ifndef MAXTABLE
#define MAXTABLE INT_MAX
#endif

/* Define to 1 to enable backtracking extension */
#ifndef YYBTYACC
#define YYBTYACC 1
#endif

/* Define to 1 if you want to perform memory-leak testing. */
#if defined(_DEBUG) && !defined(YY_NO_LEAKS) 
#define YY_NO_LEAKS 1
#endif

[feature request] --defines flag

--defines=<file> is a bison extension equivalent to -d but specifying the header file created. The linux kernel makes use of this flag, to get the kernel to build with byacc as YACC instead of bison I had to use this patch:

--- scripts/Makefile.lib
+++ scripts/Makefile.lib
@@ -202,7 +202,7 @@
 	$(call if_changed,bison)

 quiet_cmd_bison_h = YACC    $@
-      cmd_bison_h = $(YACC) -o/dev/null --defines=$@ -t -l $<
+      cmd_bison_h = $(YACC) -o/dev/null -d -t -l $< && mv y.tab.h $@

 $(obj)/%.tab.h: $(src)/%.y FORCE
 	$(call if_changed,bison_h)
--- scripts/genksyms/Makefile
+++ scripts/genksyms/Makefile
@@ -15,14 +15,14 @@
 ifeq ($(findstring 1,$(KBUILD_ENABLE_EXTRA_GCC_CHECKS)),)

 quiet_cmd_bison_no_warn = $(quiet_cmd_bison)
-      cmd_bison_no_warn = $(YACC) --version >/dev/null; \
+      cmd_bison_no_warn = $(YACC) -V >/dev/null; \
 			  $(cmd_bison) 2>/dev/null

 $(obj)/parse.tab.c: $(src)/parse.y FORCE
 	$(call if_changed,bison_no_warn)

 quiet_cmd_bison_h_no_warn = $(quiet_cmd_bison_h)
-      cmd_bison_h_no_warn = $(YACC) --version >/dev/null; \
+      cmd_bison_h_no_warn = $(YACC) -V >/dev/null; \
 			    $(cmd_bison_h) 2>/dev/null

 $(obj)/parse.tab.h: $(src)/parse.y FORCE
--- scripts/genksyms/lex.l
+++ scripts/genksyms/lex.l
@@ -30,6 +30,7 @@

 #include "genksyms.h"
 #include "parse.tab.h"
+extern YYSTYPE yylval;

 /* We've got a two-level lexer here.  We let flex do basic tokenization
    and then we categorize those basic tokens in the second stage.  */

The last hunk is because without it an error about yylval being undeclared occurs, I don't know much about yacc or lex so I'd think there's probably a more proper fix to that issue, but that in regards to --defines, the replacement I used there is rather hacky, and I doubt it'd get accepted upstream, but if byacc supported --defines there could probably be upstream support for building the kernel with byacc as YACC instead of solely working with bison.

Segmentation fault when trying to parse bison-3.8.2 grammar

Doing experiments with byacc and trying to parse bison-3.8.2 grammar I found that byacc segfaults:

valgrind byacc-snapshots/yacc bison.y 
==754== Memcheck, a memory error detector
==754== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==754== Using Valgrind-3.17.0 and LibVEX; rerun with -h for copyright info
==754== Command:byacc-snapshots/yacc bison.y
==754== 
==754== Conditional jump or move depends on uninitialised value(s)
==754==    at 0x11AABF: copy_code (reader.c:707)
==754==    by 0x11DA24: read_declarations (reader.c:1837)
==754==    by 0x11E470: reader (reader.c:3931)
==754==    by 0x1102FB: main (main.c:878)
==754== 
==754== Use of uninitialised value of size 8
==754==    at 0x11AA13: copy_code (reader.c:709)
==754==    by 0x11DA24: read_declarations (reader.c:1837)
==754==    by 0x11E470: reader (reader.c:3931)
==754==    by 0x1102FB: main (main.c:878)
==754== 
==754== Conditional jump or move depends on uninitialised value(s)
==754==    at 0x11AA21: copy_code (reader.c:712)
==754==    by 0x11DA24: read_declarations (reader.c:1837)
==754==    by 0x11E470: reader (reader.c:3931)
==754==    by 0x1102FB: main (main.c:878)
==754== 
==754== Invalid read of size 1
==754==    at 0x11AAB3: copy_code (reader.c:706)
==754==    by 0x11DA24: read_declarations (reader.c:1837)
==754==    by 0x11E470: reader (reader.c:3931)
==754==    by 0x1102FB: main (main.c:878)
==754==  Address 0x5238f45 is 0 bytes after a block of size 101 alloc'd
==754==    at 0x4C32F6D: malloc (vg_replace_malloc.c:380)
==754==    by 0x11A3FD: get_line (reader.c:294)
==754==    by 0x11A944: next_inline (reader.c:383)
==754==    by 0x11A97C: nextc (reader.c:426)
==754==    by 0x11D4BC: read_declarations (reader.c:1821)
==754==    by 0x11E470: reader (reader.c:3931)
==754==    by 0x1102FB: main (main.c:878)
==754== 
==754== 
==754== Process terminating with default action of signal 11 (SIGSEGV)
==754==  Access not within mapped region at address 0x5634000
==754==    at 0x11AAB3: copy_code (reader.c:706)
==754==    by 0x11DA24: read_declarations (reader.c:1837)
==754==    by 0x11E470: reader (reader.c:3931)
==754==    by 0x1102FB: main (main.c:878)
==754==  If you believe this happened as a result of a stack
==754==  overflow in your program's main thread (unlikely but
==754==  possible), you can try to increase the size of the
==754==  main thread stack using the --main-stacksize= flag.
==754==  The main thread stack size used in this run was 8388608.
==754== 
==754== HEAP SUMMARY:
==754==     in use at exit: 10,947 bytes in 15 blocks
==754==   total heap usage: 18 allocs, 3 frees, 19,201 bytes allocated
==754== 
==754== LEAK SUMMARY:
==754==    definitely lost: 0 bytes in 0 blocks
==754==    indirectly lost: 0 bytes in 0 blocks
==754==      possibly lost: 0 bytes in 0 blocks
==754==    still reachable: 10,947 bytes in 15 blocks
==754==         suppressed: 0 bytes in 0 blocks
==754== Rerun with --leak-check=full to see details of leaked memory
==754== 
==754== Use --track-origins=yes to see where uninitialised values come from
==754== For lists of detected and suppressed errors, rerun with: -s
==754== ERROR SUMMARY: 4174078 errors from 4 contexts (suppressed: 0 from 0)
valgrind: line 2:   754 Segmentation fault      (core dumped) valgrind $*

Byacc with some Lemon taste

Here https://github.com/mingodad/lalr-parser-test I'm making some changes to byacc to make easier to compare between bison/byacc/lemon and would appreciate any feedback/suggestion .

The byacc parser (from https://invisible-island.net/byacc/byacc.html) here have several modifications like:

• Command line option to print naked yacc -n generate naked grammar
• Command line option to convert from yacc grammar to lemon grammar -E write lemon grammar
• Command line option to convert from yacc grammar to EBNF grammar (understood by https://www.bottlecaps.de/rr/ui) -e write ebnf grammar
• Command line option to output SQL from yacc grammar -S write grammar as sql
• Command line option to ignore all precedences -u ignore precedences
• Command line option to use lemon rule precedences -z use leftmost token for rule precedence
• Add code to accept/skip bison styles alias for non-terminals rule[alias]