nikic / php-ast Goto Github PK

Extension exposing PHP 7 abstract syntax tree

License: Other

C 34.11% PHP 65.12% JavaScript 0.06% M4 0.10% Dockerfile 0.24% Shell 0.38%

php-ast's Issues

what is php version?

Hello
what is php version for php ast? Am I run php ast for php 5.4 of after or befor ?
Or is it for special version of php?
and dose create ast for any version of php?

Thank you.

Empty list on php-ast(version30) + PHP7.1 + Windows

<?php
// php-ast.dll(php_ast-20161209-7.1-ts-vc14-x64)

//var_dump(ast\parse_code('<?php list(,) = 1;', 35)); //< OK
var_dump(ast\parse_code('<?php list(,) = 1;', 30)); //< An "CLI Stop" dialog is displayed

>	php_ast-20161209-7.1-ts-vc14-x64.dll!ast_to_zval(_zval_struct * zv, _zend_ast * ast, __int64 version) line 487	C
 	php_ast-20161209-7.1-ts-vc14-x64.dll!ast_fill_children_ht(_zend_array * ht, _zend_ast * ast, __int64 version) line 418	C
 	php_ast-20161209-7.1-ts-vc14-x64.dll!ast_to_zval(_zval_struct * zv, _zend_ast * ast, __int64 version) line 568	C
 	php_ast-20161209-7.1-ts-vc14-x64.dll!ast_fill_children_ht(_zend_array * ht, _zend_ast * ast, __int64 version) line 418	C
 	php_ast-20161209-7.1-ts-vc14-x64.dll!ast_to_zval(_zval_struct * zv, _zend_ast * ast, __int64 version) line 568	C
 	php_ast-20161209-7.1-ts-vc14-x64.dll!zif_parse_code(_zend_execute_data * execute_data, _zval_struct * return_value) line 689	C
 	php_xdebug-2.5.0-7.1-vc14-x86_64.dll!00007ffa7c296621()	unknown
 	php7ts.dll!00007ffa631e9951()	unknown
 	php7ts.dll!00007ffa63279c8a()	unknown
 	php7ts.dll!00007ffa62e25114()	unknown
 	php_xdebug-2.5.0-7.1-vc14-x86_64.dll!00007ffa7c295f88()	unknown
 	php7ts.dll!00007ffa62e56dc1()	unknown
 	php7ts.dll!00007ffa62e5687e()	unknown
 	php7ts.dll!00007ffa62e56698()	unknown
 	php.exe!00007ff6a65a1dca()	unknown
 	php.exe!00007ff6a65a1566()	unknown
 	php.exe!00007ff6a65a2879()	unknown

Will this module return t_comments?

In nikic/php-parser, for instance, it strips comments while generating the tree. Now that PHP 7 will likely be implemented using ASTs, I wonder whether comments will be kept.

nullable types: possible false positive PhanTypeComparisonFromArray

<?php

function foo() : ?array
{
	return ['a'];
}

$result = foo();
// PhanTypeComparisonFromArray array to null comparison
if ($result !== null)
{
	// PhanTypeArraySuspicious Suspicious array access to ?array
	echo $result[0];	
}

// PhanTypeArraySuspicious Suspicious array access to array
if (!is_null($result))
{
	echo $result[0];	
}

I do not expect any of the errors.
version: 0.9.0

Windows built for version 40

Hi, i was checking the latest release but cannot see the windows binaries under PECL snapshots.
Where can i download the latest version for windows?

Split ast-stubs.php into two files for php 7.0 and php 7.1?

It makes it easier to know what types to expect for a php minor version when using this module: e.g. TYPE_ITERABLE doesn't exist in php 7.0
The values are going to be different, which will confuse people using the stub file to try to understand serialization(json_encode, var_export, etc.) of a node (Not sure if AST is stable in patch versions)
\ast\AST_USE is 143 in php 7.1, but is 144 in php 7.0

Segfault with PHP 7.1 running phan test suite

$ gdb php71
(gdb) run ...phpunit...
...........................................................  63 / 282 ( 22%)
....................................................
Program received signal SIGSEGV, Segmentation fault.
0x00007fffde4e1050 in ast_array_is_list (ast=0x7fffd7411078) at /usr/src/debug/php71-php-ast-0.1.2/NTS/ast.c:244
244			if (list->child[i]->child[1] != NULL || list->child[i]->attr) {
(gdb) bt
#0  0x00007fffde4e1050 in ast_array_is_list (ast=0x7fffd7411078) at /usr/src/debug/php71-php-ast-0.1.2/NTS/ast.c:244
#1  ast_to_zval (zv=zv@entry=0x7fffffff9340, ast=ast@entry=0x7fffd7411078, version=version@entry=30) at /usr/src/debug/php71-php-ast-0.1.2/NTS/ast.c:487
#2  0x00007fffde4e149c in ast_fill_children_ht (ht=0x7fffd79e7d90, ast=ast@entry=0x7fffd74110e8, version=version@entry=30) at /usr/src/debug/php71-php-ast-0.1.2/NTS/ast.c:418
#3  0x00007fffde4e0f1c in ast_to_zval (zv=zv@entry=0x7fffffff9470, ast=ast@entry=0x7fffd74110e8, version=version@entry=30) at /usr/src/debug/php71-php-ast-0.1.2/NTS/ast.c:568
#4  0x00007fffde4e149c in ast_fill_children_ht (ht=0x7fffd7906118, ast=ast@entry=0x7fffd7411018, version=version@entry=30) at /usr/src/debug/php71-php-ast-0.1.2/NTS/ast.c:418
#5  0x00007fffde4e0f1c in ast_to_zval (zv=zv@entry=0x7ffff381aff0, ast=ast@entry=0x7fffd7411018, version=30) at /usr/src/debug/php71-php-ast-0.1.2/NTS/ast.c:568
#6  0x00007fffde4e1c0d in zif_parse_file (execute_data=<optimized out>, return_value=0x7ffff381aff0) at /usr/src/debug/php71-php-ast-0.1.2/NTS/ast.c:661
#7  0x00005555557988fa in dtrace_execute_internal (execute_data=<optimized out>, return_value=<optimized out>) at /usr/src/debug/php-7.1.0RC6/Zend/zend_dtrace.c:107
#8  0x000055555584b4fc in ZEND_DO_FCALL_SPEC_RETVAL_USED_HANDLER () at /usr/src/debug/php-7.1.0RC6/Zend/zend_vm_execute.h:1099
#9  0x00005555557f2e9b in execute_ex (ex=ex@entry=0x7fffda133040) at /usr/src/debug/php-7.1.0RC6/Zend/zend_vm_execute.h:429
#10 0x0000555555798898 in dtrace_execute_ex (execute_data=0x7fffda133040) at /usr/src/debug/php-7.1.0RC6/Zend/zend_dtrace.c:83
#11 0x000055555584bb5c in ZEND_DO_FCALL_SPEC_RETVAL_UNUSED_HANDLER () at /usr/src/debug/php-7.1.0RC6/Zend/zend_vm_execute.h:949
#12 0x00005555557f2e9b in execute_ex (ex=ex@entry=0x7fffd7981380) at /usr/src/debug/php-7.1.0RC6/Zend/zend_vm_execute.h:429
#13 0x0000555555798898 in dtrace_execute_ex (execute_data=0x7fffd7981380) at /usr/src/debug/php-7.1.0RC6/Zend/zend_dtrace.c:83
#14 0x000055555584bb5c in ZEND_DO_FCALL_SPEC_RETVAL_UNUSED_HANDLER () at /usr/src/debug/php-7.1.0RC6/Zend/zend_vm_execute.h:949
#15 0x00005555557f2e9b in execute_ex (ex=ex@entry=0x7ffff39c2460) at /usr/src/debug/php-7.1.0RC6/Zend/zend_vm_execute.h:429
#16 0x0000555555798898 in dtrace_execute_ex (execute_data=0x7ffff39c2460) at /usr/src/debug/php-7.1.0RC6/Zend/zend_dtrace.c:83
#17 0x000055555579a4e1 in zend_call_function (fci=fci@entry=0x7fffffff99f0, fci_cache=<optimized out>, fci_cache@entry=0x7fffffff99c0) at /usr/src/debug/php-7.1.0RC6/Zend/zend_execute_API.c:828
#18 0x00005555556918c9 in reflection_method_invoke (execute_data=<optimized out>, return_value=0x7ffff3819f20, variadic=0) at /usr/src/debug/php-7.1.0RC6/ext/reflection/php_reflection.c:3316
#19 0x00005555557988fa in dtrace_execute_internal (execute_data=<optimized out>, return_value=<optimized out>) at /usr/src/debug/php-7.1.0RC6/Zend/zend_dtrace.c:107
#20 0x000055555584b4fc in ZEND_DO_FCALL_SPEC_RETVAL_USED_HANDLER () at /usr/src/debug/php-7.1.0RC6/Zend/zend_vm_execute.h:1099
#21 0x00005555557f2e9b in execute_ex (ex=ex@entry=0x7ffff39a94c0) at /usr/src/debug/php-7.1.0RC6/Zend/zend_vm_execute.h:429
#22 0x0000555555798898 in dtrace_execute_ex (execute_data=0x7ffff39a94c0) at /usr/src/debug/php-7.1.0RC6/Zend/zend_dtrace.c:83
#23 0x000055555584b5ce in ZEND_DO_FCALL_SPEC_RETVAL_USED_HANDLER () at /usr/src/debug/php-7.1.0RC6/Zend/zend_vm_execute.h:1076
#24 0x00005555557f2e9b in execute_ex (ex=ex@entry=0x7ffff39a65c0) at /usr/src/debug/php-7.1.0RC6/Zend/zend_vm_execute.h:429
#25 0x0000555555798898 in dtrace_execute_ex (execute_data=0x7ffff39a65c0) at /usr/src/debug/php-7.1.0RC6/Zend/zend_dtrace.c:83
#26 0x000055555584bb5c in ZEND_DO_FCALL_SPEC_RETVAL_UNUSED_HANDLER () at /usr/src/debug/php-7.1.0RC6/Zend/zend_vm_execute.h:949
#27 0x00005555557f2e9b in execute_ex (ex=ex@entry=0x7fffda13a000) at /usr/src/debug/php-7.1.0RC6/Zend/zend_vm_execute.h:429
#28 0x0000555555798898 in dtrace_execute_ex (execute_data=0x7fffda13a000) at /usr/src/debug/php-7.1.0RC6/Zend/zend_dtrace.c:83
#29 0x000055555584bb5c in ZEND_DO_FCALL_SPEC_RETVAL_UNUSED_HANDLER () at /usr/src/debug/php-7.1.0RC6/Zend/zend_vm_execute.h:949
#30 0x00005555557f2e9b in execute_ex (ex=ex@entry=0x7ffff39a5260) at /usr/src/debug/php-7.1.0RC6/Zend/zend_vm_execute.h:429
#31 0x0000555555798898 in dtrace_execute_ex (execute_data=0x7ffff39a5260) at /usr/src/debug/php-7.1.0RC6/Zend/zend_dtrace.c:83
#32 0x000055555584bb5c in ZEND_DO_FCALL_SPEC_RETVAL_UNUSED_HANDLER () at /usr/src/debug/php-7.1.0RC6/Zend/zend_vm_execute.h:949
#33 0x00005555557f2e9b in execute_ex (ex=ex@entry=0x7ffff39345a0) at /usr/src/debug/php-7.1.0RC6/Zend/zend_vm_execute.h:429
#34 0x0000555555798898 in dtrace_execute_ex (execute_data=0x7ffff39345a0) at /usr/src/debug/php-7.1.0RC6/Zend/zend_dtrace.c:83
#35 0x000055555584bb5c in ZEND_DO_FCALL_SPEC_RETVAL_UNUSED_HANDLER () at /usr/src/debug/php-7.1.0RC6/Zend/zend_vm_execute.h:949
#36 0x00005555557f2e9b in execute_ex (ex=ex@entry=0x7ffff39345a0) at /usr/src/debug/php-7.1.0RC6/Zend/zend_vm_execute.h:429
#37 0x0000555555798898 in dtrace_execute_ex (execute_data=0x7ffff39345a0) at /usr/src/debug/php-7.1.0RC6/Zend/zend_dtrace.c:83
#38 0x000055555584bb5c in ZEND_DO_FCALL_SPEC_RETVAL_UNUSED_HANDLER () at /usr/src/debug/php-7.1.0RC6/Zend/zend_vm_execute.h:949
#39 0x00005555557f2e9b in execute_ex (ex=ex@entry=0x7ffff39345a0) at /usr/src/debug/php-7.1.0RC6/Zend/zend_vm_execute.h:429
#40 0x0000555555798898 in dtrace_execute_ex (execute_data=0x7ffff39345a0) at /usr/src/debug/php-7.1.0RC6/Zend/zend_dtrace.c:83
#41 0x000055555584bb5c in ZEND_DO_FCALL_SPEC_RETVAL_UNUSED_HANDLER () at /usr/src/debug/php-7.1.0RC6/Zend/zend_vm_execute.h:949
#42 0x00005555557f2e9b in execute_ex (ex=ex@entry=0x7ffff39345a0) at /usr/src/debug/php-7.1.0RC6/Zend/zend_vm_execute.h:429
#43 0x0000555555798898 in dtrace_execute_ex (execute_data=0x7ffff39345a0) at /usr/src/debug/php-7.1.0RC6/Zend/zend_dtrace.c:83
#44 0x000055555584bb5c in ZEND_DO_FCALL_SPEC_RETVAL_UNUSED_HANDLER () at /usr/src/debug/php-7.1.0RC6/Zend/zend_vm_execute.h:949
#45 0x00005555557f2e9b in execute_ex (ex=ex@entry=0x7fffda11ba20) at /usr/src/debug/php-7.1.0RC6/Zend/zend_vm_execute.h:429
#46 0x0000555555798898 in dtrace_execute_ex (execute_data=0x7fffda11ba20) at /usr/src/debug/php-7.1.0RC6/Zend/zend_dtrace.c:83
#47 0x000055555584b5ce in ZEND_DO_FCALL_SPEC_RETVAL_USED_HANDLER () at /usr/src/debug/php-7.1.0RC6/Zend/zend_vm_execute.h:1076
#48 0x00005555557f2e9b in execute_ex (ex=ex@entry=0x7ffff38dfc60) at /usr/src/debug/php-7.1.0RC6/Zend/zend_vm_execute.h:429
#49 0x0000555555798898 in dtrace_execute_ex (execute_data=0x7ffff38dfc60) at /usr/src/debug/php-7.1.0RC6/Zend/zend_dtrace.c:83
#50 0x000055555584b5ce in ZEND_DO_FCALL_SPEC_RETVAL_USED_HANDLER () at /usr/src/debug/php-7.1.0RC6/Zend/zend_vm_execute.h:1076
#51 0x00005555557f2e9b in execute_ex (ex=ex@entry=0x7ffff38825a0) at /usr/src/debug/php-7.1.0RC6/Zend/zend_vm_execute.h:429
#52 0x0000555555798898 in dtrace_execute_ex (execute_data=0x7ffff38825a0) at /usr/src/debug/php-7.1.0RC6/Zend/zend_dtrace.c:83
#53 0x000055555584bb5c in ZEND_DO_FCALL_SPEC_RETVAL_UNUSED_HANDLER () at /usr/src/debug/php-7.1.0RC6/Zend/zend_vm_execute.h:949
#54 0x00005555557f2e9b in execute_ex (ex=ex@entry=0x7ffff38593a0) at /usr/src/debug/php-7.1.0RC6/Zend/zend_vm_execute.h:429
#55 0x0000555555798898 in dtrace_execute_ex (execute_data=0x7ffff38593a0) at /usr/src/debug/php-7.1.0RC6/Zend/zend_dtrace.c:83
#56 0x000055555584db38 in zend_execute (op_array=op_array@entry=0x7ffff3887000, return_value=return_value@entry=0x0) at /usr/src/debug/php-7.1.0RC6/Zend/zend_vm_execute.h:474
#57 0x00005555557a96a3 in zend_execute_scripts (type=type@entry=8, retval=retval@entry=0x0, file_count=file_count@entry=3) at /usr/src/debug/php-7.1.0RC6/Zend/zend.c:1464
#58 0x0000555555746800 in php_execute_script (primary_file=0x7fffffffca10) at /usr/src/debug/php-7.1.0RC6/main/main.c:2533
#59 0x000055555584fd9c in do_cli (argc=4, argv=0x555555bc8230) at /usr/src/debug/php-7.1.0RC6/sapi/cli/php_cli.c:990
#60 0x000055555562116a in main (argc=4, argv=0x555555bc8230) at /usr/src/debug/php-7.1.0RC6/sapi/cli/php_cli.c:1378

Convert AST_CONST_ELEM and AST_PROP_ELEM to \ast\Node\Decl in version 45?

They have a name (as $node->children['name'], currently)
They have the property docComment set (not part of most of \ast\Node, but set as a dynamic property)
It may be useful to know the end line number as well, for certain applications (e.g. reporting the start and end line numbers for an issue detected for a property declaration)
ReflectionProperty exists.

Suggestion: "use function" on a statement group should set flags=USE_FUNCTION on individual nodes, like "use" on a statement group

Observed in <?php use function MyNs\{foo, bar}: The flags are 0 for the nodes for foo and bar.

var_export (\ast\parse_code('<?php use function MyNs\{foo, bar};', 40));                            
ast\Node::__set_state(array(
   'kind' => 133,
   'flags' => 0,
   'lineno' => 1,
   'children' => 
  array (
    0 => 
    ast\Node::__set_state(array(
       'kind' => 544,
       'flags' => 346,
       'lineno' => 1,
       'children' => 
      array (
        'prefix' => 'MyNs',
        'uses' => 
        ast\Node::__set_state(array(
           'kind' => 144,
           'flags' => 0,
           'lineno' => 1,
           'children' => 
          array (
            0 => 
            ast\Node::__set_state(array(
               'kind' => 542,
               'flags' => 0,
               'lineno' => 1,
               'children' => 
              array (
                'name' => 'foo',
                'alias' => NULL,
              ),
            )),
            1 => 
            ast\Node::__set_state(array(
               'kind' => 542,
               'flags' => 0,
               'lineno' => 1,
               'children' => 
              array (
                'name' => 'bar',
                'alias' => NULL,
              ),
            )),
          ),
        )),
      ),
    )),
  ),
))

In contrast, <?php use MyNs\{foo, bar};' sets the flag on individual nodes to USE_NORMAL, and

php> echo json_encode(\ast\parse_code('<?php use MyNs\{foo, bar};', 40), JSON_PRETTY_PRINT);
{
    "kind": 133,
    "flags": 0,
    "lineno": 1,
    "children": [
        {
            "kind": 544,
            "flags": 0,
            "lineno": 1,
            "children": {
                "prefix": "MyNs",
                "uses": {
                    "kind": 144,
                    "flags": 0,
                    "lineno": 1,
                    "children": [
                        {
                            "kind": 542,
                            "flags": 361,
                            "lineno": 1,
                            "children": {
                                "name": "foo",
                                "alias": null
                            }
                        },
                        {
                            "kind": 542,
                            "flags": 361,
                            "lineno": 1,
                            "children": {
                                "name": "bar",
                                "alias": null
                            }
                        }
                    ]
                }
            }
        }
    ]
}

Running the tests on Appveyor/Windows

@nikic Did you ever consider to run the tests on Windows as well? It is fairly easy.

See Jan-E#1 with these results:
https://ci.appveyor.com/project/Jan-E/php-ast/build/master.2

How to set header file include directory?

I'm trying to follow your Unix install instructions, but I get a bunch of errors ending with:

/home/mpenner/Projects/php-ast/ast.c:15:28: fatal error: zend_smart_str.h: No such file or directory
 #include "zend_smart_str.h"

I've found that file located at /home/mpenner/Programs/php7/Zend/zend_smart_str.h but I don't know how to tell configure/make where the headers are.

Could you give an example?

I haven't installed php7 into the standard directories because I want to php5.6 as well.

Here's the full output if you need it:

┌─(~/Projects/php-ast)────────────────────────────────────────(mpenner@mpenner-workstation:pts/30)─┐
└─(14:04:%)── ~/Programs/php7/scripts/phpize                                         ──(Fri,Jan29)─┘
Configuring for:
PHP Api Version:         20151012
Zend Module Api No:      20151012
Zend Extension Api No:   320151012
┌─(~/Projects/php-ast)────────────────────────────────────────(mpenner@mpenner-workstation:pts/30)─┐
└─(14:04:%)── ./configure                                                            ──(Fri,Jan29)─┘
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for a sed that does not truncate output... /bin/sed
checking for cc... cc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether cc accepts -g... yes
checking for cc option to accept ISO C89... none needed
checking how to run the C preprocessor... cc -E
checking for icc... no
checking for suncc... no
checking whether cc understands -c and -o together... yes
checking for system library directory... lib
checking if compiler supports -R... no
checking if compiler supports -Wl,-rpath,... yes
checking build system type... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
checking target system type... x86_64-unknown-linux-gnu
checking for PHP prefix... /usr
checking for PHP includes... -I/usr/include/php5 -I/usr/include/php5/main -I/usr/include/php5/TSRM -I/usr/include/php5/Zend -I/usr/include/php5/ext -I/usr/include/php5/ext/date/lib
checking for PHP extension directory... /usr/lib/php5/20121212
checking for PHP installed headers prefix... /usr/include/php5
checking if debug is enabled... no
checking if zts is enabled... no
checking for re2c... no
configure: WARNING: You will need re2c 0.13.4 or later if you want to regenerate PHP parsers.
checking for gawk... no
checking for nawk... nawk
checking if nawk is broken... no
checking whether to enable ast support... yes, shared
checking for ld used by cc... /usr/bin/ld
checking if the linker (/usr/bin/ld) is GNU ld... yes
checking for /usr/bin/ld option to reload object files... -r
checking for BSD-compatible nm... /usr/bin/nm -B
checking whether ln -s works... yes
checking how to recognize dependent libraries... pass_all
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking dlfcn.h usability... yes
checking dlfcn.h presence... yes
checking for dlfcn.h... yes
checking the maximum length of command line arguments... 1572864
checking command to parse /usr/bin/nm -B output from cc object... ok
checking for objdir... .libs
checking for ar... ar
checking for ranlib... ranlib
checking for strip... strip
checking if cc supports -fno-rtti -fno-exceptions... no
checking for cc option to produce PIC... -fPIC
checking if cc PIC flag -fPIC works... yes
checking if cc static flag -static works... yes
checking if cc supports -c -o file.o... yes
checking whether the cc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking whether -lc should be explicitly linked in... no
checking dynamic linker characteristics... GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... yes
checking whether to build static libraries... no

creating libtool
appending configuration tag "CXX" to libtool
configure: creating ./config.status
config.status: creating config.h
config.status: config.h is unchanged
┌─(~/Projects/php-ast)────────────────────────────────────────(mpenner@mpenner-workstation:pts/30)─┐
└─(14:05:%)── make                                                                   ──(Fri,Jan29)─┘
/bin/bash /home/mpenner/Projects/php-ast/libtool --mode=compile cc  -I. -I/home/mpenner/Projects/php-ast -DPHP_ATOM_INC -I/home/mpenner/Projects/php-ast/include -I/home/mpenner/Projects/php-ast/main -I/home/mpenner/Projects/php-ast -I/usr/include/php5 -I/usr/include/php5/main -I/usr/include/php5/TSRM -I/usr/include/php5/Zend -I/usr/include/php5/ext -I/usr/include/php5/ext/date/lib  -DHAVE_CONFIG_H  -g -O2   -c /home/mpenner/Projects/php-ast/ast.c -o ast.lo 
mkdir .libs
 cc -I. -I/home/mpenner/Projects/php-ast -DPHP_ATOM_INC -I/home/mpenner/Projects/php-ast/include -I/home/mpenner/Projects/php-ast/main -I/home/mpenner/Projects/php-ast -I/usr/include/php5 -I/usr/include/php5/main -I/usr/include/php5/TSRM -I/usr/include/php5/Zend -I/usr/include/php5/ext -I/usr/include/php5/ext/date/lib -DHAVE_CONFIG_H -g -O2 -c /home/mpenner/Projects/php-ast/ast.c  -fPIC -DPIC -o .libs/ast.o
In file included from /home/mpenner/Projects/php-ast/ast.c:8:0:
/home/mpenner/Projects/php-ast/php_ast.h:27:16: error: unknown type name ‘zend_string’
 #define X(str) zend_string *str_ ## str;
                ^
/home/mpenner/Projects/php-ast/ast_str_defs.h:5:2: note: in expansion of macro ‘X’
  X(kind) \
  ^
/home/mpenner/Projects/php-ast/php_ast.h:28:2: note: in expansion of macro ‘AST_STR_DEFS’
  AST_STR_DEFS
  ^
/home/mpenner/Projects/php-ast/php_ast.h:27:16: error: unknown type name ‘zend_string’
 #define X(str) zend_string *str_ ## str;
                ^
/home/mpenner/Projects/php-ast/ast_str_defs.h:6:2: note: in expansion of macro ‘X’
  X(flags) \
  ^
/home/mpenner/Projects/php-ast/php_ast.h:28:2: note: in expansion of macro ‘AST_STR_DEFS’
  AST_STR_DEFS
  ^
/home/mpenner/Projects/php-ast/php_ast.h:27:16: error: unknown type name ‘zend_string’
 #define X(str) zend_string *str_ ## str;
                ^
/home/mpenner/Projects/php-ast/ast_str_defs.h:7:2: note: in expansion of macro ‘X’
  X(lineno) \
  ^
/home/mpenner/Projects/php-ast/php_ast.h:28:2: note: in expansion of macro ‘AST_STR_DEFS’
  AST_STR_DEFS
  ^
/home/mpenner/Projects/php-ast/php_ast.h:27:16: error: unknown type name ‘zend_string’
 #define X(str) zend_string *str_ ## str;
                ^
/home/mpenner/Projects/php-ast/ast_str_defs.h:8:2: note: in expansion of macro ‘X’
  X(children) \
  ^
/home/mpenner/Projects/php-ast/php_ast.h:28:2: note: in expansion of macro ‘AST_STR_DEFS’
  AST_STR_DEFS
  ^
/home/mpenner/Projects/php-ast/php_ast.h:27:16: error: unknown type name ‘zend_string’
 #define X(str) zend_string *str_ ## str;
                ^
/home/mpenner/Projects/php-ast/ast_str_defs.h:9:2: note: in expansion of macro ‘X’
  X(name) \
  ^
/home/mpenner/Projects/php-ast/php_ast.h:28:2: note: in expansion of macro ‘AST_STR_DEFS’
  AST_STR_DEFS
  ^
/home/mpenner/Projects/php-ast/php_ast.h:27:16: error: unknown type name ‘zend_string’
 #define X(str) zend_string *str_ ## str;
                ^
/home/mpenner/Projects/php-ast/ast_str_defs.h:10:2: note: in expansion of macro ‘X’
  X(docComment) \
  ^
/home/mpenner/Projects/php-ast/php_ast.h:28:2: note: in expansion of macro ‘AST_STR_DEFS’
  AST_STR_DEFS
  ^
/home/mpenner/Projects/php-ast/php_ast.h:27:16: error: unknown type name ‘zend_string’
 #define X(str) zend_string *str_ ## str;
                ^
/home/mpenner/Projects/php-ast/ast_str_defs.h:11:2: note: in expansion of macro ‘X’
  X(endLineno) \
  ^
/home/mpenner/Projects/php-ast/php_ast.h:28:2: note: in expansion of macro ‘AST_STR_DEFS’
  AST_STR_DEFS
  ^
/home/mpenner/Projects/php-ast/php_ast.h:27:16: error: unknown type name ‘zend_string’
 #define X(str) zend_string *str_ ## str;
                ^
/home/mpenner/Projects/php-ast/ast_str_defs.h:12:2: note: in expansion of macro ‘X’
  X(params) \
  ^
/home/mpenner/Projects/php-ast/php_ast.h:28:2: note: in expansion of macro ‘AST_STR_DEFS’
  AST_STR_DEFS
  ^
/home/mpenner/Projects/php-ast/php_ast.h:27:16: error: unknown type name ‘zend_string’
 #define X(str) zend_string *str_ ## str;
                ^
/home/mpenner/Projects/php-ast/ast_str_defs.h:13:2: note: in expansion of macro ‘X’
  X(uses) \
  ^
/home/mpenner/Projects/php-ast/php_ast.h:28:2: note: in expansion of macro ‘AST_STR_DEFS’
  AST_STR_DEFS
  ^
/home/mpenner/Projects/php-ast/php_ast.h:27:16: error: unknown type name ‘zend_string’
 #define X(str) zend_string *str_ ## str;
                ^
/home/mpenner/Projects/php-ast/ast_str_defs.h:14:2: note: in expansion of macro ‘X’
  X(stmts) \
  ^
/home/mpenner/Projects/php-ast/php_ast.h:28:2: note: in expansion of macro ‘AST_STR_DEFS’
  AST_STR_DEFS
  ^
/home/mpenner/Projects/php-ast/php_ast.h:27:16: error: unknown type name ‘zend_string’
 #define X(str) zend_string *str_ ## str;
                ^
/home/mpenner/Projects/php-ast/ast_str_defs.h:15:2: note: in expansion of macro ‘X’
  X(returnType) \
  ^
/home/mpenner/Projects/php-ast/php_ast.h:28:2: note: in expansion of macro ‘AST_STR_DEFS’
  AST_STR_DEFS
  ^
/home/mpenner/Projects/php-ast/php_ast.h:27:16: error: unknown type name ‘zend_string’
 #define X(str) zend_string *str_ ## str;
                ^
/home/mpenner/Projects/php-ast/ast_str_defs.h:16:2: note: in expansion of macro ‘X’
  X(extends) \
  ^
/home/mpenner/Projects/php-ast/php_ast.h:28:2: note: in expansion of macro ‘AST_STR_DEFS’
  AST_STR_DEFS
  ^
/home/mpenner/Projects/php-ast/php_ast.h:27:16: error: unknown type name ‘zend_string’
 #define X(str) zend_string *str_ ## str;
                ^
/home/mpenner/Projects/php-ast/ast_str_defs.h:17:2: note: in expansion of macro ‘X’
  X(implements) \
  ^
/home/mpenner/Projects/php-ast/php_ast.h:28:2: note: in expansion of macro ‘AST_STR_DEFS’
  AST_STR_DEFS
  ^
/home/mpenner/Projects/php-ast/php_ast.h:27:16: error: unknown type name ‘zend_string’
 #define X(str) zend_string *str_ ## str;
                ^
/home/mpenner/Projects/php-ast/ast_str_defs.h:18:2: note: in expansion of macro ‘X’
  X(expr) \
  ^
/home/mpenner/Projects/php-ast/php_ast.h:28:2: note: in expansion of macro ‘AST_STR_DEFS’
  AST_STR_DEFS
  ^
/home/mpenner/Projects/php-ast/php_ast.h:27:16: error: unknown type name ‘zend_string’
 #define X(str) zend_string *str_ ## str;
                ^
/home/mpenner/Projects/php-ast/ast_str_defs.h:19:2: note: in expansion of macro ‘X’
  X(var) \
  ^
/home/mpenner/Projects/php-ast/php_ast.h:28:2: note: in expansion of macro ‘AST_STR_DEFS’
  AST_STR_DEFS
  ^
/home/mpenner/Projects/php-ast/php_ast.h:27:16: error: unknown type name ‘zend_string’
 #define X(str) zend_string *str_ ## str;
                ^
/home/mpenner/Projects/php-ast/ast_str_defs.h:20:2: note: in expansion of macro ‘X’
  X(offset) \
  ^
/home/mpenner/Projects/php-ast/php_ast.h:28:2: note: in expansion of macro ‘AST_STR_DEFS’
  AST_STR_DEFS
  ^
/home/mpenner/Projects/php-ast/php_ast.h:27:16: error: unknown type name ‘zend_string’
 #define X(str) zend_string *str_ ## str;
                ^
/home/mpenner/Projects/php-ast/ast_str_defs.h:21:2: note: in expansion of macro ‘X’
  X(label) \
  ^
/home/mpenner/Projects/php-ast/php_ast.h:28:2: note: in expansion of macro ‘AST_STR_DEFS’
  AST_STR_DEFS
  ^
/home/mpenner/Projects/php-ast/php_ast.h:27:16: error: unknown type name ‘zend_string’
 #define X(str) zend_string *str_ ## str;
                ^
/home/mpenner/Projects/php-ast/ast_str_defs.h:22:2: note: in expansion of macro ‘X’
  X(depth) \
  ^
/home/mpenner/Projects/php-ast/php_ast.h:28:2: note: in expansion of macro ‘AST_STR_DEFS’
  AST_STR_DEFS
  ^
/home/mpenner/Projects/php-ast/php_ast.h:27:16: error: unknown type name ‘zend_string’
 #define X(str) zend_string *str_ ## str;
                ^
/home/mpenner/Projects/php-ast/ast_str_defs.h:23:2: note: in expansion of macro ‘X’
  X(dim) \
  ^
/home/mpenner/Projects/php-ast/php_ast.h:28:2: note: in expansion of macro ‘AST_STR_DEFS’
  AST_STR_DEFS
  ^
/home/mpenner/Projects/php-ast/php_ast.h:27:16: error: unknown type name ‘zend_string’
 #define X(str) zend_string *str_ ## str;
                ^
/home/mpenner/Projects/php-ast/ast_str_defs.h:24:2: note: in expansion of macro ‘X’
  X(prop) \
  ^
/home/mpenner/Projects/php-ast/php_ast.h:28:2: note: in expansion of macro ‘AST_STR_DEFS’
  AST_STR_DEFS
  ^
/home/mpenner/Projects/php-ast/php_ast.h:27:16: error: unknown type name ‘zend_string’
 #define X(str) zend_string *str_ ## str;
                ^
/home/mpenner/Projects/php-ast/ast_str_defs.h:25:2: note: in expansion of macro ‘X’
  X(class) \
  ^
/home/mpenner/Projects/php-ast/php_ast.h:28:2: note: in expansion of macro ‘AST_STR_DEFS’
  AST_STR_DEFS
  ^
/home/mpenner/Projects/php-ast/php_ast.h:27:16: error: unknown type name ‘zend_string’
 #define X(str) zend_string *str_ ## str;
                ^
/home/mpenner/Projects/php-ast/ast_str_defs.h:26:2: note: in expansion of macro ‘X’
  X(args) \
  ^
/home/mpenner/Projects/php-ast/php_ast.h:28:2: note: in expansion of macro ‘AST_STR_DEFS’
  AST_STR_DEFS
  ^
/home/mpenner/Projects/php-ast/php_ast.h:27:16: error: unknown type name ‘zend_string’
 #define X(str) zend_string *str_ ## str;
                ^
/home/mpenner/Projects/php-ast/ast_str_defs.h:27:2: note: in expansion of macro ‘X’
  X(const) \
  ^
/home/mpenner/Projects/php-ast/php_ast.h:28:2: note: in expansion of macro ‘AST_STR_DEFS’
  AST_STR_DEFS
  ^
/home/mpenner/Projects/php-ast/php_ast.h:27:16: error: unknown type name ‘zend_string’
 #define X(str) zend_string *str_ ## str;
                ^
/home/mpenner/Projects/php-ast/ast_str_defs.h:28:2: note: in expansion of macro ‘X’
  X(left) \
  ^
/home/mpenner/Projects/php-ast/php_ast.h:28:2: note: in expansion of macro ‘AST_STR_DEFS’
  AST_STR_DEFS
  ^
/home/mpenner/Projects/php-ast/php_ast.h:27:16: error: unknown type name ‘zend_string’
 #define X(str) zend_string *str_ ## str;
                ^
/home/mpenner/Projects/php-ast/ast_str_defs.h:29:2: note: in expansion of macro ‘X’
  X(right) \
  ^
/home/mpenner/Projects/php-ast/php_ast.h:28:2: note: in expansion of macro ‘AST_STR_DEFS’
  AST_STR_DEFS
  ^
/home/mpenner/Projects/php-ast/php_ast.h:27:16: error: unknown type name ‘zend_string’
 #define X(str) zend_string *str_ ## str;
                ^
/home/mpenner/Projects/php-ast/ast_str_defs.h:30:2: note: in expansion of macro ‘X’
  X(value) \
  ^
/home/mpenner/Projects/php-ast/php_ast.h:28:2: note: in expansion of macro ‘AST_STR_DEFS’
  AST_STR_DEFS
  ^
/home/mpenner/Projects/php-ast/php_ast.h:27:16: error: unknown type name ‘zend_string’
 #define X(str) zend_string *str_ ## str;
                ^
/home/mpenner/Projects/php-ast/ast_str_defs.h:31:2: note: in expansion of macro ‘X’
  X(key) \
  ^
/home/mpenner/Projects/php-ast/php_ast.h:28:2: note: in expansion of macro ‘AST_STR_DEFS’
  AST_STR_DEFS
  ^
/home/mpenner/Projects/php-ast/php_ast.h:27:16: error: unknown type name ‘zend_string’
 #define X(str) zend_string *str_ ## str;
                ^
/home/mpenner/Projects/php-ast/ast_str_defs.h:32:2: note: in expansion of macro ‘X’
  X(default) \
  ^
/home/mpenner/Projects/php-ast/php_ast.h:28:2: note: in expansion of macro ‘AST_STR_DEFS’
  AST_STR_DEFS
  ^
/home/mpenner/Projects/php-ast/php_ast.h:27:16: error: unknown type name ‘zend_string’
 #define X(str) zend_string *str_ ## str;
                ^
/home/mpenner/Projects/php-ast/ast_str_defs.h:33:2: note: in expansion of macro ‘X’
  X(cond) \
  ^
/home/mpenner/Projects/php-ast/php_ast.h:28:2: note: in expansion of macro ‘AST_STR_DEFS’
  AST_STR_DEFS
  ^
/home/mpenner/Projects/php-ast/php_ast.h:27:16: error: unknown type name ‘zend_string’
 #define X(str) zend_string *str_ ## str;
                ^
/home/mpenner/Projects/php-ast/ast_str_defs.h:34:2: note: in expansion of macro ‘X’
  X(declares) \
  ^
/home/mpenner/Projects/php-ast/php_ast.h:28:2: note: in expansion of macro ‘AST_STR_DEFS’
  AST_STR_DEFS
  ^
/home/mpenner/Projects/php-ast/php_ast.h:27:16: error: unknown type name ‘zend_string’
 #define X(str) zend_string *str_ ## str;
                ^
/home/mpenner/Projects/php-ast/ast_str_defs.h:35:2: note: in expansion of macro ‘X’
  X(traits) \
  ^
/home/mpenner/Projects/php-ast/php_ast.h:28:2: note: in expansion of macro ‘AST_STR_DEFS’
  AST_STR_DEFS
  ^
/home/mpenner/Projects/php-ast/php_ast.h:27:16: error: unknown type name ‘zend_string’
 #define X(str) zend_string *str_ ## str;
                ^
/home/mpenner/Projects/php-ast/ast_str_defs.h:36:2: note: in expansion of macro ‘X’
  X(adaptations) \
  ^
/home/mpenner/Projects/php-ast/php_ast.h:28:2: note: in expansion of macro ‘AST_STR_DEFS’
  AST_STR_DEFS
  ^
/home/mpenner/Projects/php-ast/php_ast.h:27:16: error: unknown type name ‘zend_string’
 #define X(str) zend_string *str_ ## str;
                ^
/home/mpenner/Projects/php-ast/ast_str_defs.h:37:2: note: in expansion of macro ‘X’
  X(method) \
  ^
/home/mpenner/Projects/php-ast/php_ast.h:28:2: note: in expansion of macro ‘AST_STR_DEFS’
  AST_STR_DEFS
  ^
/home/mpenner/Projects/php-ast/php_ast.h:27:16: error: unknown type name ‘zend_string’
 #define X(str) zend_string *str_ ## str;
                ^
/home/mpenner/Projects/php-ast/ast_str_defs.h:38:2: note: in expansion of macro ‘X’
  X(insteadof) \
  ^
/home/mpenner/Projects/php-ast/php_ast.h:28:2: note: in expansion of macro ‘AST_STR_DEFS’
  AST_STR_DEFS
  ^
/home/mpenner/Projects/php-ast/php_ast.h:27:16: error: unknown type name ‘zend_string’
 #define X(str) zend_string *str_ ## str;
                ^
/home/mpenner/Projects/php-ast/ast_str_defs.h:39:2: note: in expansion of macro ‘X’
  X(alias) \
  ^
/home/mpenner/Projects/php-ast/php_ast.h:28:2: note: in expansion of macro ‘AST_STR_DEFS’
  AST_STR_DEFS
  ^
/home/mpenner/Projects/php-ast/php_ast.h:27:16: error: unknown type name ‘zend_string’
 #define X(str) zend_string *str_ ## str;
                ^
/home/mpenner/Projects/php-ast/ast_str_defs.h:40:2: note: in expansion of macro ‘X’
  X(prefix) \
  ^
/home/mpenner/Projects/php-ast/php_ast.h:28:2: note: in expansion of macro ‘AST_STR_DEFS’
  AST_STR_DEFS
  ^
/home/mpenner/Projects/php-ast/php_ast.h:27:16: error: unknown type name ‘zend_string’
 #define X(str) zend_string *str_ ## str;
                ^
/home/mpenner/Projects/php-ast/ast_str_defs.h:41:2: note: in expansion of macro ‘X’
  X(true) \
  ^
/home/mpenner/Projects/php-ast/php_ast.h:28:2: note: in expansion of macro ‘AST_STR_DEFS’
  AST_STR_DEFS
  ^
/home/mpenner/Projects/php-ast/php_ast.h:27:16: error: unknown type name ‘zend_string’
 #define X(str) zend_string *str_ ## str;
                ^
/home/mpenner/Projects/php-ast/ast_str_defs.h:42:2: note: in expansion of macro ‘X’
  X(false) \
  ^
/home/mpenner/Projects/php-ast/php_ast.h:28:2: note: in expansion of macro ‘AST_STR_DEFS’
  AST_STR_DEFS
  ^
/home/mpenner/Projects/php-ast/php_ast.h:27:16: error: unknown type name ‘zend_string’
 #define X(str) zend_string *str_ ## str;
                ^
/home/mpenner/Projects/php-ast/ast_str_defs.h:43:2: note: in expansion of macro ‘X’
  X(try) \
  ^
/home/mpenner/Projects/php-ast/php_ast.h:28:2: note: in expansion of macro ‘AST_STR_DEFS’
  AST_STR_DEFS
  ^
/home/mpenner/Projects/php-ast/php_ast.h:27:16: error: unknown type name ‘zend_string’
 #define X(str) zend_string *str_ ## str;
                ^
/home/mpenner/Projects/php-ast/ast_str_defs.h:44:2: note: in expansion of macro ‘X’
  X(catches) \
  ^
/home/mpenner/Projects/php-ast/php_ast.h:28:2: note: in expansion of macro ‘AST_STR_DEFS’
  AST_STR_DEFS
  ^
/home/mpenner/Projects/php-ast/php_ast.h:27:16: error: unknown type name ‘zend_string’
 #define X(str) zend_string *str_ ## str;
                ^
/home/mpenner/Projects/php-ast/ast_str_defs.h:45:2: note: in expansion of macro ‘X’
  X(finally) \
  ^
/home/mpenner/Projects/php-ast/php_ast.h:28:2: note: in expansion of macro ‘AST_STR_DEFS’
  AST_STR_DEFS
  ^
/home/mpenner/Projects/php-ast/php_ast.h:27:16: error: unknown type name ‘zend_string’
 #define X(str) zend_string *str_ ## str;
                ^
/home/mpenner/Projects/php-ast/ast_str_defs.h:46:2: note: in expansion of macro ‘X’
  X(type) \
  ^
/home/mpenner/Projects/php-ast/php_ast.h:28:2: note: in expansion of macro ‘AST_STR_DEFS’
  AST_STR_DEFS
  ^
/home/mpenner/Projects/php-ast/php_ast.h:27:16: error: unknown type name ‘zend_string’
 #define X(str) zend_string *str_ ## str;
                ^
/home/mpenner/Projects/php-ast/ast_str_defs.h:47:2: note: in expansion of macro ‘X’
  X(init) \
  ^
/home/mpenner/Projects/php-ast/php_ast.h:28:2: note: in expansion of macro ‘AST_STR_DEFS’
  AST_STR_DEFS
  ^
/home/mpenner/Projects/php-ast/php_ast.h:27:16: error: unknown type name ‘zend_string’
 #define X(str) zend_string *str_ ## str;
                ^
/home/mpenner/Projects/php-ast/ast_str_defs.h:48:2: note: in expansion of macro ‘X’
  X(loop) \
  ^
/home/mpenner/Projects/php-ast/php_ast.h:28:2: note: in expansion of macro ‘AST_STR_DEFS’
  AST_STR_DEFS
  ^
/home/mpenner/Projects/php-ast/php_ast.h:44:14: error: unknown type name ‘zend_ast_kind’
 extern const zend_ast_kind ast_kinds[];
              ^
/home/mpenner/Projects/php-ast/php_ast.h:46:30: error: unknown type name ‘zend_ast_kind’
 const char *ast_kind_to_name(zend_ast_kind kind);
                              ^
/home/mpenner/Projects/php-ast/php_ast.h:47:1: error: unknown type name ‘zend_string’
 zend_string *ast_kind_child_name(zend_ast_kind kind, uint32_t child);
 ^
/home/mpenner/Projects/php-ast/php_ast.h:47:34: error: unknown type name ‘zend_ast_kind’
 zend_string *ast_kind_child_name(zend_ast_kind kind, uint32_t child);
                                  ^
/home/mpenner/Projects/php-ast/php_ast.h:47:54: error: unknown type name ‘uint32_t’
 zend_string *ast_kind_child_name(zend_ast_kind kind, uint32_t child);
                                                      ^
/home/mpenner/Projects/php-ast/ast.c:15:28: fatal error: zend_smart_str.h: No such file or directory
 #include "zend_smart_str.h"
                            ^
compilation terminated.
Makefile:193: recipe for target 'ast.lo' failed
make: *** [ast.lo] Error 1

Introduce named children

PHP-Parser had a pretty printer that allowed to compile an AST (as generated by PHP-Parser) back to PHP code:
https://github.com/nikic/PHP-Parser/blob/1.x/doc/2_Usage_of_basic_components.markdown#pretty-printer

It would be great to have something like this in the php-ast extension, allowing for example to manipulate ASTs exposed by this extension and converting them back to PHP code.

Installation on Ubuntu 16.04 in wrong place

Using apt-get to install the php-ast package results in the .ini file being created at /etc/php/mods-available/ast.ini instead of /etc/php/7.0/mods-available/ast.ini like the rest of the php extensions.

This required me to copy the ast.ini file to the correct place in order to 'sudo phpenmod ast'. Then all worked as expected.

Inclusion of "utils.php" format_flags into the AST module (?)

Hi,

I wrote a script using a modified version of "utils.php" including in this repository, but on upgrade to PHP 7.1, the script stopped working due to some modifications. I've got a temporary fix, but I think it would make more sense to hardwire this functionality into the extension itself.

I can understand that certain uses might benefit from not having the flags formatted, but it could be an option to ast_parse (default true or false) to list the flags as an array of strings or numbers. Managing the exclusive/combinable options manually is a bit much for non-performance critical logic.

Unable to install on OS X 10.11.1 w/ PHP 7.0.0

I'm attempting to create a homebrew-php package for this extension, but I am having a hard time getting it to compile. Output below:

==> Cloning https://github.com/nikic/php-ast.git
Cloning into '/Library/Caches/Homebrew/php70-ast--git'...
remote: Counting objects: 41, done.
remote: Compressing objects: 100% (38/38), done.
remote: Total 41 (delta 1), reused 19 (delta 0), pack-reused 0
Unpacking objects: 100% (41/41), done.
Checking connectivity... done.
==> Checking out branch master
==> /usr/local/opt/php70/bin/phpize
==> ./configure --prefix=/usr/local/Cellar/php70-ast/HEAD --with-php-config=/usr/local/opt/php70/bin/php-config
==> make
Last 15 lines from /Users/ablyler/Library/Logs/Homebrew/php70-ast/03.make:
./php_ast.h:37:28: note: expanded from macro 'AST_STR'
#define AST_STR(str) AST_G(str_ ## str)
                     ~~~~~~^~~~~~~~~~~~
<scratch space>:52:1: note: expanded from here
str_0
^
./php_ast.h:35:52: note: expanded from macro 'AST_G'
#define AST_G(v) ZEND_MODULE_GLOBALS_ACCESSOR(ast, v)
                                                   ^
/usr/local/Cellar/php70/7.0.0/include/php/Zend/zend_API.h:173:77: note: expanded from macro 'ZEND_MODULE_GLOBALS_ACCESSOR'
#define ZEND_MODULE_GLOBALS_ACCESSOR(module_name, v) (module_name##_globals.v)
                                                                            ^
4 errors generated.
make: *** [ast.lo] Error 1
make: *** Waiting for unfinished jobs....

v0.1.5 release

In order for projects to start using new php-ast features and indicate a composer dependency, a release would be needed (e.g. after php 7.2 feature freeze)

Are any more changes planned? (I can't think of anything)
Likely need to update package.xml with documentation and new tests, optionally publish with tgz file usable by pecl

which one is better?php-ast or php parser

Hello
which one is better?php-ast or php parser
Thank you.

Version 35 is deprecated

Our build broke with v0.15.0. We are using PHP 7.1 on travis and need php-ast for etsy/phan:

/home/travis/.composer/vendor/etsy/phan/src/phan.php:27 [8192] ast\parse_code(): Version 35 is deprecated
#0  phan_error_handler()
#1  ast\parse_code() called at [/home/travis/.composer/vendor/etsy/phan/src/phan.php:27]
#2  require_once(/home/travis/.composer/vendor/etsy/phan/src/phan.php) called at [/home/travis/.composer/vendor/etsy/phan/phan:2]

Stable release

Please create a stable release of php-ast in order to meet the distribution rules of the IUS (rhel/centos) repo program, to ease installation on machines that use their compilation of php7.

Request for tagged versions in GitHub

I'd like to create a homebrew-php for your extension, but need tagged releases to do so. Can you please do this?

What is the filename used for?

ast\parse_code() accepts an optional $filename parameter but the readme doesn't say what it's for. I tried for follow the code and ended up in zend_set_compiled_filename() which talks about CG(compiled_filename) and CG(filenames_table) but couldn't get much further.

Does the filename affect the output of ast\parse_code() in any way? Is it just used in case of error, perhaps?

Expose AST of classes and functions via reflection

A benefit of an API such of this would be for analyzing code of user defined functions and classes at run time. Being able to get the AST of specific closures/functions/methods or classes/traits/interfaces would be very useful rather than having to parse an entire file as a string and attempting to locate the desired node.

Perhaps respective methods such as:

ReflectionFunctionAbstract::getAST()
ReflectionClass::getAST()

That would act equivalent to ast\parseCode($code) with the code to which the reflection is referring.

PHP 7.1 support

Make sure all PHP 7.1 features are properly supported.

Void return type: https://wiki.php.net/rfc/void_return_type (Already done as part of f199ac8)
Class constant visibility: https://wiki.php.net/rfc/class_const_visibility (Already done in 26bf9d3)
Keys in list(): https://wiki.php.net/rfc/list_keys (322e614)
Multi catch: https://wiki.php.net/rfc/multiple-catch (2d3812a)
Destructuring using []: https://wiki.php.net/rfc/short_list_syntax (322e614)
Nullable types: https://wiki.php.net/rfc/nullable_types (fc5048f)
Iterable type: https://wiki.php.net/rfc/iterable (ed51bba)
Octal string overflow: https://wiki.php.net/rfc/octal.overload-checking (not actionable)

Generally would be nice to create a new AST version and normalize to 7.0/7.1 format based on it.

Strange start line numbers for AST_IF elements

For the below code, with ast VERSION 40 (or 30)

<?php
if (true) {

} else {
}

AST_STMT_LIST @ 1
    0: AST_IF @ 4
        0: AST_IF_ELEM @ 2
            cond: AST_CONST @ 2
                name: AST_NAME @ 2
                    flags: NAME_NOT_FQ (1)
                    name: "true"
            stmts: AST_STMT_LIST @ 2
        1: AST_IF_ELEM @ 4
            cond: null
            stmts: AST_STMT_LIST @ 4

Observed: The line number of AST_IF is 4. The line number of the first AST_IF_ELEM below it is 2.
Expected: The line number of AST_IF should be 2 (4 doesn't make sense as a starting line number)

php 7.1.6 + mysql_xdevapi + ast 0.1.4 = segfault

Working on phan test suite

$ gdb php71
(gdb) run -d extension=mysql_xdevapi.so /usr/bin/phan --version
Program received signal SIGSEGV, Segmentation fault.
(gdb) bt
#0  0x00007ffff48b8f86 in strlen () from /lib64/libc.so.6
#1  0x000055555568b1ac in reflection_type_name (param=<optimized out>) at /usr/src/debug/php-7.1.6/ext/reflection/php_reflection.c:3020
#2  0x00005555556992ba in zim_reflection_type___toString (execute_data=0x7ffff3813c30, return_value=0x7fffffff9d60)
    at /usr/src/debug/php-7.1.6/ext/reflection/php_reflection.c:3046
#3  0x000055555579ccd3 in zend_call_function (fci=fci@entry=0x7fffffff9c90, fci_cache=<optimized out>, fci_cache@entry=0x7fffffff9c60)
    at /usr/src/debug/php-7.1.6/Zend/zend_execute_API.c:871
#4  0x00005555557cb3d4 in zend_call_method (object=object@entry=0x7ffff3813ac0, obj_ce=obj_ce@entry=0x555555cc7920, fn_proxy=fn_proxy@entry=0x555555cc7a50, 
    function_name=function_name@entry=0x5555558a99ab "__tostring", function_name_len=function_name_len@entry=10, retval_ptr=retval_ptr@entry=0x7fffffff9d60, param_count=0, 
    arg1=0x0, arg2=0x0) at /usr/src/debug/php-7.1.6/Zend/zend_interfaces.c:99
#5  0x00005555557e7a83 in zend_std_cast_object_tostring (readobj=0x7ffff3813ac0, writeobj=0x7fffffff9de0, type=<optimized out>)
    at /usr/src/debug/php-7.1.6/Zend/zend_object_handlers.c:1631
#6  0x00005555557a350e in _zval_get_string_func (op=op@entry=0x7ffff3813ac0) at /usr/src/debug/php-7.1.6/Zend/zend_operators.c:887
#7  0x000055555583e768 in _zval_get_string (op=<optimized out>) at /usr/src/debug/php-7.1.6/Zend/zend_operators.h:276
#8  ZEND_CAST_SPEC_CV_HANDLER () at /usr/src/debug/php-7.1.6/Zend/zend_vm_execute.h:35429
#9  0x00005555557f68bb in execute_ex (ex=<optimized out>) at /usr/src/debug/php-7.1.6/Zend/zend_vm_execute.h:429
#10 0x000055555584f68e in ZEND_DO_FCALL_SPEC_RETVAL_USED_HANDLER () at /usr/src/debug/php-7.1.6/Zend/zend_vm_execute.h:1076
#11 0x00005555557f68bb in execute_ex (ex=<optimized out>) at /usr/src/debug/php-7.1.6/Zend/zend_vm_execute.h:429
#12 0x000055555584f68e in ZEND_DO_FCALL_SPEC_RETVAL_USED_HANDLER () at /usr/src/debug/php-7.1.6/Zend/zend_vm_execute.h:1076
#13 0x00005555557f68bb in execute_ex (ex=<optimized out>) at /usr/src/debug/php-7.1.6/Zend/zend_vm_execute.h:429
#14 0x000055555584f68e in ZEND_DO_FCALL_SPEC_RETVAL_USED_HANDLER () at /usr/src/debug/php-7.1.6/Zend/zend_vm_execute.h:1076
#15 0x00005555557f68bb in execute_ex (ex=<optimized out>) at /usr/src/debug/php-7.1.6/Zend/zend_vm_execute.h:429
#16 0x000055555579cbb4 in zend_call_function (fci=0x7ffff3813a70, fci@entry=0x7fffffffa100, fci_cache=<optimized out>, fci_cache@entry=0x7fffffffa0d0)
    at /usr/src/debug/php-7.1.6/Zend/zend_execute_API.c:855
#17 0x00005555556e033e in zif_array_map (execute_data=<optimized out>, return_value=0x7ffff38137d0) at /usr/src/debug/php-7.1.6/ext/standard/array.c:5359
#18 0x000055555584f5bc in ZEND_DO_FCALL_SPEC_RETVAL_USED_HANDLER () at /usr/src/debug/php-7.1.6/Zend/zend_vm_execute.h:1099
#19 0x00005555557f68bb in execute_ex (ex=<optimized out>) at /usr/src/debug/php-7.1.6/Zend/zend_vm_execute.h:429
#20 0x000055555584f68e in ZEND_DO_FCALL_SPEC_RETVAL_USED_HANDLER () at /usr/src/debug/php-7.1.6/Zend/zend_vm_execute.h:1076
#21 0x00005555557f68bb in execute_ex (ex=<optimized out>) at /usr/src/debug/php-7.1.6/Zend/zend_vm_execute.h:429
#22 0x000055555584f68e in ZEND_DO_FCALL_SPEC_RETVAL_USED_HANDLER () at /usr/src/debug/php-7.1.6/Zend/zend_vm_execute.h:1076
#23 0x00005555557f68bb in execute_ex (ex=<optimized out>) at /usr/src/debug/php-7.1.6/Zend/zend_vm_execute.h:429
#24 0x000055555584f68e in ZEND_DO_FCALL_SPEC_RETVAL_USED_HANDLER () at /usr/src/debug/php-7.1.6/Zend/zend_vm_execute.h:1076
#25 0x00005555557f68bb in execute_ex (ex=<optimized out>) at /usr/src/debug/php-7.1.6/Zend/zend_vm_execute.h:429
#26 0x000055555584f68e in ZEND_DO_FCALL_SPEC_RETVAL_USED_HANDLER () at /usr/src/debug/php-7.1.6/Zend/zend_vm_execute.h:1076
#27 0x00005555557f68bb in execute_ex (ex=<optimized out>) at /usr/src/debug/php-7.1.6/Zend/zend_vm_execute.h:429
#28 0x000055555584fc1c in ZEND_DO_FCALL_SPEC_RETVAL_UNUSED_HANDLER () at /usr/src/debug/php-7.1.6/Zend/zend_vm_execute.h:949
#29 0x00005555557f68bb in execute_ex (ex=<optimized out>) at /usr/src/debug/php-7.1.6/Zend/zend_vm_execute.h:429
#30 0x000055555584fc1c in ZEND_DO_FCALL_SPEC_RETVAL_UNUSED_HANDLER () at /usr/src/debug/php-7.1.6/Zend/zend_vm_execute.h:949
#31 0x00005555557f68bb in execute_ex (ex=<optimized out>) at /usr/src/debug/php-7.1.6/Zend/zend_vm_execute.h:429
#32 0x000055555584c774 in ZEND_INCLUDE_OR_EVAL_SPEC_CONST_HANDLER () at /usr/src/debug/php-7.1.6/Zend/zend_vm_execute.h:3479
#33 0x00005555557f68bb in execute_ex (ex=<optimized out>) at /usr/src/debug/php-7.1.6/Zend/zend_vm_execute.h:429
#34 0x0000555555851c28 in zend_execute (op_array=op_array@entry=0x7ffff3887000, return_value=return_value@entry=0x7fffd2a5ce50)
    at /usr/src/debug/php-7.1.6/Zend/zend_vm_execute.h:474
#35 0x00005555557acd33 in zend_execute_scripts (type=-209635472, type@entry=8, retval=0x7fffd2a5ce50, retval@entry=0x0, file_count=file_count@entry=3)
    at /usr/src/debug/php-7.1.6/Zend/zend.c:1476
#36 0x0000555555748d20 in php_execute_script (primary_file=0x7fffffffca20) at /usr/src/debug/php-7.1.6/main/main.c:2537
#37 0x0000555555853e8c in do_cli (argc=5, argv=0x555555bcc200) at /usr/src/debug/php-7.1.6/sapi/cli/php_cli.c:993
#38 0x000055555562158a in main (argc=5, argv=0x555555bcc200) at /usr/src/debug/php-7.1.6/sapi/cli/php_cli.c:1381

util.php doesn't dump reference flag description for node of type AST_CLOSURE_VAR (low severity)

With the master branch or latest release of php-ast

php > echo ast_dump(\ast\parse_code('<?php function(&$x) use (&$y) {};', 40));
AST_STMT_LIST
    0: AST_CLOSURE
        flags: 0
        name: {closure}
        params: AST_PARAM_LIST
            0: AST_PARAM
                flags: PARAM_REF (1)
                type: null
                name: "x"
                default: null
        uses: AST_CLOSURE_USES
            0: AST_CLOSURE_VAR
                flags: 1
                name: "y"
        stmts: AST_STMT_LIST
        returnType: null

Observed: "flags: 1"
Expected: "flags: PARAM_REF" (Or create something similar such as USE_REF)

inconsistent dll linkage

Hi @nikic

Your extension and the astkit extension of @sgolemon throw an 'inconsistent dll linkage' error when including zend_language_scanner.h. See sgolemon/astkit#2 (comment)

I am not familiar with the zend api, but there seems to be a missing ZEND_API at this line in PHP master:
https://github.com/php/php-src/blob/master/Zend/zend_language_scanner.h#L66

Would it break anything to add it there?

Does not build against current master of PHP

/bin/sh /usr/local/src/php/php-ast/libtool --mode=compile ccache clang  -I. -I/usr/local/src/php/php-ast -DPHP_ATOM_INC -I/usr/local/src/php/php-ast/include -I/usr/local/src/php/php-ast/main -I/usr/local/src/php/php-ast -I/usr/local/php-7.0/include/php -I/usr/local/php-7.0/include/php/main -I/usr/local/php-7.0/include/php/TSRM -I/usr/local/php-7.0/include/php/Zend -I/usr/local/php-7.0/include/php/ext -I/usr/local/php-7.0/include/php/ext/date/lib  -DHAVE_CONFIG_H  -march=native -mtune=native -fPIC -pipe -fcolor-diagnostics -Qunused-arguments -O0   -c /usr/local/src/php/php-ast/ast.c -o ast.lo 
mkdir .libs
 ccache clang -I. -I/usr/local/src/php/php-ast -DPHP_ATOM_INC -I/usr/local/src/php/php-ast/include -I/usr/local/src/php/php-ast/main -I/usr/local/src/php/php-ast -I/usr/local/php-7.0/include/php -I/usr/local/php-7.0/include/php/main -I/usr/local/php-7.0/include/php/TSRM -I/usr/local/php-7.0/include/php/Zend -I/usr/local/php-7.0/include/php/ext -I/usr/local/php-7.0/include/php/ext/date/lib -DHAVE_CONFIG_H -march=native -mtune=native -fPIC -pipe -fcolor-diagnostics -Qunused-arguments -O0 -c /usr/local/src/php/php-ast/ast.c  -fPIC -DPIC -o .libs/ast.o
In file included from /usr/local/src/php/php-ast/ast.c:5:
In file included from /usr/local/php-7.0/include/php/main/php.h:35:
In file included from /usr/local/php-7.0/include/php/Zend/zend.h:33:
/usr/local/php-7.0/include/php/Zend/zend_alloc.h:57:236: warning: unknown attribute 'alloc_size' ignored [-Wunknown-attributes]
__attribute__ ((visibility("default"))) void* _emalloc(size_t size , const char *__zend_filename, const uint __zend_lineno , const char *__zend_orig_filename, const uint __zend_orig_lineno) __attribute__ ((__malloc__)) __attribute__ ((alloc_size(1)));
                                                                                                                                                                                                                                           ^
/usr/local/php-7.0/include/php/Zend/zend_alloc.h:61:250: warning: unknown attribute 'alloc_size' ignored [-Wunknown-attributes]
__attribute__ ((visibility("default"))) void* _ecalloc(size_t nmemb, size_t size , const char *__zend_filename, const uint __zend_lineno , const char *__zend_orig_filename, const uint __zend_orig_lineno) __attribute__ ((__malloc__)) __attribute__ ((alloc_size(1,2)));
                                                                                                                                                                                                                                                         ^
/usr/local/php-7.0/include/php/Zend/zend_alloc.h:62:238: warning: unknown attribute 'alloc_size' ignored [-Wunknown-attributes]
__attribute__ ((visibility("default"))) void* _erealloc(void *ptr, size_t size, int allow_failure , const char *__zend_filename, const uint __zend_lineno , const char *__zend_orig_filename, const uint __zend_orig_lineno) __attribute__ ((alloc_size(2)));
                                                                                                                                                                                                                                             ^
/usr/local/src/php/php-ast/ast.c:326:107: error: use of undeclared identifier 'ZEND_ACC_FINAL_CLASS'
 zend_register_long_constant("ast\\flags" "\\" "CLASS_FINAL", sizeof("ast\\flags" "\\" "CLASS_FINAL")-1, (ZEND_ACC_FINAL_CLASS), ((1<<0) | (1<<1)), module_number);
                                                                                                          ^
3 warnings and 1 error generated.
Makefile:180: recipe for target 'ast.lo' failed
make: *** [ast.lo] Error 1

php 7.1 : php-ast has no constant value to indicate some nullable internal types

Using the latest version of the master branch, and php 7.1.0

(Applies to ast version 35 and 40)

echo ast_dump(ast\parse_code('<?php function foo(?callable $a, ?array $b, ?string $c){}', 35));    
AST_STMT_LIST
    0: AST_FUNC_DECL
        flags: 0
        name: foo
        params: AST_PARAM_LIST
            0: AST_PARAM
                flags: 0
                type: AST_TYPE
                    flags: 270
                name: "a"
                default: null
            1: AST_PARAM
                flags: 0
                type: AST_TYPE
                    flags: 263
                name: "b"
                default: null
            2: AST_PARAM
                flags: 0
                type: AST_NULLABLE_TYPE
                    type: AST_NAME
                        flags: NAME_NOT_FQ (1)
                        name: "string"
                name: "c"
                default: null
        uses: null
        stmts: AST_STMT_LIST
        returnType: null

Observed behavior:

util.php does not dump a list of bitwised ored constant names, it dumps integers (263 = 256|7, 270 = 256|14)
There doesn't seem to be a constant in util.php for the bit flag 256?
some types, such as "string", are wrapped in AST_NULLABLE_TYPE. Others aren't (array, callable). Is this so that the output reflects the internal data structure?

Expected behavior:

php-ast extension should either

expose some constant for 256 (flags\AST_TYPE_BITFLAG_NULLABLE or something, or even AST_TYPE_CALLABLE_NULLABLE)
Wrap the AST_TYPE for array, callable, etc., in an AST_NULLABLE_TYPE, for consistency

util.php should decode and print the above example in a human readable way.

Documentation and tests completeness

I looked through the list of element types in phpast_kind.c, and there are a number of them which are obviously deprecated, such as, for example the BINARY and UNARY ops cover a few types that once had their own constant.

A great number of these elements are obvious what they do and that they are not deprecated, but some of them, such as AST_ENCAPS_LIST, I could not figure out what they are a reference to and/or if they are still valid element types.

I searched through the tests folder with GREP, and found 50 elements with no tests. Most are obvious, but some are not obvious if they are still in use, for example, exit could have been converted to a function call, the various call elements could have been combined, etc. I can figure out most of these with my own tests, but if anyone has any knowledge of what has been deprecated, please post it, and also please post if you know what is "PARAM_VARIADIC" and "AST_ENCAPS_LIST".

To make this a useful issue, I am proposing documentation of deprecated elements, as well as the addition of tests for all active element types, mostly as a basic method of documentation, so that one search for the type and find where it was used.

The list of elements with no test found by grep (some clearly deprecated elements removed):

AST_ENCAPS_LIST
AST_EXPR_LIST
AST_SWITCH_LIST
AST_TRAIT_ADAPTATIONS
AST_UNPACK
AST_UNARY_PLUS
AST_UNARY_MINUS
AST_CAST
AST_EMPTY
AST_ISSET
AST_SHELL_EXEC
AST_CLONE
AST_EXIT
AST_PRINT
AST_PRE_INC
AST_PRE_DEC
AST_POST_INC
AST_POST_DEC
AST_YIELD_FROM
AST_GLOBAL
AST_UNSET
AST_LABEL
AST_REF
AST_HALT_COMPILER
AST_THROW
AST_GOTO
AST_BREAK
AST_CONTINUE
AST_DIM
AST_STATIC_PROP
AST_ASSIGN_REF
AST_ASSIGN_OP
AST_AND
AST_OR
AST_ARRAY_ELEM
AST_NEW
AST_YIELD
AST_STATIC
AST_DO_WHILE
AST_SWITCH
AST_SWITCH_CASE
AST_TRAIT_PRECEDENCE
AST_METHOD_REFERENCE
AST_NAMESPACE
AST_TRAIT_ALIAS
AST_METHOD_CALL
AST_STATIC_CALL
AST_CONDITIONAL
AST_FOR
AST_FOREACH

Publish package on Packagist / Composer

Hello,

I think it will be a good idea to publish this package on Packagist, for the PHP stub?

Missing return_type node

It doesn't seem like function and method return types are anywhere to be found in the AST_FUNC_DECL/AST_METHOD nodes. eg.

$ php -r "print_r(ast\parse_code('<?php function test():string { }'));"
ast\Node Object
(
    [kind] => 133
    [flags] => 0
    [lineno] => 1
    [children] => Array
        (
            [0] => ast\Node Object
                (
                    [kind] => 66
                    [flags] => 0
                    [lineno] => 1
                    [children] => Array
                        (
                            [0] => ast\Node Object
                                (
                                    [kind] => 137
                                    [flags] => 0
                                    [lineno] => 1
                                    [children] => Array
                                        (
                                        )

                                )

                            [1] => 
                            [2] => ast\Node Object
                                (
                                    [kind] => 133
                                    [flags] => 0
                                    [lineno] => 1
                                    [children] => Array
                                        (
                                        )

                                )

                        )

                    [endLineno] => 1
                    [name] => test
                    [docComment] => 
                )

        )

)

There should be a 4th child there, shouldn't there?

Allow "@var" type comments on variables

The code like this allows to ignore return type set in DocBlock of makeClass method and use one in @var comment instead.

/** @var kEmail $email */
$email = $this->makeClass('....');

This is common approach when dealing with factory methods, that return class based on specific text (not necessary class name) given as input to the factory method.

This is currently a show stopper for phan/phan#313 .

minor: Changelog should mention AST_PROP_ELEM is also affected in version 50

Before (<=45), it would create docComment as an object property ($node->docComment)
After (>= 50), it would create docComment as a field ($node->children['docComment'])

Before and after this change, the property name was in $node->children['name']

Code to reproduce this:

var_dump(\ast\parse_code('<?php class A{/** @var int */ public $x = 2;}', 50));

What's the status of ZEND_ACC_GENERATOR for functions in php 7.1

The flags for AST_FUNCTION (etc?) seem to include ZEND_ACC_GENERATOR in php 7.1 (but not php 7.0, I assume due to a limitation or design change)

Right now, util.php just dumps it as an undocumented constant.

It would be convenient to know what to expect for this bit value, to have util.php dump this bit value, and whether the flag can be relied on to be there in future php patch releases.

php 7.0

php > var_dump( \ast\parse_code('<?php function foo(){yield;}', 40));
object(ast\Node)#1 (4) {
  ["kind"]=>
  int(133)
  ["flags"]=>
  int(0)
  ["lineno"]=>
  int(1)
  ["children"]=>
  array(1) {
    [0]=>
    object(ast\Node\Decl)#2 (7) {
      ["kind"]=>
      int(66)
      ["flags"]=>
      int(0)
...

php 7.1:

object(ast\Node)#1 (4) {
  ["kind"]=>
  int(132)
  ["flags"]=>
  int(0)
  ["lineno"]=>
  int(1)
  ["children"]=>
  array(1) {
    [0]=>
    object(ast\Node\Decl)#2 (7) {
      ["kind"]=>
      int(66)
      ["flags"]=>
      int(8388608)

update util.php for version 50

util.php still needs to be updated to dump the name and doc comment of nodes generated with version 50.

To do that, it needs to know which node kinds are decls
It would also be useful for code which used to use Decl to have a utility to assert something is equivalent to a Decl (e.g. in the remote chance new decls get added in the future)
(Actually, now that I think about it, there aren't dynamic property fields, so something with the field 'name' is always guaranteed to be part of a decl?)

Additionally, document that anything where kind_is_decl($node->kind) is true is guaranteed to have certain properties

Flags Documentation?

Hello!
I'm quite new to PHP internals and was wondering: where could I find an explanation for flags?

For example, I can't really deduce what NAME_FQ and NAME_NON_FQ means ^^"

Clarifying meaning of "prior to version XX"

Hi,

Just to be sure I'm not misunderstanding the README, in https://github.com/nikic/php-ast#ast-node-kinds when I read:

AST_AND:              left, right            // prior to version 20

That means that AST_AND has been removed in v30 and later, and is no longer used?

Proposal: Move PHP utilities for working with php-ast(util.php) into a namespaced composer package?

E.g. v0.1.4 could be capable of printing debug output for the stable ast version (40) from v0.1.4 of ext-ast, etc.

Any future changes (e.g. best effort at converting an \ast\node back into valid PHP code, optionally with those line numbers) could be added to those utilities.

Tes failure with PHP 7.1.4RC1

FYI, detected by Fedora QA
https://apps.fedoraproject.org/koschei/package/php-ast?collection=f27

Seems only related to line number

Can you please have a look, so we can try to avoid BC break in upcoming 7.1.4.

Nullable array/callable has flags copied from inner type, but other nullable types have a flag of 0 (minor inconsistency)

This affects the same 2 nullable types mentioned in #40

Noticed for the below snippet:

<?php
function foo(?array $arr, ?int $i) {
}

The parameters from the dump of the ast in version 40 is below (not tested in other versions):

Observed: The AST_NULLABLE_TYPE(2050) has flags 7(I assume the copied flags for array, but not absolutely sure), which duplicates the flags on AST_TYPE(1) with flags 7(TYPE_ARRAY). This is inconsistent with other types, where AST_NULLABLE_TYPE has flags 0.
Expected: AST_NULLABLE_TYPE should consistently have flags 0
Motivation: Make API slightly more consistent. People may dump the AST for a nullable array with var_export, and base their code off of that one example, getting the wrong flags for other types.
Severity: very low.

                    "children": [
                        {
                            "kind": 773,
                            "flags": 0,
                            "lineno": 2,
                            "children": {
                                "type": {
                                    "kind": 2050,
                                    "flags": 7,
                                    "lineno": 2,
                                    "children": {
                                        "type": {
                                            "kind": 1,
                                            "flags": 7,
                                            "lineno": 2,
                                            "children": []
                                        }
                                    }
                                },
                                "name": "arr",
                                "default": null
                            }
                        },
                        {
                            "kind": 773,
                            "flags": 0,
                            "lineno": 2,
                            "children": {
                                "type": {
                                    "kind": 2050,
                                    "flags": 0,
                                    "lineno": 2,
                                    "children": {
                                        "type": {
                                            "kind": 1,
                                            "flags": 4,
                                            "lineno": 2,
                                            "children": []
                                        }
                                    }
                                },
                                "name": "i",
                                "default": null
                            }
                        }
                    ]

BINARY_POW

I've noticed that there is a "BINARY_POW" operator listed in the documentation under the assign operator, but does this operator really exist?

^= is the XOR operator.

Specify versions in a way that composer can depend on version ranges

Currently, versions get removed from 0.1.x, so a composer dependency on ~0.1.5 may break in a future unknown release (e.g. 0.1.8)

Changing version numbers would make it easier to specify version ranges for packages such as phan, etc, and to allow simultaneous installations of packages using php-ast

Proposed

bump minor version when an old ast version is removed
bump major version if a PHP version (e.g. 7.0) stops being supported

Add internal factory functions to make ast Nodes and Decls?

Not sure about the order, but something like the below (No validation of $kind, $flags, etc. is expected beyond the type).
Would this be something you would be interested in adding to php-ast if there was a PR for it?

\ast\make_node(int $kind, int $flags = 0, int lineno = 0, array $children = null) : \ast\Node
\ast\make_decl(int $kind, int $flags = 0, int lineno = 0, array $children = null, string $name = null, int endLineno = 0, string $docComment = null) : \ast\Node\Decl

If a program constructs a large number of AST nodes, then having functionality to construct ast nodes in C would likely be faster than constructing an object and setting multiple properties from PHP
(e.g. converting from php-parser to php-ast, or rewriting AST nodes according to a rule, to make them easier to analyze)
Changing the constructor may cause BC issues, so factories were suggested instead

Coding Standards?

Hello,

I saw this repository and the first thing in my mind was: What about coding standards? I noticed this exposes it within the ast namespace. So far I've seen this is the first actual namespace that can be provided by php if the extension is used by php 7 in the default setup.

I would recommend following the PSR guidelines before it's too late to implement a standard as provided by: http://www.php-fig.org/psr/

The main advantage would of course be consistency. If this extension will be provided, I would suggest the following namespace: namespace PHP\AST;. This is of course open for discussion, but it would be nice if php would adapt this for it's core functions, classes and constants.

For inline HTML, AST_ECHO has a lineno of the end line, not the starting line

Not familiar enough with internals to know if this is fixable. Low priority. Tested in php 7.1.2

var_export(\ast\parse_code("Hello\nworld\nthirdline", 40))

Observed: lineno => 3.
Expected: lineno => 1.

'kind' => 282,
'lineno' => 3

Include this extension in the PHP core and provide a hook API

I want to suggest to include this extension into the core to provide an API for miscellaneous userland extensions. Main idea is to give a userland code control over the produced source-code.

Chain is following: Source Code => Tokenizer => AST => Engine hooks+PHP userland hooks => Compiler => Opcodes => Execution

Why it can be a great thing: native source-code AST (static analyzers, code beautifiers), language extensions (AOP, DbC, SQL-like native syntax).

This requires to create an userland-function, probably, register_parser_extension(ParserExtensionInterface $extension) that will add an extension as parser hook.

Next additional thing is ParserExtensionInterface interface with process(Node $node) method that will receive ast\Node class and should return a modified node or tree.

Each registered extension will be called consequently, allowing to transform the source code. Last result will be used by compiler to generate final version of opcodes.

[rant] Why are Concrete Syntax Trees so neglected in the world of PHP parsers?

Every PHP parser I can find only offers Abstract Syntax Tree.

This is poor.

CSTs are absolutely necessary for perfect reconstructibility, and perfect reconstructibility is absolutely necessary if the code transformation to be implemented MUST preserve line numbers. All those AST pretty printers around will completely mangle line numbers.

Request: Add an identifier which can be used to uniquely identify a parsed closure (e.g. in combination with line)

What is being requested:

In version 45, add an identifier that can be used to identify a closure returned by parse_code, which will be reproduceable if parse_code is called multiple times on the same PHP code

Why?

Creating this identifier from PHP is possible, but would be much slower than doing it from C (A very small fraction of \ast\Nodes are closures)
Some use cases would normally only need to process part of the tree from PHP, or process the tree in different orders
Helps with issues such as phan/phan#767 . if two closures in the same file on the same line have the same function types and return types, it's hard to differentiate them
In one phase, many files are sequentially parsed, processed, and then asts are derefenced to save memory. In the second phase, asts are parsed again to analyze.

Suggested ways of doing that

The column number (or byte offset from the start of the file), if future versions of the Zend support it (The combination of line number and starting column number would be unique)
An incrementing counter(globally unique), which is reset to 0 whenever \ast\parse_code is called.
, e.g. The combination of $closureNode->lineno and $closureNode->children['__closure_identifier']

nikic / php-ast Goto Github PK

php-ast's Issues

Recommend Projects

Recommend Topics

Recommend Org