Giter Site home page Giter Site logo

cvise's Introduction

C-Vise

Build Build LLVM nightly

About

C-Vise is a super-parallel Python port of the C-Reduce. The port is fully compatible to the C-Reduce and uses the same efficient LLVM-based C/C++ reduction tool named clang_delta.

C-Vise is a tool that takes a large C, C++ or OpenCL program that has a property of interest (such as triggering a compiler bug) and automatically produces a much smaller C/C++ or OpenCL program that has the same property. It is intended for use by people who discover and report bugs in compilers and other tools that process C/C++ or OpenCL code.

The project also contains a simple wrapper cvise-delta which simulates the same behavior as original delta tool (but in super-parallel way).

NOTE: C-Vise happens to do a pretty good job reducing the size of programs in languages other than C/C++, such as JavaScript and Rust. If you need to reduce programs in some other language, please give it a try.

NOTE: Binary pass group (--pass-group=binary) contains an experimental pass that can reduce GCC's .gcda files.

Speed Comparison

I made a comparison for couple of GCC bug reports on my AMD Ryzen 7 2700X Eight-Core Processor machine with the following results:

Test-case Size C-Vise Reduction C-Reduce Reduction Speed Up
PR92516 6.5 MB 35m 77m 220%
PR94523 2.1 MB 15m 33m 220%
PR94632 3.3 MB 20m 28m 40%
PR94937 8.5 MB 242m 303m 125%

Installation

See INSTALL.md.

Usage example

The C-Vise can be used for a reduction of a compiler crash. In this case, let's consider an existing PR94534:

Original test-case (pr94534.C file):

template<typename T>
class Demo
{
  struct
  {
    Demo* p;
  } payload{this};
  friend decltype(payload);
};

int main()
{
  Demo<int> d;
}

The program crashes in GCC, but is accepted with Clang:

$ g++ pr94534.C -c
pr94534.C: In instantiation of ‘class Demo<int>’:
pr94534.C:13:13:   required from here
pr94534.C:7:5: internal compiler error: Segmentation fault
    7 |   } payload{this};
      |     ^~~~~~~
0x10a1d8f crash_signal
	/home/marxin/Programming/gcc/gcc/toplev.c:328
0x7ffff78fef1f ???
	/usr/src/debug/glibc-2.31-4.1.x86_64/signal/../sysdeps/unix/sysv/linux/x86_64/sigaction.c:0
0xae31a8 instantiate_class_template_1
	/home/marxin/Programming/gcc/gcc/cp/pt.c:11973
...
$ clang++ pr94534.C -c

So let's build a reduction script so that it will grep for instantiate_class_template_1 on the standard error output and that it compiles with Clang:

reduce-ice.sh:

#!/bin/sh
g++ pr94534.C -c 2>&1 | grep 'instantiate_class_template_1' && clang++ -c pr94534.C

The reduction can be then run with:

$ cvise ./reduce-ice.sh pr94534.C
INFO ===< 30356 >===
INFO running 16 interestingness tests in parallel
INFO INITIAL PASSES
INFO ===< IncludesPass >===
...
template <typename> class a {
  int b;
  friend decltype(b);
};
void c() { a<int> d; }

Notes

  1. C-Vise creates temporary directories in $TMPDIR and so usage of a tmpfs directory is recommended.

  2. Each invocation of the interestingness test is performed in a fresh temporary directory containing a copy of the file that is being reduced. If your interestingness test requires access to other files, you should either copy them into the current working directory or else refer to them using an absolute path.

  3. If you copy the compiler invocation line from your build tool, remove -Werror if present. Some C-Vise passes introduce warnings, so -Werror will make those passes ineffective.

    Doing that, a reduction will typically end up faster, however, one may end up with a code snippet full of warnings that needs to be addresses after the reduction.

  4. Adding -Wfatal-errors to the interestingness test can speed up large reductions by causing the compiler to bail out quickly on errors, rather than trying to soldier on producing a result that is eventually discarded.

cvise's People

Contributors

amykhuang avatar annulen avatar avikivity avatar aytey avatar chenyang78 avatar dgrunwald-qt avatar eeide avatar fitzgen avatar janisozaur avatar jketema avatar jsonn avatar jwilk avatar marxin avatar mgorny avatar mortior avatar mpflanzer avatar mzfr avatar nickdesaulniers avatar oltolm avatar oriceemple avatar pmatos avatar pranavk avatar regehr avatar rgov avatar rnk avatar strimo378 avatar tamird avatar trofi avatar vgvassilev avatar yrnkrn avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

cvise's Issues

Cvising "PASS_BUG_INFO"

I've been cvising some C++14 code recently, and ended-up with quite a few cvise_bug_* folders (38 currently).

Is there a way for me to cvise these failures?

Basically, what I'm asking is this: if I see

ClangPass::remove-nested-function has encountered a bug:
pass error
state: 5

in PASS_BUG_INFO.TXT, is it possible for me to:

  1. manually run the clang pass
  2. know if it "failed"

? If so, then I can cvise the inputs to provide reduced failures.

Sadly, I can't share the original files as-is, because they're proprietary code; if I could cvise then, I can probably make files I can share.

Assertion failure with `lift-assignment-expr`

For the following file:

template <typename a> auto b() -> decltype(1(a())) {}

and arguments:

clang_delta --transformation=lift-assignment-expr --counter=1 unit.cpp

clang_delta crashes as follows:

clang_delta: /usr/include/llvm/Support/Casting.h:104: static bool llvm::isa_impl_cl<To, const From*>::doit(const From*) [with To = clang::AttributedStmt; From = clang::Stmt]: Assertion `Val && "isa<> used on a null pointer"' failed.

Windows support

Trying to compile this on Windows (Visual Studio 2019) doesn't work.
I use the following to configure my CMake:

set CC="C:\Program Files\VisualStudio\Vs2019\VC\Tools\MSVC\14.24.28314\bin\Hostx64\x64\cl.exe"
set CXX="C:\Program Files\VisualStudio\Vs2019\VC\Tools\MSVC\14.24.28314\bin\Hostx64\x64\cl.exe"
set LLVM_DIR=LLVM_BUILD\lib\cmake\llvm
set CLANG_DIR=LLVM_BUILD\lib\cmake\clang
C:\Program Files\VisualStudio\Vs2019\Common7\IDE\CommonExtensions\Microsoft\CMake\CMake\bin\cmake.exe -G "Visual Studio 16 2019"  -A x64 -Thost=x64  ..\cvise-master

cl : command line error D8021: invalid numeric argument '/Werror' [build\clang_delta\clang_delta.vcxproj]

After removing this flag from source/CMakeLists.txt (line 144-145), an actual build gets started.

Later we get other errors, like:

source\clang_delta\ExpressionDetector.cpp : fatal error C1128: number of sections exceeded object file format limit: compile with /bigobj [build\clang_delta\clang_delta.vcxproj]
cl : command line error D8021: invalid numeric argument '/Wno-unused-function' [build\clex\clex.vcxproj]
cl : command line error D8021: invalid numeric argument '/Wno-unused-function' [build\clex\strlex.vcxproj]

These last 2 warnings are a result of source/clex/CMakeLists.txt

I also encountered some issues with the .l files, however, this could be linked to older hacks I've added to our lexer. (I did have to add %option nounistd to the options).

With these changes, I got the C++ code compiling.
Can we get some native windows support instead?

Note: Compile times take long with a lot of warnings. (Over 14 minutes on 1 file)

Can `cvise`'s intermediate files be trusted?

I'm giving cvise a try on top of some examples I've been using with creduce -- I took the inputs and interestingness test directly from something that worked for creduce.

I let cvise run for only a few minutes, with this on stdout:

INFO ===< 69135 >===
INFO running 97 interestingness tests in parallel
INFO INITIAL PASSES
INFO ===< IncludesPass >===
INFO (0.1%, 22991 bytes)
INFO ===< UnIfDefPass >===
INFO ===< CommentsPass >===
INFO ===< IfPass >===
INFO ===< LineMarkersPass >===
INFO ===< BlankPass >===
INFO ===< ClangBinarySearchPass::replace-function-def-with-decl >===
INFO ===< ClangBinarySearchPass::remove-unused-function >===
INFO ===< LinesPass::0 >===
INFO ===< LinesPass::1 >===

I copied the input file (which cvise had modified in place) to another directory and ran the "interesting test" manually ... and it failed!

Looking at the intermediate hand by eye, I agree that the file should no longer pass the interestingness test.

Is the updated file on disk supposed to always pass the interestingness test (like it does with creduce)?

Pass got stuck

All reproduction details can be found at #26 if the mentioned pass is removed.

Running the example, using 1 thread, causes several passes to get stuck

ClangPass::callexpr-to-value has encountered a non fatal bug: pass got stuck
ClangPass::replace-callexpr has encountered a non fatal bug: pass got stuck
ClangPass::simplify-callexpr has encountered a non fatal bug: pass got stuck
ClangPass::remove-unused-enum-member has encountered a non fatal bug: pass got stuck
ClangPass::remove-enum-member-value has encountered a non fatal bug: pass got stuck
ClangPass::remove-namespace has encountered a non fatal bug: pass got stuck
ClangPass::aggregate-to-scalar has encountered a non fatal bug: pass got stuck
ClangPass::param-to-global has encountered a non fatal bug: pass got stuck
ClangPass::param-to-local has encountered a non fatal bug: pass got stuck

Currently at 18h and running.

cvise does not work on OpenBSD 6.9 on PowerPC G5 Mac

For fun I'm playing with OpenBSD 6.9 on an old PowerPC G5 Mac.

I successfully built cvise but a basic invocation crashes:

cvise --version            
Segmentation fault (core dumped) 

Here's the backtrace:

builder3$ gdb python3 /home/builder/external/cvise-install/bin/python3.8.core 

(gdb) bt
#0  0xe06cb15c in do_mkvalue () from /usr/local/lib/libpython3.8.so.0.0
#1  0xe06cb3ac in do_mktuple () from /usr/local/lib/libpython3.8.so.0.0
#2  0xe06ca9c0 in do_mkvalue () from /usr/local/lib/libpython3.8.so.0.0
#3  0xe06c9b08 in Py_BuildValue () from /usr/local/lib/libpython3.8.so.0.0
#4  0xe08fc1bc in psutil_proc_oneshot_info () from /usr/local/lib/python3.8/site-packages/psutil/_psutil_bsd.cpython-38.so
#5  0xe05744a0 in cfunction_call_varargs () from /usr/local/lib/libpython3.8.so.0.0
#6  0xe0575458 in PyCFunction_Call () from /usr/local/lib/libpython3.8.so.0.0
#7  0xe0573af8 in _PyObject_MakeTpCall () from /usr/local/lib/libpython3.8.so.0.0
#8  0xe068662c in call_function () from /usr/local/lib/libpython3.8.so.0.0
#9  0xe06836a0 in _PyEval_EvalFrameDefault () from /usr/local/lib/libpython3.8.so.0.0
#10 0xe067d584 in PyEval_EvalFrameEx () from /usr/local/lib/libpython3.8.so.0.0
#11 0xe0574910 in function_code_fastcall () from /usr/local/lib/libpython3.8.so.0.0
#12 0xe0574ad4 in _PyFunction_Vectorcall () from /usr/local/lib/libpython3.8.so.0.0
#13 0xe0686568 in call_function () from /usr/local/lib/libpython3.8.so.0.0
#14 0xe0683724 in _PyEval_EvalFrameDefault () from /usr/local/lib/libpython3.8.so.0.0
#15 0xe0687424 in _PyEval_EvalCodeWithName () from /usr/local/lib/libpython3.8.so.0.0
#16 0xe0574b54 in _PyFunction_Vectorcall () from /usr/local/lib/libpython3.8.so.0.0
#17 0xe0686568 in call_function () from /usr/local/lib/libpython3.8.so.0.0
#18 0xe0683684 in _PyEval_EvalFrameDefault () from /usr/local/lib/libpython3.8.so.0.0
#19 0xe067d584 in PyEval_EvalFrameEx () from /usr/local/lib/libpython3.8.so.0.0
#20 0xe0574910 in function_code_fastcall () from /usr/local/lib/libpython3.8.so.0.0
#21 0xe0574ad4 in _PyFunction_Vectorcall () from /usr/local/lib/libpython3.8.so.0.0
#22 0xe0574070 in PyVectorcall_Call () from /usr/local/lib/libpython3.8.so.0.0
#23 0xe05741fc in PyObject_Call () from /usr/local/lib/libpython3.8.so.0.0
#24 0xe0683948 in _PyEval_EvalFrameDefault () from /usr/local/lib/libpython3.8.so.0.0
#25 0xe0687424 in _PyEval_EvalCodeWithName () from /usr/local/lib/libpython3.8.so.0.0
#26 0xe0574b54 in _PyFunction_Vectorcall () from /usr/local/lib/libpython3.8.so.0.0
#27 0xe0686568 in call_function () from /usr/local/lib/libpython3.8.so.0.0
#28 0xe0683684 in _PyEval_EvalFrameDefault () from /usr/local/lib/libpython3.8.so.0.0
#29 0xe067d584 in PyEval_EvalFrameEx () from /usr/local/lib/libpython3.8.so.0.0
#30 0xe0574910 in function_code_fastcall () from /usr/local/lib/libpython3.8.so.0.0
#31 0xe0574ad4 in _PyFunction_Vectorcall () from /usr/local/lib/libpython3.8.so.0.0
#32 0xe0686568 in call_function () from /usr/local/lib/libpython3.8.so.0.0
#33 0xe0683684 in _PyEval_EvalFrameDefault () from /usr/local/lib/libpython3.8.so.0.0
#34 0xe0687424 in _PyEval_EvalCodeWithName () from /usr/local/lib/libpython3.8.so.0.0
#35 0xe0574b54 in _PyFunction_Vectorcall () from /usr/local/lib/libpython3.8.so.0.0
#36 0xe0686568 in call_function () from /usr/local/lib/libpython3.8.so.0.0
#37 0xe0683684 in _PyEval_EvalFrameDefault () from /usr/local/lib/libpython3.8.so.0.0
#38 0xe0687424 in _PyEval_EvalCodeWithName () from /usr/local/lib/libpython3.8.so.0.0
#39 0xe0574b54 in _PyFunction_Vectorcall () from /usr/local/lib/libpython3.8.so.0.0
#40 0xe05738bc in _PyObject_FastCallDict () from /usr/local/lib/libpython3.8.so.0.0
#41 0xe05759ac in _PyObject_Call_Prepend () from /usr/local/lib/libpython3.8.so.0.0
#42 0xe05fbeac in slot_tp_init () from /usr/local/lib/libpython3.8.so.0.0
#43 0xe05f5500 in type_call () from /usr/local/lib/libpython3.8.so.0.0
#44 0xe0573af8 in _PyObject_MakeTpCall () from /usr/local/lib/libpython3.8.so.0.0
#45 0xe068662c in call_function () from /usr/local/lib/libpython3.8.so.0.0
#46 0xe06836a0 in _PyEval_EvalFrameDefault () from /usr/local/lib/libpython3.8.so.0.0
#47 0xe067d584 in PyEval_EvalFrameEx () from /usr/local/lib/libpython3.8.so.0.0
#48 0xe0574910 in function_code_fastcall () from /usr/local/lib/libpython3.8.so.0.0
#49 0xe0574ad4 in _PyFunction_Vectorcall () from /usr/local/lib/libpython3.8.so.0.0
#50 0xe0686568 in call_function () from /usr/local/lib/libpython3.8.so.0.0
#51 0xe0683724 in _PyEval_EvalFrameDefault () from /usr/local/lib/libpython3.8.so.0.0
#52 0xe0687424 in _PyEval_EvalCodeWithName () from /usr/local/lib/libpython3.8.so.0.0
#53 0xe067d358 in PyEval_EvalCode () from /usr/local/lib/libpython3.8.so.0.0
#54 0xe06dd0c0 in run_mod () from /usr/local/lib/libpython3.8.so.0.0
#55 0xe06dabb4 in PyRun_SimpleFileExFlags () from /usr/local/lib/libpython3.8.so.0.0
#56 0xe06da5f0 in PyRun_AnyFileExFlags () from /usr/local/lib/libpython3.8.so.0.0
#57 0xe07044f4 in Py_RunMain () from /usr/local/lib/libpython3.8.so.0.0
#58 0xe0704a14 in pymain_main () from /usr/local/lib/libpython3.8.so.0.0
#59 0xe0704ba4 in Py_BytesMain () from /usr/local/lib/libpython3.8.so.0.0
#60 0x249206b4 in main () from /usr/local/bin/python3

Hopefully that means something to you... otherwise, it does repro, so I could try whatever...

Unable to run cvise from build location without installing

I just built but did not install cvise, so tired running it from the build location, but this failed with cvise being unable to find the tools for doing the passes.

I solved this by doing this:

export PATH=~/cvise/build/clex/:~/cvise/build/clang_delta:~/cvise/build:~/cvise/build/cvise:~/creduce/unifdef:~/creduce/delta/:$PATH

which also requires unifdef and topflatformat, which do not appear to be part of a normal cvise build (but they are part of a creduce build!).

I guess it needs to be documented that, if you want to run a non-installed build of cvise, you need to set $PATH, and that you need to install unifdef and topflatformat "from somewhere else".

unifdef (e.g.,) is listed in the Fedora Dockerimage, but I couldn't find this documented anywhere else.

no declaration matches ‘bool clang::RecursiveASTVisitor<Derived>::TraverseCXXAddrspaceCastExpr(clang::CXXAddrspaceCastExpr*, clang::RecursiveASTVisitor<Derived>::DataRecursionQueue*)’

trying to build cvise from source again, I see:

In file included from ../clang_delta/ClassTemplateToClass.cpp:17:
/android0/llvm-project/clang/include/clang/AST/RecursiveASTVisitor.h:2154:8: error: no declaration matches ‘bool clang::RecursiveASTVisitor<Derived>::TraverseCXXAddrspaceCastExpr(clang::CXXAddrspaceCastExpr*, clang::RecursiveASTVisitor<Derived>::DataRecursionQueue*)’

maybe out of date against ToT LLVM?

Redundant sanity passes?

I'm currently running cvise like this:

cvise --start-with-pass LinesPass::0 --timeout 600 --debug --n $(nproc) cvise.sh unit.cpp

and I'm seeing this in the log:

00:00:00 DEBUG perform sanity check...
00:00:00 DEBUG sanity check tmpdir = /mnt/baranem/cvise-ja21eq2_
00:00:46 DEBUG sanity check successful
00:00:46 INFO ===< 68660 >===
00:00:46 INFO running 24 interestingness tests in parallel
00:00:46 INFO INITIAL PASSES
00:00:46 DEBUG Creating pass root folder: /mnt/baranem/cvise-m811bv_g
00:00:46 INFO ===< LinesPass::0 >===
00:00:46 DEBUG perform sanity check...
00:00:46 DEBUG sanity check tmpdir = /mnt/baranem/cvise-6403wq4c
00:01:33 DEBUG sanity check successful

what I'm surprised at here is that cvise runs the sanity check at the start (which is really great!), but also as part of the of the start of LinesPass.

Given cvise has literally just run the sanity check and made no changes to the input file, I'm surprised it is doing the sanity check again.

This is such a minor nit, but as I noticed it, I thought it was worth sharing. It is "more obvious" to me currently because the interestingness test takes ~45 seconds, so I'm really aware that cvise spends 90 seconds "not doing work".

Initial check doesn't provide the cpp file to compile

When your reproduction fails to reproduce, the following is printed out:

  DIR=`mktemp -d`
  cp c:\path\to\t.cpp $DIR
  cd $DIR
  C:\Path\to\checkSuccess.bat c:\path\to\t.cpp
  echo $?

However, if your script relies on the fact that c:\path\to\t.cpp being passed into it, the sanity check fails as this argument isn't being passed. (At least on windows)

Python crash in `process_done_futures` (AttributeError: 'NoneType' object has no attribute 'success')

Traceback (most recent call last):
  File "/home/avj/cvise/build/cvise.py", line 229, in <module>
    reducer.reduce(pass_group, skip_initial=args.skip_initial_passes)
  File "/home/avj/cvise/build/cvise/cvise.py", line 124, in reduce
    self._run_additional_passes(pass_group["first"])
  File "/home/avj/cvise/build/cvise/cvise.py", line 151, in _run_additional_passes
    self.test_manager.run_pass(p)
  File "/home/avj/cvise/build/cvise/utils/testing.py", line 460, in run_pass
    success_env = self.run_parallel_tests()
  File "/home/avj/cvise/build/cvise/utils/testing.py", line 387, in run_parallel_tests
    quit_loop = self.process_done_futures()
  File "/home/avj/cvise/build/cvise/utils/testing.py", line 320, in process_done_futures
    if test_env.success:
AttributeError: 'NoneType' object has no attribute 'success'

Assertion failure with `remove-namespace`

For the following file:

inline namespace a {
void operator"" b(unsigned long long);
}

and arguments:

clang_delta --transformation=remove-namespace --counter=1 unit.cpp

clang_delta crashes as follows:

clang_delta: ../clang_delta/RemoveNamespace.cpp:950: void RemoveNamespace::handleOneNamedDecl(const clang::NamedDecl*, const clang::DeclContext*, const string&): Assertion `IdInfo && "Invalid IdentifierInfo!"' failed.

Improve CMake file

From @mgorny:

You should add a check that SOURCE_DIR!=BINARY_DIR somewhere. Right now this permits doing cmake . which is not going to work correctly with this.

Also, this file is really begging for a for loop.

unintentional synchronization points?

When I run cvise, sometimes it seems stalled on making progress. When I check htop, it doesn't look like my cores are doing any work related to cvise. It looks like

00:00:03 INFO ===< LinesPass::0 >===

is what's running. I'm running cvise with --n 71. Are there synchronization points where all threads need to wait on one to make progress? Bisection will eventually converge, but it seems to take quite some time to make forward progress.

Playing with ps -ef, it looks like there are 71 instances of the python interpreter running, with 0 instances of clang, clang_delta, or my reproducer shell script.

Multiple clang passes encountering 'pass error'

I was reducing a pre-processed test case with 8 jobs in parallel and 11 passes encountered a pass error. I have attached a zip archive with the created bug report for each of them.

If you need any other info, let me know.

[File deleted]

Unclear how to resolve errors

When running CVise without having all requirements, we get a message like 00:02:02 ERROR Skipping pass ClangPass::remove-namespace however, this doesn't mention what the underlying problem is. As a result, it is impossible to resolve this.

Assert of 'clang_delta'

Found on following code (t.cpp):

#include <variant>
#include <vector>
struct A {
         std::basic_string<wchar_t> m;
       };
        struct B {
         B(A a);
         std::variant<A> m;
       };
        struct C : B {
         using B::B;
       };
        void f() {
         std::vector<C> c{
     };
         c.push_back(C{
     A{
    }
     }
      );
       }

Launch script:

py -3.6 G:\llvm\build-cvise\cvise.py --clang-delta-std c++17 --n 1 --print-diff checkIfBugExists.bat t.cpp 

The error occurs within ClangPass::remove-unused-function

checkIfBugExists.bat

SET CVISE_BUG_DETECTED=0
SET CVISE_BUG_IRREPRODUCABLE=1
setlocal enableDelayedExpansion

echo "Current dir: %cd%"
echo "Copied file: %1"
C:\Path\To\LLVM_7_0_0\bin\clang-cl.exe /c /EHsc -w /std:c++17 t.cpp > compilation.out 2> compilation.err
if %ERRORLEVEL% EQU 0 (
   echo "Compilation success"
   exit /b %CVISE_BUG_IRREPRODUCABLE%
)
echo "Failed to compile"

findstr /r /c:"Wrote crash dump file" .\compilation.err
if %ERRORLEVEL% EQU 0 (
   echo "Found crash message"
   exit /b %CVISE_BUG_DETECTED%
) else (
   echo "Not found"
   exit /b %CVISE_BUG_IRREPRODUCABLE%
)

Callstack of delta.exe (Compiled with LLVM 10.0.0)

 	issue_debug_notification(message) Line 28	C++
 	__acrt_report_runtime_error(message) Line 154	C++
 	abort() Line 61	C++
 	common_assert_to_stderr<wchar_t>(expression, file_name, line_number) Line 187	C++
 	common_assert<wchar_t>(expression, file_name, line_number, return_address) Line 420	C++
 	_wassert(expression, file_name, line_number) Line 444	C++
>	RemoveUnusedFunction::doRewriting() Line 307	C++
 	RemoveUnusedFunction::HandleTranslationUnit(Ctx) Line 284	C++
 	clang::ParseAST(S, PrintStats, SkipFunctionBodies) Line 178	C++
 	TransformationManager::doTransformation(ErrorMsg, ErrorCode) Line 308	C++
 	main(argc, argv) Line 249	C++

Issue is in void RemoveUnusedFunction::doRewriting() where TheFunctionDecl seems to be nullptr. ToCounter is on -1.

cvise does not remove unused template arguments when reducing

Example is a https://gcc.gnu.org/PR95768 case reducer. I'll provide slightly different example here:

Input where cvise stopped and was not able to progress further:

extern "C" void *malloc(unsigned long);
__attribute__((__noreturn__)) void a(const char *, const char *, int);
class c {
public:
  enum e { d };
};
class f {
public:
  int aa;
  f operator=(int) {
    int g, h, j, a, k, c = aa;
    k = c;
    h = k;
    int i = h &= j;
    a = i;
    int d = a;
    g = d;
    aa = g;
    return *this;
  }
};
template <typename, int, int> using af = f;
template <int ae, int ag> using l = af<int, ae, ag>;
class m {
public:
  l<1, 1> ai;
};
class C {
public:
  m *o() { return (m *)this + 1; }
};
class n {
public:
  int ak;
  C *al;
  n(c::e p1) {
    p1 ? void() : a("", "", 0);
    al = (C *)malloc(ak);
    m *b = al->o();
    b->ai = 0;
  }
};
bool an() {
  n(c::d);
  return true;
}

Test for cvise:

#!/bin/bash

args=(
/usr/bin/x86_64-pc-linux-gnu-g++

-march=sandybridge
-O2
-Wall

-c bug.cc
-o bug.o
)
#"${args[@]}"

LANG=C "${args[@]}" -fsyntax-only -Werror || exit 1

LANG=C "${args[@]}" 2>&1 | grep 'Segmentation fault'

cvise ./test_cvise.sh bug.cc can't reduce example further.

It's not very hard to shrink it further manuall by removing unused template arguments of template <typename, int, int> using af = f;. I did it as:

--- bug.cc__	2020-06-19 18:56:41.000000000 +0100
+++ bug.cc___	2020-06-20 10:27:24.000000000 +0100
@@ -19,8 +19,8 @@
     return *this;
   }
 };
-template <typename, int, int> using af = f;
-template <int ae, int ag> using l = af<int, ae, ag>;
+using af = f;
+template <int ae, int ag> using l = af;
 class m {
 public:
   l<1, 1> ai;
--- bug.cc___	2020-06-20 10:27:24.000000000 +0100
+++ bug.cc____	2020-06-20 10:39:18.000000000 +0100
@@ -20,10 +20,10 @@
   }
 };
 using af = f;
-template <int ae, int ag> using l = af;
+using l = af;
 class m {
 public:
-  l<1, 1> ai;
+  l ai;
 };
 class C {
 public:

After that cvise ./test_cvise.sh bug.cc was able to get it down to:

extern "C" void *malloc(unsigned long);
__attribute__((__noreturn__)) void a(const char *, const char *, int);
class c {
public:
  enum e { d };
};
class f {
public:
  int aa;
  f operator=(int) {
    int g, h, j, a, k, c = aa;
    k = c;
    h = k;
    int i = h &= j;
    a = i;
    int d = a;
    g = d;
    aa = g;
    return *this;
  }
};
using l = f;
class m {
public:
  l ai;
};
class C {
public:
  m *o() { return (m *)this + 1; }
};
class n {
public:
  int ak;
  C *al;
  n(c::e p1) {
    p1 ? void() : a("", "", 0);
    al = (C *)malloc(ak);
    m *b = al->o();
    b->ai = 0;
  }
};
bool an() {
  n(c::d);
  return true;
}

I think creduce had very similar limitation.

Is it something that cvise could perform perform automatically? Or it's something up to clang's libraries to get better at?

Thank you!

void RemoveNestedFunction::getNewTmpVariableStr: Assertion `FD && "Cannot resolve DName!"' failed.

The following fails:

$ cat fsize.ii
template <class V> void operator*(V);
template <class Allocator> struct cpp_int_backend {
  void do_get_string() { this->size() * Allocator::limb_bits; }
};

$ clang_delta --transformation=remove-nested-function --counter=1 fsize.ii
clang_delta: /home/marxin/Programming/cvise/clang_delta/RemoveNestedFunction.cpp:362: void RemoveNestedFunction::getNewTmpVariableStr(clang::ASTContext&, std::string&): Assertion `FD && "Cannot resolve DName!"' failed.
Aborted (core dumped)

When are /tmp files cleared?

Hiho,

after running cvise for an extended time

> cvise --n 40 --skip-key-off --print-diff --timeout 120 --clang-delta-std=c++20 ./creduce_probe.sh ice.cpp

> cvise --version        
cvise 2.3.0 (38b42d9)

I got this error:

08:20:35 INFO (73.1%, 82693 bytes, 1644 lines)
Traceback (most recent call last):
  File "/usr/local/bin/cvise", line 286, in <module>
    reducer.reduce(pass_group, skip_initial=args.skip_initial_passes)
  File "/usr/local/share/cvise/cvise.py", line 146, in reduce
    self._run_main_passes(pass_group['main'])
  File "/usr/local/share/cvise/cvise.py", line 176, in _run_main_passes
    self.test_manager.run_pass(p)
  File "/usr/local/share/cvise/utils/testing.py", line 517, in run_pass
    success_env = self.run_parallel_tests()
  File "/usr/local/share/cvise/utils/testing.py", line 443, in run_parallel_tests
    test_env = TestEnvironment(self.state, order, self.test_script, folder,
  File "/usr/local/share/cvise/utils/testing.py", line 55, in __init__
    self.copy_files(test_case, additional_files)
  File "/usr/local/share/cvise/utils/testing.py", line 61, in copy_files
    shutil.copy(test_case, self.folder)
  File "/usr/lib/python3.9/shutil.py", line 418, in copy
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "/usr/lib/python3.9/shutil.py", line 275, in copyfile
    _fastcopy_sendfile(fsrc, fdst)
  File "/usr/lib/python3.9/shutil.py", line 166, in _fastcopy_sendfile
    raise err from None
  File "/usr/lib/python3.9/shutil.py", line 152, in _fastcopy_sendfile
    sent = os.sendfile(outfd, infd, offset, blocksize)
OSError: [Errno 28] No space left on device: '/home/marehr/develope/seqan3-build/cvise/068-cvise-clang-14-gcclib-9-coordinate_matrix_simd_test/ice.cpp' -> '/tmp/cvise-zzbvxuwp/cvise-d4rw0cdm/ice.cpp'

my /tmp folder contains

> find /tmp -maxdepth 1 -type d -name "cvise-*" | wc -l
61
> du -hcs /tmp/cvise-*
492K	/tmp/cvise-0q4bn__9
0	/tmp/cvise-1axqrcpq
2.1M	/tmp/cvise-1nsmipmz
...
3.7M	/tmp/cvise-zzbvxuwp
99M	total

> find /tmp -maxdepth 1 -type f -name "ice-*" | wc -l
22278

> du -hcs /tmp/ice-*
1.9M	/tmp/ice-000530.cpp
8.0K	/tmp/ice-000530.sh
1.2M	/tmp/ice-000769.cpp
8.0K	/tmp/ice-000769.sh
...
8.0K	/tmp/ice-fffc10.sh
1.2M	/tmp/ice-fffd0e.cpp
8.0K	/tmp/ice-fffd0e.sh
1.2M	/tmp/ice-fffee2.cpp
8.0K	/tmp/ice-fffee2.sh
16G	total

I would expect that all those files will get regularly deleted during the reduction, as I didn't specify --save-temps.

Moving input file in test?

I just added something like:

mv unit.cpp test.cpp

to my interestingness test, where unit.cpp is the input name as given to cvise.

However, this then led to:

00:00:03 DEBUG ***ADVANCE*** from 31 to 32 with chunk 1
Traceback (most recent call last):
  File "/usr/bin/cvise", line 286, in <module>
    reducer.reduce(pass_group, skip_initial=args.skip_initial_passes)
  File "/usr/share/cvise/cvise.py", line 143, in reduce
    self._run_additional_passes(pass_group['first'])
  File "/usr/share/cvise/cvise.py", line 166, in _run_additional_passes
    self.test_manager.run_pass(p)
  File "/usr/share/cvise/utils/testing.py", line 521, in run_pass
    self.process_result(success_env)
  File "/usr/share/cvise/utils/testing.py", line 546, in process_result
    shutil.copy(test_env.test_case_path, self.current_test_case)
  File "/usr/lib64/python3.8/shutil.py", line 418, in copy
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "/usr/lib64/python3.8/shutil.py", line 264, in copyfile
    with open(src, 'rb') as fsrc, open(dst, 'wb') as fdst:
FileNotFoundError: [Errno 2] No such file or directory: '/mnt/baranem/cvise-g1_ch4r0/cvise-952vdbgu/unit.cpp'

and cvise couldn't do anything (this was with cvise --version == cvise 2.3.0).

There's a good chance this is "play stupid games, win stupid prizes", but I'm creating this issue in case other people come across it.

Unrecognized arguments --flake8 on Arch

Trying to build this on Arch I get:

CMake Warning at CMakeLists.txt:52 (message):
  Pytest package not available: pytest 6.2.4

  ERROR: usage: __main__.py [options] [file_or_dir] [file_or_dir] [...]

  __main__.py: error: unrecognized arguments: --flake8

    inifile: /home/vlovich/cloudflare/cvise/setup.cfg
    rootdir: /home/vlovich/cloudflare/cvise

I tried pip install flake8 but that didn't seem to help. Removing the --flake8 from setup.cfg was a workaround but I'm not sure what the right fix is.

Unable to run from within CMake

When creating executables with CMake, the executables are created in a directory like <builddir>/delta/<Flavor> in my case is Debug.
These executables are not picked up by the CVise python script.

At the moment, I've hacked around line 70 of cvise.py the following code:

            local_folder = os.path.join(script_path, local_folder)
            path = shutil.which(prog, path=local_folder)

to become:

            local_folder = os.path.join(script_path, local_folder)
            path = shutil.which(prog, path=local_folder)
            local_folder = os.path.join(local_folder,"Debug")
            path = shutil.which(prog, path=local_folder)

that way the executables are being found

Improve Python install

Besides installing cvise to site-packages, we need to get the modules byte-compiled. Now, that's a non-trivial task and CMake is horrible at it, so it'd be best to actually use distutils/setuptools for that. Ideally, the package could be split into two, one containing common C/C++ tooling that could shared between cvise and creduce, and the other containing the Python driver.

Why byte-compiling is hard? The exact commands used depend on Python interpreter and version used. There are different commands for Python <3.5, PyPy(2), Python 3 (and PyPy3) >=3.5, Jython… and these are the implementations we used. Even Python upstream recently gets it wrong, and we have to patch Python for it.

The 'fallback' code we have for it:
https://gitweb.gentoo.org/repo/gentoo.git/tree/eclass/python-utils-r1.eclass#n595

I don't think you want to repeat it verbatim in CMakeLists.txt and maintain it going forward.

Compilation issues with `clang-10`?

Apologies for this not being the most dug-into issue, however I thought it was worth writing up in case other people hit this issue ...

I have a build of clang-10 from the following commit: llvm/llvm-project@2e95941, which is a build that works correctly against csmith-project/creduce@d83cbda (from April 20th, 2020).

However, it seems that this build of clang isn't compatible with cvise.

After C=clang CXX=clang++ cmake -DENABLE_TRANS_ASSERT=OFF ..:

-- The C compiler identification is Clang 10.0.0
-- The CXX compiler identification is Clang 10.0.0
-- Check for working C compiler: /home/avj/llvm/root/bin/clang
-- Check for working C compiler: /home/avj/llvm/root/bin/clang -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /home/avj/llvm/root/bin/clang++
-- Check for working CXX compiler: /home/avj/llvm/root/bin/clang++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found LLVM 10.0.0svn
-- Using LLVMConfig.cmake in /home/avj/llvm/root/lib/cmake/llvm
-- Using ClangConfig.cmake in /home/avj/llvm/root/lib/cmake/clang

but then when I build cvise:

/home/avj/cvise/clang_delta/TransformationManager.cpp:119:34: error: no member named 'lang_cxx20' in 'clang::LangStandard::Kind'; did you mean 'lang_cxx2a'?
      LSTD = LangStandard::Kind::lang_cxx20;
             ~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~
                                 lang_cxx2a
/home/avj/llvm/root/include/clang/Frontend/LangStandards.def:143:1: note: 'lang_cxx2a' declared here
LANGSTANDARD(cxx2a, "c++2a",
^
/home/avj/llvm/root/include/clang/Frontend/LangStandard.h:45:5: note: expanded from macro 'LANGSTANDARD'
    lang_##id,
    ^
<scratch space>:107:1: note: expanded from here
lang_cxx2a
^
/home/avj/cvise/clang_delta/TransformationManager.cpp:139:27: error: use of undeclared identifier 'Language'; did you mean 'LanguageID'?
  if (IK.getLanguage() == Language::C) {
                          ^~~~~~~~
                          LanguageID
/home/avj/llvm/root/include/clang/Basic/Builtins.h:32:6: note: 'LanguageID' declared here
enum LanguageID {
     ^
/home/avj/cvise/clang_delta/TransformationManager.cpp:139:37: error: no member named 'C' in 'clang::LanguageID'
  if (IK.getLanguage() == Language::C) {
                          ~~~~~~~~~~^
/home/avj/cvise/clang_delta/TransformationManager.cpp:140:72: error: use of undeclared identifier 'Language'; did you mean 'LanguageID'?
    Invocation.setLangDefaults(ClangInstance->getLangOpts(), InputKind(Language::C), T, PPOpts);
                                                                       ^~~~~~~~
                                                                       LanguageID
/home/avj/llvm/root/include/clang/Basic/Builtins.h:32:6: note: 'LanguageID' declared here
enum LanguageID {
     ^
/home/avj/cvise/clang_delta/TransformationManager.cpp:140:82: error: no member named 'C' in 'clang::LanguageID'
    Invocation.setLangDefaults(ClangInstance->getLangOpts(), InputKind(Language::C), T, PPOpts);
                                                                       ~~~~~~~~~~^
/home/avj/cvise/clang_delta/TransformationManager.cpp:142:32: error: use of undeclared identifier 'Language'; did you mean 'LanguageID'?
  else if (IK.getLanguage() == Language::CXX) {
                               ^~~~~~~~
                               LanguageID
/home/avj/llvm/root/include/clang/Basic/Builtins.h:32:6: note: 'LanguageID' declared here
enum LanguageID {
     ^
/home/avj/cvise/clang_delta/TransformationManager.cpp:142:32: error: no member named 'CXX' in 'clang::LanguageID'; did you mean 'clang::InputKind::CXX'?
  else if (IK.getLanguage() == Language::CXX) {
                               ^~~~~~~~~~~~~
                               clang::InputKind::CXX
/home/avj/llvm/root/include/clang/Frontend/FrontendOptions.h:165:5: note: 'clang::InputKind::CXX' declared here
    CXX,
    ^
/home/avj/cvise/clang_delta/TransformationManager.cpp:146:72: error: use of undeclared identifier 'Language'; did you mean 'LanguageID'?
    Invocation.setLangDefaults(ClangInstance->getLangOpts(), InputKind(Language::CXX), T, PPOpts, LSTD);
                                                                       ^~~~~~~~
                                                                       LanguageID
/home/avj/llvm/root/include/clang/Basic/Builtins.h:32:6: note: 'LanguageID' declared here
enum LanguageID {
     ^
/home/avj/cvise/clang_delta/TransformationManager.cpp:146:72: error: no member named 'CXX' in 'clang::LanguageID'; did you mean 'clang::InputKind::CXX'?
    Invocation.setLangDefaults(ClangInstance->getLangOpts(), InputKind(Language::CXX), T, PPOpts, LSTD);
                                                                       ^~~~~~~~~~~~~
                                                                       clang::InputKind::CXX
/home/avj/llvm/root/include/clang/Frontend/FrontendOptions.h:165:5: note: 'clang::InputKind::CXX' declared here
    CXX,
    ^
/home/avj/cvise/clang_delta/TransformationManager.cpp:148:31: error: use of undeclared identifier 'Language'; did you mean 'LanguageID'?
  else if(IK.getLanguage() == Language::OpenCL) {
                              ^~~~~~~~
                              LanguageID
/home/avj/llvm/root/include/clang/Basic/Builtins.h:32:6: note: 'LanguageID' declared here
enum LanguageID {
     ^
/home/avj/cvise/clang_delta/TransformationManager.cpp:148:31: error: no member named 'OpenCL' in 'clang::LanguageID'; did you mean 'clang::InputKind::OpenCL'?
  else if(IK.getLanguage() == Language::OpenCL) {
                              ^~~~~~~~~~~~~~~~
                              clang::InputKind::OpenCL
/home/avj/llvm/root/include/clang/Frontend/FrontendOptions.h:168:5: note: 'clang::InputKind::OpenCL' declared here
    OpenCL,
    ^
/home/avj/cvise/clang_delta/TransformationManager.cpp:176:71: error: too few arguments to function call, expected 4, have 3
                                       ClangInstance->getDiagnostics());
                                                                      ^
/home/avj/llvm/root/include/clang/Frontend/CompilerInvocation.h:154:3: note: 'CreateFromArgs' declared here
  static bool CreateFromArgs(CompilerInvocation &Res,
  ^
/home/avj/cvise/clang_delta/TransformationManager.cpp:179:42: error: use of undeclared identifier 'Language'; did you mean 'LanguageID'?
                               InputKind(Language::OpenCL),
                                         ^~~~~~~~
                                         LanguageID
/home/avj/llvm/root/include/clang/Basic/Builtins.h:32:6: note: 'LanguageID' declared here
enum LanguageID {
     ^
/home/avj/cvise/clang_delta/TransformationManager.cpp:179:42: error: no member named 'OpenCL' in 'clang::LanguageID'; did you mean 'clang::InputKind::OpenCL'?
                               InputKind(Language::OpenCL),
                                         ^~~~~~~~~~~~~~~~
                                         clang::InputKind::OpenCL
/home/avj/llvm/root/include/clang/Frontend/FrontendOptions.h:168:5: note: 'clang::InputKind::OpenCL' declared here
    OpenCL,
    ^
14 errors generated.
make[2]: *** [clang_delta/CMakeFiles/clang_delta.dir/TransformationManager.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....

Is this a known issue?

Missing documentation: An example

I'm trying to get this working on Windows. However, I'm having a lot of struggles.
Having some documentation of how this is actualy intended to work on linux could help a lot.
For example:

py -3.6 checkSuccess.sh t.cpp + some example content of checkSuccess.sh

Assertion failure with `simple-inliner`

For the following file:

struct a;
template <typename> using b = a;
struct a {
  template <typename c> auto d(c) {
    using e = b<c>;
    return e{};
  }
} f;
auto g() -> a {
  a h;
  f.d([] {}).d(g());
  return h;
}

and arguments:

clang_delta --transformation=simple-inliner --counter=1 unit.cpp

clang_delta crashes as follows:

clang_delta: ../clang_delta/SimpleInliner.cpp:283: virtual void SimpleInliner::HandleTranslationUnit(clang::ASTContext&): Assertion `TheStmt && "NULL TheStmt!"' failed.

`clang_delta`: `query-instances` with `getQualifiedNameAsString`

Sorry if this is a "dumb" question, but what's the status of making changes to clang_delta in the cvise repo? I am not sure how closely this repo tracks creduce, or if cvise is happy to have feature extensions to clang_delta.

Why am I asking? Well, I am interested (in some circumstances) in running clang_delta to get the instance number as well as the function name for a given transformation.

For some inputs, I know a priori a function I want to preserve, but I want to replace every other function with the declaration. If query-instances could not only give the number of instances, but also the function name for each instance, that would be awesome.

If I had this, I could do something like:

# query all of the instances in our original file
clang_delta --query-instances=replace-function-def-with-decl test.cpp

# record the index of the function I care about; let's say it is index 3

# replace from the first to the second (leaving the third alone!)
clang_delta --transformation=replace-function-def-with-decl --counter=1 --to-counter=2 test.cpp > intermediate.cpp

# re-query to get the remaining numbers
clang_delta --query-instances=replace-function-def-with-decl intermediate.cpp

# validate that the original function should now be index 1

# replace from the second to the total number
clang_delta --transformation=replace-function-def-with-decl --counter=2 --to-counter=99 intermediate.cpp > final.cpp

# the file final.cpp now *only* has a body for the function that was at index 3 in the original file

I've taken a look at this, and it seems that it is achievable by using getQualifiedNameAsString inside of addOneFunctionDef.

If I was to wrap this up as a feature request, is it something that cvise would be happy to accept?

Assertion failure with `param-to-local`

For the following file:

struct a {
  a(a &b) = default;
};

and arguments:

clang_delta --transformation=param-to-local --counter=1 unit.cpp

clang_delta crashes as follows:

clang_delta: ../clang_delta/RewriteUtils.cpp:764: bool RewriteUtils::addLocalVarToFunc(const string&, clang::FunctionDecl*): Assertion `Body && "NULL body for a function definition!"' failed.

`IncludesPass` and `CommentsPass` have issues with files using "non-standard" encodings

For:

cat unit.c
/*
* für
*/ 
file unit.c
unit.c: ISO-8859 text

running cvise --start-with-pass IncludesPass gives:

00:00:00 INFO ===< 118085 >===
00:00:00 INFO running 24 interestingness tests in parallel
00:00:00 INFO INITIAL PASSES
00:00:00 INFO ===< IncludesPass >===
00:00:00 WARNING IncludesPass has encountered a non fatal bug: pass got stuck
00:00:00 INFO ===< UnIfDefPass >===
00:00:00 INFO ===< CommentsPass >===
00:00:00 WARNING CommentsPass has encountered a non fatal bug: pass got stuck
00:00:00 INFO ===< IfPass >===
Traceback (most recent call last):
  File "/usr/bin/cvise", line 286, in <module>
    reducer.reduce(pass_group, skip_initial=args.skip_initial_passes)
  File "/usr/share/cvise/cvise.py", line 143, in reduce
    self._run_additional_passes(pass_group['first'])
  File "/usr/share/cvise/cvise.py", line 166, in _run_additional_passes
    self.test_manager.run_pass(p)
  File "/usr/share/cvise/utils/testing.py", line 503, in run_pass
    self.state = self.current_pass.new(self.current_test_case, self.check_sanity)
  File "/usr/share/cvise/passes/ifs.py", line 36, in new
    bs = BinaryState.create(self.__count_instances(test_case))
  File "/usr/share/cvise/passes/ifs.py", line 22, in __count_instances
    for line in in_file.readlines():
  File "/usr/lib64/python3.8/codecs.py", line 322, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xfc in position 6: invalid start byte

(this was after editing cvise/utils/testing.py to set GIVEUP_CONSTANT to 1)

and creates:

Package: cvise 2.3.0
Git version: unknown
LLVM version: 12.0.0
System: uname_result(system='Linux', node='vistrrdslin0001', release='5.12.13-1-default', version='#1 SMP Mon Jun 28 06:37:23 UTC 2021 (74bd8c0)', machine='x86_64', processor='x86_64')
***************************************************

IncludesPass has encountered a bug:
pass got stuck
state: 2

Please consider tarring up cvise_bug_0
and creating an issue at https://github.com/marxin/cvise/issues and we will try to fix the bug.

***************************************************

and

Package: cvise 2.3.0
Git version: unknown
LLVM version: 12.0.0
System: uname_result(system='Linux', node='vistrrdslin0001', release='5.12.13-1-default', version='#1 SMP Mon Jun 28 06:37:23 UTC 2021 (74bd8c0)', machine='x86_64', processor='x86_64')
***************************************************

CommentsPass has encountered a bug:
pass got stuck
state: -1

Please consider tarring up cvise_bug_1
and creating an issue at https://github.com/marxin/cvise/issues and we will try to fix the bug.

***************************************************

Segmentation violation with multiple passes

For the following file:

template <typename...> struct a;
template <typename> struct b { b(...); };
namespace {
auto bx() -> b<a<>> {
  using by = b<int>;
  by bz;
  return bz;
}
} // namespace

(yes, it seems cvise itself can't make that smaller!) oops, didn't have -Wno-unused!

The following holds:

#!/bin/bash

set -eu

clang_delta="/home/avj/clones/cvise/master/build/clang_delta/clang_delta"
passes="aggregate-to-scalar callexpr-to-value param-to-global param-to-local remove-enum-member-value remove-namespace remove-nested-function remove-unused-function replace-callexpr simplify-callexpr union-to-struct local-to-global param-to-global"
file=unit.cpp
pass_no=1

g++ -Wall -Werror -Wno-unused -c $file

for pass_name in $passes; do
    ret=0
    ${clang_delta} --transformation=${pass_name} --counter=${pass_no} ${file} || ret=$?

    if [ $ret -ne 139 ]; then
        exit 1
    fi
done

echo "All failed"

exit 0

# EOF

That is, for all of passes aggregate-to-scalar, param-to-local, remove-nested-function, local-to-global, union-to-struct, remove-namespace and param-to-global, clang_delta crashes.

Infinitely expanding file with `remove-nested-function`

For the following file:

struct a {
  template <typename aj> void b(aj) {}
};
auto c() -> a;
auto d() {
  c().b([] {
    a d = c();
    return d;
  });
}

cvise appears to get stuck in an "infinite" loop running:

clang_delta --transformation=remove-nested-function --counter=<idx>

If you run clang_delta --transformation=remove-nested-function --counter=1 manually 10 times (with the new input being the last output), then you get something like this:

struct a {
  template <typename aj> void b(aj) {}
};
auto c() -> a;
auto d() {
  c().b([] {
    a __trans_tmp_10 = c();
    a __trans_tmp_9 = __trans_tmp_10;
    a __trans_tmp_8 = __trans_tmp_9;
    a __trans_tmp_7 = __trans_tmp_8;
    a __trans_tmp_6 = __trans_tmp_7;
    a __trans_tmp_5 = __trans_tmp_6;
    a __trans_tmp_4 = __trans_tmp_5;
    a __trans_tmp_3 = __trans_tmp_4;
    a __trans_tmp_2 = __trans_tmp_3;
    a __trans_tmp_1 = __trans_tmp_2;
    a d = __trans_tmp_1;
    return d;
  });
}

While I was cvising one file, this count got up to 5812 before I killed it; the tail of the cvise log looked like this:

01:27:39 INFO (-545.8%, 608185 bytes, 6636 lines)
01:27:40 INFO (-545.9%, 608282 bytes, 6637 lines)
01:27:41 INFO (-546.0%, 608379 bytes, 6638 lines)
01:27:42 INFO (-546.1%, 608476 bytes, 6639 lines)
01:27:43 INFO (-546.2%, 608573 bytes, 6640 lines)
01:27:44 INFO (-546.3%, 608670 bytes, 6641 lines)
01:27:45 INFO (-546.4%, 608767 bytes, 6642 lines)
01:27:46 INFO (-546.5%, 608864 bytes, 6643 lines)
01:27:48 INFO (-546.6%, 608961 bytes, 6644 lines)
01:27:49 INFO (-546.7%, 609058 bytes, 6645 lines)
01:27:50 INFO (-546.8%, 609155 bytes, 6646 lines)
01:27:51 INFO (-546.9%, 609252 bytes, 6647 lines)
01:27:52 INFO (-547.0%, 609349 bytes, 6648 lines)
01:27:53 INFO (-547.1%, 609446 bytes, 6649 lines)
01:27:54 INFO (-547.2%, 609543 bytes, 6650 lines)
01:27:56 INFO (-547.3%, 609640 bytes, 6651 lines)
01:27:57 INFO (-547.5%, 609737 bytes, 6652 lines)
01:27:58 INFO (-547.6%, 609834 bytes, 6653 lines)
01:27:59 INFO (-547.7%, 609931 bytes, 6654 lines)
01:28:00 INFO (-547.8%, 610028 bytes, 6655 lines)
01:28:01 INFO (-547.9%, 610125 bytes, 6656 lines)
01:28:02 INFO (-548.0%, 610222 bytes, 6657 lines)
01:28:03 INFO (-548.1%, 610319 bytes, 6658 lines)
01:28:05 INFO (-548.2%, 610416 bytes, 6659 lines)
01:28:06 INFO (-548.3%, 610513 bytes, 6660 lines)

and cvise first stated doing --transformation=remove-nested-function at 00:06:27:

00:06:16 INFO ===< ClangPass::remove-namespace >===
00:06:20 INFO ===< ClangPass::aggregate-to-scalar >===
00:06:20 INFO ===< ClangPass::param-to-global >===
00:06:25 INFO ===< ClangPass::param-to-local >===
00:06:27 INFO ===< ClangPass::remove-nested-function >===

and tried no other passes -- that is, it spent 1h21m doing remove-nested-function with the file slowly getting larger every time.

No module named {pytest|'pebble'}

Following up on #12. Today I pulled, and reran cmake, which failed.

$ cmake .. -G Ninja -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++
...
  Pytest package not available: /usr/bin/python3.6: No module named pytest
  ModuleNotFoundError: No module named 'pebble'

but this worked with my previous checkout. In my shell history, I have:

$ pip3 install pebble pytest

so I thought I had these installed correctly? I tried wiping out my build dir and cmake cache files, but it made no difference.

Rerunning:

$ pip3 install pebble pytest
Requirement already satisfied: pebble in /usr/local/google/home/ndesaulniers/.local/lib/python3.7/site-packages (4.5.1)
Requirement already satisfied: pytest in /usr/local/google/home/ndesaulniers/.local/lib/python3.7/site-packages (5.4.1)
Requirement already satisfied: attrs>=17.4.0 in /usr/lib/python3/dist-packages (from pytest) (19.3.0)
Requirement already satisfied: pluggy<1.0,>=0.12 in /usr/local/google/home/ndesaulniers/.local/lib/python3.7/site-packages (from pytest) (0.13.1)
Requirement already satisfied: packaging in /usr/lib/python3/dist-packages (from pytest) (19.1)
Requirement already satisfied: wcwidth in /usr/local/google/home/ndesaulniers/.local/lib/python3.7/site-packages (from pytest) (0.1.9)
Requirement already satisfied: importlib-metadata>=0.12; python_version < "3.8" in /usr/local/google/home/ndesaulniers/.local/lib/python3.7/site-packages (from pytest) (1.6.0)
Requirement already satisfied: py>=1.5.0 in /usr/local/google/home/ndesaulniers/.local/lib/python3.7/site-packages (from pytest) (1.8.1)
Requirement already satisfied: more-itertools>=4.0.0 in /usr/local/google/home/ndesaulniers/.local/lib/python3.7/site-packages (from pytest) (8.2.0)
Requirement already satisfied: zipp>=0.5 in /usr/local/google/home/ndesaulniers/.local/lib/python3.7/site-packages (from importlib-metadata>=0.12; python_version < "3.8"->pytest) (3.1.0)

Those seem to mention python3.7, but the cmake failures for pytest mention python3.6. Is that related?

Build detection of Python and Pytest

From @mgorny:

  • You should really use the interpreter found by PythonInterp (PYTHON_EXECUTABLE if I'm not mistaken) instead of hardcoding python3 which will not respect user overrides.
  • Do you really want to enforce pytest on everyone, even if there's no intention of running tests?

Oh, and this really needs to be either purely informational, or come with a switch to disable the checks. Otherwise, you're going to prevent people from building binary packages without installing runtime dependencies.

cvise --help fails

this is a regression in 1.5, compared to 1.4:

cvise --help
Traceback (most recent call last):
File "debian/cvise/usr/bin/cvise", line 166, in
passes_group.add_argument("--pass-group", type=str, choices=get_available_pass_groups(), help="Set of passes used during the reduction")
File "debian/cvise/usr/bin/cvise", line 113, in get_available_pass_groups
CVise.parse_pass_group_dict(pass_group_dict, set(), None, None, None, None, None)
TypeError: parse_pass_group_dict() takes 7 positional arguments but 8 were given

Do you handle access specifier?

I see in reductions that code like

class test : private bar
{
public:
  statement1;
  statement2;
};

will often be collapsed to

class test : private bar
{
public: statement1;
statement2;
};

The problem is that the statement1 line could be removed, but will trigger a build error because statement2 isn't accessible any more.

I would expect that all access specifier will be removed at a pretty early stage as they most often shouldn't matter.

That means class should be replaced by struct and any access specifier should be removed from a class.

/*class*/ struct test : /*private*/ public bar
{
/*public:*/ statement1;
statement2;
};

cannot find external program {clang_delta|clex|clang-format}

Hello!

I ran cvise today and get a bunch of warnings:

00:00:00 ERROR Prereqs not found for pass ClangPass::vector-to-array
00:00:00 ERROR cannot find external program clang_delta
00:00:00 ERROR Prereqs not found for pass ClangPass::remove-try-catch
00:00:00 ERROR cannot find external program clang-format
00:00:00 ERROR Prereqs not found for pass IndentPass::regular
00:00:00 ERROR cannot find external program clex

cvise still ran to completion, but it looks like some passes were skipped because these are missing.

clang-format is in my $PATH; clang_delta and clex are not. (I haven't heard of those latter two) Not sure if this is an issue with relative paths based on how I ran cvise:

$ cd <dir that's not cvise sources>
$ /<abs path to>/cvise/cvise.py --n 71 ./repro.sh slub.i   

or if I've probably not enabled clang-tools-extra for my build of llvm?

ERROR Prereqs not found for pass

so first shot at cvise:

$ /android1/cvise/cvise.py ./repro.sh compat2.i 
ERROR Prereqs not found for pass ClangBinarySearchPass::replace-function-def-with-decl
ERROR Prereqs not found for pass ClangBinarySearchPass::remove-unused-function
ERROR Prereqs not found for pass ClangBinarySearchPass::replace-function-def-with-decl
ERROR Prereqs not found for pass ClangBinarySearchPass::remove-unused-function
ERROR Prereqs not found for pass ClangPass::remove-unused-function
ERROR Prereqs not found for pass ClangPass::callexpr-to-value
ERROR Prereqs not found for pass ClangPass::replace-callexpr
ERROR Prereqs not found for pass ClangPass::simplify-callexpr
ERROR Prereqs not found for pass ClangPass::remove-unused-enum-member
ERROR Prereqs not found for pass ClangPass::remove-enum-member-value
ERROR Prereqs not found for pass ClangBinarySearchPass::remove-unused-var
ERROR Prereqs not found for pass ClangPass::remove-namespace
ERROR Prereqs not found for pass ClangPass::aggregate-to-scalar
ERROR Prereqs not found for pass ClangPass::param-to-global
ERROR Prereqs not found for pass ClangPass::param-to-local
ERROR Prereqs not found for pass ClangPass::remove-nested-function
ERROR Prereqs not found for pass ClangPass::union-to-struct
ERROR Prereqs not found for pass ClangPass::return-void
ERROR Prereqs not found for pass ClangPass::simple-inliner
ERROR Prereqs not found for pass ClangPass::reduce-pointer-level
ERROR Prereqs not found for pass ClangPass::lift-assignment-expr
ERROR Prereqs not found for pass ClangPass::copy-propagation
ERROR Prereqs not found for pass ClangPass::callexpr-to-value
ERROR Prereqs not found for pass ClangPass::simplify-callexpr
ERROR Prereqs not found for pass ClangPass::remove-unused-function
ERROR Prereqs not found for pass ClangPass::remove-unused-enum-member
ERROR Prereqs not found for pass ClangPass::remove-enum-member-value
ERROR Prereqs not found for pass ClangBinarySearchPass::remove-unused-var
ERROR Prereqs not found for pass ClangPass::simplify-if
ERROR Prereqs not found for pass ClangPass::reduce-array-dim
ERROR Prereqs not found for pass ClangPass::reduce-array-size
ERROR Prereqs not found for pass ClangPass::move-function-body
ERROR Prereqs not found for pass ClangPass::simplify-comma-expr
ERROR Prereqs not found for pass ClangPass::simplify-dependent-typedef
ERROR Prereqs not found for pass ClangPass::replace-simple-typedef
ERROR Prereqs not found for pass ClangPass::replace-dependent-typedef
ERROR Prereqs not found for pass ClangPass::replace-one-level-typedef-type
ERROR Prereqs not found for pass ClangPass::remove-unused-field
ERROR Prereqs not found for pass ClangPass::instantiate-template-type-param-to-int
ERROR Prereqs not found for pass ClangPass::instantiate-template-param
ERROR Prereqs not found for pass ClangPass::template-arg-to-int
ERROR Prereqs not found for pass ClangPass::template-non-type-arg-to-int
ERROR Prereqs not found for pass ClangPass::reduce-class-template-param
ERROR Prereqs not found for pass ClangPass::remove-trivial-base-template
ERROR Prereqs not found for pass ClangPass::class-template-to-class
ERROR Prereqs not found for pass ClangPass::remove-base-class
ERROR Prereqs not found for pass ClangPass::replace-derived-class
ERROR Prereqs not found for pass ClangPass::remove-unresolved-base
ERROR Prereqs not found for pass ClangPass::remove-ctor-initializer
ERROR Prereqs not found for pass ClangPass::replace-class-with-base-template-spec
ERROR Prereqs not found for pass ClangPass::simplify-nested-class
ERROR Prereqs not found for pass ClangPass::remove-unused-outer-class
ERROR Prereqs not found for pass ClangPass::empty-struct-to-int
ERROR Prereqs not found for pass ClangPass::remove-pointer
ERROR Prereqs not found for pass ClangPass::reduce-pointer-pairs
ERROR Prereqs not found for pass ClangPass::remove-array
ERROR Prereqs not found for pass ClangPass::rename-fun
ERROR Prereqs not found for pass ClangPass::rename-param
ERROR Prereqs not found for pass ClangPass::rename-var
ERROR Prereqs not found for pass ClangPass::rename-class
ERROR Prereqs not found for pass ClangPass::rename-cxx-method
ERROR Prereqs not found for pass ClangPass::combine-global-var
ERROR Prereqs not found for pass ClangPass::combine-local-var
ERROR Prereqs not found for pass ClangPass::simplify-struct-union-decl
ERROR Prereqs not found for pass ClangPass::move-global-var
ERROR Prereqs not found for pass ClangPass::unify-function-decl
ERROR Prereqs not found for pass ClexPass::rename-toks
ERROR Prereqs not found for pass ClexPass::delete-string
Missing prerequisites for passes ClangPass::lift-assignment-expr, ClangPass::reduce-pointer-pairs, ClangPass::remove-unresolved-base, ClexPass::rename-toks, ClexPass::rm-toks-3, ClangPass::simplify-nested-class, ClangPass::aggregate-to-scalar, ClangPass::rename-fun, ClangPass::simplify-dependent-typedef, ClangPass::rename-cxx-method, ClangPass::remove-enum-member-value, ClangPass::replace-one-level-typedef-type, ClangPass::template-non-type-arg-to-int, ClangPass::simplify-callexpr, ClangPass::rename-class, ClangPass::rename-param, ClangPass::callexpr-to-value, ClangPass::remove-ctor-initializer, ClangBinarySearchPass::replace-function-def-with-decl, ClangPass::vector-to-array, ClangPass::copy-propagation, ClangPass::param-to-local, ClexPass::rm-toks-11, ClangPass::remove-base-class, ClangPass::replace-derived-class, ClangPass::remove-namespace, ClangPass::remove-array, ClangPass::instantiate-template-param, ClexPass::rm-toks-4, ClangPass::remove-trivial-base-template, ClangPass::simplify-struct, ClangPass::replace-array-access-with-index, ClangPass::remove-unused-enum-member, ClangPass::replace-simple-typedef, ClangPass::reduce-pointer-level, ClangPass::move-function-body, ClangPass::replace-dependent-name, ClexPass::delete-string, ClexPass::rm-toks-14, ClangPass::remove-unused-outer-class, ClangPass::simplify-recursive-template-instantiation, ClexPass::rm-toks-15, ClexPass::rm-toks-10, ClangPass::remove-unused-function, ClangPass::replace-array-index-var, ClangPass::reduce-array-dim, ClangPass::combine-global-var, ClangPass::simplify-comma-expr, ClangPass::instantiate-template-type-param-to-int, ClexPass::rm-toks-12, ClangPass::remove-try-catch, ClexPass::rm-toks-8, ClangPass::move-global-var, ClangPass::replace-class-with-base-template-spec, ClangPass::simple-inliner, ClangPass::reduce-class-template-param, ClexPass::rm-toks-5, ClexPass::rm-toks-2, ClangPass::replace-callexpr, ClangPass::simplify-if, ClangPass::remove-nested-function, ClexPass::rm-toks-9, ClangPass::simplify-struct-union-decl, ClangPass::replace-undefined-function, ClangPass::remove-pointer, ClangPass::replace-dependent-typedef, ClexPass::rm-tok-pattern-4, ClexPass::define, ClangPass::unify-function-decl, ClangPass::empty-struct-to-int, ClexPass::rm-toks-6, ClangPass::rename-var, ClangPass::local-to-global, ClangBinarySearchPass::remove-unused-function, ClangPass::remove-addr-taken, ClangPass::template-arg-to-int, ClangPass::combine-local-var, ClangPass::remove-unused-field, ClangPass::class-template-to-class, ClangPass::param-to-global, ClangPass::reduce-array-size, ClangPass::return-void, ClexPass::rm-toks-16, ClexPass::rm-toks-1, ClexPass::rm-toks-13, ClangBinarySearchPass::remove-unused-var, ClangPass::union-to-struct, ClexPass::rm-toks-7!

am I missing a dependency, or invoking this incorrectly? I built cvise via:

$ mkdir build && cd build
$ cmake .. -G Ninja -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++

or should I not be doing out of tree builds?

"--clang-delta-std" ignored?

This isn't a big deal to me, but I just ran cvise like this:

cvise --debug --clang-delta-std c++11 --n 97 cvise.sh ac.cpp |& cat > log.txt

but I then noticed the following in the log file:

grep "/usr/libexec/cvise/clang_delta" log.txt | egrep -oh "std=[^ ]*" | sort | uniq -c | sort -n
   1002 std=c++98
   4870 std=c++17

So we've done a 1002 instances of c++98 and 4870 of c++17 -- but none of c++11.

Have I missed something, or is this a bug?

This is with:

cvise --version
cvise 2.1.0

from 2.1.0+git.20210111.972f4ba-1.2 on Tumbleweed.

cvise from git seems to miss an import: NameError: name 'subprocess' is not defined

Ran today's cvise: and observerd the crash:

$ cvise --remove-pass IfPass ./test_cvise.sh *.c
00:00:01 INFO ===< 834890 >===
00:00:01 INFO running 4 interestingness tests in parallel
00:00:01 INFO INITIAL PASSES
00:00:01 INFO ===< IncludesPass >===
00:00:16 INFO ===< UnIfDefPass >===
00:00:41 INFO ===< CommentsPass >===
00:00:45 INFO ===< LineMarkersPass >===
00:00:45 INFO ===< BlankPass >===
00:00:55 INFO (0.0%, 683496 bytes)
00:00:59 INFO ===< ClangBinarySearchPass::replace-function-def-with-decl >===
Traceback (most recent call last):
  File "/usr/share/cvise/passes/clangbinarysearch.py", line 51, in count_instances
    proc = subprocess.run(cmd, universal_newlines=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
NameError: name 'subprocess' is not defined

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/bin/cvise", line 265, in <module>
    reducer.reduce(pass_group, skip_initial=args.skip_initial_passes)
  File "/usr/share/cvise/cvise.py", line 122, in reduce
    self._run_additional_passes(pass_group["first"])
  File "/usr/share/cvise/cvise.py", line 145, in _run_additional_passes
    self.test_manager.run_pass(p)
  File "/usr/share/cvise/utils/testing.py", line 458, in run_pass
    self.state = self.current_pass.new(self.current_test_case)
  File "/usr/share/cvise/passes/clangbinarysearch.py", line 31, in new
    self.detect_best_standard(test_case)
  File "/usr/share/cvise/passes/clangbinarysearch.py", line 20, in detect_best_standard
    instances = self.count_instances(test_case)
  File "/usr/share/cvise/passes/clangbinarysearch.py", line 52, in count_instances
    except subprocess.SubprocessError:
NameError: name 'subprocess' is not defined

'cvise' sometimes passes non-existent files to 'clex'

Initially I noticed it as core dumps filling the disk at late stages of cvise run. Crashes usually happen around ClexPass::rm-toks-<N> phases.

Example crash dump of clex:

$ file core.580492
core.580492: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from '/usr/libexec/cvise/clex rm-toks-2 10 /tmp/cvise-yohufd05/cvise-7sc5myxt/main.c', real uid: 1000, effective uid: 1000, real gid: 100, effective gid: 100, execfn: '/usr/libexec/cvise/clex', platform: 'x86_64'

$ gdb --quiet /usr/libexec/cvise/clex core.580492
Reading symbols from /usr/libexec/cvise/clex...
Reading symbols from /usr/lib/debug//usr/libexec/cvise/clex.debug...
[New LWP 580492]
Core was generated by `/usr/libexec/cvise/clex rm-toks-2 10 /tmp/cvise-yohufd05/cvise-7sc5myxt/main.c'.
Program terminated with signal SIGABRT, Aborted.
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
50        return ret;
(gdb) bt
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1  0x00007f4461671537 in __GI_abort () at abort.c:79
#2  0x00007f446167140f in __assert_fail_base (fmt=0x7f44617d3078 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n",
    assertion=0x559dc86042f2 "in", file=0x559dc86041cc "../cvise-9999/clex/driver.c", line=473, function=<optimized out>)
    at assert.c:92
#3  0x00007f44616803c2 in __GI___assert_fail (assertion=0x559dc86042f2 "in",
    file=0x559dc86041cc "../cvise-9999/clex/driver.c", line=473, function=0x559dc8604324 <__PRETTY_FUNCTION__.1> "main")
    at assert.c:101
#4  0x0000559dc86003e2 in main (argc=<optimized out>, argv=0x7ffe196361c8) at ../cvise-9999/clex/driver.c:473
(gdb) frame 4
#4  0x0000559dc86003e2 in main (argc=<optimized out>, argv=0x7ffe196361c8) at ../cvise-9999/clex/driver.c:473
473       assert(in);
(gdb) list
468       int tok_index;
469       int ret = sscanf(argv[2], "%d", &tok_index);
470       assert(ret == 1);
471       // printf ("file = '%s'\n", argv[3]);
472       FILE *in = fopen(argv[3], "r");
473       assert(in);
474       yyin = in;
475
476       max_toks = initial_length;
477       tok_list = (struct tok_t *)malloc(max_toks * sizeof(struct tok_t));

No graceful file check handling. If I add a bit of debugging backtrace looks like that:

--- /home/slyfox/dev/git/cvise/cvise/passes/clex.py     2020-07-22 22:01:16.000000000 +0100
+++ /usr/share/cvise/passes/clex.py     2020-07-23 23:01:36.000000000 +0100
@@ -2,4 +2,5 @@
 import shutil
 import tempfile
+import traceback

 from cvise.passes.abstract import AbstractPass, PassResult
@@ -19,4 +20,7 @@

     def transform(self, test_case, state, process_event_notifier):
+        if not os.path.isfile(test_case):
+            traceback.print_stack()
+
         tmp = os.path.dirname(test_case)
         with tempfile.NamedTemporaryFile(mode="w", delete=False, dir=tmp) as tmp_file:```
00:02:05 INFO ===< ClexPass::rm-toks-8 >===
  File "/usr/bin/cvise", line 265, in <module>
    reducer.reduce(pass_group, skip_initial=args.skip_initial_passes)
  File "/usr/share/cvise/cvise.py", line 125, in reduce
    self._run_main_passes(pass_group["main"])
  File "/usr/share/cvise/cvise.py", line 155, in _run_main_passes
    self.test_manager.run_pass(p)
  File "/usr/share/cvise/utils/testing.py", line 474, in run_pass
    success_env = self.run_parallel_tests()
  File "/usr/share/cvise/utils/testing.py", line 409, in run_parallel_tests
    future = pool.schedule(test_env.run, timeout=self.timeout)
  File "/usr/lib/python3.7/site-packages/pebble/pool/process.py", line 90, in schedule
    self._check_pool_state()
  File "/usr/lib/python3.7/site-packages/pebble/pool/base_pool.py", line 91, in _check_pool_state
    self._update_pool_state()
  File "/usr/lib/python3.7/site-packages/pebble/pool/base_pool.py", line 100, in _update_pool_state
    self._start_pool()
  File "/usr/lib/python3.7/site-packages/pebble/pool/process.py", line 63, in _start_pool
    self._pool_manager.start()
  File "/usr/lib/python3.7/site-packages/pebble/pool/process.py", line 192, in start
    self.worker_manager.create_workers()
  File "/usr/lib/python3.7/site-packages/pebble/pool/process.py", line 364, in create_workers
    self.new_worker()
  File "/usr/lib/python3.7/site-packages/pebble/pool/process.py", line 377, in new_worker
    None, worker_process, True, self.worker_parameters, self.workers_channel)
  File "/usr/lib/python3.7/site-packages/pebble/common.py", line 141, in launch_process
    process.start()
  File "/usr/lib/python3.7/multiprocessing/process.py", line 112, in start
    self._popen = self._Popen(self)
  File "/usr/lib/python3.7/multiprocessing/context.py", line 223, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "/usr/lib/python3.7/multiprocessing/context.py", line 277, in _Popen
    return Popen(process_obj)
  File "/usr/lib/python3.7/multiprocessing/popen_fork.py", line 20, in __init__
    self._launch(process_obj)
  File "/usr/lib/python3.7/multiprocessing/popen_fork.py", line 74, in _launch
    code = process_obj._bootstrap()
  File "/usr/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
    self.run()
  File "/usr/lib/python3.7/multiprocessing/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/lib/python3.7/site-packages/pebble/pool/process.py", line 407, in worker_process
    payload.function, *payload.args, **payload.kwargs)
  File "/usr/lib/python3.7/site-packages/pebble/common.py", line 174, in process_execute
    return function(*args, **kwargs)
  File "/usr/share/cvise/utils/testing.py", line 103, in run
    ProcessEventNotifier(self.pid_queue))
  File "/usr/share/cvise/passes/clex.py", line 23, in transform
    traceback.print_stack()

As if occasionally temp dir was cleaned before worker processes finished execution.

Better support for `--preserve-routine` in `clang_delta`

In #44, functionality was introduced that allowed the user to run clang_delta (with --transformation=replace-function-def-with-decl), but to say "do not change this routine".

We have a few extensions that are worth implementing:

  • Extend --query-instances to not only print the instance count, but also the names of the possible routines (which then allows a user to know which name to use as part of --preserve-routine).
  • Support the use of the "full" name (e.g., in the case of templated methods in C++) -- currently templated names do not use the full expansion
  • Allow the user to provide a regex over a simple string, and therefore preserve any matching routine

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.