Comments (23)
Hello.
It seems to me as an unusual temporary directory (expected would be C:\Users\USERNAME\AppData\Local\Temp
). Can you please check the usual one?
Unfortunately, I don't have a Windows machine I can test it on.
from cvise.
It seems to me as an unusual temporary directory
Yeah, now you point that out, that is odd. c:\\Users\\xxx\\Desktop\\help
is the directory in which the source file and script calling
the compiler live. %TEMP%
and %TMP%
do point to C:\Users\xxx\AppData\Local\Temp
. I'll have a look to see why it's not picking up on that.
from cvise.
Hi again,
It seems that Windows 11 doesn't like a file to be opened twice. The following fails with the same error:
import tempfile
import shutil
with tempfile.NamedTemporaryFile(mode='w+') as x:
shutil.copy(some_file_that_exists, x.name)
In addition temp directory is not being used due to the following line:
Line 16 in 00bdd8c
from cvise.
Interesting! Anyway, it's something we can probably address with the new API provided by Python 3.12:
https://docs.python.org/3/library/tempfile.html#tempfile.NamedTemporaryFile
where we can set delete_on_close=False
and then call tmp_file.close()
in the with
block.
Any chance you could test the suggested change on your system, please?
from cvise.
Yes, that works if I close both tmp_file
after the flush and backup
sometime before shutil.copy(test_case, backup.name)
.
from cvise.
I also need to make similar changes to the clang
and clex
passes, otherwise those passes hang.
from cvise.
Great, then let me prepare a proper PR during the weekend that would address that.
from cvise.
Thanks for your help. Here's what I did:
diff --git a/cvise/passes/clang.py b/cvise/passes/clang.py
index 1247c88..1e714f1 100644
--- a/cvise/passes/clang.py
+++ b/cvise/passes/clang.py
@@ -21,7 +21,7 @@ class ClangPass(AbstractPass):
def transform(self, test_case, state, process_event_notifier):
tmp = os.path.dirname(test_case)
- with tempfile.NamedTemporaryFile(mode='w', delete=False, dir=tmp) as tmp_file:
+ with tempfile.NamedTemporaryFile(mode='w', delete=False, dir=tmp, delete_on_close=False) as tmp_file:
args = [
self.external_programs['clang_delta'],
f'--transformation={self.arg}',
@@ -36,9 +36,11 @@ class ClangPass(AbstractPass):
stdout, _stderr, returncode = process_event_notifier.run_process(cmd)
if returncode == 0:
tmp_file.write(stdout)
+ tmp_file.close()
shutil.move(tmp_file.name, test_case)
return (PassResult.OK, state)
else:
+ tmp_file.close()
os.unlink(tmp_file.name)
if returncode == 255 or returncode == 1:
return (PassResult.STOP, state)
diff --git a/cvise/passes/clex.py b/cvise/passes/clex.py
index 64b9000..352fc52 100644
--- a/cvise/passes/clex.py
+++ b/cvise/passes/clex.py
@@ -20,14 +20,16 @@ class ClexPass(AbstractPass):
def transform(self, test_case, state, process_event_notifier):
tmp = os.path.dirname(test_case)
- with tempfile.NamedTemporaryFile(mode='w', delete=False, dir=tmp) as tmp_file:
+ with tempfile.NamedTemporaryFile(mode='w', delete=False, dir=tmp, delete_on_close=False) as tmp_file:
cmd = [self.external_programs['clex'], str(self.arg), str(state), test_case]
stdout, _stderr, returncode = process_event_notifier.run_process(cmd)
if returncode == 51:
tmp_file.write(stdout)
+ tmp_file.close()
shutil.move(tmp_file.name, test_case)
return (PassResult.OK, state)
else:
+ tmp_file.close()
os.unlink(tmp_file.name)
return (
PassResult.STOP if returncode == 71 else PassResult.ERROR,
diff --git a/cvise/passes/lines.py b/cvise/passes/lines.py
index 2eb0d48..6c67c59 100644
--- a/cvise/passes/lines.py
+++ b/cvise/passes/lines.py
@@ -15,9 +15,10 @@ class LinesPass(AbstractPass):
def __format(self, test_case, check_sanity):
tmp = os.path.dirname(test_case)
- with tempfile.NamedTemporaryFile(mode='w+', dir=tmp) as backup, tempfile.NamedTemporaryFile(
- mode='w+', dir=tmp
+ with tempfile.NamedTemporaryFile(mode='w+', dir=tmp, delete_on_close=False) as backup, tempfile.NamedTemporaryFile(
+ mode='w+', dir=tmp, delete_on_close=False
) as tmp_file:
+ backup.close()
with open(test_case) as in_file:
try:
cmd = [self.external_programs['topformflat'], self.arg]
@@ -29,6 +30,7 @@ class LinesPass(AbstractPass):
if not line.isspace():
tmp_file.write(line)
tmp_file.flush()
+ tmp_file.close()
# we need to check that sanity check is still fine
if check_sanity:
from cvise.
There's also a similar problem in the clangbinarysearch
pass, which I had missed initially, as I ran with a test case that was already reduced.
from cvise.
Can you please test the pull request I've just created?
from cvise.
On your PR I'm seeing it hanging in ClangPass::remove-unused-function
that probably means there's still some file that cannot be accessed. Before any of mine or your changes I also saw this with ClangBinarySearchPass
, but that works with your PR, so there must be some subtle difference between the passes. It's not clear to me what though, the code seems pretty much identical.
from cvise.
Thanks for the testing effort! Yeah, the passes seem very much the same. Can you please terminate the program and show me the corresponding back-traces?
from cvise.
The backtrace I'm getting is:
Exception ignored in: <Finalize object, dead>
Traceback (most recent call last):
File "C:\Users\jketema\AppData\Local\Programs\Python\Python312\Lib\multiprocessing\util.py", line 224, in __call__
res = self._callback(*self._args, **self._kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\jketema\AppData\Local\Programs\Python\Python312\Lib\multiprocessing\managers.py", line 873, in _decref
conn = _Client(token.address, authkey=authkey)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\jketema\AppData\Local\Programs\Python\Python312\Lib\multiprocessing\connection.py", line 507, in Client
answer_challenge(c, authkey)
File "C:\Users\jketema\AppData\Local\Programs\Python\Python312\Lib\multiprocessing\connection.py", line 935, in answer_challenge
message = connection.recv_bytes(256) # reject large message
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\jketema\AppData\Local\Programs\Python\Python312\Lib\multiprocessing\connection.py", line 215, in recv_bytes
buf = self._recv_bytes(maxlength)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\jketema\AppData\Local\Programs\Python\Python312\Lib\multiprocessing\connection.py", line 304, in _recv_bytes
waitres = _winapi.WaitForMultipleObjects(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
KeyboardInterrupt:
00:00:28 INFO Exiting now ...
from cvise.
Note: eventually cvise does continue after printing:
00:05:42 WARNING ClangPass::remove-unused-function has encountered a non fatal bug: pass got stuck
from cvise.
Can you please add more debugging output to the pass:
diff --git a/cvise/passes/clang.py b/cvise/passes/clang.py
index 990184d..c6cbc4b 100644
--- a/cvise/passes/clang.py
+++ b/cvise/passes/clang.py
@@ -33,7 +33,8 @@ class ClangPass(AbstractPass):
logging.debug(' '.join(cmd))
- stdout, _, returncode = process_event_notifier.run_process(cmd)
+ stdout, stderr, returncode = process_event_notifier.run_process(cmd)
+ print(stdout[:64], stderr, returncode)
if returncode == 0:
tmp_file.write(stdout)
tmp_file.close()
and test it?
from cvise.
The output is:
Error: The counter value exceeded the number of transformation i 1
Where print(cmd)
gives:
['C:/Users/xxx/Desktop/cvise/tools/cvise2/libexec\\cvise\\clang_delta.EXE', '--transformation=remove-unused-function', '--counter=505', 'C:\\Users\\xxx\\AppData\\Local\\Temp\\cvise-ClangPass-remove-unused-function-leyd0zb3\\cvise-yp5abkhu\\test.cpp']
And test.cpp
contains:
template <class a, class...> bool b = __is_nothrow_constructible(a);
class c;
class B {
template <typename... d> B() noexcept(b<c, d...>);
};
class e : B {
};
from cvise.
Error: The counter value exceeded the number of transformation i 1
Yeah, that's expected output, but I don't see why should it become stuck. Please attach a full reduction log: (with --debug
option).
from cvise.
There's very little log data:
...
00:00:15 INFO ===< ClangBinarySearchPass::remove-unused-function >===
00:00:15 DEBUG available transformation opportunities for c++98: 1, took: 0.02 s
00:00:15 DEBUG available transformation opportunities for c++11: 1, took: 0.03 s
00:00:15 DEBUG available transformation opportunities for c++14: 1, took: 0.02 s
00:00:15 DEBUG available transformation opportunities for c++17: 1, took: 0.02 s
00:00:15 DEBUG available transformation opportunities for c++20: 1, took: 0.02 s
00:00:15 DEBUG available transformation opportunities for c++2b: 1, took: 0.02 s
00:00:15 INFO using C++ standard: c++2b with 1 transformation opportunities
00:00:16 DEBUG Creating pass root folder: C:\Users\xxx\AppData\Local\Temp\cvise-ClangPass-remove-unused-function-0sj3kcmo
00:00:16 INFO ===< ClangPass::remove-unused-function >===
00:06:38 WARNING ClangPass::remove-unused-function has encountered a non fatal bug: pass got stuck
00:06:38 DEBUG Please consider tarring up cvise_bug_2 and creating an issue at https://github.com/marxin/cvise/issues and we will try to fix the bug.
00:06:51 DEBUG Creating pass root folder: C:\Users\xx\AppData\Local\Temp\cvise-BalancedPass-curly-ze7mdy62
00:06:51 INFO ===< BalancedPass::curly >===
...
from cvise.
The problem goes away when I close the file in the location where the unlink was at the beginning of the else
branch in the clang pass. And similarly for the clex pass.
from cvise.
Oh, got catch!
Apparently, it's something that produces an exception:
https://docs.python.org/3/library/os.html#os.remove
On Windows, attempting to remove a file that is in use causes an exception to be raised; on Unix, the directory entry is removed but the storage allocated to the file is not made available until the original file is no longer in use.
Please try the latest version of the #139.
from cvise.
The latest version works for me!
from cvise.
Great, thanks for testing!
from cvise.
Thanks for fixing!
from cvise.
Related Issues (20)
- remove-unused-outer-class is very restrictive on what it removes HOT 1
- cvise stops intermittently HOT 7
- Parallelism doesn't do much HOT 9
- When are shortcuts taken into account? HOT 3
- clang_delta: […] clang::Expr::ClassifyImpl(…) const: Assertion `isLValue()' failed. HOT 15
- cvise: running only the "light" transformations HOT 2
- ability to use /bin/sh for --commands HOT 2
- Release new version HOT 1
- Understanding C-Vise Performance with multiple cores and Comparison with C-Reduce HOT 7
- Idiomatic way to avoid reductions on some parts of the code? HOT 2
- C-Vise doesn't build with recent LLVM hashes HOT 1
- cvise fails to build with LLVM 18.1.0 HOT 7
- Req: cvise use with runtime errors HOT 1
- LLVM 19: Build failures HOT 8
- CVise fills temp directory on Windows HOT 1
- cvise doesn't reliably kill things upon timeout. HOT 3
- Add the ability to control when `cvise_extra` directories are created. HOT 2
- Cvise stopped abnormally midway HOT 5
- Obscure crash error message HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cvise.