artifacts_folder FileNotFoundError in combination with pytest-parallel

[microsoft]

artifacts_folder FileNotFoundError in combination with pytest-parallel

[mxschmitt]

Should be fixed by #97

[geiregjo]

We've been running the tests in Ubuntu for a couple of months. We decided to start running them on Windows because of numerous reasons. And we noticed that we have the same issue. And it's fixed the same way as suggested in my first post. Fix #97 does not capture the permission issue.

___________________________________ ERROR at teardown of test_upload_file[ActionOrigin.TOOLBAR-temp_dirs0] ____________________________________ 

path = 'C:\\Users\\ubuntu\\AppData\\Local\\Temp\\playwright-pytest-vs4qcn13'
onerror = <function TemporaryDirectory._rmtree.<locals>.onerror at 0x00000257D0DE1FC0>

    def _rmtree_unsafe(path, onerror):
        try:
            with os.scandir(path) as scandir_it:
                entries = list(scandir_it)
        except OSError:
            onerror(os.scandir, path, sys.exc_info())
            entries = []
        for entry in entries:
            fullname = entry.path
            if _rmtree_isdir(entry):
                try:
                    if entry.is_symlink():
                        # This can only happen if someone replaces
                        # a directory with a symlink after the call to
                        # os.scandir or entry.is_dir above.
                        raise OSError("Cannot call rmtree on a symbolic link")
                except OSError:
                    onerror(os.path.islink, fullname, sys.exc_info())
                    continue
                _rmtree_unsafe(fullname, onerror)
            else:
                try:
>                   os.unlink(fullname)
E                   PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\u
buntu\\AppData\\Local\\Temp\\playwright-pytest-vs4qcn13\\83cde3cf4414efd29848e7ff566f2d6e.webm'

C:\Python310\lib\shutil.py:615: PermissionError

During handling of the above exception, another exception occurred:

func = <built-in function unlink>
path = 'C:\\Users\\ubuntu\\AppData\\Local\\Temp\\playwright-pytest-vs4qcn13\\83cde3cf4414efd29848e7ff566f2d6e.webm'
exc_info = (<class 'PermissionError'>, PermissionError(13, 'The process cannot access the file because it is being used by another process'), <t
raceback object at 0x00000257D07B1200>)

    def onerror(func, path, exc_info):
        if issubclass(exc_info[0], PermissionError):
            def resetperms(path):
                try:
                    _os.chflags(path, 0)
                except AttributeError:
                    pass
                _os.chmod(path, 0o700)

            try:
                if path != name:
                    resetperms(_os.path.dirname(path))
                resetperms(path)

                try:
>                   _os.unlink(path)
E                   PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\u
buntu\\AppData\\Local\\Temp\\playwright-pytest-vs4qcn13\\83cde3cf4414efd29848e7ff566f2d6e.webm'

C:\Python310\lib\tempfile.py:820: PermissionError

During handling of the above exception, another exception occurred:

launch_browser = <function launch_browser.<locals>.launch at 0x00000257D042AC20>

    @pytest.fixture(scope="session")
    def browser(launch_browser: Callable[[], Browser]) -> Generator[Browser, None, None]:
        browser = launch_browser()
        yield browser
        browser.close()
>       artifacts_folder.cleanup()

C:\Python310\lib\site-packages\pytest_playwright\pytest_playwright.py:186:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
C:\Python310\lib\tempfile.py:850: in cleanup
    self._rmtree(self.name, ignore_errors=self._ignore_cleanup_errors)
C:\Python310\lib\tempfile.py:832: in _rmtree
    _shutil.rmtree(name, onerror=onerror)
C:\Python310\lib\shutil.py:747: in rmtree
    return _rmtree_unsafe(path, onerror)
C:\Python310\lib\shutil.py:617: in _rmtree_unsafe
    onerror(os.unlink, fullname, sys.exc_info())
C:\Python310\lib\tempfile.py:823: in onerror
    cls._rmtree(path, ignore_errors=ignore_errors)
C:\Python310\lib\tempfile.py:832: in _rmtree
    _shutil.rmtree(name, onerror=onerror) 
C:\Python310\lib\shutil.py:747: in rmtree
    return _rmtree_unsafe(path, onerror)
C:\Python310\lib\shutil.py:598: in _rmtree_unsafe
    onerror(os.scandir, path, sys.exc_info())
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
 
path = 'C:\\Users\\ubuntu\\AppData\\Local\\Temp\\playwright-pytest-vs4qcn13\\83cde3cf4414efd29848e7ff566f2d6e.webm'
onerror = <function TemporaryDirectory._rmtree.<locals>.onerror at 0x00000257D10CDAB0>

    def _rmtree_unsafe(path, onerror):
        try:
>           with os.scandir(path) as scandir_it:
E           NotADirectoryError: [WinError 267] The directory name is invalid: 'C:\\Users\\ubuntu\\AppData\\Local\\Temp\\playwright-pytest-vs4qcn
13\\83cde3cf4414efd29848e7ff566f2d6e.webm'

C:\Python310\lib\shutil.py:595: NotADirectoryError 
----------------------------------------------------

[geiregjo]

Can this issue be reopened?

[mxschmitt]

Can this issue be reopened?

Can you file a separate issue and reference to this one? With a repro etc.

[geiregjo]

I'll take another look. I worked around the issue by fixing it in a fork. But this was only temporary. I also stopped using pytest-parallel and was surprised that I'm still experiencing this issue... Maybe the cleanup starts before the process writing the .webm file releases the file?