Comments (25)
webmaster128
commented on August 24, 2024
1
Ahhhhahahaha muhaa – boom, there is a logic error:
When a manifest is newly created it is not checked if the object is already in cache or not. Instead it is assumed the cache object does not exists. This happens because only nodirect was used before and then you switch to direct mode or manifests have been cleaned.
Temporary workaround would be:
- Clear cache entirely
- Use
CLCACHE_NODIRECT
I think I'll provide a patch as soon as frerich is back and the PRs list is reduced a bit.
from clcache.
webmaster128
commented on August 24, 2024
1
Updated analysis:
When a manifest is newly created it is not checked if the object is already in cache or not. Instead it is assumed the cache object does not exists. This happens because
only nodirect was used before and then you switch to direct mode ormanifests have been cleaned.
The cachekey for directmode and non-directmode are entirely different:
- direct mode cachekey: manifest key + hash of all includes
- non-direct mode cachekey: compiler + command line + preprocessed output
So the issue occurs when
- The build system build two files with the same direct mode cachekey at the same time (unlikely thing, should not happen)
- A manifest is cleaned but the compiler artifacts remain alive (likely thing for full caches)
from clcache.
frerich
commented on August 24, 2024
Three questions to trace this down:
- Which clcache version are you using? Please make sure that you're using the latest version ,which has some fixes related to concurrent access to the object cache.
- Do you use parallel compilation (e.g. by invoking
clwith more than one source file, or by running multiple invocations ofclin parallel) using the same cache directory? - Is your cache directory on a local disk or is it a shared network drive used by multiple hosts?
from clcache.
cyberwjf
commented on August 24, 2024
- I'm using the latest version of clcache.
- seems I'm using parallel compilation.
- It's on my local disk.
Let me try to avoid parallel compilation.
BRs,
Carl Wang
On Fri, Jul 8, 2016 at 8:39 PM, Frerich Raabe [email protected]
wrote:
Three questions to trace this down:
Which clcache version are you using? Please make sure that you're
using the latest version ,which has some fixes related to concurrent access
to the object cache.
2.Do you use parallel compilation (e.g. by invoking cl with more than
one source file, or by running multiple invocations of cl in parallel)
using the same cache directory?
3.Is your cache directory on a local disk or is it a shared network
drive used by multiple hosts?—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#155 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/AA4FeSRvodvOk2LP9NpKmDkyZfBzR9rqks5qTkT5gaJpZM4JH5QV
.
from clcache.
frerich
commented on August 24, 2024
Can you reproduce this issue? If so, can you please set the environment variable CLCACHE_LOG=1, then reproduce the error and then send the printed log files? That might shed some light.
from clcache.
Liyra
commented on August 24, 2024
Same error here, following the info with the log variable:
Traceback (most recent call last): (TaskId:22)
File "clcache.py", line 1443, in (TaskId:22)
File "clcache.py", line 1328, in main (TaskId:22)
File "clcache.py", line 1357, in processCompileRequest (TaskId:22)
File "clcache.py", line 1422, in processDirect (TaskId:22)
File "clcache.py", line 1419, in (TaskId:22)
File "clcache.py", line 1261, in postprocessNoManifestMiss (TaskId:22)
File "clcache.py", line 1178, in addObjectToCache (TaskId:22)
File "clcache.py", line 317, in setEntry (TaskId:22)
File "clcache.py", line 647, in copyOrLink (TaskId:22)
FileExistsError: [WinError 183] Cannot create a file when that file already exists: 'D:\clcache\objects\2f\2f583e9f9181cc2578f56d07f6bf42d6\object.tmp' -> 'D:\clcache\objects\2f\2f583e9f9181cc2578f56d07f6bf42d6\object' (TaskId:22)
C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Parsing given commandline '[ #CommandLine# ]' (TaskId:22)
C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Expanded commandline '[ #CommandLine# ]' (TaskId:22)
C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Compiler source files: #SourcePath#
C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Compiler object file: #ObjFile# (TaskId:22)
C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Invoking real compiler as 'C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\cl_original.exe'
C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Real compiler returned code 0 (TaskId:22)
C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Adding file #ObjFile# to cache using key 2f583e9f9181cc2578f56d07f6bf42d6 (TaskId:22)
Failed to execute script clcache (TaskId:22)
(TaskId:22)
C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Child: ' #CommandLine# '
C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Child: ' #CommandLine# '
C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Child: ' #CommandLine# '
C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Child: ' #CommandLine# '
C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V120\Microsoft.CppCommon.targets(341,5): error MSB6006: "CL.exe" exited with code -1.
from clcache.
Jimilian
commented on August 24, 2024
@Liyra , can you describe your environment? Pure clcache + cl.exe? Incredibuild+clcache + cl.exe? Local run or Continuous integration? If CI, how much executions do you have in same time?
from clcache.
Liyra
commented on August 24, 2024
I use only clcache 3.2 with visual studio community 2013 (I don't use Incredibuild), on my local machine, just clicking on the build solution button in visual, so I assume it's compiling in parallel. The solution I am working on is composed of more or less 100 projects, and this error message occurs only for 4-5.
Hope it helps !
from clcache.
Jimilian
commented on August 24, 2024
@Liyra, @webmaster128 suggested workaround that works for me - use CLCACHE_NODIRECT=1.
from clcache.
webmaster128
commented on August 24, 2024
@Liyra, @webmaster128 suggested workaround that works for me - use CLCACHE_NODIRECT=1.
The workaround is for missing include files, not for this issue here (existing cache "object" files). If disabling direct mode helps anyway, then direct mode is responsible for this issue.
from clcache.
Liyra
commented on August 24, 2024
I have the same errors with the CLCACHE_NODIRECT variable set to 1, but the errors occurs for almost every project instead of 4-5.
from clcache.
webmaster128
commented on August 24, 2024
@Liyra Is it possible, that your build environment builds the same file twice at the same time?
from clcache.
Liyra
commented on August 24, 2024
I really don't know, I just use the default parameters in visual studio, I will assume that everything is correctly handled and only compiled once.
from clcache.
Liyra
commented on August 24, 2024
After having deleted my cache folder manually, it doesn't change anything, the problem is still here and occurs on each (or almost each) project.
from clcache.
webmaster128
commented on August 24, 2024
Could you please add a current stacktrace? This should be similar but different from the one above, since processDirect should not be called anymore.
from clcache.
Liyra
commented on August 24, 2024
Here you go:
17> Traceback (most recent call last):
17> File "clcache.py", line 1443, in <module>
17> File "clcache.py", line 1328, in main
17> File "clcache.py", line 1355, in processCompileRequest
17> File "clcache.py", line 1431, in processNoDirect
17> File "clcache.py", line 1193, in processCacheHit
17> File "clcache.py", line 328, in cachedCompilerOutput
17> FileNotFoundError: [Errno 2] No such file or directory: 'D:\\clcache\\objects\\98\\98a70d87d6a357b65abc721fabbe7640\\output.txt'
17> C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Parsing given commandline '[]'
17> C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Expanded commandline '[]'
17> C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Compiler source files: []
17> C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Compiler object file: .obj
17> C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Reusing cached object for key 98a70d87d6a357b65abc721fabbe7640 for output file .obj
17> Failed to execute script clcache
17> C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Found real compiler binary at 'C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\cl_original.exe'
17> Traceback (most recent call last):
17> File "clcache.py", line 1443, in <module>
17> File "clcache.py", line 1328, in main
17> File "clcache.py", line 1355, in processCompileRequest
17> File "clcache.py", line 1431, in processNoDirect
17> File "clcache.py", line 1193, in processCacheHit
17> File "clcache.py", line 328, in cachedCompilerOutput
17> FileNotFoundError: [Errno 2] No such file or directory: 'D:\\clcache\\objects\\77\\77d53a5273822136e5759776a3a3c9cb\\output.txt'
17> C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Arguments we care about: '[]'
17> C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Parsing given commandline '[]'
17> C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Expanded commandline '[]'
17> C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Compiler source files: []
17> C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Compiler object file: .obj
17> C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Reusing cached object for key 77d53a5273822136e5759776a3a3c9cb for output file .obj
17> Failed to execute script clcache
17> C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Found real compiler binary at 'C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\cl_original.exe'
17> C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Arguments we care about: '[]'
17> Traceback (most recent call last):
17> File "clcache.py", line 1443, in <module>
17> File "clcache.py", line 1328, in main
17> File "clcache.py", line 1355, in processCompileRequest
17> File "clcache.py", line 1431, in processNoDirect
17> File "clcache.py", line 1193, in processCacheHit
17> File "clcache.py", line 328, in cachedCompilerOutput
17> FileNotFoundError: [Errno 2] No such file or directory: 'D:\\clcache\\objects\\f0\\f00f4fdb42aae261cc1b449917497c7c\\output.txt'
17> C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Parsing given commandline '[]'
17> C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Expanded commandline '[]'
17> C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Compiler source files: []
17> C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Compiler object file: .obj
17> C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Reusing cached object for key f00f4fdb42aae261cc1b449917497c7c for output file .obj
17> Failed to execute script clcache
17>
17> C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Child: []'
17> C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Child: []'
17> Traceback (most recent call last):
17> File "clcache.py", line 1443, in <module>
17> File "clcache.py", line 1328, in main
17> File "clcache.py", line 1355, in processCompileRequest
17> File "clcache.py", line 1431, in processNoDirect
17> File "clcache.py", line 1193, in processCacheHit
17> File "clcache.py", line 328, in cachedCompilerOutput
17> FileNotFoundError: [Errno 2] No such file or directory: 'D:\\clcache\\objects\\46\\4698db611e6f2bee6314c804ab111bce\\output.txt'
17> Failed to execute script clcache
17> Traceback (most recent call last):
17> File "clcache.py", line 1443, in <module>
17> File "clcache.py", line 1328, in main
17> File "clcache.py", line 1355, in processCompileRequest
17> File "clcache.py", line 1431, in processNoDirect
17> File "clcache.py", line 1193, in processCacheHit
17> File "clcache.py", line 328, in cachedCompilerOutput
17> FileNotFoundError: [Errno 2] No such file or directory: 'D:\\clcache\\objects\\91\\918f34a29c459335f40ad03c91bdad4c\\output.txt'
17> Failed to execute script clcache
17> Traceback (most recent call last):
17> File "clcache.py", line 1443, in <module>
17> File "clcache.py", line 1328, in main
17> File "clcache.py", line 1355, in processCompileRequest
17> File "clcache.py", line 1431, in processNoDirect
17> File "clcache.py", line 1193, in processCacheHit
17> File "clcache.py", line 328, in cachedCompilerOutput
17> FileNotFoundError: [Errno 2] No such file or directory: 'D:\\clcache\\objects\\41\\41ccbefdf6a9a9e7bc30276932410f92\\output.txt'
17> Failed to execute script clcache
17> Traceback (most recent call last):
17> File "clcache.py", line 1443, in <module>
17> File "clcache.py", line 1328, in main
17> File "clcache.py", line 1355, in processCompileRequest
17> File "clcache.py", line 1431, in processNoDirect
17> File "clcache.py", line 1193, in processCacheHit
17> File "clcache.py", line 328, in cachedCompilerOutput
17> FileNotFoundError: [Errno 2] No such file or directory: 'D:\\clcache\\objects\\11\\1150c14a794b5ab9ff6bdf9b8bf99738\\output.txt'
17> Failed to execute script clcache
17> Traceback (most recent call last):
17> File "clcache.py", line 1443, in <module>
17> File "clcache.py", line 1328, in main
17> File "clcache.py", line 1355, in processCompileRequest
17> File "clcache.py", line 1431, in processNoDirect
17> File "clcache.py", line 1193, in processCacheHit
17> File "clcache.py", line 328, in cachedCompilerOutput
17> FileNotFoundError: [Errno 2] No such file or directory: 'D:\\clcache\\objects\\8a\\8a135e981b561ed3cc206b76f169756e\\output.txt'
17> Failed to execute script clcache
17> Traceback (most recent call last):
17> File "clcache.py", line 1443, in <module>
17> File "clcache.py", line 1328, in main
17> File "clcache.py", line 1355, in processCompileRequest
17> File "clcache.py", line 1431, in processNoDirect
17> File "clcache.py", line 1193, in processCacheHit
17> File "clcache.py", line 328, in cachedCompilerOutput
17> FileNotFoundError: [Errno 2] No such file or directory: 'D:\\clcache\\objects\\78\\78ee10533f6188c37a57c46f0c7238db\\output.txt'
17> Failed to execute script clcache
17> C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Found real compiler binary at 'C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\cl_original.exe'
17> C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Arguments we care about: '[]'
17> C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Parsing given commandline '[]'
17> C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Expanded commandline '[]'
17> C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Compiler source files: []'
17> C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Compiler object file: .obj
17> C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Reusing cached object for key 4698db611e6f2bee6314c804ab111bce for output file .obj
17> C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Found real compiler binary at 'C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\cl_original.exe'
17> C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Arguments we care about: '[]'
17> C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Parsing given commandline '[]'
17> C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Expanded commandline '[]'
17> C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Compiler source files: []'
17> C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Compiler object file: .obj
17> C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Reusing cached object for key 918f34a29c459335f40ad03c91bdad4c for output file .obj
17> C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Found real compiler binary at 'C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\cl_original.exe'
17> C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Arguments we care about: '[]'
17> C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Parsing given commandline '[]'
17> C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Expanded commandline '[]'
17> C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Compiler source files: []'
17> C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Compiler object file: .obj
17> C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Reusing cached object for key 41ccbefdf6a9a9e7bc30276932410f92 for output file .obj
17> C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Found real compiler binary at 'C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\cl_original.exe'
17> C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Arguments we care about: '[]'
17> C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Parsing given commandline '[]'
17> C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Expanded commandline '[]'
17> C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Compiler source files: []'
17> C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Compiler object file: .obj
17> C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Reusing cached object for key 1150c14a794b5ab9ff6bdf9b8bf99738 for output file .obj
17> C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Found real compiler binary at 'C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\cl_original.exe'
17> C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Arguments we care about: '[]'
17> C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Parsing given commandline '[]'
17> C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Expanded commandline '[]'
17> C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Compiler source files: []'
17> C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Compiler object file: .obj
17> C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Reusing cached object for key 8a135e981b561ed3cc206b76f169756e for output file .obj
17> C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Found real compiler binary at 'C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\cl_original.exe'
17> C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Arguments we care about: '[]'
17> C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Parsing given commandline '[]'
17> C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Expanded commandline '[]'
17> C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Compiler source files: []'
17> C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Compiler object file: .obj
17> C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\clcache.py Reusing cached object for key 78ee10533f6188c37a57c46f0c7238db for output file .obj
17>C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V120\Microsoft.CppCommon.targets(341,5): error MSB6006: "CL.exe" exited with code -1.
from clcache.
webmaster128
commented on August 24, 2024
@Liyra Could you please open a new issue for this? Looks like an unrelated problem.
from clcache.
sasobadovinac
commented on August 24, 2024
I am seeing this here
https://ci.appveyor.com/project/sasobadovinac/freecad/build/1.0.238
https://ci.appveyor.com/project/sasobadovinac/freecad/build/1.0.260
and some test builds after this...
This is with release 3.2.0 and I did switch between direct and nodirect mode in both cases, but also the cache is full in both cases, mostly because of the switching :)
I am posting mostly just if it can help to better understand the issue and I can run some more tests if needed...
from clcache.
frerich
commented on August 24, 2024
@sasobadovinac Thanks for sharing your findings! Learning more about different scenarios when this issue manifests will surely be helpful.
You mentioned that you tried it with version 3.2.0 -- could you maybe see whether the recently released version 3.3.0 works better? There were some major improvements to how the object cache and the manifest repository is accessed, it might be that this resolves your issue.
from clcache.
akleber
commented on August 24, 2024
I think I see this problem also, but additionally in other forms. While working on the lockfile locking mechanism I build our project with a stock 3.3.0 on a somewhat faster machine (with the default mutex based locking).
File "clcache.py", line 1624, in <module>
File "clcache.py", line 1493, in main
File "clcache.py", line 1526, in processCompileRequest
File "clcache.py", line 1587, in processDirect
File "clcache.py", line 1614, in getOrSetArtifacts
File "clcache.py", line 1363, in addObjectToCache
File "clcache.py", line 322, in setEntry
File "clcache.py", line 816, in copyOrLink
FileExistsError: [WinError 183] Cannot create a file when that file already exists: 'D:\\j2\\clcache\\objects\\7c\\7c1208d0f399fc48095c24c1b8847eec\\object.tmp' -> 'D:\\j2\\clcache\\objects\\7c\\7c1208d0f399fc48095c24c1b8847eec\\object'
and
File "clcache.py", line 1624, in <module>
File "clcache.py", line 1493, in main
File "clcache.py", line 1526, in processCompileRequest
File "clcache.py", line 1571, in processDirect
File "clcache.py", line 1423, in postprocessUnusableManifestMiss
File "clcache.py", line 1399, in createOrUpdateManifest
File "clcache.py", line 157, in setManifest
OSError: [Errno 22] Invalid argument: 'D:\\j2\\clcache\\manifests\\d3\\d36bc9c50622da9f845552a1cfa4bc7e.json'
and
File "clcache.py", line 1624, in <module>
File "clcache.py", line 1493, in main
File "clcache.py", line 1526, in processCompileRequest
File "clcache.py", line 1587, in processDirect
File "clcache.py", line 1606, in getOrSetArtifacts
File "clcache.py", line 1382, in processCacheHit
File "clcache.py", line 815, in copyOrLink
File "shutil.py", line 115, in copyfile
OSError: [Errno 22] Invalid argument: 'kernel\\kernel\\maths\\CMakeFiles\\IDOMaths.dir\\fft\\kernel\\awake.c.obj.tmp'
These were three separate runs which aborted after these error. They occurred pretty early on in our build. We use ninja by cmake to build so only single files get compiled. I am not quite sure if the cache was empty or partially filled in the three cases (have to take better notes). The cache was on a local disk. The build machines use 24 cores and the build happens on a SSD.
As the three errors are all OS errors I would assume the problems is related but I have no idea what it might be. When working on a slower VM, the same build had no such errors.
from clcache.
akleber
commented on August 24, 2024
My problems mentioned in the post above do no occur on a Windows 10 build node. So I think this has nothing to do with clcache but rather with a problem on our build node. I am guessing some kind of OS version, OS updates, Python version, Driver version incompatibility. Or something completely different. When I know something more I will report back.
from clcache.
akleber
commented on August 24, 2024
It get weirder every day... I can see my above mentioned errors using clcache packaged to cl.exe with pyinstaller using Python 3.5.2-64bit, 3.5.1-32bit or 3.4.4-32bit on Windows 7.
When using clcache with Python 3.4.4-32bit using the clcache.bat approach, so without packaging it using pyinstaller, I do not get any errors.
So in our environment using pyinstaller leads to some weird sporadic random problems.
Another fact I absolutely do not understand is, that during some runs with using pyinstaller/clcache I have Windows errors like the ones above during a javac compile step in our build which should absolutely not be influenced by pyinstaller/clcache as javac is called by ninja and absolutely not by clcache. Still investigating...
from clcache.
frerich
commented on August 24, 2024
This is getting really interesting. Maybe it would be worthwhile to see whether cx_freeze works better than PyInstaller? It appears to be faster, too (cf. #232).
from clcache.
frerich
commented on August 24, 2024
I think this was fixed by #237
from clcache.
Liyra
commented on August 24, 2024
I have no more errors with or without CLCACHE_NODIRECT, so I think @frerich correctly guessed.
from clcache.
Related Issues (20)
- Error installing latest on Windows 10 HOT 5
- Renaming build directory cause cache misses HOT 2
- Clcache performance issues with big projects HOT 5
- What does "passed for external debug" mean? HOT 1
- Syntax error (printTraceStatement) running on Windows 10 HOT 2
- Please state that python 2 is no-longer supported in the readme.
- Autoclean feature documentation HOT 2
- Error "Cannot find file at '..\\lib\clcache\clcache\clcache.exe'" HOT 1
- ClCache mess up with MSBuild cache because of STATS.TXT HOT 2
- Can't use clcache from terminal when using VS HOT 3
- Visual Studio IDE w/ CMake w/ clcache always rebuilds HOT 5
- Enabling clcache overrides the local property-sheets
- Migrate LGTM.com installation from OAuth to GitHub App
- Enable long paths support on Windows 10
- Warning ModuleNotFoundError
- Update Visual Studio Integration Docs
- CLCACHE_PROFILE ignored when clcache is installed as .exe from chocolatey or pip HOT 1
- ThreadPoolExecutor / concurrent.futures.as_completed prevent cProfile from providing actual info
- Alternative CMake Integration HOT 4
- Problems with ASM identification, using clcache on CMAKE
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from clcache.