Archiving fails in Swift5

[dreymonde]

Archiving fails in Swift5

[snoozemoose]

I've found the offending piece of code and also a workaround for the issue. As it turns out, this is a compiler bug in Swift 5. I have reported it here: https://bugs.swift.org/browse/SR-10716. The code that triggers the bug is this:
var directoryURL = DiskFolderStorage.url(forFolder: "offlineMetadataFolder", in: .documentDirectory, domainMask: .userDomainMask)
let diskStorage = DiskFolderStorage(folderURL: directoryURL, diskStorage: DiskStorage.main.asStorage(), filenameEncoder: .base64)

And the whole issue was resolved when I replaced it with this code:
let diskStorage = DiskStorage.main.folder("offlineMetadataFolder", in: .documentDirectory, domainMask: .userDomainMask)

[dreymonde]

This is such a bizarre issue. Since you found the workaround, I don't think there's anything I should change, but please let me know if you'll see something similar in the future!

[jbmaxwell]

Actually, I was having this *** DESERIALIZATION FAILURE (please include this section in any bug report) *** error and assumed it was related to running an old, pods version of Shallows. But after upgrading to the SPM version I'm still seeing it (tried cleaning, deleting derivedData, etc.). I don't see this all the time, but it's pretty regular and often requires a full clean/delete derivedData to get it to build again.
@snoozemoose how did you track down the offending line? I'm not clear how/why your workaround works... Any help appreciated.
Oh, and to clarify: this happens every time I build, not just on archiving... Development is completely stuck on this error (and I see there's no activity on the Swift bug report).

[snoozemoose]

@jbmaxwell I don’t remember the details about how I tracked down the bug but I recall that somewhere I realized I wasn’t using the APIs the same way as the documentation suggested. I think I did some divide and conquer debugging where I commented out stuff until I found the line that broke compilation. Was over a year ago, don’t remember exactly I’m afraid. Is the workaround not helping in your case?

[jbmaxwell]

@snoozemoose, thanks for the reply! The problem is I use Shallows pretty extensively in my project, and I can't find any calls using the url(forFolder:in:domainMask:) syntax you have in your original (before the workaround). The only place I do find it is in older—pre-SPM—code, and that's in the actual Shallows pod code itself, not my code. So I'm thinking maybe it isn't a workaround for my situation... I'm stumped. The pod version sometimes manages to build, and sometimes doesn't. Very weird. I also tried using the Legacy Build system, as recommended elsewhere, but no luck...

[snoozemoose]

This makes me a bit worried since my app also relies on this framework for some key features. AFAIK, the swift compiler bug that I’ve reported should only happen when compiling with optimization so you might be experiencing some new bug. Is it possible for you to try to comment out Shallows calls and get it to compile repeatedly as a way to find out which call that is causing the problem?

[jbmaxwell]

Researching that error I found references to problems with subclasses (not specific to Shallows). In my case it wasn't a subclass, but we did have a custom Storage (i.e., implementing StorageProtocol), so I wondered whether it was perhaps a type inference issue. I was able to work around that class and remove it from the target, which did stop the build errors. It's been building consistently since. I haven't had time to dig into why that class was a problem, which is a bit of a concern, but I'm able to get back to work at least.

[dreymonde]

Hi everyone! I’m about to dive into this issue myself too, as I clearly see this is a problem. Just to be clear, you don’t know which exact code is causing the problem, right?

[dreymonde]

Would be awesome if you could reproduce this issue in a test project that you could share, that would help big time

[dreymonde]

@jbmaxwell can you also post here the full error message that Xcode gives you?

[jbmaxwell]

In this issue #17 (comment) I tracked it down to the attached file (a custom Storage class). I'm on a deadline for the next few days, so can't really get further into it, but I could probably include it again to just get the error. Give me a second...

[jbmaxwell]

<unknown>:0: error: fatal error encountered while reading from module 'Shallows'; please file a bug report with your project and the crash log

*** DESERIALIZATION FAILURE (please include this section in any bug report) ***
(see "While..." info below)
Stack dump:
---
1.	Apple Swift version 5.2.4 (swiftlang-1103.0.32.9 clang-1103.0.32.53)
2.	Contents of /var/folders/ws/w7z1fxvs4x1cr2p5f3hwqg9h0000gn/T/sources-803fc3:
---
3.	While running pass #484 SILModuleTransform "MandatorySILLinker".
4.	While deserializing SIL function "$s8Shallows15StorageProtocolPAAE02asB0AA0B0Vy3KeyQz5ValueQzGyF"
0  swift                    0x00000001057eb4ea PrintStackTraceSignalHandler(void*) + 42
1  swift                    0x00000001057eacc0 SignalHandler(int) + 352
2  libsystem_platform.dylib 0x00007fff734bb5fd _sigtramp + 29
3  libsystem_platform.dylib 0x00007fa767ac0e00 _sigtramp + 4099954720
4  libsystem_c.dylib        0x00007fff73391808 abort + 120
5  swift                    0x000000010221b440 std::__1::__vector_base<swift::Diagnostic, std::__1::allocator<swift::Diagnostic> >::clear() (.16933) + 0
6  swift                    0x0000000102229e10 swift::DeclDeserializer::deserializeAnyFunc(llvm::ArrayRef<unsigned long long>, llvm::StringRef, bool) + 0
7  swift                    0x0000000102247d52 swift::SILDeserializer::readSILFunctionChecked(llvm::PointerEmbeddedInt<unsigned int, 31>, swift::SILFunction*, llvm::StringRef, bool, bool) + 11202
8  swift                    0x0000000102268247 swift::SILDeserializer::lookupSILFunction(swift::SILFunction*, bool) + 679
9  swift                    0x0000000101e825fd swift::SILLinkerVisitor::maybeAddFunctionToWorklist(swift::SILFunction*) + 253
10 swift                    0x0000000101e82884 swift::SILLinkerVisitor::processFunction(swift::SILFunction*) + 244
11 swift                    0x0000000101d55740 (anonymous namespace)::SILLinker::run() + 304
12 swift                    0x0000000101c46d09 swift::SILPassManager::execute() + 2329
13 swift                    0x0000000101c4c038 swift::runSILDiagnosticPasses(swift::SILModule&) + 2264
14 swift                    0x0000000101849622 swift::CompilerInstance::performSILProcessing(swift::SILModule*, swift::UnifiedStatsReporter*) + 66
15 swift                    0x0000000101547070 performCompileStepsPostSILGen(swift::CompilerInstance&, swift::CompilerInvocation&, std::__1::unique_ptr<swift::SILModule, std::__1::default_delete<swift::SILModule> >, bool, llvm::PointerUnion<swift::ModuleDecl*, swift::SourceFile*>, swift::PrimarySpecificPaths const&, bool, int&, swift::FrontendObserver*, swift::UnifiedStatsReporter*) + 1712
16 swift                    0x000000010153ceba swift::performFrontend(llvm::ArrayRef<char const*>, char const*, void*, swift::FrontendObserver*) + 48426
17 swift                    0x00000001014b4b73 main + 1283
18 libdyld.dylib            0x00007fff732c2cc9 start + 1
error: Abort trap: 6 (in target 'Spliqs' from project 'Spliqs')

[dreymonde]

Closing this along with #17
Feel free to reopen if this happens again!