Notion Importer hangs on large .zip file import (1.1.0)

[obsidianmd]

Notion Importer hangs on large .zip file import (1.1.0)

[lishid]

That's what I'm suspecting - If you open the zip, do you see nested zips? Can you show me what it looks like especially their file sizes?

[athenaprime]

I was able to export the top-level pages as HTML separately, but I had to separate a number of them because apparently, if you have templates created by other people in Notion, you get some amount of their data (which is very very troubling). It isn't the prettiest or most organized thing, but I can access my information, which is the important part.

[lishid]

Ahh! Was the zip file an attachment (instead of exported notes)?

[sutrisno00]

I have the same issue with 3.5 gb file. Tried twice.

Appreciate any pointers. Many thanks

[joshuatazrein]

This is an interesting problem, we'll need to do some testing with larger data files to reproduce.

[lishid]

It might be because large files take a while to do the initial parsing, so it would look like it's done. As seem from the screenshot there is no "done" button which shows up when it is finished.

I've pushed an update where the importer will show that it is crawling the files. This should help give a sense of progress.

[athenaprime]

Having the same issue here, although I don't know anything about coding this stuff. I exported the Notion html into a huge .zip file with a very long name (so long that only 7-zip could open it rather than the Windows default). I left the .zip intact and used the Importer and it hangs on the same window as the original poster. The file tree is present in Obsidian and in the file structure in the main vault folder, but there are no files.

Should I have unzipped everything somewhere? The instructions say to leave the .zip as is.

[joshuatazrein]

You are correct to leave everything as the .zip.

I'm not sure exactly where the problem is stemming from, but for now you could try exporting top-level pages individually, then importing those by selecting them all. If you try that does it fix the issue?

[lishid]

Ok with the release of v1.1.1 we should have a better idea what the importer is doing - can you guys test again and let me know what this new version does for you? Thanks!

[rjayasin]

Ok with the release of v1.1.1 we should have a better idea what the importer is doing - can you guys test again and let me know what this new version does for you? Thanks!

After updating to 1.1.1 the importer is stuck here:

The same error appears in the developer console.

[sutrisno00]

Just tried - same result. Hang for a while then say done with all zeros

[joshuatazrein]

It might be an issue with reading the .zip itself. My other guess is that any files over 2GB cause the ArrayBuffer error when reading as binary, but that should just skip the files. If you remove all files over 2 GB does it work as expected?

[rjayasin]

@lishid you were right about nested zip files: I had a large zip file (~1 GB) attached to one of my notion pages which I deleted and the import was able to proceed. Thanks!

[wpalm]

I'm also having problems importing pages which includes .zip attachments, but if I rename the .zip files to .zip_ it works as expected.

[joshuatazrein]

This is because if the importer detects a nested .zip it assumes the zip is a Notion database and tries to parse it. We just need to validate the name making sure it has a Notion ID at the end of the .zip:

// notion.ts, processZips()
if (entry.extension === 'zip' && getNotionId(entry.name)) {
   await processZips(ctx, [entry], callback);
}

[lishid]

Should be fixed now!

[Sinled]

Should be fixed now!

is it 1.1.2 version?

It is still wont import correctly - i have 2.2gb notion export archive, that has two zip files (*part-1.zip and *part-2.zip) inside.

Import says that it is completed succesfuly, but only two zip archive that were inside notion archive are added to attachments folder, no notes are present in destination folder

if i try to import parts, it fails to import with error Error: Folder already exists. on first part

[joshuatazrein]

It seems like the .zip importer is incorrectly treating those .zips as attachments. I would suggest modifying the importer to look for part-x as title names for nested zips.