Error unpacking new BFLYT Nintendo Switch

[tyulis]

Error unpacking new BFLYT Nintendo Switch

[Tyulis]

I am working on it, looks like many things are changed and extended...

[Tyulis]

Just some unsupported stuff, some things were added by Switch games into the 3DS or WiiU formats. That's hopefully fixed, don't hesitate if you find any other issue.
(BTW, be aware that the output is a bit... Scary. I might make something more visual someday.)

[Migushthe2nd]

Thanks, I am able to unpack the BFLYT's now, however my Switch crashes when loading the home menu with the modified repacked file. I tried just unpacking and repacking the file and that also results in my Switch crashing. There are no errors when (un)packing. Wexos is able to open the modified file without crashing and the contents are the same as the original file.
By opening the files in a Hex editor the differences can be seen.
The two files (original and repack):
https://drive.google.com/drive/folders/114dObM6fwqFb0itZTYxjI8nJKzjY-gJ7?usp=sharing

[Migushthe2nd]

Also just so you know, I'm not using the default latest Wexos toolbox version. I'm using a modified version (WexosF) by Exelix of the Qcean team. In his download message for WexosF he noted that the patched version can edit 6.x (meaning 6.x firmware) bflyt, which the files are. The original Wexos can edit 5.x and <5.x bflyt's fine. Here's WexosF:
https://cdn.discordapp.com/attachments/491700344691097645/509442969884033025/WexosF.zip

[Tyulis]

Okay, I made a brand new implementation, ways better, with JSON output.
So it works quite good with Switch files but there is just a few problems with prior versions, I think I'll commit that in the afternoon when everything will be fixed

[Tyulis]

Done !
Theorically, it should work just fine, but don't hesitate if you encounter any other issue.
Thanks for your help !

[Migushthe2nd]

I can't thank you enough! It works as expected now, really, thanks a lot!
I was able to modify my file to my needs.

[Migushthe2nd]

However, there seems to be another file that can't be unpacked correctly: RdtBtnIconGame.bflyt

--------.\RdtBtnIconGame.bflyt--------
No compression
BFLYT file found
Extracting...
Format version : 8.6.0.0
Number of sections : 97
Traceback (most recent call last):
  File ".\3DSkit.py", line 249, in <module>
    result = main(args, opts)
  File ".\3DSkit.py", line 169, in main
    extract_files(filename, args.big, args.format, args.verbose, opts)
  File ".\3DSkit.py", line 112, in extract_files
    unpack.extract(filename, file, format, endian, verbose, opts)
  File "S:\Mijn Drive\Things\NSW\Themes\Very nice theme\tools\3DSkit-master\unpack\__init__.py", line 16, in extract
    unpacker = cls(*args)
  File "S:\Mijn Drive\Things\NSW\Themes\Very nice theme\tools\3DSkit-master\util\utils.py", line 18, in __new__
    return self.main(*args, **kwargs)
  File "S:\Mijn Drive\Things\NSW\Themes\Very nice theme\tools\3DSkit-master\unpack\BFLYT.py", line 66, in main
    self.readdata(data)
  File "S:\Mijn Drive\Things\NSW\Themes\Very nice theme\tools\3DSkit-master\unpack\BFLYT.py", line 108, in readdata
    name, node = self.readsection(data, magic, startpos)
  File "S:\Mijn Drive\Things\NSW\Themes\Very nice theme\tools\3DSkit-master\unpack\BFLYT.py", line 123, in readsection
    name, node = self.readmat1(data, startpos)
  File "S:\Mijn Drive\Things\NSW\Themes\Very nice theme\tools\3DSkit-master\unpack\BFLYT.py", line 226, in readmat1
    flagnode['alpha blending'] = ALPHA_BLENDS[alphablend]
IndexError: tuple index out of range

Download:
https://drive.google.com/file/d/1S9XMC2jBD2qvSUrF5usNzcHZ77RXJRgr/view?usp=sharing

I extracted all of the bflyt's in Residentmenu, turns out there are even more errors:
https://gist.github.com/Migushthe2nd/63beb72a73ba909cc6da0af54c7c8db5

[Tyulis]

Okay, that's a known problem, but I though it was only on 7.2 as I never encountered it in another version.
I make a quick fix waiting to find from where it comes

[Migushthe2nd]

I updated my comment with a gist link

[Tyulis]

Okay, there was a stupid encoding error, a version difference that I hadn't figured out, a few other stupid errors, and finally something that I could never find before found a meaning ^^
So you can try, it should work a bit better.

[Migushthe2nd]

Yeah most of the files work now, except for a few: HudBatteryNum.bflyt for example:
(This is the same error for all files that won't unpack yet)

(python ..\3DSkit.py -x "S:\Mijn Drive\Things\NSW\Themes\Very nice theme\tools\3DSkit-master\blyt-\RdtProgressBarErr.bflyt" -v  )

--------S:\Mijn Drive\Things\NSW\Themes\Very nice theme\tools\3DSkit-master\blyt-\RdtProgressBarErr.bflyt--------
No compression
BFLYT file found
Extracting...
Format version : 8.6.0.0
Number of sections : 11
Traceback (most recent call last):
  File "..\3DSkit.py", line 249, in <module>
    result = main(args, opts)
  File "..\3DSkit.py", line 169, in main
    extract_files(filename, args.big, args.format, args.verbose, opts)
  File "..\3DSkit.py", line 112, in extract_files
    unpack.extract(filename, file, format, endian, verbose, opts)
  File "S:\Mijn Drive\Things\NSW\Themes\Very nice theme\tools\3DSkit-master\unpack\__init__.py", line 16, in extract
    unpacker = cls(*args)
  File "S:\Mijn Drive\Things\NSW\Themes\Very nice theme\tools\3DSkit-master\util\utils.py", line 18, in __new__
    return self.main(*args, **kwargs)
  File "S:\Mijn Drive\Things\NSW\Themes\Very nice theme\tools\3DSkit-master\unpack\BFLYT.py", line 66, in main
    self.readdata(data)
  File "S:\Mijn Drive\Things\NSW\Themes\Very nice theme\tools\3DSkit-master\unpack\BFLYT.py", line 108, in readdata
    name, node = self.readsection(data, magic, startpos)
  File "S:\Mijn Drive\Things\NSW\Themes\Very nice theme\tools\3DSkit-master\unpack\BFLYT.py", line 139, in readsection
    name, node = self.readprt1(data, startpos)
  File "S:\Mijn Drive\Things\NSW\Themes\Very nice theme\tools\3DSkit-master\unpack\BFLYT.py", line 414, in readprt1
    entry['extra'] = self.unpack_from('48s', data, startoffset + extraoffset)[0].hex()
NameError: name 'startoffset' is not defined

File:
https://drive.google.com/file/d/1I55lOCFBff_C_K0ANj4QAFbXWnV6dbuS/view?usp=sharing
But for real, thank you for your hard work.

[Tyulis]

No problem, thank you for your patience and your help ^^
If the paddings don't matter, it's theoretically fixed.

[Migushthe2nd]

Yep, all files unpack now without errors. I tried booting with every file repacked and it works fine. Thanks man! I'll open this issue again if I ever find other Switch bflyts erroring.

[Migushthe2nd]

Some files seem to return this error after being unpacked and repacked quite a few times. This only happens when unpacking:

--------RdtBalloon.bflyt--------                                                                    
No compression                                                                                      
BFLYT file found                                                                                    
Extracting...                                                                                       
Format version : 8.6.0.0                                                                            
Number of sections : 45                                                                             
Traceback (most recent call last):                                                                  
  File "..\3DSkit.py", line 249, in <module>                                                        
    result = main(args, opts)                                                                       
  File "..\3DSkit.py", line 169, in main                                                            
    extract_files(filename, args.big, args.format, args.verbose, opts)                              
  File "..\3DSkit.py", line 112, in extract_files                                                   
    unpack.extract(filename, file, format, endian, verbose, opts)                                   
  File "K:\NSW\Themes\Very nice theme\tools\3DSkit-master\unpack\__init__.py", line 16, in extract  
    unpacker = cls(*args)                                                                           
  File "K:\NSW\Themes\Very nice theme\tools\3DSkit-master\util\utils.py", line 18, in __new__       
    return self.main(*args, **kwargs)                                                               
  File "K:\NSW\Themes\Very nice theme\tools\3DSkit-master\unpack\BFLYT.py", line 66, in main        
    self.readdata(data)                                                                             
  File "K:\NSW\Themes\Very nice theme\tools\3DSkit-master\unpack\BFLYT.py", line 108, in readdata   
    name, node = self.readsection(data, magic, startpos)                                            
  File "K:\NSW\Themes\Very nice theme\tools\3DSkit-master\unpack\BFLYT.py", line 133, in readsection
    name, node = self.readwnd1(data, startpos)                                                      
  File "K:\NSW\Themes\Very nice theme\tools\3DSkit-master\unpack\BFLYT.py", line 335, in readwnd1   
    framenode['material'] = self.matnames[material]                                                 
IndexError: list index out of range                                                                 

E.g. RdtBalloon.bflyt:
https://drive.google.com/file/d/1DqmzoZDQDpDbOndvdDnCaZ_6t3gWaCC-/view?usp=sharing

[Tyulis]

I don't have any time for the moment, but I'll try to look at this as soon as I can, no problem ;)

[Tyulis]

Got a little time ^^"
I couldn't reproduce it on any file except your bugged one, obviously you have not changed the material ?
I can't point out why this can happen, does it still happen with an unmodified file ?
Another time, sorry for the long response time...

[Migushthe2nd]

Ok so I also was unable to reproduce this aswell, but I've only seen it happen with the RdtBalloon file so far. I compared the original json to my modified one, and the only difference is the x_translation of N_Card.

Another thing I noticed is that some repacked (even unmodified) files do make the switch crash. One of the files I'm talking about:
https://drive.google.com/file/d/1w_xusvVHO9RRegOWvxaY6Iu4C1zoK_ly/view?usp=sharing (the original one. You could repack it and look at the differences)
PslBtnCardPlayer from playerSelect