retreev/panglib

Series of tools to interact with Pangya PC MMO game files

panglib's Issues

Add ability to save DAT file to disk

Add the ability to save a loaded (and changed) instance of DATFile back to an IFF file.

ToDo:

• Add a Save method to save an DATFile instance to disk.

Add SelfDesign/UCC utilities

Pangya selfdesign files are more than corrupted image files, they contain the information for front, back and the selfdesign icon.

It would be nice to have a utility to read in the files and convert them to actual image formats, or even save them back to image files.

Implement LZ77 compression

Currently, PangLib.Utilities only supports decompression of LZ77 (not even sure if that is LZ77 exactly) bytestreams.

We also need the custom masked LZ77 compression to compress PAK archives back to a file.

Add tests for PangLib.IFF functionality

The libraries functionality needs to be tested.

Note: The focus is not lying on properly parsing game files, as the implementation should be sufficient and any arbitrary structure can be passed to an IFFFile, even completely unrelated to Pangya.

To Do:

• Set up NUnit project PangLib.IFF.Tests
• Create a simple TestStructure used for the tests
• Test writing of files (verify a field using a BinaryReader)
• Test reading of files (using a IFFFile<TestStructure> instance)

JP IFF files are not working - System.InvalidCastException

Description: I am trying to load a Caddie.iff (from pangya_jp).

• Binding ID: 0
• Version: 13

Problem: I get "System.InvalidCastException: 'The record length (248) mismatches the length of the passed structure (260)'"

Code snippet used:

static int Main(String[] args)
{

    IFFFile<Caddie> IFF = new IFFFile<Caddie>(File.Open("Caddie.iff", FileMode.Open));

    Console.ReadKey();

    return 0;

}

Here the Caddie.iff file I am trying to load: Caddie.zip

Add PETScript parser

PET.Models.Frame contains a Script which is a set of commands executed at the corresponding frame.

Example of PETScript included in a Frame:

*showbone("E_WI_11")*hidebone("E_WI_11_SUB") *hideclub *hidebone("E_Cannon" "E_Telescope" "E_Ship" "E_killerwhale") *ptex("__facetexture__" "e_def_reset.png") *showbone("E_WI_11")*hidebone("E_WI_11_SUB")

Write up a PETScriptParser that returns an AST, which can then be used by other tools to execute the given commands with their arguments.

Add Encoding field to PETFile

Currently, string encoding is all over the place for Puppet files, early code iterations use Encoding.UTF8, later ones use Encoding.ASCII.

To solve and improve this, add a Encoding (with Encoding.ASCII as a default) field to PETFile and pass the encoding to helpers where necessary.

Add SPR utilities

The SPR (Spray/Sprite) file format is a custom scripting format defining particle animations.

Examples and a reference can be found at https://docs.pangya.golf/pc/file-formats/spr

Write up PangLib.SPR.SPRFile parsing the file and returning an AST, which can then be used by other tools to execute the given commands with their arguments.

Add ability to create a file-less instance

So there's no dependency on a self-design file pre-existing.

To Do:

• Add file-less constructor for UCCFile

Switch away from System.Drawing.Common

System.Drawing.Common is dependent on GDI+, which either requires you to run Windows on the host machine the library is built/executed on, or instances of libgdiplus.

Instead of using this system namespace, we can switch to an alternative like SkiaSharp which has cross-platform bindings.

The refactor itself should be relatively easy, the used methods are almost the same, except for being prefixed with SK.

SkiaSharp Documentation:
https://docs.microsoft.com/en-us/dotnet/api/skiasharp?view=skiasharp-1.68.0

To Do:

• Switch from System.Drawing.Bitmap to SkiaSharp.SKBitmap

Add parsing of FANM sections

FANM is (aside of SMTL, which is always empty) the final section that we need to parse to handle all PET file sections.

We assume that section to be related to Face Animation. Containing probably not just reference to named planes and meshes it replaces textures on, but also conditional sub-sections with even more data.

Fix parsing of bpet files

Currently, any file with the extension .bpet cannot be parsed.

As a test, trying to parse e_def.bpet from Season 8, throws following exception:

Unable to read beyond the end of the stream.
   at System.IO.MemoryStream.InternalReadInt32()
   at PangLib.PET.Helpers.CollisionBoxReader.ReadAllCollisionBoxes(BinaryReader sectionReader, Version version) in D:\Projects\pangyatools\PangLib\PangLib.PET\Helpers\CollisionBoxReader.cs:line 36
   at PangLib.PET.PETFile.Parse(Stream data, Boolean catchExceptions) in D:\Projects\pangyatools\PangLib\PangLib.PET\PETFile.cs:line 131

Add ability to import images to UCCFile parts

Instead of just being able to export images, we should also be able to import images to a UCCFile.

To Do:

• Add import of images to specified UCC file sections

Use structs instead of classes for DataModels

Not even sure why I used classes in the first place, probably just a minor oversight on my end here.

structs make way more sense for these.

Note: With using structs we lose the ability to have default values on initializing an instance, so List<>s have to be created in the proper helper.

Add *bin utilities

There are file types related to model collection/maps, namely:

• *.gbin
• *.sbin

Handling all file types also involves these files of course, so we need fitting utilities and documentation for them!

Add ability to save IFF file to disk

Add the ability to save a loaded (and changed) instance of IFFFile back to an IFF file. It of course needs to respect the header format too.

ToDo:

• Add a Save method to save an IFFFile instance to disk.

Remove file encoding guessing from DATFile

With using Stream in the constructor we can't even pass a file name anymore, aside that we've been guessing it anyway.

Move guessing of encoding to PangLib.Utilities instead, so people don't have to search for the encodings themself.

To Do:

• Add Encoding to DATFile constructor
• Move GetEncodingFromFileName to a static helper inside PangLib.Utilities

Add LZ77 compression utilities

Add (de)compression utilities using LZ77, PangLib.PAK requires it to properly parse the files inside PAK files.

It also needs a custom implementation supporting another "mode" of (de)compression, which ORs 0xC8 to the flag/mask.

Add ability to create file-less instance

Don't depend on a file path to be passed to the IFFFile constructor.

ToDo:

• Add a file path-less constructor to IFFFile
• Add a Load method which takes a path to load in an IFF file

Use Streams instead of file paths

This is a refactoring issue related to basically all my implementations of PangLib libraries.

Instead of passing a filePath to the constructor of a *File class, pass a Stream to it instead. These can be used by the BinaryReader just fine.

Please open seperate Pull Requests for each library you are refactoring!

To-Do List:

• PangLib.DAT
• PangLib.IFF
• PangLib.PAK
• PangLib.PET
• PangLib.UCC
• PangLib.UpdateList

Use fixed size arrays instead of Lists in DataModels

While Lists make sense for the root PETFile, we should use fixed size arrays in the DataModels.

In basically all cases we have a *count property available in a file which we could use to get a properly sized array.

Add ability to save UCC file to disk

So imported images, and changed UCC files in general can be properly utilized by games and tools!

To Do:

• Add the ability to save an instance of UCCFile back to a file

Add ability to save *pet files to disk

Add the ability to save a loaded (and changed) instance of PETFile back to a PET file. It of course needs to respect the header format too.

ToDo:

• Add a Save method and *Writer helpers to save a PETFile instance to disk.

Fix parsing of apet files

Currently, any file with the extension .apet cannot be parsed.

As a test, trying to parse e_def.apet from Season 8, throws following exception:

Unable to read beyond the end of the stream.
   at System.IO.BinaryReader.FillBuffer(Int32 numBytes)
   at System.IO.BinaryReader.ReadSingle()
   at PangLib.PET.Helpers.AnimationReader.ReadAllAnimations(BinaryReader sectionReader, Version version) in D:\Projects\pangyatools\PangLib\PangLib.PET\Helpers\AnimationReader.cs:line 36
   at PangLib.PET.PETFile.Parse(Stream data, Boolean catchExceptions) in D:\Projects\pangyatools\PangLib\PangLib.PET\PETFile.cs:line 116

Add SEQ utilities

The SEQ (Sequence) file format is a custom scripting format defining particle animation sequences (calling SPR files, basically).

It shares the same format as SPR files, with the difference that SEQ files can contain a frame/ms count in front of a command.

Examples and a reference can be found at https://docs.pangya.golf/pc/file-formats/seq

Write up PangLib.SEQ.SEQFile parsing the file and returning an AST, which can then be used by other tools to execute the given commands with their arguments.

Add optional exception handling to parsing loop

If there are structures that differ from general format (like *.apet and *.bpet) we might run into exceptions that interrupt full parsing of the file until we implement proper parsing of that filetype and it's special structures.

We should add an option to parse a file continously, effectively handling an exception that comes up.

Add ability to create file-less instance

Don't depend on a file path to be passed to the DATFile constructor.

ToDo:

• Add a file path-less constructor to DATFile
• Add a Load method which takes a path to load in an DAT file

Add parsing of EXTR sections

Looks like I had an oversight and there is another section EXTR only present in *.bpet files.

From first looks at the section, it looks like it only contains a character table, maybe allowed characters for naming inside a specific file set.

Add *pet utilities

There are a vast amount of file types related to models and animation, namely:

• *.pet
• *.apet
• *.bpet
• *.mpet

Handling all file types also involves these files of course, so we need fitting utilities and documentation for them!