Giter Site home page Giter Site logo

spriggit's Introduction

Spriggit

A tool to facilitate converting Bethesda plugin files to a text based format that can be stored in Git. Large scale projects can then live in Github, and accept Pull Requests from many developers.

The goal is to help modders store their files in a versioning system that allows them to easily iterate in the same way that programmers do with their code.

Spriggit Demo

Check out the Documentation Official Documentation for more detailed explainations of Spriggit, and more typical use cases and examples.

Reasons to Use Git for Mods

Git is an extremely powerful versioning and iteration tool that almost all programmers use when working. It's what powers the world of coding to be able to iterate new code quickly and collaborate easily.

Some things Git can help you do when developing your mod:

  • Keep track of the many versions of your mod, without resorting to Dropbox folder hell.
  • Create a living "changelog" as you work
  • Be able to go back in time and view your mod exactly as it was at any point in history
  • Stamp your mod with version tags, letting you see how it looked at any one from the past
  • Easily experiment on side branches without worrying your stable setup
  • Share your work on Github, allowing people to see your mod's development progress
  • Collaborate easily, by allowing others to contribute to your mod via Pull Requests
  • More easily merge the work of multiple developers with Git Merge technology

The Workflow:

An Individual Modder

  • Create a Git Repository to hold your mod
  • Create a Bethesda plugin with existing normal tools of choice
  • Use Spriggit to convert the .esp/m/l files from your Bethesda workspace, to .yaml or .json files inside your Git Repository
  • Make commits in Git.
    "Added all the bandit Npc definitions" "Fixed the Powerblade damage to be more reasonable"
  • Upload your mod, in its text format, up to Github (or your host of preference)

Many Collaborators

Other modders, whether on your team or just helpful people out in the world can help collaborate and participate in your mod's development.

  • They can clone the mod via Git to their computers
  • Use Spriggit to convert from the .yaml or .json files to a Bethesda plugin
  • Open the Bethesda plugin with the game, or other tools
  • Modify the mod and help work on something
  • Use Spriggit to convert back to text format
  • Make commits in Git
  • Upload their improvements to Github
  • Initiate a Pull Request to ask that you consider their changes
  • You can discuss with them about further changes, or merge their improvements into your mod

Example Output

Record Data is Stored as Plaintext

Json or Yaml formatting is currently supported.

Here is a snippet of what a record file might look like if Yaml output is used:

FormKey: 087835:Skyrim.esm
EditorID: JewelryNecklaceGoldGems
ObjectBounds:
  First: -3, -9, 0
  Second: 3, 9, 1
Name: Gold Jeweled Necklace
WorldModel:
  Male:
    Model:
      File: Armor\AmuletsandRings\GoldAmuletGemsGO.nif
      Data: 0x020000000300000000000000A4E51E5364647300D8C674AFC031228D64647300D8C674AFB8EC307B64647300262C333B
PickUpSound: 08AB15:Immersive Sounds - Compendium.esp
PutDownSound: 08AB16:Immersive Sounds - Compendium.esp
Race: 013749:Skyrim.esm
Keywords:
- 06BBE9:Skyrim.esm
- 08F95A:Skyrim.esm
- 0A8664:Skyrim.esm
- 10CD0A:Skyrim.esm
Armature:
- 09171F:Skyrim.esm
Value: 485
Weight: 0.5

This file is more palatable to Git and can support diff tools and similar functionality.

Mods are Split into Folders

Rather than having one large file of all of a mod's data, Spriggit splits a mod into a folder of Yaml/Json files.

A typical mod folder structure might look like:

Some/Folder/
   RecordData.yaml          -  The mod header
   Weapons/                 -  Folder for all the weapons
      GlassDagger.yaml      -  File dedicated to the record Glass Dagger
      IronLongsword.yaml    -  Seperate file for the Iron Longsword
   Npcs/                    -  Folder for all the weapons
      Goblin.yaml           -  File dedicated to the Goblin's data

This folder structure helps organize git diffs to be more meaningful. If a new record is added, then this will be seen as a new file. If a record is modified, it will be a modified file. Similar to wanting to avoid having a program's code be in one large monolith file, having smaller bite sized files helps navigate and digest changes being made.

Installation

Spriggit UI

This is a self-contained WPF application, which can only run on Windows. You can add links between a mod file and where it should be translated to, and then sync back and forth with one click.

image image

Spriggit CLI

Spriggit comes with a Command Line Interface that can be used to convert from Betheseda Plugins to Git Repositories, and back. Note that the UI can also accept these CLI commands, and so this variant is meant for linux machines that cannot handle the UI. It is not self contained, so it might complain about needing a specific .NET runtime downloaded.

This would use Spriggit to convert from a SkyrimSE mod to Yaml, and put it in your Git Repository.

.\Path\To\Spriggit.CLI.exe serialize --InputPath "C:\Games\steamapps\common\Skyrim Special Edition\Data\SomeMod.esp" --OutputPath "C:\MyGitRepository\SomeMod.esp" --GameRelease SkyrimSE --PackageName Spriggit.Yaml

And this would convert it back and overwrite the original mod file.

.\Path\To\Spriggit.CLI.exe deserialize --InputPath "C:\Users\Levia\Downloads\SpriggitOutput\SomeMod.esp" --OutputPath "C:\MyGitRepository\SomeMod.esp"

Spriggit Translation Packages

The logic for actually doing the translation to/from an esp is not housed or packaged directly with the CLI or UI. Rather, the logic exists in NuGet packages that are downloaded and then used to do the translation.

Mechanics

When Spriggit is asked to do a serialization from a Bethesda plugin file to text format, you must provide it with the name of a NuGet package to use for the translation. If a specific version is supplied, it will use that, otherwise it will grab the latest version of that package. As it exports the text files, the NuGet package name and version are included.

When Spriggit is asked to do a deserialization from text to a Bethesda Plugin, no specification of a NuGet package is needed. Rather, it looks in the text files for the listed package + version that was used to make the files in the first place, and downloads that package to use.

As such, a Spriggit CLI/UI can translate to/from many different translation formats/styles, rather than being bound to the translation logic that existed when the CLI/UI was built.

Reasoning

This separation is an important aspect of keeping Spriggit flexible as record definitions evolve.

Especially early on during a game's release, the record definitions are constantly being upgraded, adjusted, and fixed. The separation allows Spriggit to always grab the older version of a translation, and use that to deserialize text that contains older definitions.

If very old versions get deserialized, the older nuget packages will be downloaded and used to read them. And, if the user re-serializes them with the latest nuget packages, they will be "upgraded" to the latest text definitions.

Example

Let's take a simple example of a typo. Let's say Haelth was accidentally used as a field name in v1.1 of Spriggit, and a plugin was serialized into text with that typo. Eventually someone notices, and fixes it to Health in v1.2. How does the old file that contained Haelth get properly read anymore?

It will be able to be read, because the original file will have been stamped with v1.1. During deserialization of the file, Spriggit will see that and download that v1.1 nuget package, and use that to read the file. The Haelth field will be read in as expected by the v1.1 package, yielding a proper esp output.

Customization

Spriggit uses Mutagen systems under the hood, and leans on the Mutagen.Bethesda.Serialization library to convert to/from Yaml and Json.

One of the features of Mutagen.Bethesda.Serialization is that it allows for customization of naming, file structure, and other similar things. If you utilize this to make your own customization, you will need to upload the results to nuget.org for people to grab and use.

More documentation will follow on how to upload your own customization package to Nuget so that it can be used by Spriggit. For now, two packages exist "built in":

  • Spriggit.Yaml.[GameName]
  • Spriggit.Json.[GameName]

These are what will be used to do any translations, unless the user specifies the name of a customization package they wish to use instead.

Check out the Documentation Official Documentation for more detailed explainations of Spriggit, and more typical use cases and examples.

spriggit's People

Contributors

epicestgamer avatar kuropon avatar metacubed avatar noggog avatar xnyo avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

spriggit's Issues

Live conversion

Option to watch the esp file and convert to git automatically. The reverse could be offered, but of course would be more dangerous, and harder with the tooling probably locking down that file a bit more

Improve warning for unknown package

putting --PackageName TestBooks.yaml resulted in an error message of Could not get identity for SpriggitMeta. It should say something more along the lines of "Could not find package named TestBooks.yaml" or something like that. We could get the identity, but we couldn't find the package associated with that identity

Have Spriggit UI support .spriggit configs

Maybe add a checkbox to make config with given settings.

On load, if config exists, populate settings with those values.

When changed, save config out with new settings

Esp move on serialization

Some users like committing both the text files and the raw esp. This is sort of a one way usage of Spriggit, where the user would be mostly interested in the diff results of spriggit. They would rarely actually deserialize from text back to esp, but instead just use the raw esp directly. In this setup, rather than requiring they run Spriggit and copy the file by hand each time, spriggit could potentially do that esp copy for them so it's a one click thing

No warning when a form type is skipped

Currently using Spriggit for the Community Patch.

Added a new form type Package (PACK) with 4 additional records.

Serialized the plugin.

No errors during serialisation. The total form count is updated but the PACK forms were omitted.

If Spriggit encounters a data type is can't decode it should throw and error and cancel the process. Had I not checked this, the mod could've been released with missing data.

To Replicate

Post Conversion Double Check

People might have some paranoia that using spriggit might corrupt their esp that they've worked so hard on. So we could do a doubleback check when serializing. Serialize into a temp folder, deserialize back into another temp area.. and do a byte comparison with the original file to ensure nothing changed. Only if this passed would the serialization results actually get published to the end location? This would give some trust in the system, and users would be immediately alerted if there was any corruption or data loss.

This would suffer from any customizations that omit fields like LastModified, as the comparison would then always fail, but on non-interesting fields

Initial TryGetMetaInfo call should be via current version's package

Unhandled exception. Microsoft.CSharp.RuntimeBinder.RuntimeBinderException: The best overloaded method match for 'Spriggit.Yaml.Skyrim.EntryPoint.TryGetMetaInfo(string, Noggog.WorkEngine.IWorkDropoff, System.IO.Abstractions.IFileSystem, Noggog.IO.ICreateStream, System.Threading.CancellationToken)' has some invalid arguments
at CallSite.Target(Closure, CallSite, Object, String, IWorkDropoff, IFileSystem, ICreateStream, CancellationToken)
at System.Dynamic.UpdateDelegates.UpdateAndExecute6[T0,T1,T2,T3,T4,T5,TRet](CallSite site, T0 arg0, T1 arg1, T2 arg2, T3 arg3, T4 arg4, T5 arg5)
at Spriggit.Core.DynamicEntryPoint.TryGetMetaInfo(String inputPath, IWorkDropoff workDropoff, IFileSystem fileSystem, ICreateStream streamCreator, CancellationToken cancel) in D:\a\Spriggit\Spriggit\Spriggit.Core\DynamicEntryPoint.cs:line 71
at Spriggit.Engine.EngineEntryPoint.TryGetMetaInfo(String inputPath, IWorkDropoff workDropoff, IFileSystem fileSystem, ICreateStream streamCreator, CancellationToken cancel) in D:\a\Spriggit\Spriggit\Spriggit.Engine\EngineEntryPoint.cs:line 73
at Spriggit.Engine.GetMetaToUse.Get(SpriggitSource source, String spriggitPluginPath, CancellationToken cancel) in D:\a\Spriggit\Spriggit\Spriggit.Engine\GetMetaToUse.cs:line 41
at Spriggit.Engine.SpriggitEngine.Deserialize(String spriggitPluginPath, FilePath outputFile, SpriggitSource source, CancellationToken cancel) in D:\a\Spriggit\Spriggit\Spriggit.Engine\SpriggetEngine.cs:line 68
at Spriggit.CLI.Runner.Run(DeserializeCommand deserializeCommand) in D:\a\Spriggit\Spriggit\Spriggit.CLI.Lib\Runner.cs:line 56
at Program.<>c.<<

$>b__0_0>d.MoveNext() in D:\a\Spriggit\Spriggit\Spriggit.CLI\Program.cs:line 11

NavmeshGeometry GridCell array serialized as single unparsed GridCell

When exporting a STAT record containing a navmesh from a Starfield .esm, the gridcells sub-container is not split into separate gridcells, but rather contains a single gridcell with the unparsed data for all the gridcells.

Spriggit 0.18; SF1Edit 4.1.5d

It gives the impression that gridcells are parsed, but they are not. My suggestion would be to either parse them, or not include the gridcell object at all (as what is labeled a "GridCell" is in fact an "UnparsedGridCellArray"

testnavmesh.zip

The attached esm has an example of a static with a navmesh that should contain 4 gridcells according to SF1edit, but in the spriggit yaml it is stored as one.

Made up example:

Current output
(non-data commented)

  GridArrays:
  - GridCell:
    - 2    # Length
    - 0    # Delimiter
    - 13
    - 14
    - 6    # Length
    - 0    # delimiter
    - 12
    - 13
    - 14
    - 18
    - 20
    - 0
    - 0    # Length
    - 0    # delimiter

Expected Output:

  GridArrays:
  - GridCell:
    - 13
    - 14
  - GridCell:
    - 12
    - 13
    - 14
    - 18
    - 20
    - 0

Alternatively if such parsing is undesired, the structure is inaccurate - the GridCell node should be collapsed into parent:

Non-ideal alternative

  GridArrays:
  - 2
  - 0
  - 13
  - 14
  - 6
  - 0
  - 12
  - 13
  - 14
  - 18
  - 20
  - 0
  - 0
  - 0

(I have yet to see if Mutagen has the same issue)

No output in console when running Spriggit.exe in "cli mode"

Spriggit CLI output:

PS F:\spriggit-out> & '..\MyStarfieldModpack\tools\SpriggitCLI 0.4\Spriggit.CLI.exe' serialize -i "F:\SteamLibrary\steamapps\common\Starfield\Data\OldMars.esm" -o OldMars
[22:20:18.361 INF] Command to serialize
[22:20:18.427 INF] Found .spriggit config at F:\spriggit-out\.spriggit
[22:20:18.676 INF] Loaded .spriggit config with SpriggitMetaSerialize { PackageName = Spriggit.Yaml, Version = 0.4, Release = Starfield }
[22:20:18.678 INF] Getting entry point for SpriggitMeta { Source = Spriggit.Yaml.0.4, Release = Starfield }
[22:20:18.681 INF] Getting first identity for SpriggitMeta { Source = Spriggit.Yaml.0.4, Release = Starfield }
[22:20:18.684 INF] Cached first identity for SpriggitMeta { Source = Spriggit.Yaml.0.4, Release = Starfield }: Spriggit.Yaml.Starfield.0.4.0
[22:20:18.689 INF] Constructing entry point for Spriggit.Yaml.Starfield.0.4.0
[22:20:18.726 INF] Getting target framework directory for C:\Users\Flayan\AppData\Local\Temp\Spriggit\Sources\Spriggit.Yaml.Starfield.0.4.0\Spriggit.Yaml.Starfield.0.4.0
[22:20:18.728 INF] Framework directory located: C:\Users\Flayan\AppData\Local\Temp\Spriggit\Sources\Spriggit.Yaml.Starfield.0.4.0\Spriggit.Yaml.Starfield.0.4.0\lib\net7.0
[22:20:18.729 INF] Loading plugin: C:\Users\Flayan\AppData\Local\Temp\Spriggit\Sources\Spriggit.Yaml.Starfield.0.4.0\Spriggit.Yaml.Starfield.0.4.0\lib\net7.0\Spriggit.Yaml.Starfield.dll
[22:20:19.323 INF] Retrieving default assembly
[22:20:19.378 INF] Finding entry point type
[22:20:19.379 INF] Looking for typically named entry point classes
[22:20:19.386 INF] Creating entry point object
[22:20:19.387 INF] Cached entry point for Spriggit.Yaml.Starfield.0.4.0
[22:20:19.389 INF] Starting to serialize from F:\SteamLibrary\steamapps\common\Starfield\Data\OldMars.esm to F:\spriggit-out\OldMars
[22:20:21.516 INF] Finished serializing
PS F:\spriggit-out>

Spriggit exe output

PS F:\spriggit-out> & '..\MyStarfieldModpack\tools\Spriggit 0.4\Spriggit.exe' serialize -i "F:\SteamLibrary\steamapps\common\Starfield\Data\OldMars.esm" -o OldMars
PS F:\spriggit-out>

Pre-convert

If app is open, preconvert the git contents to a temp esp somewhere, and then if the user hits convert, it'll be instant. If the files change, then of course wipe and redo

Serializing doesn't warn about duplicate editor ids (Version 0.7.0)

Use case: My modlist renames a spell tome from both Odin and Apocalypse to "SpellTomeConjuration5ConjureKyrkrim". I noticed it was missing when doing a round trip delta patch. On checking the yaml files, there is one entry in Book/ and it's the Apocalypse one. The Odin one was never serialized and the log doesn't show any errors when processing. Should it behave like CK and append DUPLICATE etc rather than destroying data?

Only oddity that stands out was during first serialize:
2023-10-09 21:49:54.109 -07:00 [INF] Getting target framework directory for C:\Users\ra2ph\AppData\Local\Temp\e3thvthc.v4e\Mutagen.Bethesda.Serialization.SourceGenerator.1.3.0
2023-10-09 21:49:54.109 -07:00 [INF] Could not locate framework directory

Optional to build unlocalised

It might be useful to have an option to build the plugin un-localised.

In this scenario, it would ignore the "Localised" header flag, only take the text string for the required language and write it directly into the plugin rather than creating .strings files.

Sync Overwrite Warnings

If the target has a date newer than the source that's going to overwrite it, then warn user. Offer a way to turn off these warnings if the user is experienced

Port nuget config analyzer logic

Since nuget is being used, often people will have a Nuget.Config without nuget.org. We would like to detect and fix this in spriggit, like we do in Synthesis

Records with reflection data do not export?

Just found this tool the other day and was looking at using it as an alternative to xEdit+scripts to export the record data from ESM files. Seems to work nicely except that it appears that it will not export records that contain reflection data. I understand that reflection data is not well-understood at this time and its not really safe to change it, but it would still be helpful to be able to export records that contain reflection data so that other properties in the record can be altered even if we treat the reflection data as an opaque blob.

ESP backup

Optional feature to store X amount of binary esps in a backup area somewhere. That way on accidental sync, someone can retrieve the old file.

Will need to think about how the recovery is exposed to the users... do we have some fancy recovery UI? Or just have some help tooltip that opens the folder for them to see and grab the file themselves manually?

Spriggit version post-merge syncer

If a merge has occurred where one side has updated spriggit versions, there might be issues. Would be good to check and re-run the older side to the newer version

Avoid Nuget Installation Requirement

Right now we're using the Nuget SDK in a way that needs NuGet.config to exist with sensible targets to work. Might be good to research if there's a way to fall back intelligently if those dont exist, so that Spriggit can work in environments that dont have nuget installed

Omit ModHeader Stats

Stuff like NumRecords, NextFormID, and OverriddenForms are derivative, and will only be merge conflict headache

Add hyper compatible 2nd entry point interface

Right now, the interface is a bit dangerous, as it's using fancy objects like IFileSystem and IWorkEngineDropoff for certain features. Would be nice to define a more safe but limited alternative using only basic types. This would be used as a fallback if the primary entry point could not connect

Preprocessor "pipeline"

For example, CK randomizes script properties. Some people might want that ordered, and so a preprocessor cmd could be added that did the job before flowing into the spriggit call.

Would be nice to have this in the UI somehow to where it could run these for the user as part of the single click

Why is Framework directory located repeatedly

If targeting a new entry point that doesn't exist in the temp folder yet, there's some spam:

[01:20:42.888 INF] Framework directory located: C:\Users\Levia\AppData\Local\Temp\5aus2qdl.nh0\Mutagen.Bethesda.Serialization.Yaml.1.0.0\lib\net7.0
[01:20:42.890 INF] Getting target framework directory
[01:20:42.891 INF] Framework directory located: C:\Users\Levia\AppData\Local\Temp\5aus2qdl.nh0\Mutagen.Bethesda.Skyrim.0.42.0\lib\net7.0
[01:20:42.912 INF] Getting target framework directory
[01:20:42.913 INF] Framework directory located: C:\Users\Levia\AppData\Local\Temp\5aus2qdl.nh0\Noggog.CSharpExt.2.60.0\lib\net7.0
[01:20:42.916 INF] Getting target framework directory
[01:20:42.917 INF] Framework directory located: C:\Users\Levia\AppData\Local\Temp\5aus2qdl.nh0\NuGet.Versioning.6.5.0\lib\netstandard2.0
[01:20:42.920 INF] Getting target framework directory
[01:20:42.921 INF] Framework directory located: **C:\Users\Levia\AppData\Local\Temp\5aus2qdl.nh0\SharpZipLib.1.4.2\lib\net6.0**

Github workflow to push new version when Mutagen updates

We need to periodically push new Spriggit.Yaml and Spriggit.Json packages when mutagen updates with new definitions. We can either do this by hand, or have github actions somehow do this for us automatically.

Maybe we dont want to have it push a new version -every- time mutagen gets a new version, but perhaps every day/week we check and push a new package if new one is found.

Could not locate entry point for: SpriggitMeta { Source = Spriggit.yaml, Release = Starfield }

When trying to serialize an ESM from Starfield, I receive the error below. It's the same error in the UI. It works fine for a Fallout 4 ESM though.

[12:30:14.987 INF] Spriggit version 0.18.0+5a68a8cad
[12:30:14.990 INF] Getting entry point for SpriggitMeta { Source = TestBooks.yaml, Release = Starfield }
[12:30:14.996 INF] Getting first identity for SpriggitMeta { Source = TestBooks.yaml, Release = Starfield }
[12:30:14.998 INF] No version specified.  Checking NuGet repositories for latest version
[12:30:14.998 INF]   Looking in repo nuget.org
[12:30:15.139 INF]   GET https://api.nuget.org/v3-flatcontainer/testbooks.yaml.starfield/index.json
[12:30:15.364 INF]   NotFound https://api.nuget.org/v3-flatcontainer/testbooks.yaml.starfield/index.json 222ms
[12:30:15.372 INF]   Looking in repo Microsoft Visual Studio Offline Packages
[12:30:15.381 INF] No version specified.  Checking NuGet repositories for latest version
[12:30:15.382 INF]   Looking in repo nuget.org
[12:30:15.384 INF]   GET https://api.nuget.org/v3-flatcontainer/testbooks.yaml/index.json
[12:30:15.507 INF]   NotFound https://api.nuget.org/v3-flatcontainer/testbooks.yaml/index.json 122ms
[12:30:15.508 INF]   Looking in repo Microsoft Visual Studio Offline Packages
[12:30:15.509 WRN] Could not get identity for SpriggitMeta { Source = TestBooks.yaml, Release = Starfield }
it\Spriggit.Engine\SpriggetEngine.cs:line 40
   at Spriggit.CLI.Runner.Run(SerializeCommand serializeCommand) in D:\a\Spriggit\Spriggit\Spriggit.CLI.Lib\Runner.cs:line 91
   at Program.<>c.<<<Main>$>b__0_1>d.MoveNext() in D:\a\Spriggit\Spriggit\Spriggit.CLI\Program.cs:line 12
--- End of stack trace from previous location ---
   at Program.<Main>$(String[] args) in D:\a\Spriggit\Spriggit\Spriggit.CLI\Program.cs:line 9
   at Program.<Main>(String[] args)

I tried v0.17 and v0.18.

Post Merge FormID Collision Fixer

Write logic that can detect when there is duplicate FormIDs, and offer to reassign one of them, tracking down existing usages and pointing them appropriately. This might be written in Spriggit, or be a more generic tool in the Mutagen toolset that Spriggit just makes use of

Spriggit target package config file

Have a file that a repo can include, such as .spriggit that outlines what Spriggit translation nuget package to use, with a specific version optional.

This would override the normal UI setting and "drive" the target package setting. If the repo's file changes, the UI would see it. If the UI is used to change it, it would hit this file (maybe in addition to the normal spriggit settings file)

The CLI should also pick up on this, making the target package parameters optional for serialize calls

Round-trip mismatch for empty ESP

This is an edge case I ran into when developing my project. It affects the mod header information for empty mods.

  1. Create a new ESP in xEdit for a known-good baseline.
    • Note that the "Record Count" in the mod header is 0, since there are no records.
    • Note that the "Next Record ID" in the mod header is 0x800.
  2. Serialize the ESP into YAML with Spriggit.
    • Note that the ModHeader object in ReportData.yaml does not have Stats.
    • This is expected, because they have default values.
  3. Deserialize the same YAML back into an ESP.
  4. Check the new ESP in xEdit.
    • The "Record Count" in the mod header is changed to 1, but there are no records.
    • The "Next Record ID" in the mod header is changed to 0x801.
    • The list of masters is cleaned to remove unused masters.

Not sure if this issue report belongs in Spriggit or Mutagen.

Serializing/Deserializing QUST records does not save ANAM Next Alias ID (Version 0.7.0)

QUST records do not serialize their ANAM value into the yaml, so when it's rebuilt in deserialize it just scans the existing aliases to get the next value. According to Elminster this value is important in case they deleted aliases and should be preserved, so it needs to be explicitly saved and restored.

Steps to reproduce:
Create a patch just changing some quest text in nearly any vanilla Skyrim quest. Ex: 00025185 CR02. Save to git and back to esp, and the ANAM Next Alias ID is 18, not 19 as in vanilla records.

For deserialization, infer output file name from ModKey so they are never different

The ModKey entry in the RecordData.yaml file contains the name of the ESx from which the serialized code was generated. When transforming the code back into an ESx, the generated file name should match that entry.

For the deserialize command, change --OutputPath to accept just a directory path, not the full file path. The file path should be calculated by appending the ModKey to the OutputPath.

This will prevent errors due to the entries being out of sync.

Check AoT that target package is viable

If somehow an unknown package, or unknown version is targeted, we can see if we can find the nuget package ahead of time and show some errors proactively

Passthrough Testing

There's some problems with testing passthroughs, as the default Spriggit.Yaml.[X] package doesn't have certain things like EnforceRecordOrder enabled, so it doesn't quite pass through byte by byte as expected

I think either we can make EnforceRecordOrder more of a runtime parameter, rather than a package compile time customization. OR, we can improve the passthrough testing suite to reorder records (sort by FormID, perhaps?)

Audit nuget pulldowns

Noticing that old versions of some packages are being downloaded, like Mutagen.Bethesda.Kernel v0.13.3, which is ancient. Test to make sure it's Spriggit doing this pulldown. and then see if it can it be avoided?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.