alvazir/habasi

Habasi - TES3 Plugin Merging Tool

habasi's Issues

Ignore-Important-Errors Flag Failing and asking to use Ignore-Important-Errors

Hello! We've recently started using Habasi on the momw-patches repository in order to compile a merged version of the patches we host there. This worked great for a while. However, last night, we started getting unexpected failures.

The problem: DeltaPlugin serializes XSCL at the end of a FRMR. According to the spec XSCL should come after the ID. This causes Habasi to fail.

We were using --ignore-important-errors to bypass this and merge together those plugins which weren't affected by this. That just stopped working as of our last job.

Would it be possible to make habasi do this in a fashion that doesn't rely so strictly on the ordering of subrecords?

This is really two separate problems, as --ignore-important-errors seems to just not do anything at all now. Here's the job logs:

Log is being written into "/usr/bin/habasi.log"
Global list options: mode = keep, config = openmw.cfg, ignore_important_errors
Gathering plugins from game configuration file "openmw.cfg"
Plugin list was expanded with contents of load order due to "use_load_order" flag
Processing list "MergedLoadOrder.esp" with options: mode = complete_replace, config = openmw.cfg, use_load_order, strip_masters, exclude_deleted_records, ignore_important_errors
  Processing plugin "/builds/modding-openmw/momw-patches/01 BCOM+Imperial Towns Revamp Patch/BCOM+Imperial Towns Revamp Patch.esp"
Output plugin "MergedLoadOrder.esp": masters will not be stripped due to encountering external reference
  Processing plugin "/builds/modding-openmw/momw-patches/02 UvirithsLegacy+TR Patch/UvirithsLegacy+TR Patch.esp"
Failed to process list "MergedLoadOrder.esp, no_dry_run, use_load_order, exclude_deleted_records, complete_replace, strip_masters, ignore_errors, no_insufficient_merge"
Caused by:
    Failed to process plugin
    Failed to load plugin "/builds/modding-openmw/momw-patches/02 UvirithsLegacy+TR Patch/UvirithsLegacy+TR Patch.esp": Unexpected Tag: CELL::XSCL
    Use --ignore-important-errors to ignore
    Consider reporting the error to add this tag to the list of unexpected tags to skip by default

Failed Job: https://gitlab.com/modding-openmw/momw-patches/-/jobs/5617347498

Successful one: https://gitlab.com/modding-openmw/momw-patches/-/jobs/5570279215

I believe the failure of --ignore-important-plugins is a regression introduced by f74a5b5.

TES3::FORM subrecord causes failures

OpenMW extended the TES3 subrecord to also allow a FORM subrecord indicating the format version of omwaddon; these tags cause habasi to fail when such plugins are encountered.

Failed to process list "MergedLoadOrder.esp, no_dry_run, use_load_order, exclude_deleted_records, complete_replace, strip_masters, ignore_errors, no_insufficient_merge"

Caused by:
0: Failed to process plugin
1: Failed to load plugin "/home/sk3shun-8/openmw/openmw_mods/crassified navigation/Crassified Navigation.esp": Unexpected Tag: TES3::FORM
Fix the problem or add "--ignore-important-errors"(may rarely cause unexpected behaviour) to ignore

Implicit OpenMW Data folder is not used

OpenMW has a 'hidden' data path used by the OpenMW-CS which is not listed in the main openmw.cfg. This results in plugins which openmw knows how to find, being invisible to Habasi.

In almost all cases, the actual path used matches https://openmw.readthedocs.io/en/latest/reference/modding/paths.html#savegames with the exception that the subdir is data instead of saves.

For example on my system it's: /home/sk3shun-8/.local/share/openmw/data/

Since these are consistent across platforms it should be relatively simple to also read this directory in the case that an openmw.cfg load order is detected.

Journal Infos produced by OpenMW don't merge

Failed to process list "tmp.omwaddon, Balmora Club.omwaddon, bc2.omwaddon"

Caused by:
0: Failed to process plugin
1: Failed to process records from plugin "bc2.omwaddon": Error: "4924ff3bafe644ecbe4ad1690d369063" info record's kind is different to "BC_Zyrchyal" dialogue's
Fix the problem or add "--ignore-important-errors"(may rarely cause unexpected behaviour) to ignore

The bc2 plugin has one Journal Topic, "BC_Zyrchyal", and one Journal Info for it, "hello world" (index: 10).

The bc2 plugin works in the OpenMW engine, though I can't test it on the original so I don't know if it works there or not.

DeltaPlugin and Habasi merged plugins

I just saw this recently and had a question about something in the release announcement:

There are also record mergers: TES3Merge, DeltaPlugin, Merged Lands, all the leveled lists mergers etc. I only use leveled lists merging, though I've made sure that all of those(tested with TES3Merge, DeltaPlugin, Jobasha) would continue working with merged plugins the same way as with unmerged. It's done by keeping different variants of mergeable records in a merged plugin by default.

Are you sure that this works with DeltaPlugin? While I can't speak for other tools, DeltaPlugin uses a custom internal representation of the plugin data which, among other things, stores records in a mapping using the record ids as keys, so it will discard all but the last one it reads (much like OpenMW does when loading them).

Also, I'd love to help cultivate more of a rust ecosystem for openmw/morrowind libraries. Unfortunately, DeltaPlugin's internals make that difficult for the most part, but I can offer up the openmw-cfg crate in case you find it helpful, which is a library providing functions that interact with the openmw VFS and plugin load order (and I recently added support for VFS files in BSA archives via bsatool_rs.

Plugin Names with comma in their name are impossible to specify

Maybe I'm overlooking something, but there seems to be no way to specify plugin names like "Umbra, Blademaster.esp" to an '-m' merge command without habasi failing because it thinks I just specified two non-existant plugins.

I can just rename the plugin and its fine, but these days plugin names carry a certain relevancy since they are used to sort load order with plox/mlox.

My suggestion would be to support an escape sequence \,.