kruithne/wow.export

📦 Export Toolkit for World of Warcraft

📦 wow.export

wow.export is a node-webkit port of Marlamin's WoW Export Tools which provides tools for extracting and converting files from the World of Warcraft game client or public CDN servers.

Features

• Supports both Retail and Classic game clients.
• Complete online support allowing streaming of all files without a client.
• Full 3D preview of both M2 and WMO (doodads included) game models.
• Export into a variety of formats (with more coming soon).
• Overhead map viewer with easy exporting of terrain, textures and objects.
• Includes Blender add-on for advanced map/model importing.
• Preview and export all sound files from the game client.
• Export all video files (cinematics) from the game client.
• Locale support for all 13 languages supported by the client.
• Convert local M2/BLP files by drag-dropping them onto the application.

Installing

To install wow.export, navigate to the site and download the latest version. That's it!

OSX/Linux: We are currently not producing builds targeted for non-Windows builds. If you wish to use wow.export on OSX or Linux, you will need to compile your own build from the source. See GH-1 for known issues.

Updating

When an update to wow.export is available, you will be prompted in the application to update. This process is done entirely automatically once you accept the update!

Building (Developers Only)

• 🔨 Building wow.export requires Node 15.3.0 or above.
• 🧙‍ For building on Windows, node-gyp prerequisites may be required.
• 🍷 For building Windows builds on platforms other than Windows, Wine 1.6 or above is required.

git fetch https://github.com/Kruithne/wow.export.git
npm install

# This will list available builds.
node ./build.js

# This will compile -all- available builds.
node ./build.js *

# Substitute <BUILD> for the build(s) you wish to compile, space-delimitated.
node ./build.js <BUILD1> <BUILD2> ...

Debugging (Developers Only)

Note: Debugging is currently only supported on Windows.

To debug wow.export, compile a win-x64-debug build using the build script. This will produce a bare-bones build using the SDK framework and without any production polish. Upon starting the debug version, DevTools will be automatically launched alongside the application.

For the debug build, source code will not be compiled, rather a symlink is created. This means changes to the source code are instantly reflected in the application, simply run chrome.runtime.reload() in DevTools console to refresh sources (pressing F5 does not drop references and will lead to memory leaks).

To quickly launch the debug build from the project directory, run node debug from your terminal. This will also handle compilation of .scss files both on startup and when changes are detected (allowing styling to be hot-reloaded via DevTools).

wow.export's Issues

BLTE magic error when parsing root file for retail

Failed to load CASC: Error: [BLTE] Invalid magic: 0
    at new BLTEReader (D:\wow.export\src\js\casc\blte-reader.js:42:10)
    at CASCLocal.parseRootFile (D:\wow.export\src\js\casc\casc-source.js:110:16)
    at CASCLocal.loadRoot (D:\wow.export\src\js\casc\casc-source-local.js:185:37)
    at async CASCLocal.load (D:\wow.export\src\js\casc\casc-source-local.js:85:3)
    at async D:\wow.export\src\js\ui\source-select.js:41:4
    at async Object.block (D:\wow.export\src\js\core.js:67:2) {
  [stack]: 'Error: [BLTE] Invalid magic: 0\n' +
    '    at new BLTEReader (D:\\wow.export\\src\\js\\casc\\blte-reader.js:42:10)\n' +
    '    at CASCLocal.parseRootFile (D:\\wow.export\\src\\js\\casc\\casc-source.js:110:16)\n' +
    '    at CASCLocal.loadRoot (D:\\wow.export\\src\\js\\casc\\casc-source-local.js:185:37)\n' +
    '    at async CASCLocal.load (D:\\wow.export\\src\\js\\casc\\casc-source-local.js:85:3)\n' +
    '    at async D:\\wow.export\\src\\js\\ui\\source-select.js:41:4\n' +
    '    at async Object.block (D:\\wow.export\\src\\js\\core.js:67:2)',
  [message]: '[BLTE] Invalid magic: 0'
}

Unknown cause.

Improve recent local build list

Reduce user clicks by including the build with the locally selected directories, allowing users to load a local build with just one click.

Model Exporter

• Export raw models.
• Export models as OBJ.
• Export WMOs with doodads.
• Filtering of WMO mesh groups/doodad sets.
• Toggle exporting of collision files.
• 3D preview of selected model

Add PNG export to 3D model viewer

Given that we are rendering the 3D model viewer with a canvas, it should be trivial to add a PNG export with transparency support to the 3D model viewer. Why not?

WMO Groups/Doodad Sets

WMO (Warcraft Multi-Object) models have both groups (individual meshes) which work like M2 geosets, and doodad sets, which are collections of M2 doodads.

Both of these need to be customizable in the display, as well as respected by the exporters.

Implement 'Videos' tab

Using native video streaming capabilities, implement a tab for previewing and exporting video files from the game files. As far as I know, this is just the .avi format.

Classic Support

Classic has not yet been tested. Test and ensure it works with wow.export.

Toggle collision file exporting

Models that come with collision files (.phys?) need to be accessible via a toggle. Since this is not a normal feature, it can be hidden in the configuration panel and disabled by default.

User Configuration

• Create a configuration system where values are loaded by a built-in default and user changes are persisted on disk within the user data directory.
• Implement an interface for getting/setting configuration values. A configuration value does not need to exist to be requested or set.
• Design a user interface allowing user-friendly configuration of values. Changes should not immediately be applied, allowing users to cancel/save.
• Feed configuration directly into the reactive system, allowing changes to values to automatically propagate.

Some textures are not wrapped correctly on 3D previews

Some textures on 3D models for the preview are not correctly wrapped. An example of this can be seen clearly on world/azeroth/burningsteppes/passivedoodads/trees/burningmidtree04.m2.

Finalize updater

The following things needs to be completed in the updater:

• The updater binary attempts to overwrite itself during file apply; see that this either doesn't halt the process or that it skips itself due to expected permission errors.
• The updater application needs to start the wow.export.exe binary (or respective binary for other platforms).
• The updater needs to clean up after itself.
• The updater needs to terminate itself once done.

Enable preview toggle for 3D model viewer

A feature that existed in the original version was a checkbox that could be toggled to enable/disable the automatic previewing of 3D models. This is especially useful for people using the online CASC more or those with slower computers.

This feature needs to be replicated for wow.export.

Define geosets for all character models

• Human Female
• Human Male
• Dwarf Female
• Dwarf Male
• Night Elf Female
• Night Elf Male
• Gnome Female
• Gnome Male
• Draenei Female
• Draenei Male
• Worgen Female
• Worgen Male
• Orc Female
• Orc Male
• Undead Female
• Undead Male
• Tauren Female
• Tauren Male
• Troll Female
• Troll Male
• Blood Elf Female
• Blood Elf Male
• Goblin Female
• Goblin Male
• Pandaren Female
• Pandaren Male
• Void Elf Female
• Void Elf Male
• Lightforged Dranei Female
• Lightforged Dranei Male
• Dark Iron Dwarf Female
• Dark Iron Dwarf Male
• Kul Tiran Female
• Kul Tiran Male
• Nightborne Female
• Nightborne Male
• Highmountain Tauren Female
• Highmountain Tauren Male
• Mag'har Orc Female
• Mag'har Orc Male
• Zandalari Troll Female
• Zanalari Troll Male

Persist recent local installation selections

For the local source select, the most recent local installations the user has selected should be displayed within the option. Currently, this is a placeholder widget.

Blend alpha correctly for 3D models on model viewer

On the 3D model viewer, textures that have transparency are not rendered correctly. Additionally, textures that have an alpha channel that should not be alpha blended are rendering transparent. Apply the correct blending modes.

Show info message if a loaded model doesn't have any mesh to render

Some models don't have any mesh data. Instead of just loading them blankly, show the user an informative message that the model does not have any 3D data.

Create the super secret awesome feature

👀

Local file conversion for M2 / WMO

Feeding into the same export system that will be used for the 'Models' tab to export models, we should introduce a drag-and-drop handler that supports local M2 and WMO files for local conversion.

Application crashes when changing locale on remote CASC

Cannot assign to read only property 'locale' of object '# <CASCRemote>'

Drag-and-drop support

Chromium's default behavior with files being dropped onto the application is to open them. To prevent this from happening, ondragover and ondrop events have been disabled at src/app.js.

Further into development, a better solution for this would be to allow files such as .blp and .m2 to be dropped onto the application for local conversion.

Add keyboard navigation support to listbox component

Coming from the old version, users may expect keyboard controls- namely the arrow keys- to work in navigating a listbox component.

Implement 'Models' tab

Implement the 'Models' tab with a listbox filtering to contain files ending with the extensions .m2 and .wmo. Additional, WMO LOD files should be filtered out, which will require additional work to the listfile filtering system.

Write feature list for README.md

Include a list of available features that wow.export offers in the README.md.

Handle errors when side-loading textures for 3D preview

A recent change made it so that in the 3D preview, textures are loaded asynchronously and then bound to their respective texture during runtime. In the case of an error while loading these textures, there is no handling and will result in an application crash.

Copy/paste functionality for listboxes

Something that was recently implemented into WET is the ability to use the keyboard shortcut CTRL + C to copy the current selection of a focused listbox.

Given the custom nature of our listbox components, it is not possible to natively track the focus state of the component, which makes this slightly trickier to implement. In the internet of 1:1 feature parity, this should be included.

Drag/drop support not functioning correctly on invalid files

• If the valid file counter is zero for drag/drop operations, the overlay is incorrectly displayed.
• Dropping 'zero' valid files still processes nothing with visual feedback.

Preview of encrypted BLP causes application hang

item/objectcomponents/head/helm_armor_explorer_d_01_pam.m2 causes the application to hang upon preview.

M2 loading performance

There appears to be some issues with M2 loading performance currently. Potentially geoset related.

Persist user-selected CDN choice

On the source-select screen, the default selected CDN is chosen by a static pre-programmed constant. For a better user experience, if the user changes the default choice, it should be remembered and defaulted to.

Skin selector for 3D model preview and exporter

A drop-down menu should be added to the 3D model viewer allowing users to select which skin, out of those available on the model, should be used; it currently defaults to using the first entry.

The skin selected by the user should also be used when passing the object over to an exporter.

This feature is not something that was available in the original version, but since the infrastructure is already in-place, it is borderline trivial to implement pending UI layout.

DB2 Reader Interface

For a number of features, we require access and reading capabilities of DB2 files from the game files.

Implement 'Maps' tab

Implement a 'Maps' tab which displays a list of available maps read from relative DB2 file. The list should include both the name of the map as well as the internal name. This may require additions to the listbox component to support cleanly.

Enable/disable all geosets buttons

It is currently not possible to enable/disable all geosets in one click. Implement something allowing this.

Local file conversion for MDX

Something that was added into WET recently was the ability to locally convert MDX files. This feature should be ported over, exposed via the drag-and-drop interface.

Add placeholder texture for missing/replaceable textures

Models which have replaceable or missing textures are rendering transparent instead of using a placeholder texture.

Selected listbox option is not persisted when changing tabs

If an option has been selected in a listbox, switching to another tab and then back again will clear the selection from the listbox. Since the preview is generally persisted through this action, it is visually awkward to lose it.

Map Overview

Instead of having to select individual tiles from a list, we want a map widget that shows a complete overlay of the selected map split into tiles. Users can select tiles from the map which are then exported when requested.

3D Model Viewer

A simple 3D model viewer needs to be created which allows previewing of our internal 3D format. The component should including basic controls such as mouse movement and zooming.

Error: Invalid locale set in configuration, defaulting to enUS

For unknown reasons, setting the locale to Chinese (1024) counts as an invalid locale. Potentially the case for all non enUS localizations.

M2 Geoset Control

M2 models are comprised of individual meshes known as geosets. The user should be able to toggle which of these is shown in the preview (exporters need to respect this too).

Source selection

• Clicking on 'Open Local Installation', users should be prompted with a file dialog that allows them to select a directory. This directory needs to be validate to contain a valid .build.info file.
• Clicking 'Use Blizzard CDN' needs to download the CDNS and Versions configuration files from the patch server.
• Users need to be prompted to select a product when opening a local or remote installation. This selection should appear within the same clicked panel.
• When selecting a region server, the downloaded CDNS file needs to be parsed. Loop through all available CDNs and find the first available one.

CASC Initialization

• Download updated listfile and cache on disk. Remote location of the listfile should be something the user can configure.
• Filter and index listfile for optimized access of certain extensions.
• Build a shared CASC interface that works seamlessly for local and remote sources.

Add encryption error detection to all file readers

Rudimentary support for detecting files that fail to decrypt due to a missing decryption key has been added to the M2 loader. Better underlying support for this error should be added to the BLTE class and detected by all preview/exporters with an informational error.

Previewing textures files has broken since BLP data URL changes

After changing some inner workings of how the BLPFile class transfers data through a base64 data URL, the preview feature of the textures tab appears to have broken.

3D Model Exporting

Models need to be exportable in a variety of formats, specifically WaveFront OBJ to start with. An internal 3D format will be created which an exporter should then translate to external files in the requested format.

Scrolling while selecting a multi-range on a listbox causes produces invalid selections

Steps to reproduce:

• Select an item on a listbox.
• Scroll up or down any number of items.
• Hold SHIFT and attempt to select a range.

Local CASC installations won't load in (release) win-x64 builds

The obscure error in question is

[15:40:17] Failed to load CASC: ReferenceError: core is not defined
    at Object.load (chrome-extension://hjnamambhlpahhgnmkmdplebghbjncpm/src/app.js:1:46959)
    at async chrome-extension://hjnamambhlpahhgnmkmdplebghbjncpm/src/app.js:1:51307
    at async Object.block (chrome-extension://hjnamambhlpahhgnmkmdplebghbjncpm/src/app.js:1:12301) {
  [stack]: 'ReferenceError: core is not defined\n' +
    '    at Object.load (chrome-extension://hjnamambhlpahhgnmkmdplebghbjncpm/src/app.js:1:46959)\n' +
    '    at async chrome-extension://hjnamambhlpahhgnmkmdplebghbjncpm/src/app.js:1:51307\n' +
    '    at async Object.block (chrome-extension://hjnamambhlpahhgnmkmdplebghbjncpm/src/app.js:1:12301)',
  [message]: 'core is not defined'
}

Doesn't happen in debug builds.

Decryption keys are read incorrectly from encrypted BLTE chunks

As per the title, decryption key names are read incorrectly from encrypted BLTE chunks. Each hexadecimal character in the string needs reversing.

Finalize cross-platform support

wow.export was originally built with cross-platform support in mind, however due to 99% of the potential users being on Windows, the demand for it wasn't worth the development time of finishing it.

While we're not distributing builds for platforms other than Windows, the support for building them has still been implemented in the form of osx-x64 and linux-x64 build targets.

node ./build.js osx-x64 linux-x64

This issue tracks current known issues regarding non-Windows builds which must be resolved before we start providing them pre-built to users.

• linux-x64 does not run on Ubuntu as it's detected as a "shared application library".
• osx-x64 has not been tested and will likely explode.
• updater/updater.js needs support for process monitoring on non-Windows platforms.
• The updater application has not been tested on non-Windows platforms.

Rename 'Sounds' section to 'Audio'

'Sounds' implies sound files, but music is also included. Rename the section to 'Audio' to be more correct.