inwc3/jmpq3

Native Java mpq archive library

jmpq3's Issues

Allow building mpqs without a listfile

if all blocks can be identified by provided listfiles or similar.
I added some log for now to tell the user that the DefaultListFile was used and rebuilding isn't supported.

just warning for `insertByteArray` when listFile contains filename

JMpqEditor.java line 650 and 675

Make JMPQ mpqs work with ENT and MMH

can i use Multithreading read w3x or write?

can i use Multithreading read w3x or write?

Make keepHeaderOffset configurable and correct

@DrSuperGood
https://github.com/inwc3/JMPQ3/blob/master/src/main/java/systems/crigges/jmpq3/JMpqEditor.java#L95

It's always true and cannot be changed by the user. Just changing it doesn't work properly though, because the headerOffset value isn't changed when the offset shall not be kept.

https://github.com/inwc3/JMPQ3/blob/master/src/main/java/systems/crigges/jmpq3/JMpqEditor.java#L701

Nothing important, but for rebuilding protected mpqs this will be required I think.

insertFile parameters confusion + vague error

JMPQEditor.java\insertFile:

javadoc "dest" parameter does not match actual method parameter name
"dest" should rather indicate the target path inside the mpq
the method should rather take the harddisk source file first and then the mpq destination
is String as archive path specificator better than File/Path? Path btw is the successor to File
while we are at it, give option for any InputStream like you did with extractFile?

MpqFile.java\writeFileAndBlock: Files.readAllBytes is caught and muted with vague "Internal JMpq Error", IO errors should be extraverted

NegativeArraySizeException

We get the following exception when trying to extract the w3u data from a map in Wurst:

java.lang.NegativeArraySizeException
	at systems.crigges.jmpq3.MpqFile.getSectorAsByteArray(MpqFile.java:334)
	at systems.crigges.jmpq3.MpqFile.extractToOutputStream(MpqFile.java:116)
	at systems.crigges.jmpq3.MpqFile.extractToBytes(MpqFile.java:75)
	at systems.crigges.jmpq3.JMpqEditor.extractFileAsBytes(JMpqEditor.java:519)

(19:34:05) Frotty: bekomem so was bei ner recht großen asia map
(19:34:25) Frotty: kommt immer negativearraysize wenn ich bei runmap die auswähle
(19:34:27) Frotty: andere maps gehen
(19:34:41) Frotty: hast ne idee?
(19:36:35) Frotty: https://mega.nz/#!CZw0xIrR!CevOolg-bcSPuAONFM56brs4Ovd9QSl2yuOMUXmN15c
(19:36:37) Frotty: das ist die map

@Frotty do you want to look at it? At the moment I don't have the nerves to debug the mpq library ;)

JMPQ-Version: com.github.inwc3:JMPQ3:1.5.3

sometime can not save w3x!

java.nio.BufferOverflowException
at java.base/java.nio.DirectByteBuffer.put(DirectByteBuffer.java:410)
at java.base/java.nio.ByteBuffer.put(ByteBuffer.java:906)
at systems.crigges.jmpq3.MpqFile.writeFileAndBlock(MpqFile.java:348)
at systems.crigges.jmpq3.MpqFile.writeFileAndBlock(MpqFile.java:277)
at systems.crigges.jmpq3.JMpqEditor.close(JMpqEditor.java:799)
at systems.crigges.jmpq3.JMpqEditor.close(JMpqEditor.java:705)
at systems.crigges.jmpq3.JMpqEditor.close(JMpqEditor.java:701)

Remove empty space from mpq table when saving

Which hopefully fixes the issues with ENT and MMH.
Comparison:

Listfile null exception

When closing an mpq, JMpqEditor.calcNewTableSize may throw NullException on listFile since apparently it can be null at this point. The constructor tries to check if there is a listfile and extracts it but may fail in doing so, which is caught and listFile is set/remains null. Also the whole writing part in the JMPQ.close method will be unnecessarily done if you do not add files to the mpq.

optimize performance

Use multithreading to perform compression algorithms！

Now the maps are all 500MB.

export MpqFile from ENCRYPTED Block

i made this code ago. but i have a mpq file with all encrypted block.

JMPQ3/src/main/java/systems/crigges/jmpq3/JMpqEditor.java

Line 627 in 849f0fb

public MpqFile getMpqFileByBlock(BlockTable.Block block) throws IOException {

i skip the if . and the start is negative. on this line

JMPQ3/src/main/java/systems/crigges/jmpq3/MpqFile.java

Line 105 in 849f0fb

buf.position(start);

the mpq file is a big one. size 1.6GB

how to open jmpq3.jar

file is not running my pc is windows7
i used by cmd
file is not open

i want really use this software

No MPQ archive in file

Hi,

this map fools the parser with fake MPQ headers into thinking it's not an MPQ archive (but it can be played): https://www.hiveworkshop.com/threads/green-td-pro-deathmatch.319693/

Can we fix it?

java.lang.IllegalArgumentException: Size exceeds Integer.MAX_VALUE at extractAllFiles

I want to extract all files from StarCraft II/Campaigns/Liberty.SC2Campaign/base.SC2Assets ( Wings of liberty) size 3,5G.
I got the following Exception:

Exception in thread "main" java.lang.IllegalArgumentException: Size exceeds Integer.MAX_VALUE
at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:869)
at systems.crigges.jmpq3.JMpqEditor.searchHeader(JMpqEditor.java:166)
at systems.crigges.jmpq3.JMpqEditor.(JMpqEditor.java:126)
at systems.crigges.jmpq3test.Main.main(Main.java:30)

update gradle

http://stackoverflow.com/questions/26281466/eclipse-gradle-plugin-could-not-fetch-model-of-type

also it uses your local paths in .settings\gradle.prefs

edit: Also the properties like GROUP referenced in build.gradle are missing. Should there be a pom.xml?

Allow saving of MPQs without rebuilding

As a user, I would like to be able to save an MPQ Archive without having to rebuild the entire archive so that I can avoid data loss from incomplete listfiles and to improve the performance of saving the archive.

(to be clear: I intend to code this)

WoW mpqs not fully supported

Hello,
I'm trying out this library to open archive for World of Warcraft Vanilla. However, it seems not all the files are correctly read from the archives.
Is there something different ?
Thanks,
Warkdev

Modularize the library

Hello guys,

Do you have any plans to modularize this library with the Java 9 JPMS system ?

Regards,

Warkdev

Cannot handle file attributes

E.g. load, then close war3.mpq
The attributes file and section are gone and wc3 will not load.

How to rewrite w3x file?

Hi all!
I'm trying to add files as in the instructions and after I open it with any mpq editor, these files don't appear, and if I open it again in java or export, then the files are there.

MPQ file manipulation tests do not check if files were manipulated.

The tests do not check if any files were manipulated. When there was an error in the file key generation algorithm the result was that all tests passed because there were no files in any of the MPQs. Reading files from the test MPQs should verify they exist after extraction and also that the extracted content matches a sha-1 hash of the expected content.

Handling of incompressible sectors/files incomplete

Right now it only seems to work if the file is either 0 bytes, or compressible.
Incompressible sectors produce an exception and in case of incompressible files, they would still carry the COMPRESSED flag.

Tempfile swamp

JMPQ creates tons of temp files and does not clean up them partially respectively temp files are difficult to begin with when the thread/application crashes/ends abruptly. Can it not rather use a fixed folder and refresh it on initialization or bestow the folder choice upon the user as he/she may want to use multiple instances of JMPQ in parallel.

bad header size

Hi,

ran JMPQ on a larger sample of maps. 59 out of 857 could not be parsed due to a reported bad header size. Can we fix it?

Example: Forest Defense 0.21g https://www.hiveworkshop.com/threads/forest-defense-0-21g.300554/

Add Readme

Attempting to open a file that does not exist creates that file unnecessarily

When JMPQ3 attempts to open a file that does not exist, it creates a 0 byte file with the name that was attempted to be opened.

This is due to the use of FileChannel: https://docs.oracle.com/javase/7/docs/api/java/nio/channels/FileChannel.html#open(java.nio.file.Path,%20java.nio.file.OpenOption...)

Therefore as a user, I'd like the program to check for the existence of the file and throw an exception immediately before attempting to open it so that I don't get files created when I try to open a file that doesn't exist.

cant get a wav file from w3x file

cant get a wav file from w3x file

File not properly released after JMpqEditor.close

After having used an mpq file, the path is still somewhat locked because of the FileChannel.map method.

see http://stackoverflow.com/questions/2972986/how-to-unmap-a-file-from-memory-mapped-using-filechannel-in-java

Add command line functionality

specifying mpqs, listfiles, input/output dir etc.