daemonengine/crunch

Advanced DXTc texture compression and transcoding library and tool, upgraded with Unity improvements, added features, extended system and hardware support, deeply tested.

crunch's Issues

Small change to make it compatible with Unity again

See this post:
https://forum.unity.com/threads/crunch-textures.522787/#post-4282222

If you change that field to 1, Unity won't complain about an old crunch version.
https://github.com/DaemonEngine/crunch/blob/master/inc/crnlib.h#L229

I tested that change with your version locally and it works for me. I can load the crn files at runtime.

crunch is known to not properly grok all png format variants

I don't have samples to provide at this time, but crunch is known to not properly grok all png format variants. So, while this bug is not fixed, it's better to convert png to another format (example: tga) before trying to convert them with crunch.

the help of crunch says "-grayscalsampling" without e

Reported by julius on Xonotic mapping channel:

the help of crunch says "-grayscalsampling" when the correct parm seems to be "-grayscalesampling"

known macOS fixes to pull

I noticed there are some macOS-related fixes in that fork: https://github.com/xuewuli/crunch

Those fixes are mixed with other related features so some manpower is required to extract the fixes we may want before merging them.

Currently, the crn lib (the part integrated in the engine) is known to build on macOS, but the crunch tool is known to not build yet, so we may be interested in some of these fixes.

I open that issue to not forget about those fixes and their availability.

Speed is much slower than with modified (not by me) BinomialLLC 2018 version

I've decided to test compressing speed (with default settings in both cases) and it was at least 3 times slower. Compiled latest version with VS2017 x64 build. I've attached both version (that I've compiled and old one) so anybody can test it by yourself.
crunch.zip
P.S. It seems that problem happened coz I was using debug version. Release version has same speed like modified BinomialLLC 2018. Reuploaded file with release version.

How to properly compile/decompile DirectDraw Surface [L8] files?

Hi, after compile from dds to crn and back this https://workupload.com/file/rdn35kWyDTp image not the same and looks like inverted. I've tried different settings but with no luck. Is it possible to make this tool to proccess such files by default without problems plz?

Crunch fails to open files on NFS and SSHFS network file systems

Crunch fails to open files on NFS and SSHFS network file systems.

I verified it on both macOS and FreeBSD with NFS, and on Linux with SSHFS:

$ pwd
/home/illwieckz/Mapping/Unvanquished/UnvanquishedAssets/src/tex-pk01_src.dpkdir/textures/shared_pk01_src

$ file door01a_d.png
door01a_d.png: PNG image data, 256 x 512, 8-bit/color RGB, non-interlaced

$ crunch -file door01a_d.png -out door01a_d.crn
crunch: Advanced DXTn Texture Compressor (Unity format variant)

Brought to you by:
- 2014-2022 Daemon Developers and contributors
  https://github.com/DaemonEngine/crunch
- 2017-2018 Alexander Suvorov and Unity Software Inc.
  https://github.com/Unity-Technologies/crunch/tree/unity
- 2010-2017 Richard Geldreich, Jr. and Binomial LLC and contributors
  https://github.com/BinomialLLC/crunch

crnlib version v1.04U x64 Built from git-1479e08

Warning: No files found: /usr/home/illwieckz/Mapping/Unvanquished/UnvanquishedAssets/src/tex-pk01_src.dpkdir/textures/shared_pk01_src/door01a_d.png

While it works on local filesystem:

$ cp -av door01a_d.png /tmp
door01a_d.png -> /tmp/door01a_d.png

$ crunch -file /tmp/door01a_d.png -out /tmp/door01a_d.crn
crunch: Advanced DXTn Texture Compressor (Unity format variant)

Brought to you by:
- 2014-2022 Daemon Developers and contributors
  https://github.com/DaemonEngine/crunch
- 2017-2018 Alexander Suvorov and Unity Software Inc.
  https://github.com/Unity-Technologies/crunch/tree/unity
- 2010-2017 Richard Geldreich, Jr. and Binomial LLC and contributors
  https://github.com/BinomialLLC/crunch

crnlib version v1.04U x64 Built from git-1479e08

Reading source texture: "/tmp/door01a_d.png"
Texture successfully loaded in 0.006s
Source texture: 256x512, Levels: 1, Faces: 1, Format: R8G8B8
Apparent type: 2D map, Flags: R G B Non-Flipped 
Generating mipmaps using filter "kaiser"
Generated 9 mipmap levels in 0.030s
Writing DXT1 texture to file: "/tmp/door01a_d.crn"
Compressing using quality level 128
Processing: 100%              
Texture successfully written in 0.091s
Texture successfully processed in 0.122s
Input texture: 256x512, Levels: 1, Faces: 1, Format: R8G8B8
Input pixels: 131072, Input file size: 219476, Input bits/pixel: 13.396
Output texture: 256x512, Levels: 10, Faces: 1, Format: DXT1
Output pixels: 174763, Output file size: 32467, Output bits/pixel: 1.486

Total time: 0.128s
1 total file(s) successfully processed, 0 file(s) skipped, 0 file(s) failed.

Exit status: 0

We can notice the absolute file path is properly computed anyway.

On Windows the crunch command line tool properly open files stored on native CIFS network share, so that looks specific to Unix APIs and/or NFS storage.

merging to upstream

I move the issue there so we can remove the old unvanquished fork.

There was some PR made on BinomialLLC's side but they were not merged:

• BinomialLLC#12
• BinomialLLC#13
  • BinomialLLC#25 by someone else was merged instead
• BinomialLLC#14

Since they officially stated they don't support crunch any more, it's likely to never happens.

Then Unity people took the code and improved it a lot, so I replayed the commits on their tree with hope they will upstream them, I also added another commit to fix an issue they introduced:

• Unity-Technologies#2
• Unity-Technologies#3
• Unity-Technologies#5
• Unity-Technologies#7
• Unity-Technologies#8
• Unity-Technologies#9
• Unity-Technologies#10
• Unity-Technologies#13

Basically the commits fix compilation on 64 bit, fix compilation on Linux, fix build outside of Visual Studio, and add an improvement: to renormalize on the top of mip-level too using the -rtopmip option with the --renormalize one, allow user-defined CRND_ASSERT, fix malloc on macOS and FreeBSD.

On two patches that may attempt to do the same things, what's better?

On February 16 of 2014 @gimhael committed the patch 4a63c5a named “Linux64 compilation fixes.” doing this:

-  size_t new_capacity = min_new_capacity;
+  ptr_bits_t new_capacity = min_new_capacity;
   if ((grow_hint) && (!math::is_power_of_2(new_capacity)))
     new_capacity = math::next_pow2(new_capacity);

I submitted this patch as PR BinomialLLC#13 to @BinomialLLC on July 23 of 2017 in hope to get it upstreamed but the patch never got merged neither the submission got an answer for unknown reason. This PR is tracked on our side in #2.

I notice today that @richgel999 merged on October 9 of 2020 the PR BinomialLLC#25 with the patch 3a445a9 by @griffin2000 committed on November 13 of 2018 and named “Avoid ambiguity error on GCC” that seems to touch the same code, but doing different things instead:

       size_t new_capacity = min_new_capacity;
-      if ((grow_hint) && (!math::is_power_of_2(new_capacity)))
-         new_capacity = math::next_pow2(new_capacity);
+	  if ((grow_hint) && (!math::is_power_of_2((uint64)new_capacity)))
+		  new_capacity = math::next_pow2((uint64)new_capacity);

1. Are the two patches meant to fix the same bug?
2. Which implementation is the best one?

Don't use fast math

See:

• Unity-Technologies#16 by @blaztinn

He said:

Without fast math we get the same output for the same input on all the platforms and architectures.

Otherwise the shasum of output was different on different machines that compressed the same input.

So we may want to disable fast-math, and maybe even pass -nofast-math to be 100% sure some environment variables doesn't introduce it again. The related patch only affects the legacy Makefile so someone has to implement it in CMake as well.

crn_decomp.h(598): Assertion failure: "i < m_size" on crn files created with Unity 5

crn.zip
I've attached crn file and very old version of crunch tool that allows to unpack (crn2dds) such file but it's useless for new version of Unity so plz add such files support in current version if possible.

Another fix for an issue affecting the encoder on macOS

See:

• Unity-Technologies#14 by @blaztinn

The commit 4bb735f7966eedf414aa8e24533967cffd5bbcdb only fixed the threading limit on the decoder part. Use the limit also in the encoder.

Without this change we got random black squares on converted textures when compressed on Macos with 24 threads.

The issue was not reproducible on Windows but I've added the limit there also in case it is a race issue that is harder to trigger on Windows.

@ArctypeZach you may also be interested in testing that so we can merge it.

Keep some branches for history

There is some branch that are kept to help to understand the history:

• unity branch is on-par with current Unity's unity branch;
• binomiallc branch is on-par with BinomialLLC's master branch before Unity added their commit;
• googlecode branch is the very old BinomialLLC's tree from Google Code repository;
• unvanquished branch is on par with current Unvanquished's master branch (some custom commits on the very old branch from Google Code repository).

The master branch merged unity branch on master then the cherry-picked patches from the unvanquished branch plus some more fix (to fix compilation on 64 bit, to fix compilation on linux, and to fix build outside of Visual Studio) and improvements (to renormalize on the top of mip-level too using the -rtopmip option with the --renormalize one).

Note that BinomialLLC officially stated they don't support crunch anymore, so it's really better to follow Unity at this point.