hxim / paq8px Goto Github PK
View Code? Open in Web Editor NEWPAQ8PX compression archiver
PAQ8PX compression archiver
PAQ8PX README --------- COPYRIGHT --------- Copyright (C) 2008 Matt Mahoney, Serge Osnach, Alexander Ratushnyak, Bill Pettis, Przemyslaw Skibinski, Matthew Fite, wowtiger, Andrew Paterson, Jan Ondrus, Andreas Morphis, Pavel L. Holoborodko, Kaido Orav, Simon Berger, Neill Corlett, Márcio Pais, Andrew Epstein, Mauro Vezzosi, Zoltán Gotthardt, Sebastian Lehmann, Moisés Cardona, Byron Knoll We would like to express our gratitude for the endless support of many contributors who encouraged PAQ8PX development with ideas, testing, compiling, debugging: LovePimple, Skymmer, Darek, Stephan Busch, m^2, Christian Schneider, pat357, Rugxulo, Gonzalo, a902cd23, pinguin2, Luca Biondi - and many others. ------- LICENCE ------- This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details at Visit <http://www.gnu.org/copyleft/gpl.html>. For a summary of your rights and obligations in plain language visit https://tldrlegal.com/license/gnu-general-public-license-v2 ----- ABOUT ----- PAQ is a series of experimental lossless data compression software that have gone through collaborative development to top rankings on several benchmarks measuring compression ratio (although at the expense of speed and memory usage). PAQ8PX (this branch) is one of the longest living branch of the PAQ series started by Jan Ondrus in 2009. PAQ8PX is suitable and recommended for file entropy estimation. However - being an experimental compressor - it is not recommended for production use or for file archival purposes. For history and current state of development visit https://encode.su/threads/342-paq8px ----------------- HOW DOES IT WORK? ----------------- Data compression is carried out by compressing the input file(s) bit by bit using context mixing i.e. mixing predictions from many models. For a longer (technical) explanation see the DOC file. --------------- PAQ8PX ARCHIVES --------------- A PAQ8PX archive stores a file or multiple files in a highly compressed format. You can recognize a PAQ8PX archive from its file extension. The file extension reflects the exact PAQ8PX version that created the archive. You may also examine the first few bytes of the file. If it reads "paq8px" then it is (most probably) a PAQ8PX archive. Exact version information cannot be inferred from the archive content. A PAQ8PX archive may contain multiple files, but once created, you cannot add to or remove files from the archive. In single file mode (i.e. when compressing just one file) only file contents are stored. File paths, file names, creation or modification dates, file attributes, permissions and other metadata are not preserved. In multiple file mode however, you may store such metadata in the archive (see the help screen for more details). Notes on pre-training: (1) The exe pre-training function (see the command line flag "e") uses the paq8px.exe file itself to pre-train the exe prediction model, so expect the resulting archives to be different created by two different paq8px.exe executables - even when built with the same command line options from the same source. When the exe is different, the predictions will be slightly different, thus the archives will be (completely) different. As a result you must have the very executable that created an archive in order to extract its contents when you use this pre-training option. (2) If you use the text pre-training function (see the command line flag "t"), the word list (english.dic) and expression list (english.exp) files are only used for pre-training the prediction model before the actual file compression would begin. However their content is not stored in the archive. Since these files are required to pre-train the model before decompression as well, you will need to have them when extracting files from your archive. More notes: Files and archives larger than 2 GB are not (yet) supported. PAQ8PX archives are not compatible with previous or future PAQ8PX releases. ---------- HOW TO USE ---------- This software is portable, you don't need to install it. You may copy paq8px.exe to the folder where your files to be compressed are located and launch PAQ8PX from the command line (cmd.exe in Windows, a terminal of your choice in Linux, macOS and Android). ---------------------- COMMAND LINE INTERFACE ---------------------- A graphical user interface is not provided, PAQ8PX runs from the command line. See the help screen for command line options. For the help screen just launch PAQ8PX from the command line without any options. ----------- SOURCE CODE ----------- https://github.com/hxim -------------- HOW TO COMPILE -------------- Building PAQ8PX requires a C++17 capable C++ compiler: https://en.cppreference.com/w/cpp/compiler_support#cpp17 Windows If you are a Windows user you don't need to compile the source. Just grab the latest executable from the https://encode.su/threads/342-paq8px thread. If you would like to build an executable yourself you may use the Visual Studio solution file or in case of Mingw-w64 see the "build-mingw-w64-generic-publish.cmd" batch file in the build subfolder. Linux/macOS gcc/clang users on Linux/macOS may use the following commands to build: sudo apt-get install build-essential zlib1g-dev cmake make cd build cmake .. make The following compilers were tested and verified to compile/work correctly in this release or in an earlier release: - Visual Studio 2017 Community Edition (Windows) - Visual Studio 2019 Community Edition (Windows) verified @v207fix1 - MinGW-w64 7.1.0 (Windows) - MinGW-w64 7.2.0 (Windows) - MinGW-w64 8.1.0 (Windows) verified @v187fix1 - MinGW-w64 9.3.0 (Windows) verified @v207fix1 - GCC 7.3.0 (macOS) - GCC 7.2.0 (Ubuntu) - GCC 8.3.0 (Lubuntu) verified @v187fix1 - GCC 9.3.0 (Ubuntu) verified @v207fix1 - clang 8.0.0-3 (Lubuntu) verified @v187fix1 - clang 10.0.0-4 (ubuntu) verified @v207fix1 - aarch64-linux-gnu-gcc 9.4.0 (Ubuntu) @v207fix1 Note: We build and test 64-bit releases, 32-bit releases are seldom built or tested. A known limitation of 32-bit releases is the 2 GB memory barrier. As a consequence, compression and decompression with 32-bit releases may not work ("out of memory") on level 8 and above. -------------- FOR DEVELOPERS -------------- When you make a new release: - Please update the version number in the "Versioning" section in the paq8px.cpp source file. - Please append a short description of your modifications to the CHANGELOG file. - If you publish an executable, always publish a static build (link the required library files statically). - Always publish a build for the general public (e.g. don't use -march=native). - Before publishing your build, please carry out some basic tests. Run your tests with asserts on (remove the "NDEBUG" preprocessor directive).
cmake ..
-- The C compiler identification is GNU 9.3.0
-- The CXX compiler identification is GNU 9.3.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found ZLIB: /usr/lib/x86_64-linux-gnu/libz.so (found version "1.2.11")
CMake Error at CMakeLists.txt:40 (add_executable):
Syntax error in cmake code at
/home/slava/Загрузки/paq8px-master/CMakeLists.txt:40
when parsing string
model\BlockModel.cpp
Invalid character escape '\B'.
CMake Error at CMakeLists.txt:40 (add_executable):
Syntax error in cmake code at
/home/slava/Загрузки/paq8px-master/CMakeLists.txt:40
when parsing string
model\ContextModelBlock.cpp
Invalid character escape '\C'.
-- Configuring incomplete, errors occurred!
See also "/home/slava/Загрузки/paq8px-master/build/CMakeFiles/CMakeOutput.log".
Hi! I'm not very experienced in building software myself and the thread in the README is outdated. When I try to compile using VS19 I get the following error:
1>LINK : warning LNK4098: defaultlib 'LIBCMT' conflicts with use of other libs; use /NODEFAULTLIB:library
1>ContextModel.obj : error LNK2001: unresolved external symbol "public: static unsigned char const * const DECAlpha::rel_op" (?rel_op@DECAlpha@@2QBEB)
1>ContextModel.obj : error LNK2001: unresolved external symbol "public: static enum DECAlpha::InstructionFormat const * const DECAlpha::formats" (?formats@DECAlpha@@2QBW4InstructionFormat@1@B)
1>paq8px.obj : error LNK2001: unresolved external symbol "public: static void __cdecl DECAlpha::Unshuffle(unsigned int &)" (?Unshuffle@DECAlpha@@SAXAEAI@Z)
1>paq8px.obj : error LNK2001: unresolved external symbol "public: static void __cdecl DECAlpha::Shuffle(unsigned int &)" (?Shuffle@DECAlpha@@SAXAEAI@Z)
1>paq8px.obj : error LNK2001: unresolved external symbol "public: static bool __cdecl DECAlpha::IsValidInstruction(unsigned int)" (?IsValidInstruction@DECAlpha@@SA_NI@Z)
1>C:\Users\snack\Downloads\paq8px-master\paq8px-master\x64\Release\paq8px.exe : fatal error LNK1120: 5 unresolved externals
1>Done building project "paq8px.vcxproj" -- FAILED.
I'm gonna assume that this is because of missing dependencies. Please add either more detailed build instructions or link a new build. Thank you!
It is possible to add the function to compress a folder like the version 124. Thank's.
If you recommend the use of this project in production, is there a Python 3.7 interface that can be had for it? Thanks.
I'm currently getting two errors. First is with cmake at build-linux.sh:
CMake Error: Parse error in command line argument: NDEBUG
Should be: VAR:type=value
Then if just using "cmake -G "Unix Makefiles" ..", I get (Debian12 aarch64):
cc1plus: error: unknown value ‘nocona’ for ‘-march’
cc1plus: note: valid arguments are: armv8-a armv8.1-a armv8.2-a armv8.3-a armv8.4-a armv8.5-a armv8.6-a armv8.7-a armv8.8-a armv8-r armv9-a native
Properly adjusting the compiler flags paq8px works as expect in a Raspberry Pi.
Hi,
Readme says that for windows I should just download the .exe file and run it in the folder where my files are. However I cannot find any .exe, neither here nor in kaitz's. Is there another way to install/ simple build on windows, perhaps with python?
Thanks a lot!
Hi! Do you think you could post an example of using this compression directly on a char array? Something along the lines of: compress(char* buffer, int size, int compression_mode, int compression_type);
Thank you so much!
Hi again,
In order to optimize compression of small texts (and may be some email headers with no random data or keys, just words, separators and ascii numbers) for English, Spanish and Portuguese, what would be the best approach? Can I insert some dictionaries to paq8px?
Hi all,
I'd like to use paq8px for email compression before over-the-air transmission on a production system. Do you think it might be safe? Today I use xz, but paq8px gives me significant gains, sometimes 20% more compression.
Hello~. It seems the paq8qx project performs so excellent in compression ratio. But I want to implement it into
compress big data(.e.g., 1GB ) text compress. And the process speed is kB/s, which is too slow to practice in real.
Could you so nicely to give me some advises in how to optimize the speed in text compress?
Ideally 32 and 64 bit variants
here's a profile from https://software.intel.com/content/www/us/en/develop/tools/vtune-profiler.html
measured on an intel platform (avx2 build, gcc 9.2.0)
hopefully, it might be useful for someone to investigate where the bottlenecks are
paq8px.exe -8 data.dat // default CM
overview
paq8px.exe -8l data.dat // default CM
overview (ltsm)
paq8px.exe -8 data.dat // default CM
tree
paq8px.exe -8 data.dat // default CM
tree (ltsm)
What is the maximum file size supported? Afaik it's mentioned nowhere.
Is any of these warnings worrying?
/home/rafael2k/files/rhizomatica/hermes/paq8px/ContextMap.cpp:22:27: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
22 | p[1 + ((c >> 5) & 1)] = 1 + ((c >> 4) & 1);
| ^
/home/rafael2k/files/rhizomatica/hermes/paq8px/HashElementForContextMap.hpp:7:11: note: at offset 0 to object ‘bitState’ with size 1 declared here
7 | uint8_t bitState; // state of bit0 (1st slot) / state of bit2 (2nd slot) / state of bit5 (in slot 2)
| ^
/home/rafael2k/files/rhizomatica/hermes/paq8px/ContextMap.cpp:23:27: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
23 | p[3 + ((c >> 4) & 3)] = 1 + ((c >> 3) & 1);
| ^
/home/rafael2k/files/rhizomatica/hermes/paq8px/HashElementForContextMap.hpp:7:11: note: at offset 0 to object ‘bitState’ with size 1 declared here
7 | uint8_t bitState; // state of bit0 (1st slot) / state of bit2 (2nd slot) / state of bit5 (in slot 2)
| ^
/home/rafael2k/files/rhizomatica/hermes/paq8px/ContextMap.cpp:26:27: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
26 | p[1 + ((c >> 2) & 1)] = 1 + ((c >> 1) & 1);
| ^
/home/rafael2k/files/rhizomatica/hermes/paq8px/HashElementForContextMap.hpp:7:11: note: at offset 0 to object ‘bitState’ with size 1 declared here
7 | uint8_t bitState; // state of bit0 (1st slot) / state of bit2 (2nd slot) / state of bit5 (in slot 2)
| ^
/home/rafael2k/files/rhizomatica/hermes/paq8px/ContextMap.cpp:27:27: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
27 | p[3 + ((c >> 1) & 3)] = 1 + (c & 1);
| ^
/home/rafael2k/files/rhizomatica/hermes/paq8px/HashElementForContextMap.hpp:7:11: note: at offset 0 to object ‘bitState’ with size 1 declared here
7 | uint8_t bitState; // state of bit0 (1st slot) / state of bit2 (2nd slot) / state of bit5 (in slot 2)
| ^
Using built-in specs.
COLLECT_GCC=g++
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/5/lto-wrapper
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu 5.4.0-6ubuntu116.04.5' --with-bugurl=file:///usr/share/doc/gcc-5/README.Bugs --enable-languages=c,ada,c++,java,go,d,fortran,objc,obj-c++ --prefix=/usr --program-suffix=-5 --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-vtable-verify --enable-libmpx --enable-plugin --with-system-zlib --disable-browser-plugin --enable-java-awt=gtk --enable-gtk-cairo --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-5-amd64/jre --enable-java-home --with-jvm-root-dir=/usr/lib/jvm/java-1.5.0-gcj-5-amd64 --with-jvm-jar-dir=/usr/lib/jvm-exports/java-1.5.0-gcj-5-amd64 --with-arch-directory=amd64 --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --enable-objc-gc --enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu16.04.5)
Thread model: posix
gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1
root@641d9216354d:/paq8px# g++ paq8px.cpp -DUNIX -static -o paq8px
paq8px.cpp:617:0: warning: "UNIX" redefined
#define UNIX //to compile for Unix, Linux, Solairs, MacOS / Darwin, etc)
^
:0:0: note: this is the location of the previous definition
paq8px.cpp:955:25: error: macro "putc" requires 2 arguments, but only 1 given
virtual void putc(U8 c) = 0;
^
paq8px.cpp:956:67: error: macro "putc" requires 2 arguments, but only 1 given
void append(const char* s) { for (int i = 0; s[i]; i++)putc(s[i]); }
^
paq8px.cpp:960:41: error: macro "putc" requires 2 arguments, but only 1 given
void put32(U32 x){putc((x >> 24) & 255); putc((x >> 16) & 255); putc((x >> 8) & 255); putc(x & 255);}
^
paq8px.cpp:960:64: error: macro "putc" requires 2 arguments, but only 1 given
void put32(U32 x){putc((x >> 24) & 255); putc((x >> 16) & 255); putc((x >> 8) & 255); putc(x & 255);}
^
paq8px.cpp:960:86: error: macro "putc" requires 2 arguments, but only 1 given
void put32(U32 x){putc((x >> 24) & 255); putc((x >> 16) & 255); putc((x >> 8) & 255); putc(x & 255);}
^
paq8px.cpp:960:101: error: macro "putc" requires 2 arguments, but only 1 given
void put32(U32 x){putc((x >> 24) & 255); putc((x >> 16) & 255); putc((x >> 8) & 255); putc(x & 255);}
^
paq8px.cpp:979:17: error: macro "putc" requires 2 arguments, but only 1 given
void putc(U8 c) { fputc(c, file); }
^
paq8px.cpp:1051:17: error: macro "putc" requires 2 arguments, but only 1 given
void putc(U8 c) {
^
paq8px.cpp:1061:27: error: macro "putc" requires 2 arguments, but only 1 given
(file_on_disk).putc(c);
^
paq8px.cpp:7017:47: error: macro "putc" requires 2 arguments, but only 1 given
if (mode==COMPRESS) archive->putc(x2>>24);
^
paq8px.cpp:7036:22: error: macro "putc" requires 2 arguments, but only 1 given
archive->putc(c);
^
paq8px.cpp:7088:25: error: macro "putc" requires 2 arguments, but only 1 given
archive->putc(x1>>24); // Flush first unequal byte of range
^
paq8px.cpp:7876:35: error: macro "putc" requires 2 arguments, but only 1 given
out->putc((blockresidual>>8)&255);
^
paq8px.cpp:7877:31: error: macro "putc" requires 2 arguments, but only 1 given
out->putc(blockresidual &255);
^
paq8px.cpp:7940:18: error: macro "putc" requires 2 arguments, but only 1 given
out->putc(g);
^
paq8px.cpp:7941:30: error: macro "putc" requires 2 arguments, but only 1 given
out->putc(skipRGB?r:g-r);
^
paq8px.cpp:7942:30: error: macro "putc" requires 2 arguments, but only 1 given
out->putc(skipRGB?b:g-b);
^
paq8px.cpp:7944:55: error: macro "putc" requires 2 arguments, but only 1 given
for (int j=0; j<width%3; j++) out->putc(in->getc());
^
paq8px.cpp:7957:32: error: macro "putc" requires 2 arguments, but only 1 given
out->putc(skipRGB?r:b-r);
^
paq8px.cpp:7958:20: error: macro "putc" requires 2 arguments, but only 1 given
out->putc(b);
^
paq8px.cpp:7959:32: error: macro "putc" requires 2 arguments, but only 1 given
out->putc(skipRGB?g:b-g);
^
paq8px.cpp:7971:34: error: macro "putc" requires 2 arguments, but only 1 given
out->putc(en.decompress());
^
paq8px.cpp:7990:18: error: macro "putc" requires 2 arguments, but only 1 given
out->putc(g);
^
paq8px.cpp:7991:30: error: macro "putc" requires 2 arguments, but only 1 given
out->putc(skipRGB?r:g-r);
^
paq8px.cpp:7992:30: error: macro "putc" requires 2 arguments, but only 1 given
out->putc(skipRGB?b:g-b);
^
paq8px.cpp:7993:18: error: macro "putc" requires 2 arguments, but only 1 given
out->putc(a);
^
paq8px.cpp:7995:55: error: macro "putc" requires 2 arguments, but only 1 given
for (int j=0; j<width%4; j++) out->putc(in->getc());
^
paq8px.cpp:8008:32: error: macro "putc" requires 2 arguments, but only 1 given
out->putc(skipRGB?r:b-r); out->putc(b); out->putc(skipRGB?g:b-g); out->putc(a);
^
paq8px.cpp:8008:46: error: macro "putc" requires 2 arguments, but only 1 given
out->putc(skipRGB?r:b-r); out->putc(b); out->putc(skipRGB?g:b-g); out->putc(a);
^
paq8px.cpp:8008:72: error: macro "putc" requires 2 arguments, but only 1 given
out->putc(skipRGB?r:b-r); out->putc(b); out->putc(skipRGB?g:b-g); out->putc(a);
^
paq8px.cpp:8008:86: error: macro "putc" requires 2 arguments, but only 1 given
out->putc(skipRGB?r:b-r); out->putc(b); out->putc(skipRGB?g:b-g); out->putc(a);
^
paq8px.cpp:8021:34: error: macro "putc" requires 2 arguments, but only 1 given
out->putc(en.decompress());
^
paq8px.cpp:8092:42: error: macro "putc" requires 2 arguments, but only 1 given
if (mode==FDECOMPRESS) out->putc(c[5]);
^
paq8px.cpp:8241:29: error: macro "putc" requires 2 arguments, but only 1 given
out->putc(diffCount[index]);
^
paq8px.cpp:8242:19: error: macro "putc" requires 2 arguments, but only 1 given
out->putc(window);
^
paq8px.cpp:8243:18: error: macro "putc" requires 2 arguments, but only 1 given
out->putc(index);
^
paq8px.cpp:8249:77: error: macro "putc" requires 2 arguments, but only 1 given
for (int i=0; i<diffCount[index]; i++) out->putc(diffByte[indexLIMIT+i+1]);
^
paq8px.cpp:8316:57: error: macro "putc" requires 2 arguments, but only 1 given
if (mode==FDECOMPRESS) out->putc(diffByte[diffIndex]);
^
paq8px.cpp:8468:23: error: macro "putc" requires 2 arguments, but only 1 given
out->putc(hdrsize>>8);
^
paq8px.cpp:8469:24: error: macro "putc" requires 2 arguments, but only 1 given
out->putc(hdrsize&255);
^
paq8px.cpp:8470:18: error: macro "putc" requires 2 arguments, but only 1 given
out->putc(bsize);
^
paq8px.cpp:8471:24: error: macro "putc" requires 2 arguments, but only 1 given
out->putc(clearpos>>8);
^
paq8px.cpp:8472:25: error: macro "putc" requires 2 arguments, but only 1 given
out->putc(clearpos&255);
^
paq8px.cpp:8473:21: error: macro "putc" requires 2 arguments, but only 1 given
out->putc(codesize);
^
paq8px.cpp:8509:61: error: macro "putc" requires 2 arguments, but only 1 given
if (code==maxcode+1) { if (phase==1) out->putc(j); else diffpos++; }
^
paq8px.cpp:8536:23: error: macro "putc" requires 2 arguments, but only 1 given
out->putc(hdrsize>>8);
^
paq8px.cpp:8537:24: error: macro "putc" requires 2 arguments, but only 1 given
out->putc(hdrsize&255);
^
paq8px.cpp:8538:22: error: macro "putc" requires 2 arguments, but only 1 given
out->putc(255-bsize);
^
paq8px.cpp:8539:30: error: macro "putc" requires 2 arguments, but only 1 given
out->putc((clearpos>>8)&255);
^
paq8px.cpp:8540:25: error: macro "putc" requires 2 arguments, but only 1 given
out->putc(clearpos&255);
^
paq8px.cpp:8575:44: error: macro "putc" requires 2 arguments, but only 1 given
if (mode==FDECOMPRESS) out->putc(codesize);
^
paq8px.cpp:8602:37: error: macro "putc" requires 2 arguments, but only 1 given
if (mode==FDECOMPRESS) out->putc(0);
^
paq8px.cpp:8794:57: error: macro "putc" requires 2 arguments, but only 1 given
if (mode==FDECOMPRESS) out->putc(en.decompress());
^
paq8px.cpp:9068:22: error: macro "putc" requires 2 arguments, but only 1 given
archive->putc(0);
^
paq8px.cpp:9069:97: error: macro "putc" requires 2 arguments, but only 1 given
archive->putc(level | (trainEXE << 4) | (trainTXT << 5) | (adaptive << 6) | (skipRGB << 7));
^
paq8px.cpp: In member function 'void ProgramChecker::print() const':
paq8px.cpp:736:131: warning: format '%u' expects argument of type 'unsigned int', but argument 3 has type 'long long unsigned int' [-Wformat=]
printf("Time %1.2f sec, used %I64u MB (%I64u bytes) of memory\n",double(clock()-start_time)/CLOCKS_PER_SEC,maxmem>>20,maxmem);
^
paq8px.cpp:736:131: warning: format '%u' expects argument of type 'unsigned int', but argument 4 has type 'U64 {aka long long unsigned int}' [-Wformat=]
paq8px.cpp: In member function 'void Array<T, Align>::create(U32)':
paq8px.cpp:812:15: error: 'uintptr_t' was not declared in this scope
data=(T*)(((uintptr_t)ptr+pad) & ~(uintptr_t)pad);
^
paq8px.cpp:812:25: error: expected ')' before 'ptr'
data=(T*)(((uintptr_t)ptr+pad) & ~(uintptr_t)pad);
^
paq8px.cpp:812:52: error: expected ')' before ';' token
data=(T*)(((uintptr_t)ptr+pad) & ~(uintptr_t)pad);
^
paq8px.cpp: At global scope:
paq8px.cpp:950:21: warning: defaulted and deleted functions only available with -std=c++11 or -std=gnu++11
virtual ~File() = default;
^
paq8px.cpp:955:29: error: variable or field 'putc' declared void
virtual void putc(U8 c) = 0;
^
paq8px.cpp:979:37: error: variable or field 'putc' declared void
void putc(U8 c) { fputc(c, file); }
^
paq8px.cpp:979:37: error: expected ';' at end of member declaration
paq8px.cpp:1018:51: warning: non-static data member initializers only available with -std=c++11 or -std=gnu++11
const U32 MAX_RAM_FOR_TMP_CONTENT = 64 * 1024 * 1024; //64 MB (per file)
^
paq8px.cpp:1062:3: error: variable or field 'putc' declared void
}
^
paq8px.cpp:1062:3: error: expected ';' at end of member declaration
paq8px.cpp:1031:3: error: looser throw specifier for 'virtual FileTmp::~FileTmp()'
~FileTmp() {close();}
^
paq8px.cpp:950:11: error: overriding 'virtual File::~File() throw ()'
virtual ~File() = default;
^
paq8px.cpp: In copy constructor 'StationaryMap::StationaryMap(const StationaryMap&)':
paq8px.cpp:797:3: error: 'Array<T, Align>::Array(const Array<T, Align>&) [with T = unsigned int; int Align = 16]' is private
Array(const Array&); // no copy or assignment
^
paq8px.cpp:2082:7: error: within this context
class StationaryMap {
^
paq8px.cpp: In function 'void im24bitModel(Mixer&, int, int, int)':
paq8px.cpp:3684:146: note: synthesized method 'StationaryMap::StationaryMap(const StationaryMap&)' first required here
static StationaryMap Map[nMaps] = {12, 12, 12, 12, 10, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 0};
^
paq8px.cpp:2087:3: note: after user-defined conversion: StationaryMap::StationaryMap(int, int, int)
StationaryMap(int BitsOfContext, int BitsPerContext = 8, int Rate = 0): Data(1<<(BitsOfContext+BitsPerContext)), Context(0), Mask(1<<BitsPerContext), bCount(1), B(1) {
^
paq8px.cpp: In function 'void wavModel(Mixer&, int, ModelStats*)':
paq8px.cpp:5125:23: warning: extended initializer lists only available with -std=c++11 or -std=gnu++11
static Array pr{32}; // [3][2]
^
paq8px.cpp:5125:27: error: in C++98 'pr' must be initialized by constructor, not by '{...}'
static Array pr{32}; // [3][2]
^
paq8px.cpp:5126:22: warning: extended initializer lists only available with -std=c++11 or -std=gnu++11
static Array n{2};
^
paq8px.cpp:5126:24: error: in C++98 'n' must be initialized by constructor, not by '{...}'
static Array n{2};
^
paq8px.cpp:5127:28: warning: extended initializer lists only available with -std=c++11 or -std=gnu++11
static Array counter{2};
^
paq8px.cpp:5127:30: error: in C++98 'counter' must be initialized by constructor, not by '{...}'
static Array counter{2};
^
paq8px.cpp:5128:25: warning: extended initializer lists only available with -std=c++11 or -std=gnu++11
static Array F{49492}; //[49][49][2]
^
paq8px.cpp:5128:33: error: in C++98 'F' must be initialized by constructor, not by '{...}'
static Array F{49492}; //[49][49][2]
^
paq8px.cpp:5129:25: warning: extended initializer lists only available with -std=c++11 or -std=gnu++11
static Array L{4949}; //[49][49]
^
paq8px.cpp:5129:31: error: in C++98 'L' must be initialized by constructor, not by '{...}'
static Array L{4949}; //[49][49]
^
paq8px.cpp: In member function 'void exeModel::Train()':
paq8px.cpp:5993:17: error: 'MAX_PATH' was not declared in this scope
char filename[MAX_PATH+1];
^
paq8px.cpp:5994:34: error: 'filename' was not declared in this scope
if ((i=GetModuleFileName(NULL, filename,MAX_PATH)) && i<=MAX_PATH){
^
paq8px.cpp:5994:51: error: 'GetModuleFileName' was not declared in this scope
if ((i=GetModuleFileName(NULL, filename,MAX_PATH)) && i<=MAX_PATH){
^
paq8px.cpp: In member function 'void ContextModel::Train()':
paq8px.cpp:6794:17: error: 'MAX_PATH' was not declared in this scope
char filename[MAX_PATH+12];
^
paq8px.cpp:6795:34: error: 'filename' was not declared in this scope
if ((i=GetModuleFileName(NULL, filename,MAX_PATH)) && i<=MAX_PATH){
^
paq8px.cpp:6795:51: error: 'GetModuleFileName' was not declared in this scope
if ((i=GetModuleFileName(NULL, filename,MAX_PATH)) && i<=MAX_PATH){
^
paq8px.cpp: In member function 'int Encoder::code(int)':
paq8px.cpp:7017:36: error: 'class File' has no member named 'putc'
if (mode==COMPRESS) archive->putc(x2>>24);
^
paq8px.cpp: In member function 'void Encoder::compress(int)':
paq8px.cpp:7036:16: error: 'class File' has no member named 'putc'
archive->putc(c);
^
paq8px.cpp: In member function 'void Encoder::flush()':
paq8px.cpp:7088:14: error: 'class File' has no member named 'putc'
archive->putc(x1>>24); // Flush first unequal byte of range
^
paq8px.cpp: In function 'void encode_cd(File*, File*, U64, int)':
paq8px.cpp:7876:8: error: 'class File' has no member named 'putc'
out->putc((blockresidual>>8)&255);
^
paq8px.cpp:7877:8: error: 'class File' has no member named 'putc'
out->putc(blockresidual &255);
^
paq8px.cpp: In function 'void encode_bmp(File*, File*, U64, int)':
paq8px.cpp:7940:12: error: 'class File' has no member named 'putc'
out->putc(g);
^
paq8px.cpp:7941:12: error: 'class File' has no member named 'putc'
out->putc(skipRGB?r:g-r);
^
paq8px.cpp:7942:12: error: 'class File' has no member named 'putc'
out->putc(skipRGB?b:g-b);
^
paq8px.cpp:7944:40: error: 'class File' has no member named 'putc'
for (int j=0; j<width%3; j++) out->putc(in->getc());
^
paq8px.cpp: In function 'U64 decode_bmp(Encoder&, U64, int, File*, FMode, U64&)':
paq8px.cpp:7957:14: error: 'class File' has no member named 'putc'
out->putc(skipRGB?r:b-r);
^
paq8px.cpp:7958:14: error: 'class File' has no member named 'putc'
out->putc(b);
^
paq8px.cpp:7959:14: error: 'class File' has no member named 'putc'
out->putc(skipRGB?g:b-g);
^
paq8px.cpp:7971:14: error: 'class File' has no member named 'putc'
out->putc(en.decompress());
^
paq8px.cpp: In function 'void encode_im32(File*, File*, U64, int)':
paq8px.cpp:7990:12: error: 'class File' has no member named 'putc'
out->putc(g);
^
paq8px.cpp:7991:12: error: 'class File' has no member named 'putc'
out->putc(skipRGB?r:g-r);
^
paq8px.cpp:7992:12: error: 'class File' has no member named 'putc'
out->putc(skipRGB?b:g-b);
^
paq8px.cpp:7993:12: error: 'class File' has no member named 'putc'
out->putc(a);
^
paq8px.cpp:7995:40: error: 'class File' has no member named 'putc'
for (int j=0; j<width%4; j++) out->putc(in->getc());
^
paq8px.cpp: In function 'U64 decode_im32(Encoder&, U64, int, File*, FMode, U64&)':
paq8px.cpp:8008:14: error: 'class File' has no member named 'putc'
out->putc(skipRGB?r:b-r); out->putc(b); out->putc(skipRGB?g:b-g); out->putc(a);
^
paq8px.cpp:8008:40: error: 'class File' has no member named 'putc'
out->putc(skipRGB?r:b-r); out->putc(b); out->putc(skipRGB?g:b-g); out->putc(a);
^
paq8px.cpp:8008:54: error: 'class File' has no member named 'putc'
out->putc(skipRGB?r:b-r); out->putc(b); out->putc(skipRGB?g:b-g); out->putc(a);
^
paq8px.cpp:8008:80: error: 'class File' has no member named 'putc'
out->putc(skipRGB?r:b-r); out->putc(b); out->putc(skipRGB?g:b-g); out->putc(a);
^
paq8px.cpp:8021:14: error: 'class File' has no member named 'putc'
out->putc(en.decompress());
^
paq8px.cpp: In function 'U64 decode_exe(Encoder&, U64, File*, FMode, U64&)':
paq8px.cpp:8092:33: error: 'class File' has no member named 'putc'
if (mode==FDECOMPRESS) out->putc(c[5]);
^
paq8px.cpp: In function 'int encode_zlib(File*, File*, U64, int&)':
paq8px.cpp:8241:8: error: 'class File' has no member named 'putc'
out->putc(diffCount[index]);
^
paq8px.cpp:8242:8: error: 'class File' has no member named 'putc'
out->putc(window);
^
paq8px.cpp:8243:8: error: 'class File' has no member named 'putc'
out->putc(index);
^
paq8px.cpp:8249:47: error: 'class File' has no member named 'putc'
for (int i=0; i<diffCount[index]; i++) out->putc(diffByte[indexLIMIT+i+1]);
^
paq8px.cpp: In function 'int decode_zlib(File, U64, File*, FMode, U64&)':
paq8px.cpp:8316:33: error: 'class File' has no member named 'putc'
if (mode==FDECOMPRESS) out->putc(diffByte[diffIndex]);
^
paq8px.cpp: In function 'int encode_gif(File*, File*, U64, int&)':
paq8px.cpp:8468:8: error: 'class File' has no member named 'putc'
out->putc(hdrsize>>8);
^
paq8px.cpp:8469:8: error: 'class File' has no member named 'putc'
out->putc(hdrsize&255);
^
paq8px.cpp:8470:8: error: 'class File' has no member named 'putc'
out->putc(bsize);
^
paq8px.cpp:8471:8: error: 'class File' has no member named 'putc'
out->putc(clearpos>>8);
^
paq8px.cpp:8472:8: error: 'class File' has no member named 'putc'
out->putc(clearpos&255);
^
paq8px.cpp:8473:8: error: 'class File' has no member named 'putc'
out->putc(codesize);
^
paq8px.cpp:8509:55: error: 'class File' has no member named 'putc'
if (code==maxcode+1) { if (phase==1) out->putc(j); else diffpos++; }
^
paq8px.cpp:8536:8: error: 'class File' has no member named 'putc'
out->putc(hdrsize>>8);
^
paq8px.cpp:8537:8: error: 'class File' has no member named 'putc'
out->putc(hdrsize&255);
^
paq8px.cpp:8538:8: error: 'class File' has no member named 'putc'
out->putc(255-bsize);
^
paq8px.cpp:8539:8: error: 'class File' has no member named 'putc'
out->putc((clearpos>>8)&255);
^
paq8px.cpp:8540:8: error: 'class File' has no member named 'putc'
out->putc(clearpos&255);
^
paq8px.cpp: In function 'int decode_gif(File*, U64, File*, FMode, U64&)':
paq8px.cpp:8575:31: error: 'class File' has no member named 'putc'
if (mode==FDECOMPRESS) out->putc(codesize);
^
paq8px.cpp:8602:31: error: 'class File' has no member named 'putc'
if (mode==FDECOMPRESS) out->putc(0);
^
paq8px.cpp: In function 'void transform_encode_block(Filetype, File*, U64, Encoder&, int, char*, int, float, float, U64)':
paq8px.cpp:8671:68: warning: format '%u' expects argument of type 'unsigned int', but argument 2 has type 'U64 {aka long long unsigned int}' [-Wformat=]
printf("Transform fails at %I64u, skipping...\n", diffFound-1);
^
paq8px.cpp: In function 'void compressRecursive(File*, U64, Encoder&, char*, int, float, float)':
paq8px.cpp:8731:150: warning: unknown conversion type character 'I' in format [-Wformat=]
printf(" %-11s | %-16s |%10I64u bytes [%I64u - %I64u]",blstr,typenames[(type==ZLIB && (info>>24)>=PNG8)?info>>24:type],len,begin,detected_end-1);
^
paq8px.cpp:8731:150: warning: format '%u' expects argument of type 'unsigned int', but argument 4 has type 'U64 {aka long long unsigned int}' [-Wformat=]
paq8px.cpp:8731:150: warning: format '%u' expects argument of type 'unsigned int', but argument 5 has type 'U64 {aka long long unsigned int}' [-Wformat=]
paq8px.cpp:8731:150: warning: too many arguments for format [-Wformat-extra-args]
paq8px.cpp: In function 'void compress(const char*, U64, Encoder&)':
paq8px.cpp:8762:76: warning: format '%u' expects argument of type 'unsigned int', but argument 2 has type 'U64 {aka long long unsigned int}' [-Wformat=]
printf("Compressed from %I64u to %I64u bytes.\n",filesize,en.size()-start);
^
paq8px.cpp:8762:76: warning: format '%u' expects argument of type 'unsigned int', but argument 3 has type 'U64 {aka long long unsigned int}' [-Wformat=]
paq8px.cpp: In function 'U64 decompressRecursive(File*, U64, Encoder&, FMode, int)':
paq8px.cpp:8794:37: error: 'class File' has no member named 'putc'
if (mode==FDECOMPRESS) out->putc(en.decompress());
^
paq8px.cpp: In function 'void decompress(const char*, U64, Encoder&)':
paq8px.cpp:8827:45: warning: format '%u' expects argument of type 'unsigned int', but argument 3 has type 'U64 {aka long long unsigned int}' [-Wformat=]
printf(" %s %I64u -> ", filename, filesize);
^
paq8px.cpp:8832:63: warning: format '%u' expects argument of type 'unsigned int', but argument 2 has type 'U64 {aka long long unsigned int}' [-Wformat=]
else if (mode==FCOMPARE && r) printf("differ at %I64u\n",r-1);
^
paq8px.cpp: In function 'int putsize(String&, String&, const char*, int)':
paq8px.cpp:8861:34: warning: format '%u' expects argument of type 'unsigned int', but argument 3 has type 'U64 {aka long long unsigned int}' [-Wformat=]
sprintf(blk, "%I64u\t", len);
^
paq8px.cpp: In function 'int main(int, char**)':
paq8px.cpp:9068:16: error: 'class File' has no member named 'putc'
archive->putc(0);
^
paq8px.cpp:9069:16: error: 'class File' has no member named 'putc'
archive->putc(level | (trainEXE << 4) | (trainTXT << 5) | (adaptive << 6) | (skipRGB << 7));
^
paq8px.cpp:9102:48: warning: format '%u' expects argument of type 'unsigned int', but argument 2 has type 'U64 {aka long long unsigned int}' [-Wformat=]
printf("\nFile list (%I64u bytes)\n", len);
^
paq8px.cpp:9113:75: warning: format '%u' expects argument of type 'unsigned int', but argument 2 has type 'U64 {aka long long unsigned int}' [-Wformat=]
printf("Compressed from %I64u to %I64u bytes.\n",len,en.size()-start);
^
paq8px.cpp:9113:75: warning: format '%u' expects argument of type 'unsigned int', but argument 3 has type 'U64 {aka long long unsigned int}' [-Wformat=]
paq8px.cpp:9156:87: warning: format '%u' expects argument of type 'unsigned int', but argument 5 has type 'long long unsigned int' [-Wformat=]
printf("\n%d/%d Filename: %s (%I64u bytes)\n", i+1, files, fname[i], fsize[i]);
^
paq8px.cpp:9160:87: warning: format '%u' expects argument of type 'unsigned int', but argument 2 has type 'U64 {aka long long unsigned int}' [-Wformat=]
printf("\nTotal %I64u bytes compressed to %I64u bytes.\n", total_size, en.size());
^
paq8px.cpp:9160:87: warning: format '%u' expects argument of type 'unsigned int', but argument 3 has type 'U64 {aka long long unsigned int}' [-Wformat=]
Text block is not detected if data is like TEXT-DEFAULT-(...)
I have this issue on 2 very different devices (one amd64 one i686), both running debian testing.
build$ cmake ..
-- The C compiler identification is GNU 12.2.0
-- The CXX compiler identification is GNU 12.2.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found ZLIB: /usr/lib/i386-linux-gnu/libz.so (found version "1.2.13")
-- IPO / LTO enabled
-- Configuring done
-- Generating done
-- Build files have been written to: build
build$ make
[ 1%] Building CXX object CMakeFiles/paq8px.dir/ProgramChecker.cpp.o
...
[ 98%] Building CXX object CMakeFiles/paq8px.dir/StateMapPair.cpp.o
[100%] Linking CXX executable paq8px
during GIMPLE pass: graphite
during GIMPLE pass: graphite
SmallStationaryContextMap.cpp: In member function ‘__ct_base .constprop’:
text/EnglishStemmer.cpp: In member function ‘isVowel’:
SmallStationaryContextMap.cpp:3:1: internal compiler error: Ungültiger Maschinenbefehl
3 | SmallStationaryContextMap::SmallStationaryContextMap(const Shared* const sh, const int bitsOfContext, const int inputBits, const int rate, const int scale) :
| ^
text/EnglishStemmer.cpp:526:6: internal compiler error: Ungültiger Maschinenbefehl
526 | bool EnglishStemmer::isVowel(const char c) {
| ^
0x94d3bec internal_error(char const*, ...)
???:0
0xb7cc0042 isl_ctx_alloc_with_options
???:0
0xb7cc0272 isl_ctx_alloc
???:0
0x9362f86 graphite_transform_loops()
???:0
Please submit a full bug report, with preprocessed source (by using -freport-bug).
Please include the complete backtrace with any bug report.
See <file:///usr/share/doc/gcc-12/README.Bugs> for instructions.
make[3]: *** [/tmp/ccKDW8Wc.mk:35: /tmp/ccA5LC32.ltrans11.ltrans.o] Fehler 1
make[3]: *** Es wird auf noch nicht beendete Prozesse gewartet....
0x94d3bec internal_error(char const*, ...)
???:0
0xb7cc0042 isl_ctx_alloc_with_options
???:0
0xb7cc0272 isl_ctx_alloc
???:0
0x9362f86 graphite_transform_loops()
???:0
Please submit a full bug report, with preprocessed source (by using -freport-bug).
Please include the complete backtrace with any bug report.
See <file:///usr/share/doc/gcc-12/README.Bugs> for instructions.
make[3]: *** [/tmp/ccKDW8Wc.mk:26: /tmp/ccA5LC32.ltrans8.ltrans.o] Fehler 1
lto-wrapper: fatal error: make returned 2 exit status
compilation terminated.
/usr/bin/ld: error: lto-wrapper failed
collect2: error: ld returned 1 exit status
make[2]: *** [CMakeFiles/paq8px.dir/build.make:1586: paq8px] Fehler 1
make[1]: *** [CMakeFiles/Makefile2:83: CMakeFiles/paq8px.dir/all] Fehler 2
make: *** [Makefile:91: all] Fehler 2
On Ubuntu, it builds fine. So I suspect a bug in the debian variant of the link-time optimizer. But I have 0 experience with all of this so I hope you know what to do next.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.