Giter Site home page Giter Site logo

Comments (18)

fcorbelli avatar fcorbelli commented on May 20, 2024

The stdin is rewritted into 56.2
You can find the pre release here
www.francocorbelli.it/zpaqfranz.cpp

This version include a full size dd (!) for Windows to make images of drives (with admin privileges of course) creating a batch file
It seems to work
Almost no error handling etc

zpaqfranz a d:\thebackup.zpaq c: -dd

The very last version (not released yet) image drive without dd and stdin, reading directly disk sectors

from zpaqfranz.

fcorbelli avatar fcorbelli commented on May 20, 2024

Ps even with version 56.1 should work (slow)
BUT
you must give a name for the stdin stream

Something | zpaqfranz a thearchive.zpaq NAMEOFSTREAM -stdin

Then into thearchive.zpaq a NAMEOFSTREAM (or whatever) file will be stored
As said 56.1 use a full on byte at time (aka should work everytime,but slow, about 8 MB/s)
56.2 (more complex) runs up to 270MB/s if you use something like -minsize 1MB with -dd (the bs parameter)

from zpaqfranz.

Erol-2022 avatar Erol-2022 commented on May 20, 2024

Hello Mr. Corbelli,

Thanks for your hard work. Much appreciated. Sorry if I missed the parameter specifiying the stream name.

With your help, I managed to get it work :

D:\>dd if=sample.txt | zpaqfranz.exe a output.zpaq stream -stdin
zpaqfranz v56.1j-JIT-L (HW BLAKE3), SFX64 v55.1, (15 Nov 2022)
franz:-stdin
40587: REBUILDING STDIN
40828: IMPORT FROM STDIN
Creating output.zpaq at offset 0 + 0
Adding stream from stdin with 16 threads @ 2022-11-29 12:47:45
rawwrite dd for windows version 0.6beta3.
Written by John Newbigin <[email protected]>
This program is covered by terms of the GPL Version 2.

2378+1 records in
2378+1 records out
1 +added, 0 -removed.

0 + (1.217.704 -> 1.217.700 -> 3.336) = 3.336 @ 9.29 MB/s

0.125 seconds (00:00:00) (all OK)

D:\>dir output.zpaq

 Directory of D:\

29.11.2022  12:47             3.336 output.zpaq
               1 File(s)          3.336 bytes

I will do other testings on the dd capability of zpaqfranz.

Thanks again,

Erol

from zpaqfranz.

fcorbelli avatar fcorbelli commented on May 20, 2024

As said 56.2 is MUCH faster
You can download from my italian homesite (not sure if even an updated exe is here, just try, it is safe, my own site afterall :)

from zpaqfranz.

Erol-2022 avatar Erol-2022 commented on May 20, 2024

Hello Mr. Corbelli,

Thanks, I can download the source code :

D:>wget http://www.francocorbelli.it/zpaqfranz.cpp

Any link to your website to receive the latest executable?

from zpaqfranz.

fcorbelli avatar fcorbelli commented on May 20, 2024

I am not sure, but you can try
www.francocorbelli.it/zpaqfranz.exe

I really do not remember if build script upload the exe too (I am now on smartphone)

from zpaqfranz.

Erol-2022 avatar Erol-2022 commented on May 20, 2024

Hello Mr. Corbelli,

Thanks, I downloaded the latest executable from your website but upon running, it reports :

“This version of this file is not compatible with the version of Windows you’re running”

OS : Windows 10 64-bit Version 10.0.19044.2251

I decided to build the executable with Msys2 :

wget https://github.com/fcorbelli/zpaqfranz/archive/refs/tags/55.16.tar.gz
tar xvf 55.16.tar.gz zpaqfranz-55.16/
cd zpaqfranz-55.16/
mv zpaqfranz.cpp zpaqfranz.cpp.original
wget http://www.francocorbelli.it/zpaqfranz.cpp
cd WINDOWS/
cp -p ../*.cpp .
make

After some warning messages, Msys2 managed to compile the source code :

C:/msys64/mingw64/include/winnt.h:6151: note: this is the location of the previous definition
 6151 | #define IO_REPARSE_TAG_CLOUD (__MSABI_LONG(0x9000001A))
      |
zpaqfranz.cpp:30932: warning: "IO_REPARSE_TAG_CLOUD_1" redefined
30932 | #define IO_REPARSE_TAG_CLOUD_1 0x9000101A
      |
C:/msys64/mingw64/include/winnt.h:6152: note: this is the location of the previous definition
 6152 | #define IO_REPARSE_TAG_CLOUD_1 (__MSABI_LONG(0x9000101A))
      |
zpaqfranz.cpp:30933: warning: "IO_REPARSE_TAG_CLOUD_2" redefined
30933 | #define IO_REPARSE_TAG_CLOUD_2 0x9000201A
      |
C:/msys64/mingw64/include/winnt.h:6153: note: this is the location of the previous definition
 6153 | #define IO_REPARSE_TAG_CLOUD_2 (__MSABI_LONG(0x9000201A))
      |
zpaqfranz.cpp:30934: warning: "IO_REPARSE_TAG_CLOUD_3" redefined
30934 | #define IO_REPARSE_TAG_CLOUD_3 0x9000301A
      |
C:/msys64/mingw64/include/winnt.h:6154: note: this is the location of the previous definition
 6154 | #define IO_REPARSE_TAG_CLOUD_3 (__MSABI_LONG(0x9000301A))
      |
zpaqfranz.cpp:30935: warning: "IO_REPARSE_TAG_CLOUD_4" redefined
30935 | #define IO_REPARSE_TAG_CLOUD_4 0x9000401A
      |
C:/msys64/mingw64/include/winnt.h:6155: note: this is the location of the previous definition
 6155 | #define IO_REPARSE_TAG_CLOUD_4 (__MSABI_LONG(0x9000401A))
      |
zpaqfranz.cpp:30936: warning: "IO_REPARSE_TAG_CLOUD_5" redefined
30936 | #define IO_REPARSE_TAG_CLOUD_5 0x9000501A
      |
C:/msys64/mingw64/include/winnt.h:6156: note: this is the location of the previous definition
 6156 | #define IO_REPARSE_TAG_CLOUD_5 (__MSABI_LONG(0x9000501A))
      |
zpaqfranz.cpp:30937: warning: "IO_REPARSE_TAG_CLOUD_6" redefined
30937 | #define IO_REPARSE_TAG_CLOUD_6 0x9000601A
      |
C:/msys64/mingw64/include/winnt.h:6157: note: this is the location of the previous definition
 6157 | #define IO_REPARSE_TAG_CLOUD_6 (__MSABI_LONG(0x9000601A))
      |
zpaqfranz.cpp:30938: warning: "IO_REPARSE_TAG_CLOUD_7" redefined
30938 | #define IO_REPARSE_TAG_CLOUD_7 0x9000701A
      |
C:/msys64/mingw64/include/winnt.h:6158: note: this is the location of the previous definition
 6158 | #define IO_REPARSE_TAG_CLOUD_7 (__MSABI_LONG(0x9000701A))
      |
zpaqfranz.cpp:30939: warning: "IO_REPARSE_TAG_CLOUD_8" redefined
30939 | #define IO_REPARSE_TAG_CLOUD_8 0x9000801A
      |
C:/msys64/mingw64/include/winnt.h:6159: note: this is the location of the previous definition
 6159 | #define IO_REPARSE_TAG_CLOUD_8 (__MSABI_LONG(0x9000801A))
      |
zpaqfranz.cpp:30940: warning: "IO_REPARSE_TAG_CLOUD_9" redefined
30940 | #define IO_REPARSE_TAG_CLOUD_9 0x9000901A
      |
C:/msys64/mingw64/include/winnt.h:6160: note: this is the location of the previous definition
 6160 | #define IO_REPARSE_TAG_CLOUD_9 (__MSABI_LONG(0x9000901A))
      |
zpaqfranz.cpp:30941: warning: "IO_REPARSE_TAG_CLOUD_A" redefined
30941 | #define IO_REPARSE_TAG_CLOUD_A 0x9000A01A
      |
C:/msys64/mingw64/include/winnt.h:6161: note: this is the location of the previous definition
 6161 | #define IO_REPARSE_TAG_CLOUD_A (__MSABI_LONG(0x9000A01A))
      |
zpaqfranz.cpp:30942: warning: "IO_REPARSE_TAG_CLOUD_B" redefined
30942 | #define IO_REPARSE_TAG_CLOUD_B 0x9000B01A
      |
C:/msys64/mingw64/include/winnt.h:6162: note: this is the location of the previous definition
 6162 | #define IO_REPARSE_TAG_CLOUD_B (__MSABI_LONG(0x9000B01A))
      |
zpaqfranz.cpp:30943: warning: "IO_REPARSE_TAG_CLOUD_C" redefined
30943 | #define IO_REPARSE_TAG_CLOUD_C 0x9000C01A
      |
C:/msys64/mingw64/include/winnt.h:6163: note: this is the location of the previous definition
 6163 | #define IO_REPARSE_TAG_CLOUD_C (__MSABI_LONG(0x9000C01A))
      |
zpaqfranz.cpp:30944: warning: "IO_REPARSE_TAG_CLOUD_D" redefined
30944 | #define IO_REPARSE_TAG_CLOUD_D 0x9000D01A
      |
C:/msys64/mingw64/include/winnt.h:6164: note: this is the location of the previous definition
 6164 | #define IO_REPARSE_TAG_CLOUD_D (__MSABI_LONG(0x9000D01A))
      |
zpaqfranz.cpp:30945: warning: "IO_REPARSE_TAG_CLOUD_E" redefined
30945 | #define IO_REPARSE_TAG_CLOUD_E 0x9000E01A
      |
C:/msys64/mingw64/include/winnt.h:6165: note: this is the location of the previous definition
 6165 | #define IO_REPARSE_TAG_CLOUD_E (__MSABI_LONG(0x9000E01A))
      |
zpaqfranz.cpp:30946: warning: "IO_REPARSE_TAG_CLOUD_F" redefined
30946 | #define IO_REPARSE_TAG_CLOUD_F 0x9000F01A
      |
C:/msys64/mingw64/include/winnt.h:6166: note: this is the location of the previous definition
 6166 | #define IO_REPARSE_TAG_CLOUD_F (__MSABI_LONG(0x9000F01A))
      |

The executable :

# ls -l zpaqfranz.exe
-rwxr-xr-x 1 Erol Domain Users 3979776 Nov 29 14:10 zpaqfranz.exe

# ./zpaqfranz.exe | grep v56
zpaqfranz v56.2d-JIT-L (HW BLAKE3), SFX64 v55.1, (28 Nov 2022)

I will test the new exe built with Msys2.

from zpaqfranz.

fcorbelli avatar fcorbelli commented on May 20, 2024

I confirm that the .EXE is not good
As mentioned I did not remember the script

You need to compile from the .cpp (this IS good) with something like

g++ -O3  zpaqfranz.cpp -o zpaqfranz -pthread -static

Or even

g++ -O3  zpaqfranz.cpp -o zpaqfranz 

from zpaqfranz.

Erol-2022 avatar Erol-2022 commented on May 20, 2024

Hello Mr.Corbelli,

g++ -O3  zpaqfranz.cpp -o zpaqfranz : zpaqfranz.exe

g++ -O3  zpaqfranz.cpp -o zpaqfranz -pthread -static : zpaqfranz.exe.second-build

make : zpaqfranz.exe.first-build

The make file is producing the smallest executable.

# ls -l z*.exe*
-rwxr-xr-x 1 Erol Domain Users 4267521 Nov 29 15:08 zpaqfranz.exe
-rwxr-xr-x 1 Erol Domain Users 3979776 Nov 29 14:10 zpaqfranz.exe.first-build
-rwxr-xr-x 1 Erol Domain Users 4767652 Nov 29 15:07 zpaqfranz.exe.second-build

Thanks for your help.

from zpaqfranz.

fcorbelli avatar fcorbelli commented on May 20, 2024

In the comment (of source code) you will find the suggested way to compile zpaqfranz

TARGET EXAMPLES

Windows 64 (g++ 7.3.0)
g++ -O3  zpaqfranz.cpp -o zpaqfranz 

Windows 64 (g++ 10.3.0) MSYS2
g++ -O3  zpaqfranz.cpp -o zpaqfranz -pthread -static

Windows 64 (g++, Hardware Blake3 implementation)
In this case, of course, linking the .S file is mandatory
g++ -O3 -DHWBLAKE3 blake3_windows_gnu.S zpaqfranz.cpp -o zpaqfranz -pthread -static

Windows 64 (g++, Hardware Blake3 implementation PLUS HW SHA1)
g++ -O3 -DHWBLAKE3 -DHWSHA1 blake3_windows_gnu.s zpaqfranz.cpp sha1ugo.obj -o zpaqfranzhw -pthread -static

Windows 32 (g++ 7.3.0 64 bit)
c:\mingw32\bin\g++ -m32 -O3 zpaqfranz.cpp -o zpaqfranz32 -pthread -static

Windows 64 (g++ 7.3.0), WITH cloud paq
g++ -O3 -DSERVER zpaqfranz.cpp -o zpaqfranz -lwsock32 -lws2_32

FreeBSD (11.x) gcc 7
gcc7 -O3 -march=native -Dunix zpaqfranz.cpp -lstdc++ -pthread -o zpaqfranz -static -lm

FreeBSD (12.1) gcc 9.3.0
g++ -O3 -march=native -Dunix zpaqfranz.cpp  -pthread -o zpaqfranz -static-libstdc++ -static-libgcc

FreeBSD (11.4) gcc 10.2.0
g++ -O3 -march=native -Dunix zpaqfranz.cpp  -pthread -o zpaqfranz -static-libstdc++ -static-libgcc -Wno-stringop-overflow

FreeBSD (11.3) clang 6.0.0
clang++ -march=native -Dunix zpaqfranz.cpp  -pthread -o zpaqfranz -static

OpenBSD 6.6 clang++ 8.0.1
OpenBSD 7.1 clang++ 13.0.0
clang++ -Dunix -O3 -march=native zpaqfranz.cpp -o zpaqfranz -pthread -static

Arch Linux
I strongly advise against using zpaqfranz on this Linux distro(s).
There is a bizarre policy on compiling executables.
Obviously no one forbids it (runs fine), but just don't ask for help.

Debian Linux (10/11) gcc 8.3.0
ubuntu 21.04 desktop-amd64 gcc  10.3.0
manjaro 21.07 gcc 11.1.0
g++ -O3 -Dunix zpaqfranz.cpp  -pthread -o zpaqfranz -static

QNAP NAS TS-431P3 (Annapurna AL314) gcc 7.4.0
g++ -Dunix zpaqfranz.cpp  -pthread -o zpaqfranz -Wno-psabi

Fedora 34 gcc 11.2.1
Typically you will need some library (out of a fresh Fedora box)
sudo dnf install glibc-static libstdc++-static -y;
Then you can compile, via Makefile or "by hand"
(do not forget... sudo!)

CentoOS
Please note:
"Red Hat discourages the use of static linking for security reasons. 
Use static linking only when necessary, especially against libraries provided by Red Hat. "
Therefore a -static linking is often a nightmare on CentOS => change the Makefile
g++ -O3 -Dunix zpaqfranz.cpp  -pthread -o zpaqfranz

Solaris 11.4 gcc 7.3.0
OmniOS r151042 gcc 7.5.0
Beware: -DSOLARIS and some different linking options
g++ -O3 -march=native -DSOLARIS zpaqfranz.cpp -o zpaqfranz  -pthread -static-libgcc -lkstat

MacOS 11.0 gcc (clang) 12.0.5, INTEL
MacOS 12.6 gcc (clang) 13.1.6, INTEL
Please note:
The -std=c++11 is required, otherwise you have to change half a dozen lines (or -DANCIENT). 
No -static here
"Apple does not support statically linked binaries on Mac OS X. 
(...) Rather, we strive to ensure binary 
compatibility in each dynamically linked system library and framework
(AHAHAHAHAHAH, note by me)
Warning: Shipping a statically linked binary entails a significant compatibility risk. 
We strongly recommend that you not do this..."
Short version: Apple does not like -static, so compile with
g++ -Dunix  -O3 -march=native zpaqfranz.cpp -o zpaqfranz -pthread  -std=c++11

Mac PowerPC with gcc4.x
Look at -DBIG (for BIG ENDIAN) and -DANCIENT (old-compiler)
g++ -O3 -DBIG -DANCIENT -Dunix -DNOJIT zpaqfranz.cpp -o zpaqfranz -pthread

Apple Macintosh (M1/M2)
Untested (yet), should be
g++ -Dunix  -O3 -DNOJIT zpaqfranz.cpp -o zpaqfranz -pthread  -std=c++11

ESXi (gcc 3.4.6)
Note: not fully developed ( extract() with minimum RAM need to be implemented )
g++ -O3 -DESX zpaqfranz.cpp -o zpaqfranz6  -pthread -static -s

from zpaqfranz.

fcorbelli avatar fcorbelli commented on May 20, 2024

If you want please leave a review on
https://sourceforge.net/projects/zpaqfranz/
and put a star on github (if you haven't already).
Any comment or suggestion is welcome
Thanks for collaboration

from zpaqfranz.

Erol-2022 avatar Erol-2022 commented on May 20, 2024

Hello Mr. Corbelli,

Thanks, You have a new star in your GitHub account. I will post a review to the SourceForge after creating an account.

Here is my latest report. Compiling the source code with Msys2 :

 wget https://github.com/fcorbelli/zpaqfranz/archive/refs/tags/55.16.tar.gz
 tar xvf 55.16.tar.gz
 cd WINDOWS/
 wget http://www.francocorbelli.it/zpaqfranz.cpp
 gcc --version
 gcc.exe (Rev6, Built by MSYS2 project) 12.2.0

 g++ -O3  zpaqfranz.cpp -o zpaqfranz -pthread -static

 # ls -l zpaqfranz.exe
 -rwxr-xr-x 1 Erol None 4769446 Nov 29 20:36 zpaqfranz.exe

 # ldd zpaqfranz.exe
        ntdll.dll => /c/Windows/SYSTEM32/ntdll.dll (0x776f0000)
        kernel32.dll => /c/Windows/system32/kernel32.dll (0x775d0000)
        KERNELBASE.dll => /c/Windows/system32/KERNELBASE.dll (0x7fefd260000)
        ADVAPI32.dll => /c/Windows/system32/ADVAPI32.dll (0x7feff900000)
        msvcrt.dll => /c/Windows/system32/msvcrt.dll (0x7feff750000)
        sechost.dll => /c/Windows/SYSTEM32/sechost.dll (0x7fefd6a0000)
        RPCRT4.dll => /c/Windows/system32/RPCRT4.dll (0x7fefee60000)
        SHELL32.dll => /c/Windows/system32/SHELL32.dll (0x7fefd6c0000)
        SHLWAPI.dll => /c/Windows/system32/SHLWAPI.dll (0x7fefedd0000)
        GDI32.dll => /c/Windows/system32/GDI32.dll (0x7feff170000)
        USER32.dll => /c/Windows/system32/USER32.dll (0x774d0000)
        LPK.dll => /c/Windows/system32/LPK.dll (0x7fefee50000)
        USP10.dll => /c/Windows/system32/USP10.dll (0x7fefea60000)

The static linking eliminates the extra dependencies like the new universal C run-time library.

After booting off from a Windows PE environment designed for Windows 10, I shrinked the root partition hosting the operating system to reduce the archiving time.

Operating system backup to Network drive ( WinPE system ) :

D:\zpaqfranz>zpaqfranz.exe a Z:\Win10.zpaq C: -image
zpaqfranz v56.2e-JIT-L archiver, SFX64 v55.1, (29 Nov 2022)
franz:-image
42165: REBUILDING IMAGE filename to image_c.img
Creating Z:/Win10.zpaq at offset 0 + 0
Adding drive image with 8 threads @ 2022-11-29 21:26:02
42743: opening drive c:
42751: image size 24.941.253.632 (23.23 GB)
38385: WARN expected                   4 getted      24.941.252.608 for image_c.img
1 +added, 0 -removed.

0 + (24.941.253.632 -> 18.471.624.704 -> 8.671.225.708) = 8.671.225.708 @ 28.14 MB/s

38271: HOUSTON expected 24.941.253.632, done 49.882.505.216, diff 24.941.251.584
38271: Corrupted source files? Lost connection? Cannot access? Media full?
38271: =>The updated .zpaq archive is almost certainly incompleted

845.344 seconds (000:14:05) (with errors)

Operating system backup to Ramdisk ( WinPE system ) :

D:\zpaqfranz>zpaqfranz.exe a R:\Win10.zpaq C: -image
zpaqfranz v56.2e-JIT-L archiver, SFX64 v55.1, (29 Nov 2022)
franz:-image
42165: REBUILDING IMAGE filename to image_c.img
Creating R:/Win10.zpaq at offset 0 + 0
Adding drive image with 8 threads @ 2022-11-29 22:09:32
42743: opening drive c:
42751: image size 24.941.253.632 (23.23 GB)
38385: WARN expected                   4 getted      24.941.252.608 for image_c.img
1 +added, 0 -removed.

0 + (24.941.253.632 -> 18.471.624.704 -> 8.671.219.249) = 8.671.219.249 @ 67.98 MB/s

38271: HOUSTON expected 24.941.253.632, done 49.882.505.216, diff 24.941.251.584
38271: Corrupted source files? Lost connection? Cannot access? Media full?
38271: =>The updated .zpaq archive is almost certainly incompleted

349.906 seconds (000:05:49) (with errors)

The final zpaq archive :

29.11.2022 22:15 8.671.219.249 Win10.zpaq

How to extract the zpaq archive after formatting the C partition?

D:\zpaqfranz>zpaqfranz.exe x R:\Win10.zpaq -to C:\ -space
zpaqfranz v56.2e-JIT-L archiver, SFX64 v55.1, (29 Nov 2022)
franz:Do not check free space/writeability
R:/Win10.zpaq:
1 versions, 1 files, 35.543 fragments, 1.113 blocks, 8.671.219.249 bytes (8.08 GB)
Extracting 24.941.252.608 bytes (23.23 GB) in 1 files (0 folders) with 8 threads

17519: access denied             C:\
17620: percorso does not exists <<C:\>>
        99.66% 00:00:00 (  23.15 GB) of (  23.23 GB)  171.77 MB/sec
********************************************************************************************************************
Something STRANGE happened. Archive seems corrupt. Media full?
WRITTEN BYTES       18.529.820.672
EXPECTED            18.529.886.208
********************************************************************************************************************
139.734 seconds (000:02:19) (with errors)

The image file cannot be opened with 7-Zip :

30.11.2022 01:09 24.047.497.216 image_c.img

I guess you need to add an option to extract directly the operating system backup to the drive C: ( or any other one. )

from zpaqfranz.

fcorbelli avatar fcorbelli commented on May 20, 2024

I am getting a lot of problems with windows,unable to report correctly the size of a partition (!)
Work in progress for -image
In this release I think I can fix
In the next a double buffering to speed up
Thanks for your help

from zpaqfranz.

fcorbelli avatar fcorbelli commented on May 20, 2024

How to extract the zpaq archive after formatting the C partition?

Simply, you can't :)
Imaging C partition is NOT (yet) something like Acronis or Macrium
It is (or should be) a full-backup of C: that, in case of emergency, you need to

  1. Restore (extract from .zpaq the .img)
  2. Mount with something else (es. OSFMount https://www.osforensics.com/tools/mount-disk-images.html), then copy-and-paste your files
  3. OR open the .img with 7zip (supposing NTFS format), then extract with 7zip
  4. OR write back the image with "something" (example dd) into a virtual machine, or even the "real" HW (booting from USB key, for example)

Into the future (aka: no way :) a restore of C:
56_2e.zip
The attached pre-release now have a -buffer parameter too (zpaq's default 4KB is not so good for high speed NVMes, usually 32-64KB are faster)

zpaqfranz a k:\copyofc.zpaq c: -image -buffer 1MB -key pippo

from zpaqfranz.

Erol-2022 avatar Erol-2022 commented on May 20, 2024

Hello Mr. Corbelli,

Thanks for your efforts, much appreciated. I will try your new release.

In order to avoid the additional step to restore a complete raw image, kindly could please bring back the -stdin option? A separate dd tool could do the job with the condition that you add an extra -stdout option to your tool.

from zpaqfranz.

fcorbelli avatar fcorbelli commented on May 20, 2024

The -stdin is present, as the -dd (the embedded dd)
It is unfortunately NOT possible to make a "-stdout" option, due deduplication
Or, at least, I do not know how to do :)
It is possible (the w command) only if the available RAM is bigger than the uncompressed image

from zpaqfranz.

fcorbelli avatar fcorbelli commented on May 20, 2024

BTW it is impossible because the output of zpaq is NOT streamed, but random (seeked)

Basically the archives (.zpaq files) are read sequentially, from beginning to end
The output instead is not sequential, a stream, but the individual files are gradually filled with data. This requires seeks and ultimately prevents stdout

Also, incidentally, explains why the extraction of complex archives on systems with magnetic disks is not as fast as SSDs (where seeks are much faster)

It is possible, in theory, to do the opposite
That is to create an ordered list of blocks, then randomly read (with seek) the .zpaq archive, then write output sequentially

Very complex, very slow and, actually, I don't find it particularly interesting at the moment (actually yes, but to test the archives, rather than to extract them)

from zpaqfranz.

Erol-2022 avatar Erol-2022 commented on May 20, 2024

Hello Mr. Corbelli,

Thanks for your explanation, I understand that data stored in the zpaq archives is not sequential.

Here is my new report :

Booting off from WinPE and creating a new virtual drive with Microsoft's VssSampleProvider :

https://learn.microsoft.com/en-us/windows/win32/vss/vsssampleprovider-tool-and-sample

X:\Windows\System32\Tools64\vstor64>vstorcontrol.exe install
Installing Virtual Bus Driver... Succeeded

X:\Windows\System32\Tools64\vstor64>vstorcontrol create fixeddisk -newimage c:\new.img -size 524288000
Virtual Storage v10.0.0.0

Creating backing file c:\new.img ... Succeeded
Creating drive backed by c:\new.img ...
Waiting for drive initialization...............
Erasing Drive ...Succeeded
Drive     : \\.\PhysicalDrive1
Drive ID  : {B7B739C2-73EB-11ED-B1AF-9107B3883313}
Type      : Fixed Disk
Size      : 500MB
No volumes on the drive

Formatting the new drive and copying some files and directories. The virtual drive is identified with the symbol H:

Testing the new release of zpaqfranz :

D:\zpaqfranz>zpaqfranz.exe a C:\test1.zpaq H: -image
zpaqfranz v56.2f-JIT-L (HW BLAKE3), SFX64 v55.1, (30 Nov 2022)
franz:-image
42165: REBUILDING IMAGE filename to image_h.img
42743: opening drive h:
42751: image size 521.142.272 (497.00 MB)
Creating C:/test1.zpaq at offset 0 + 0
Adding drive image with 4 threads @ 2022-12-04 19:05:39
1 +added, 0 -removed.

0 + (521.142.272 -> 56.706.598 -> 23.660.343) = 23.660.343 @ 23.56 MB/s

21.109 seconds (000:00:21) (all OK)

04.12.2022  19:06        23.660.343 test1.zpaq

Extracting the raw image file from the zpaq archive :

D:\zpaqfranz>md C:\backup

D:\zpaqfranz>zpaqfranz.exe x C:\test1.zpaq -to C:\backup
zpaqfranz v56.2f-JIT-L (HW BLAKE3), SFX64 v55.1, (30 Nov 2022)
C:/test1.zpaq:
1 versions, 1 files, 798 fragments, 6 blocks, 23.660.343 bytes (22.56 MB)
Extracting 521.138.176 bytes (497.00 MB) in 1 files (0 folders) with 4 threads
         1.08% 00:01:00 (   5.35 MB) of ( 497.00 MB)    5.35 MB/sec

2.985 seconds (000:00:02) (all OK)

04.12.2022  22:05       521.138.176 image_h.img

With 7-Zip, I can extract all the files and directories from the image file image_h.img

Removing the virtual drive :

X:\Windows\System32\Tools64\vstor64>vstorcontrol.exe remove \\.\PhysicalDrive1
Virtual Storage v10.0.0.0

Removing...
Drive     : \\.\PhysicalDrive1
Drive ID  : {B7B739C2-73EB-11ED-B1AF-9107B3883313}
Type      : Fixed Disk
Size      : 500MB

Thanks for the new release.

from zpaqfranz.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.