niklasrosenstein/pbzx

Fork of the pbzx stream parser (www.tonymacx86.com/general-help/135458-pbzx-stream-parser.html)

👋 Hi, I'm Niklas. I like Python, Kubernetes and Tooling.

pbzx's Issues

parse_args() is not parsing args correctly

For example, the following command which reads the pbzx stream from stdin

tar zxOf Xcode.xip Content | pbzx -n - | cpio -i

will result

failed to open: -

Here's my fix

diff --git a/pbzx.c b/pbzx.c
index d01ef35..bd1d066 100644
--- a/pbzx.c
+++ b/pbzx.c
@@ -38,12 +38,13 @@ struct options {
     bool noxar;    /* The input data is not a XAR archive but the pbzx Payload. */
     bool help;     /* Print usage with details and exit. */
     bool version;  /* Print version and exit. */
+    const char * filename;
 };

 /* Prints usage information and exit. Optionally, displays an error message and
  * exits with an error code. */
 static void usage(char const* error) {
-    fprintf(stderr, "usage: pbzx [-v] [-h] [-n] [-] [filename]\n");
+    fprintf(stderr, "usage: pbzx [-v] [-h] [-n] <-|filename>\n");
     if (error) {
         fprintf(stderr, "error: %s\n", error);
         exit(EINVAL);
@@ -70,20 +71,18 @@ static void version() {
 /* Parses command-line flags into the #options structure and adjusts the
  * argument count and values on the fly to remain only positional arguments. */
 static void parse_args(int* argc, char const** argv, struct options* opts) {
-    for (int i = 0; i < *argc; ++i) {
-        /* Skip arguments that are not flags. */
-        if (argv[i][0] != '-') continue;
+
+    opts->filename = NULL;
+
+    for (int i = 1; i < *argc; ++i) {
+
+        if (argv[i][0] != '-' && !opts->filename) opts->filename = argv[i];
         /* Match available arguments. */
-        if      (strcmp(argv[i], "-")  == 0) opts->stdin = true;
+        else if (strcmp(argv[i], "-")  == 0) opts->stdin = true;
         else if (strcmp(argv[i], "-n") == 0) opts->noxar = true;
         else if (strcmp(argv[i], "-h") == 0) opts->help = true;
         else if (strcmp(argv[i], "-v") == 0) opts->version = true;
-        else usage("unrecognized flag");
-        /* Move all remaining arguments to the front. */
-        for (int j = 0; j < (*argc-1); ++j) {
-            argv[j] = argv[j+1];
-        }
-        (*argc)--;
+
     }
 }

@@ -200,17 +199,23 @@ static inline size_t cpio_out(char *buffer, size_t size) {

 int main(int argc, const char** argv) {
     /* Parse and validate command-line flags and arguments. */
+
     struct options opts = {0};
+    char const* filename = NULL;
+
     parse_args(&argc, argv, &opts);
+
     if (opts.version) version();
     if (opts.help) usage(NULL);
-    if (!opts.stdin && argc < 2)
+    if (opts.stdin) {
+        fprintf(stderr, "reading <stdin>\n");
+    } else if (opts.filename) {
+        fprintf(stderr, "reading \"%s\"\n", opts.filename);
+        filename = opts.filename;
+    } else {
         usage("missing filename argument");
-    else if ((!opts.stdin && argc > 2) || (opts.stdin && argc > 1))
-        usage("unhandled positional argument(s)");
+    }

-    char const* filename = NULL;
-    if (argc >= 2) filename = argv[1];

     /* Open a stream to the payload. */
     struct stream stream;

Publish to Homebrew

Awesome project! Could we publish this to Homebrew, to make it easier to install?

binary please

Not every person on every computer has installed brew or developer tools, but is interested to extract contents from some installers. So please provide a binary.

undefined reference to `xar_extract_tostream_init'

clang -llzma -lxar pbzx.c -o pbzx
/tmp/pbzx-7ed43e.o: In function stream_open': pbzx.c:(.text+0x9f1): undefined reference to xar_extract_tostream_init'
/tmp/pbzx-7ed43e.o: In function stream_read': pbzx.c:(.text+0xaef): undefined reference to xar_extract_tostream'
/tmp/pbzx-7ed43e.o: In function stream_close': pbzx.c:(.text+0xc78): undefined reference to xar_extract_tostream_end'
clang-5.0: error: linker command

BUG: Incorrect interpretation of file format

pbzx/pbzx.c

Line 248 in bf536e1

uint64_t flags = stream_read_64(&stream);

What the code calls flags is actually the block size of uncompressed input.

pbzx/pbzx.c

Lines 257 to 266 in bf536e1

	while (flags & 1 << 24) {
	flags = stream_read_64(&stream);
	length = stream_read_64(&stream);
	char plain = (length == 0x1000000);
	stream_read(xbuf, min(XBSZ, (uint32_t) length), &stream);
	/* Validate the header. */
	if (!plain && strncmp(xbuf, "\xfd""7zXZ\0", 6) != 0) {
	fprintf(stderr, "Header is not <FD>7zXZ<00>\n");
	return 1;
	}

Here flags inside the loop is actually the size of uncompressed data, length is the size of compressed data. plain should be flags == length. Input error is not properly checked too: https://github.com/NiklasRosenstein/pbzx/blob/bf536e167f2e514866f91d7baa0df1dff5a13711/pbzx.c

According to compression_tool(1):

The file starts with a 4-byte header 'p','b','z',<algo>, where <algo> indicates the algorithm used to compress data. The header is followed by the 64-bit block size in bytes. Then for each block, we have 64-bit uncompressed size (will batch the block size, except possibly for the last block), 64-bit compressed size, and the compressed payload. If both uncompressed and compressed sizes for a block are equal, the payload is stored uncompressed. All 64-bit values are stored big-endian. Valid values for <algo> are: 'z' for zlib, 'x' for lzma, '4' for lz4, and 'e' for lzfse.

You can see my implementation of pbzx here: https://github.com/jakwings/unpkg

I don't know how you would like to structure the code, so maybe it is up to you. No big deal at present I guess unless Apple devs start using a non-default block size.

Works with HighSierra 10.13.6 and open Mojave Update 10.14.2.pkg -> Payload

First: thanks the developer for the source code.

Payload file: macOSUpd10.14.2.dmg -> macOSUpd10.14.2.pkg (2.46 GB) -> macOSUpd10.14.2.pkg (1.65 GB) -> Payload (pbzx file).

Open Payload file with Hex Fiend app and verify the first line: pbz

https://support.apple.com/kb/DL1987?viewlocale=en_US&locale=en_US

https://ridiculousfish.com/hexfiend/

I tried opening the file macOSUpd10.14.2.pkg - 2.46 GB with Pacifist 3.6.1 but gave the error:

Crashed Thread:        5  Dispatch queue: com.apple.root.default-qos

Exception Type:        EXC_CRASH (SIGABRT)
Exception Codes:       0x0000000000000000, 0x0000000000000000
Exception Note:        EXC_CORPSE_NOTIFY

Application Specific Information:
*** Terminating app due to uncaught exception 'NSFileHandleOperationException', reason: '*** -[NSConcreteFileHandle writeData:]: Invalid argument'
terminating with uncaught exception of type NSException
abort() called

1 - I extracted the macOSUpd10.14.2.pkg - 2.46 GB with The Unarchiver.app 4.0.0 and extracted the following files.

https://itunes.apple.com/br/app/the-unarchiver/id425424353?l=en&mt=12

SecureBoot.pkg
macOSUpd10.14.2.RecoveryHDUpdate.pkg
macOSUpd10.14.2.pkg (1.65 GB)
macOSBrain.pkg
FullBundleUpdate.pkg
FirmwareUpdate.pkg
EmbeddedOSFirmware.pkg
Distribution

2 - Now the files that are inside macOSUpd10.14.2.pkg (1.65 GB) can be seen in the Finder -> right button mouse -> Show Package Contents -> copy Payload file to another folder/directory.

3 - Install Macports (need XCode installed). My system is HighSierra: MacPorts-2.5.4-10.13-HighSierra.pkg and reboot Mac OS system.

https://guide.macports.org/#installing.macports

4 - Update Macports - Finder -> Applications -> Utilities -> Terminal
sudo port selfupdate

5 - Install XZ (lzma.h) - /opt/local/include/lzma.h
sudo port install xz

6 - Install XAR (xar.h) - /opt/local/include/xar/xar.h
sudo port install xar

7 - Compile pbzx.c - source code in /Downloads/pbzx-master/pbzx.c
cd Downloads/pbzx-master/
clang -llzma -lxar -I /opt/local/include pbzx.c -o pbzx

8 - Copy Payload file to /Downloads/pbzx-master/ .

Executable files that are not in the system path need to be executed by putting before ". /" - Unix rules.
./pbzx -n Payload | cpio -i

9 - The following directories with their respective files will be created inside directory/folder /Downloads/pbzx-master/

Applications - 352,6 MB
bin - 4,9 MB
Library - 78,9 MB
private - 630 KB
sbin - 2,4 MB
System - 4,93 GB
usr - 528,2 MB