Page 1 of 1

pbzx writing truncated output in some cases

PostPosted: Mon Feb 26, 2018 4:47 am
by ryandesign
Hello, I'm writing to report what I believe is a problem in pbzx version 20170731. I think it may not be properly processing the last chunk of data, because in some cases the resulting output file is truncated.

I'm using pbzx to extract the xip files that Apple uses to distribute Xcode 8 and later on the Apple developer site. First, the Content pbzx file and Metadata plist are extracted from the xip file:

Code: Select all
xar -xf Xcode8.3.2.xip

Then the Content file is decompressed into a cpio archive using pbzx:

Code: Select all
pbzx < Content > Content.cpio

Then the inside the Content.cpio archive is extracted:

Code: Select all
gnucpio -i < Content.cpio

This method has worked to extract several different versions of Xcode, including 8.0, 8.3.1, 8.3.2, and 9.2. gnucpio prints a message like "23702616 blocks", exits with exit status 0 to indicate all was well, and all the files appear to have been extracted correctly.

(I am using GNU cpio 2.12 as installed by MacPorts, instead of the version of cpio provided by macOS, because the version provided by macOS generates lots of error messages that it "Can't create" certain files, though the errors appear to be spurious since the files are in fact created.)

But I'm encountering a problem trying to extract the Xcode 8.3 or 8.3.3 xip files specifically. The cpio archive resulting from extracting the Content pbzx file inside either of those xip files appears to be truncated. It does not end with the "TRAILER!!!" that proper cpio archives end with. If I use macOS cpio to extract it, it prints the error message "cpio: (Empty error message)", exits with exit status 1, and the last file in the cpio archive -- -- is extracted incompletely (it's truncated). If I use GNU cpio instead, the error message is "gnucpio: premature end of file", the exit status is 2, and the version.plist file isn't extracted at all.

If instead I double-click the Xcode 8.3 or 8.3.3 xip files in the Finder and allow Archive Utility to extract them, they are extracted correctly, including the complete version.plist file, which makes me think that the archive is not corrupted, but that pbzx isn't reading it correctly.

Re: pbzx writing truncated output in some cases

PostPosted: Thu Mar 15, 2018 5:40 pm
by ryandesign
I see that you released a new version of pbzx.c on March 10, 2018, but it does not appear to fix this issue.

Re: pbzx writing truncated output in some cases

PostPosted: Tue Nov 12, 2019 5:43 am
by ryandesign
I see that you have released a new version of ota, which includes pbzx functionality, but that "It's simple enough to add a "pbzx only" feature, to just extract - I leave that as an exercise to the reader". So essentially you no longer offer the pbzx tool that you used to offer. And I can't build your old pbzx tool anymore, because the pbzx.c on your site (from March 19, 2018) is not compatible with the 02_decompress.c on your site (from August 7, 2019). So what should one do now if one simply wants to decompress pbxz data, like one could do with your old pbzx tool?

Re: pbzx writing truncated output in some cases

PostPosted: Thu Nov 14, 2019 7:40 am
by morpheus
Let me get on it and restore the pbzx per your problem report, while I'm at it.