OTA saga continues - a followup to the followup

A simple script to recreate the iOS/TvOS filesystem on your Mac

Jonathan Levin, (@Morpheus______), http://newosxbook.com/ - 02/17/2016

About

If you've read the first part of this and the second part, you know by now that part II had a nice set of tools which worked very well on iOS OTA files, along with a shell script to retrieve the base file from a jailbroken device (over SSH), and then apply the bxpatch on it

So what changed?

For reasons known only to the Cupertino folk, they now offer FULL OTAs - that is, not differential updates necessitating a base prerequisitve version, but rather a huge archive with everything in it. This entirely defeats the purpose of encrypting the filesystem in an IPSW. It was already an issue that the GID key is inaccessible during OTA updates, at least with diffs one had to have the base system. All of a sudden, and actually not long after my two articles, Apple started putting the full OTAs out. No idea why. But hey - thanks guys!

Now, you might say, big deal - since we always have SOME version of iOS jailbroken, even in 64-bit (and 32-bit is not an issue, thanks to @Xerub and @iH8sn0w - you rock, guys!). Not to mention the boot loader components are IMG3/IMG4 and encrypted. And I would agree. But then, not so for the Apple TV^*! The TV's IPSW is encrypted.. But.. wait.. what's this on the iPhone Wiki?

So you can grab the OTA update as is, and start unpacking the full "TvOS" (spoiler - it's 90% identical to iOS 9, minus HeadBoard, the default TV* apps, and a daemon or two) - without having a base OS image, which we don't, for lack of a public TvOS jailbreak (but not for long, I suspect :-)

Problem

Somehow, the Pbzx file in the OTA - the huge payload file in payloadv2 - has some weird chunks in it which don't actually contain xz. This could be a bug in my unpacker, but I checked, and checked again, and something doesn't make sense. Because the pbzx unpacker crashes, you end up with a partial xz'ed payload, which of course means only 350MB of the very large 1.9GB image.

Solution

I fixed both the pbzx unpacker (to skip non xz chunks) and the OTA extractor (to skip invalid entries). Along the way I also introduced an improvement in the latter to do the whole thing over mmap(2)

TL;DR

You can now get most of the TvOS image:

morpheus@Zephyr (~/Downloads/TvOS/AssetData/payloadv2) %./pbzx  < payload > pb.xz  
Flags: 0x1000000
Chunk #1 (flags: 1000000, length: 6066008 bytes) 
Chunk #2 (flags: 1000000, length: 14240912 bytes) 
Chunk #3 (flags: 1000000, length: 14231772 bytes) 
Chunk #4 (flags: 1000000, length: 16777216 bytes) 
Warning: Can't find XZ header. Instead have 0x16c3201e(?).. This is likely not XZ data.
Chunk #5 (flags: 1000000, length: 16777216 bytes) 
Warning: Can't find XZ header. Instead have 0xf52ffff9(?).. This is likely not XZ data.
Chunk #6 (flags: 1000000, length: 14112620 bytes) 
Chunk #7 (flags: 1000000, length: 3866128 bytes) 
... # Business as usual in the modified version
Chunk #114 (flags: 1000000, length: 6347484 bytes) 
Chunk #115 (flags: 1000000, length: 3935192 bytes) 
Chunk #116 (flags: e7cfca, length: 2839680 bytes)
morpheus@Zephyr (~/Downloads/TvOS/AssetData/payloadv2) %ls -l pb.xz                               
                                         
-rw-r--r--  1 morpheus  staff  756774712 Feb 17 18:27 pb.xz
morpheus@Zephyr (~/Downloads/TvOS/AssetData/payloadv2) %xz --decompress pb.xz                                               18:30
morpheus@Zephyr (~/Downloads/TvOS/AssetData/payloadv2) %ls -l pb                                                            18:31
-rw-r--r--  1 morpheus  staff  1911017418 Feb 17 18:27 pb
# and extraction..
morpheus@Zephyr (~/Downloads/TvOS/AssetData/payloadv2/TvOS) % otaa -e '*' ../pb                                      18:46
... # So we lose one or two .. :-(
Corrupt entry (0xaa0103f3 at pos 95438849).. skipping
... # But quickly get back ot it :-)
..

morpheus@Zephyr (~/Downloads/TvOS/AssetData/payloadv2/TvOS) %ls -lR | more                                                  18:47

total 16
drwxr-xr-x  18 morpheus  staff  612 Feb 17 18:46 Applications
drwxr-xr-x   3 morpheus  staff  102 Feb 17 18:46 System
drwxr-xr-x   2 morpheus  staff  136 Feb 17 18:46 bin
-rw-r--r--   1 morpheus  staff   11 Feb 17 18:46 etc
drwxr-xr-x   3 morpheus  staff  102 Feb 17 18:46 private
drwxr-xr-x   2 morpheus  staff  374 Feb 17 18:46 sbin
-rw-r--r--   1 morpheus  staff   15 Feb 17 18:46 tmp
drwxr-xr-x   8 morpheus  staff  272 Feb 17 18:46 usr

./Applications:
total 0
drwxr-xr-x   4 morpheus  staff   306 Feb 17 18:46 AdSheetTV.app
drwxr-xr-x  41 morpheus  staff  1598 Feb 17 18:46 HeadBoard.app
drwxr-xr-x   3 morpheus  staff   204 Feb 17 18:46 PBLinkHandler.app
drwxr-xr-x  40 morpheus  staff  1530 Feb 17 18:46 PineBoard.app
drwxr-xr-x  44 morpheus  staff  6392 Feb 17 18:46 Podcasts.app
drwxr-xr-x  40 morpheus  staff  2278 Feb 17 18:46 Setup.app
drwxr-xr-x  40 morpheus  staff  1530 Feb 17 18:46 SiriViewService.app
drwxr-xr-x   3 morpheus  staff   238 Feb 17 18:46 StoreDemoViewService.app
drwxr-xr-x  40 morpheus  staff  1496 Feb 17 18:46 TVAirPlay.app
drwxr-xr-x  40 morpheus  staff  3366 Feb 17 18:46 TVAppStore.app
...
-rw-r--r--  1 morpheus  staff   46172 Feb 17 18:46 TiSerialFlasher-06.16.txt

./usr/standalone/firmware/STDP2700:
total 24
-rw-r--r--  1 morpheus  staff  8780 Feb 17 18:46 runtime

Get the updated OTAPack HERE

MOXiI 2 is coming! Really! Just ... taking more time than I had hoped..

Hey - I'm working on it. Aren't all these articles a clear indication of progress? ;-) Oh, and - I'm covering TvOS too, if you're wondering. I'm already at 50% more NEW stuff over the first edition.. In the meantime, have I mentioned Technologeeks' OS X and iOS for Reverse Engineers training discusses this stuff?

Sources

OTA.c (updated) PBZX (updated)

Greets

@Xerub: Because you're awesome for giving us the keys to 32-bit (and fixing my code :-)

@iH8sn0w: Because where would we be without that iBoot exploit? :-)

AAPL: For supplying us with awesome products, and the unencrypted OTAs to appreciate them. Ignore my German friend's rants about this bug :-)

* - The watch doesn't have this OTA "feature" . But then, not even JBers are interested in the Watch