iOS introduced Over-The-Air (OTA) updates sometime back in 5.x, which was after the 1st edition of MOXiI was already published. As such was the case, this very important discussion was left out of where it should have been included (Chapter 6). With the 2nd edition in the works, OTAs will naturally be included - but since this information is immediately useful (as you'll see shortly), I figured it made sense to publish it as an article.
Why should you care? (Target Audience)
As you probably know, iOS software images (the well known .IPSWs, which I did cover in the book) are encrypted. Thanks to the hard work of the folks at the iPhoneWiki, decryption keys are made available fairly regularly for A4 and A5 devices - the former via limera1n, and the latter through an undisclosed but pretty well known exploit somewhere in the boot chain. The problem, however, is that no keys exist for A6+ devices, most notably the 64-bit devices (5S, 6, iPad Air, etc). This is exacerbated by Apple effectively phasing out A4 with 8, and the very likely end-of-line for A5 when iOS 9 is likely to be announced this summer.
Thankfully, OTA updates exist and are unencrypted. This means that if you have a jailbroken device of version 8.1.2 or lower, you can do two pretty amazing things:
Obtain the OTA update, and - using the techniques demonstrated here - unpack it so as to reverse engineer it to your heart's content
Update the iOS filesystem image manually - while maintaining the jailbreak. That's actually a long and very delicate process which, if done improperly, can render your device unbootable and force you to restore to the latest, non jailbreakable version. If carried out correctly, however, you get the latest and greatest iOS - without sacrificing a jailbreak. A manual update is effectively indistinguishable from a stock update, provided you don't mess with the kernelcache or dyld, thanks to which the jailbreak remains feasible. In fact, this is important enough to merit a nice div of its own:
If you ARE going to try this at home on a real device - caveat - if you accidentally update dyld or the core libs you might end up locking yourself out of the user mode component of a given jailbreak. If you touch the kernelcache you will render the apticket.der invalid - which is even worse, since iBoot will refuse to boot you - and force you to restore, when the only IPSWs Apple presently signs are no longer jailbreakable. At least, publicly. A future post will demonstrate how to do so - but it requires meticulous attention to detail. Believe me.
Both the above are great, but especially the latter. In the case of 8.2, for example, that gives you the Apple Watch support, in case you absolutely must send your heartbeat to your friends (Awwww). But going forward, this will likely be useful for whatever 9.x brings - if only to reverse the hell out of it. So let's get started, but before that - the usual disclaimer:
If you know me, you know I'm all for keeping iOS jailbreakable - despite Apple's herculean efforts. I wouldn't be publishing this if I thought Apple could plug this. Since an OTA is started from user mode when iOS is already running - unlike a full restore/upgrade which is carried out via iBoot - it does not require (and, in fact, can't access) the UID/GID keys. This also means that Apple can't easily encrypt the OTA updates. At least not on the present generation of devices.
OTA Update bundles
OTA update bundles are zip files which Apple makes readily available for download at http://appldnld.apple.com/. The zip files are cryptically named, in what's probably a SHA-1 of their "real" name. The convention appears to be:
Fortunately, you can find all the files easily through the iPhoneWiki OTA page - and download them easily. You don't even have to fake a User-Agent (Have I mentioned, Cupertino Syndrome? Lock the door (encrypt the fs), but leave the Window wide open :).
Downloading this (or any other) update, an unpacking the zip, expect to see the same directory structure, which is something like this:
META-INF/: Subdirectory containing a single file, com.apple.ZipMetadata.plist, which is a binary plist:
all in all, nothing too interesting.
Info.plist - As with any bundle, the required manifest. This is a standard Plist, which looks like the following:
AssetData - Where the update binaries, etc are.
AssetData has an internal directory structure, like so:
Info.plist - Once again (*sigh*) we have a plist here, containing similar keys to the previous one, but with the addition of a SystemUpdatePathMap dictionary, containing the names of the non-filesystem images (i.e. baseband, RAMdisk, glyphs, etc) which (from what I can see) aren't part of the update, or are present only in patch form)
boot/ - A subdirectory containing actual non-filesystem images. These are all in the boot/Firmware/... directory, and contain the kernelcache.release.boardId, iBec/iBSS (in boot/Firmware/dfu), and iBoot and friends (in boot/Firmware/all_flash/all_flash.boardIdap.production.. These are all in im4p (or, pre-A7, IMG3) format, which means that they're encrypted. :-( Apple can encrypt those because they're handled during the boot process, wherein the GID key is still accessible.
boot/ also contains a BuildManifest.plist, a lengthy manifest containing the update rules for the various im4ps. I'll leave this rather detailed description for the book..
payloadv2/patches: A directory containing deltae to be patched. This is, of course, assuming, that the versions you're patching from are exactly the ones which came with stock iOS. These are in BXDIFF format, specifying offsets and patch data.
payloadv2/prepare_payload: A PBZX file (see later), containing an (encrypted) IM4P. Likely used by iBoot and friends
payloadv2/removed.txt: Specifying a list of files which will be removed (potentially replaced), one per line.
payloadv2/links.txt: Specifying a list of symbolic links to link to (=) and add (+).
payloadv2/payload: More on that in a moment
Various BOM files
The BOM files
Apple uses the venerable NeXTSTEP Bill-Of-Materials (BOM) proprietary format for their bundles. The private Bom.Framework (which likely hasn't changed from NextSTEP, aside from 64-bit support..) handles these files, and you can view them on OS X using the lsbom(8) command, or package them yourself with mkbom(8). The files are:
Bill of materials of files pre-application of payload
Bill of materials of files post-application of payload
Bill of materials of files in payload
An additional file, payload.bom.signature protects payload.bom from tampering, though in practice that's meaningless (binaries in BOM have to be codesigned anyway, and if you're on a jailbroken device this, too, can be bypassed). I've discussed the BOM format in the book (Chapter 6), so no sense adding more here. What's more interesting is the payload file itself.
The payload and pbzx files
The patches in the payloadv2 directory are only applicable to files which already exist on the system, and - by means of patch application - can be updated to a new file version. iOS updates, however, often contain new functionality (again, the Watch support in 8.2 being a great example).
New functionality can be found in the payloadv2/payload directory. Inspecting the file reveals a "pbzx" magic, like so:
The <FD>7zX you see in the first row, and the YZ at the end of the file are clearly indicative of the XZ compression header and trailer. That Apple adopted LZMA is not surprising, considering that it's already used in the kernelcaches (as of 10.10/8.0). Outright using XZ was a surprise. But that makes our life easier - grabbing the XZ sources from tukaani.org, you can quickly build the XZ binary which will decompress it. But just stripping the pbzx encapsulation won't work - the XZ data is separated into chunks - somewhat akin to how Apple compresses DMGs, if you saw the other article of mine. So you need a small "extractor" for these chunks, before you can get to apply XZ on them. I've just the tool for that, which you can compile yourself from this listing:
EDIT: There's a fix and better version of this now - q.v. Part III
And use like so:
The compression is evident - and quite impressive (1.2GB compressed into 423MB - some 65% - not bad, considering this is binary data). And we now have a single archive - that p - which is the last part of the bundle we need to deal with, and is obviously not a VAX executable..
At this point, it's obvious that "p" is some proprietary format, of some archive. This is further corroborated by attempting to view it with more:
It would make sense if this were a cpio or (Apple's favorite) pax archive, but it's apparently neither (pax actually uses cpio internally, and complains about name lengths being out of range when fed this). Some of my readers are reverse-engineers, so this makes for a nice (and easy) exercise in reversing. Observe the following:
File names are clearly visible (highlighted), as are their contents. Because there is no NULL byte in between the filename and the contents, it follows there must be some length specifier for the name.
Likewise, it follows there has to be a length specifier for the size of the file
Using od makes it a bit easier to figure things out - specifically, where the file contents are raw XML and not a binary plist:
How does the file format fall apart so quickly? Well, in this case, it's possible to cheat, using lsbom(1):
(If you're wondering why the endian-ness seems off - thank PPC)
If you just want the struct, it's something like:
and you can get the source of my OTA payload extractor (that is, past the pbxz and the xz) right here. Using it, it super simple:
The code is quick and dirty (I haven't removed the anchor, etc), but - it works. And it can extract all the files from the update payload (and you can patch the rest). Happy reversing :-)