Spitting Image:
    MacOS & *OS Software Images

No software is perfect, and as Lubarsky's Law of Cybernetic Entomology states, "there's always one more bug". Bugs are proportional to the size of the code base, and in a code base as vast as that of an operating system, they are numerous and oftentimes severe. This is especially true when the bugs aren't just functional bugs, but security vulnerabilities. Software updates - once restricted just to major OS versions - have therefore become commonplace. MacOS and the *OS variants support sophisticated software update mechanisms, which can automatically locate, download and deploy software updates. The mechanisms are somewhat different between the OS variants.

MacOS and *OS software images are also of immense value to enthusiasts, researchers, and jailbreakers. Whether it's to look for ways to "tweak" the system, find upcoming features, uncover files of importance, or reverse binary diffs in updates in order to find vulnerabilities, the ability to download the filesystem image and extract files greatly simplifies the process and lessens the need to have a running instance of it on a device. That *OS software images are of importance was also noted by Apple, who chose to encrypt them with the device-class specific GID keys for most of iOS's lifetime. It was only around iOS 10 that this practice was ceased, whether by choice or mistake. Since then, all but the iBoot components are unencrypted.

This chapter (originally intended for Volume II, then ending up fitting better as an Appendix for Volume I) discusses software images in detail. Looking first at the MacOS realm, we peek behind the system installation process, and that of software updates. Then, *OS software images are explored, with the various files corresponding to the SoC's components' software. A special discussion of *OS's Over The Air (OTA) images follows, along with a demonstration of unpacking selected files from them. Finally, the aspects of "personalization" through APTickets, which lock an installation to a particular device given Apple's blessing, are discussed. Although started with *OS variants, personalization has spread to BridgeOS enabled devices, and will become standard for new Macs.

MacOS Software Images

Apple has moved the MacOS software images to the App Store, and new releases of MacOS can now be downloaded as any other app. The "app", in this case, is called Install MacOS codeName.app, and the binary launched by the app is InstallAssistantTool. The InstallAssistantTool provides a standardized GUI for the user to select which volume to install MacOS onto, and - along with several other helper daemons in the app's OSInstallerSetup.framework/Resources/ - prepares the system for installation.

To enable the InstallAssistantTool binary unfettered access to the filesystem, it possesses several of the most powerful SIP entitlements:

The Installer's SharedSupport directory contains three HFS+ formatted DMGs, chunk lists (for two of them), and an accompanying InstallInfo.plist. The property list defines the roles of the DMGs, shown in Table A-2:

The InstallInfo.plist directs the installation process to the OS Installer key, which is set to OSInstall.mpkg, found in InstallESD.dmg's Packages/ directory. The .mpkg contains a Distribution file with the required scripts and UI (as shown in 5-4). Additional packages in InstallESD.dmg are Core.pkg (the full root filesystem) and the secure boot and firmware packages, discussed later.

Creating a standalone update disk image

A standalone update disk image can prove highly useful when installing MacOS onto a machine with no network access, or no prior MacOS image - for example, a virtual machine. With little work, the .dmg files inside the MacOS installer application can be converted onto a bootable DVD or USB storage. Hidden in the Installer application's Resources is a small binary, createinstallmedia, which creates a bootable image by overwriting the volume specified with the --volume argument. An additional argument, --nointeraction, can be specified to disable prompts and automate the process when scripting.

MacOS Software Updates

Installing a new version of MacOS is commonly an annual experience, but quite a few updates are published by Apple throughout the year. The user can manually check for software updates by using apple → About this Mac → Software Updates, which brings up System Preferences.app's Software Update tab. The default settings are to check and download new updates automatically, but install only system data files and security updates, which are likely to be critically important, yet with a low chance of disrupting the system.

Software Update.app

The software updating responsibilities of MacOS are performed by the daemons contained in the Resources/ folder of Software Update.app:

• softwareupdated: is the main daemon handling the updates. It is set to start periodically (via an XPC activity) three times a day, or when manually triggered through Darwin notifications. It claims the com.apple.softwareupdated XPC service, with which clients communicate over Objective-C classes and NSXPC messages.
• suhelperd: handles various auxilliary tasks, such as client authorization, credentials, and file maintenance on clients' behalf. It claims the com.apple.suhelperd port, over which it provides the legacy (and unentitled) MIG subsystem 675309 to clients. The subsystem consists of 39 messages, which map to SoftwareUpdate.framework _suhelperd_client_XXX exports. The framework also provides an Objective-C interface through the SUHelperProxy class.



Table A-3: The MIG subsystem exported by SoftwareUpdate.framework

#	_suhelperd_client export	SUHelperProxy selector
675309	..authorize_tool	authorizeTool:forRights:
675310	..extend_rights
675311	..disconnect_client	disconnect
675312	..prepare_for_logout_and_install	prepareForLogoutAndInstall:
675313	..prepare_loginwindow_for_postlogout_install _no_console_user	(...same name, mixed case..)
675314	..check_and_fix_dir_permissions	checkAndFixPermissionsAtPath:owner:
675315	..register_product_file	registerProductFile:forProductKey:firmware: trustLevel:keepOriginal:inForeground:
675316	..client_register_personalized_manifests	registerPersonalizedManifests:forProductKey: inForeground:
675317	..make_queues	makeQueues
675318	..move_installed_printers_to_library	moveInstalledPrintersToLibraryFromPath:
675319	..remove_metadata_cache	removeMetadataCacheFromUpdates
675320	..move_metadata_cache_to_updates	moveMetadataCacheToUpdatesFromPath:
675321	..move_ppd_cache_to_updates	movePPDVersionCacheToUpdatesFromPath:
675322	..remove_updates_index	removeIndexFromUpdates
675323	..read_updates_index	readUpdatesIndex
675324	..write_updates_index	writeUpdatesIndex:
675325	..create_directory_for_product	createDirectoryForProductKey:Firmware:
675326	..remove_dist_for_product	removeDistForProductKey:withFilename:
675327	..remove_product	removeProductDirectoryForKey:
675328	..remove_recovery_install_result	removeRecoveryInstallResult
675329	..remove_downloaded_packages_for_product	removePackagesForLocalReferences:forProductKey:

 

Table A-3: The MIG subsystem exported by SoftwareUpdate.framework (cont.)

#	_suhelperd_client export	SUHelperProxy selector
675330	..digest_for_package	getDigest:forPackageAtURL:
675331	..set_any_user_preferences	setObject:forAnyUserPreference:
675332	..clear_any_user_preference	clearAnyUserPreference:
675333	..update_any_user_preferences	SUHelperProxy updateAnyUserPreferences
675334	..create_updates_available_cookie	createUpdatesAvailableCookie
675335	..remove_updates_available_cookie	removeUpdatesAvailableCookie
675336	..clear_critical_update_notification_date	clearCriticalUpdateNotificationDate
675337	..set_app_store_auto_update	setAppStoreAutoUpdate:
675338	..set_macos_auto_update	setMacOSAutoUpdate:
675339	..deletepref_indomain	deletePref:inDomain:
675340	..clear_catalog_to_production_and_notify	clearCatalogToProductionAndNotify
675341	..unenroll_from_seed_program	unenrollFromSeedProgram
675342	..stash_login_credentials	stashLoginCredentialsEnablingFLO:
675343	..commit_login_credentials	commitLoginCredentialsDisablingFLO: hasBaseSystemUpdates:
675344	..stash_commit_apfs_fde_key	stashAndCommitAPFSFDEKey
675345	..set_applestaged_upgrade_run_flo	setAppleStagedUpgradeShouldFLORun:
675346	..configure_progress_phases	configureProgressPhasesAll:
675347	..retrieve_url_credentials	lookupURLCredentialInSystemKeychainForHost:port:
675348	..arm_basesystem_updates_mechanism	armBaseSystemUpdates:productKey: recoveryUpdatePackageURL:error:



• softwareupdate_download_service: Handles the downloaded updates. It is a relatively light wrapper over the SUDownloadServiceAgent class. The daemon registers the com.apple.softwareupdate_download_service XPC, which clients interface with using the SUDownloadServiceAgent objective-C class.
• softwareupdate_notify_agent: Handles notifications when new software updates are detected. It starts through Darwin notifications (e.g. com.apple.SoftwareUpdate.[updates/OSXAvailable]), and uses the SUAppStoreAgentNotifier Objective-C class to relay these to the App Store.app.
• softwareupdate_firstrun_tasks: is run once on system startup, and picks up any installations which required a system reboot at some point.

The Software Update.app itself merely provides the user interface, by launching the App Store.app on its "Updates" tab. A command-line interface is provided by softwareupdate(8). The CLI offers a couple of other useful options, including (as of Darwin 17) --history (to show installed updates) which can be coupled with --all (to show updates installed by other clients, such as storedownloadd, the Mac App Store daemon). This capability, like most of the rest, is achieved through the SoftwareUpdate.framework's SUUpdateServiceDaemon object (specifically, getInstallHistoryWithReply:) over NSXPC to softwareupdated itself.

The combined history of all software updates can also be directly viewed through its backing file, /Library/Receipts/InstallHistory.plist. The system_profiler also provides this functionality through its SPInstallHistoryReporter.spreporter. Reversing the plugin reveals very simple code, which calls on the private PackageKit.framework to retrieve this information programmatically (straight from the file, without any IPC), as shown in Listing A-4:

#import <Foundation/Foundation.h>

/* -framework PackageKit -F /System/Library/PrivateFrameworks -lobjc */

// Retrieved from jtool -d objc -v PackageKit
@interface PKInstallHistory : NSObject
// No properties..
// 1 instance variables
// /* 0 */  NSString _path;             // @"NSString"
// 2 class methods
/* 0 */ + (id)historyOnVolume:(id)vol;  // Protocol 123d4f801
/* 1 */ + (id)defaultHistory;           // Protocol 123d4f8fa
// 4 instance methods
/* 0 */ - (id)initWithPath:(id)path;    // Protocol @24@0:8@16
/* 1 */ - (void)dealloc;                // Protocol v16@0:8
/* 2 */ - (id)addInstallRequest:(id)req;// Protocol c24@0:8@16
/* 3 */ - (id)installedItems;           // Protocol @16@0:8
@end

int main (int argc, char **argv)
{
   NSMutableArray *items = 
	(NSMutableArray *) [[PKInstallHistory defaultHistory] installedItems];

   // Quick and dirty way to show items: (-framework CoreFoundation)
   CFShow(items);
}

A verbose log containing the details of installation processes can be found in /var/log/install.log. The SoftwareUpdate.framework supports even more verbose logging when the SUDebugVeryVerbose key is set in /Library/Preferences/com.apple.SoftwareUpdate.plist.

The Software Catalog

The com.apple.SoftwareUpdate.plist provides the configuration for the update mechanism, including statistics on the last time updates were checked, and updates pending install. This property list is not meant to be handled directly, with the softwareupdate(8) utility providing its main interface. Alternatively, with this being a CFPreferences file, defaults(1) can be used as well. Output A-5 shows a sample of this property list:

#
# The property list is binary, so display with jlutil:
#
morpheus@Bifröst (~)$ jlutil /Library/Preferences/com.apple.SoftwareUpdate.plist
 LastResultCode: 0
   LastAttemptSystemVersion: 10.14.4 (18E226)
   SkipLocalCDN: false
   LastUpdatesAvailable: 2
   LastRecommendedUpdatesAvailable: 2
   LastAttemptBuildVersion: 10.14.4 (18E226)
   RecommendedUpdates[0]: 
      Identifier: macOS 10.14.5 Update
      Product Key: 041-57074
      Display Name: macOS 10.14.5 Update
      Display Version:  

   RecommendedUpdates[1]: 
      Identifier: MobileDeviceSU
      Product Key: 041-62886
      Display Name: iTunes Device Support Update
      Display Version:  

   LastFullSuccessfulDate: 2019-05-21T20:23:28Z
   PrimaryLanguages[0]: en-US
   PrimaryLanguages[1]: en
   LastSessionSuccessful: true
   LastBackgroundSuccessfulDate: 2019-05-20T21:10:12Z
   LastSuccessfulDate: 2019-05-21T20:23:28Z

#
# Using softwareupdate -l, this is presented to the user
#
morpheus@Zephyr (~)$ softwareupdate -l
Software Update Tool

Finding available software
Software Update found the following new or updated software:
   * macOS 10.14.5 Update- 
        macOS 10.14.5 Update ( ), 2740918K [recommended] [restart]
   * MobileDeviceSU- 
        iTunes Device Support Update ( ), 105795K [recommended] [restart]

The most important configuration parameter is the CatalogURL, which specifies the location of Apple's published software catalog. This URL is retrieved by the local system, which uses its contents to determine which updates are available, as well as download them. Apple defines three catalogs - the normal (release), beta and developer (early beta) seeds. Release systems are set to the normal catalog, but the CatalogURL can be manually modified (using defaults or softwareupdate -[set/clear]-catalog). The CatalogURL strings are also hardcoded into SoftwareUpdate.framework itself, and can easily be found by applying strings(1) and grep(1)ing for https://swscan.apple.com/content URLs, which will reveal URLs with versions from the latest through 10.9, and then the feline names down to Leopard. The public beta seed can be obtained by appending "beta" to the latest MacOS version, the customer seed by "customerseed", and the developer program (normally requiring a registered developer login) - by appending "seed".

The software catalog is a gzipped property list. When served by Apple its MIME type, application/x-apple.sucatalog+xml, is claimed by the Software Update.app. The main key of the property list is a Products dictionary. Each of the dictionary keys is the update identifier (###-#####), and its value is another dictionary, containing a ServerMetadataURL, and an array of one or more Packages dictionaries. Each package dictionary specifies a Digest, Size, URL of the update. This is shown in Listing A-6:

CatalogVersion: +2
ApplePostURL: http://swpost.apple.com/stats
IndexDate: 2019-05-31T21:26:55Z
Products:
   ..
   041-57074:
      ServerMetadataURL: http://swcdn.apple.com/content/downloads/13/38/
		041-57074/o46jekrijxrkfybffroi02an8yxqr5mbjt/macOSUpd10.14.5.smd

      ..
      Packages[0]:
      Digest: 6fc7c52aae9740b0eb850d10e8acc2b2792ba134
      Size: 1662914144
      MetadataURL: https://swdist.apple.com/content/downloads/13/38/
       041-57074/o46jekrijxrkfybffroi02an8yxqr5mbjt/macOSUpd10.14.5.pkm
      URL: http://swcdn.apple.com/content/downloads/13/38/041-57074/
       o46jekrijxrkfybffroi02an8yxqr5mbjt/macOSUpd10.14.5.pkg
      IntegrityDataURL: http://swcdn.apple.com/content/downloads/13/38/
       041-57074/o46jekrijxrkfybffroi02an8yxqr5mbjt/macOSUpd10.14.5.pkg.integrityDataV1
      IntegrityDataSize: 5792
      ExtendedMetaInfo:
	  ProductType: macOS
	  AutoUpdate: NO
	  ProductVersion: 10.14.5
          PostDate: 2019-05-20T17:00:30Z
      Distributions:
	  English: https://swdist.apple.com/content/downloads/13/38/
	 041-57074/o46jekrijxrkfybffroi02an8yxqr5mbjt/041-57074.English.dist

The private Seeding.framework's seedutil is an undocumented tool hiding in its Resources/, which enables the enrollment of the Mac in different seed programs. The tool is so tiny it can be easily decompiled in full, the following listing shows a portion of its source, and the full reversed source can be found at the book's companion website^[1].

void printProgram() {
  CFPropertyListRef sp = CFPreferencesCopyValue(CFSTR("SeedProgram"),
		       CFSTR("com.apple.seeding"),
		       kCFPreferencesAnyUser,
		       kCFPreferencesCurrentHost);

  NSString *sdcat = [SDCatalogUtilities  _currentCatalog];

  CFPropertyListRef nssfm = CFPreferencesCopyValue(CFSTR("NSShowFeedbackMenu"),
		       kCFPreferencesAnyApplication,
		       kCFPreferencesAnyUser,
		       kCFPreferencesCurrentHost);
  CFPropertyListRef dsoo = CFPreferencesCopyValue(CFSTR("DisableSeedOptOut"),
		       CFSTR("com.apple.SoftwareUpdate"),
		       kCFPreferencesAnyUser,
		       kCFPreferencesCurrentHost);

  printf("Program: %ld\n", [sp longValue]);
  int bis  =  [SDBuildInfo currentBuildIsSeed];
  printf("Build is  seed: %s\n", bis ? "YES" : "NO");
  printf("CatalogURL: %s\n",  [sdcat UTF8String]);
  printf ("NSShowFeedbackMenu: %s\n", nssfm? "YES": "NO");
  printf ("DisableSeedOptOut: %s\n", dsoo? "YES": "NO");
}

int main (int argc, char **argv) {
   if (getuid()) { puts ("Must be run as root"); exit(1); }

   if (argc == 1) { usage(); return 0; }

   NSString *arg = [NSString stringWithUTF8String:argv[1] ];

   if ([arg isEqualToString:@"enroll"]) {
	if (argc > 2) {
	  printf("Enrolling...\n");
          [SDSeedProgramManager
	   enrollInSeedProgram: 
		[SDSeedProgramManager _seedProgramForString:
		  [NSString stringWithUTF8String:argv[1]]]];
	   }
	else {  usage(); exit(1);}
   }
   else if ([arg isEqualToString:@"unenroll"]) {
	puts ("Unenrolling...\n");
	[SDSeedProgramManager unenrollFromSeedProgram];
  }
  else if ([arg isEqualToString:@"current"]) {
	void *csp = [SDSeedProgramManager currentSeedProgram];
	NSString *str = [SDSeedProgramManager _stringForSeedProgram:csp];

	printf("Currently enrolled in: %s\n\n", [str UTF8String]);
	printProgram ();
  }
  else if ([arg isEqualToString:@"migrate"]) {
	printf("Migrating seed program settings\n\n");
	[SDSeedProgramMigrator migrateSeedProgramSettings];
	printProgram ();
  }
  else if ([arg isEqualToString:@"fixup"]) {
	printf("Fixing up seed program settings\n");
	if (![SDSeedProgramMigrator fixupSeedProgramSettings])
		printf("No seed program was set; did nothing\n");
  }
  return 0;
}

A particularly useful seed program is the DeveloperSeed - enabling early access to MacOS beta releases when /Users/Shared/.SeedEnrollment.plist exists and contains a SeedProgram key set to a DeveloperSeed string value. The "MacOS Developer Beta Access Utility" Apple provides to registered developers performs this automatically, in addition to obtaining the Developer Beta bundle identifier from the IASBundleID found in a property list at http://configration.apple.com/configuration/macos/seeding/content/content-1.0.plist. The identifier is then embedded in the x-apple.systempreferences:com.apple.preferences.softwareupdate) private URL as the installMajorOSBundle= argument, which brings up System Preferences.app to start the download.

Software Update Format

Software updates are downloaded (via softwareupdate's temporary directory in /var/folders) to the SIP restricted /Library/Updates, in subfolders of the form ###-#####. The root folder also contains an index.plist, which specifies keys to InstallAtLogout and InstallLater, according to the update requirement and user choice. The indvidual updates' files may be .tar archives or .dmg disk images, but are more commonly .pkg files:

# Display folder contents, and index.plist:
#
morpheus@Bifröst(/Library/Updates)$ ls -F
041-57074/             041-62886/             ProductMetadata.plist  index.plist
morpheus@Bifröst(/Library/Updates)$ jlutil index.plist
   InstallAtLogout[0]: 041-47074
   InstallAtLogout[1]: 041-62886
   ProductPaths:
	041-57074: 041-57074
	041-62886: 041-62886
#
# The update files (all restricted by SIP)
#
morpheus@Bifröst(/Library/Updates)$ ls -lFH 041-47074 | more
total 5590992
...  12K May 20 22:10 041-57074.English.dist               # installer-gui-script Plist
..  601B May 20 22:10 041-57074.English.extraInfo          # plist with ExtendedMetaInfo & ServerMetadataURL
..  707K May 20 22:10 BridgeOSBrain.pkg                    # UpdateBrainService.xpc arm64 bundle
..  312M May 20 22:21 BridgeOSUpdateCustomer.pkg           # arm64 image
..   57M May 20 22:11 EmbeddedOSFirmware.pkg               # eOS image (for T1 devices)
..  165M May 20 22:25 FirmwareUpdate.pkg                   # EFI firmware image
..   98M May 20 22:24 FullBundleUpdate.pkg                 # 
..  224B May 20 22:51 PersonalizedManifests/               # im4m ticket for T2's ECID
..   67K May 20 22:13 SecureBoot.pkg                       # /usr/standalone/i386/SecureBoot.bundle
..   67K May 20 22:13 macOSBrain.pkg                       # /usr/libexec/atomicupdatetool
..  459M May 20 22:20 macOSUpd10.14.5.RecoveryHDUpdate.pkg # DMG image for recovery partition
..  1.5G May 20 22:50 macOSUpd10.14.5.pkg                  # Core OS

As explained in I/5, the .pkg format is a special case of xar(1). As with standard packages, the .xar contains a mandatory PackageInfo, and the actual update contents, either a .dmg, or a Bom and Payload. The Payload is usually compressed with pbzx, a custom packer wherein individual frames may be compressed by XZ or left uncompressed. An open sourced implementation of the unpacker can be found on the book's companion website^[2].

Once uncompressed, the payload files are simple cpio archives, in the legacy pre-SVR4 ("odc") format. A simple application of cpio -ivd (in an empty directory) will unpack from stdin (-i), preserving directories (-d) and verbosely detailing contents (-v).

With Apple's move to coprocessor enabled devices, MacOS updates also contain packages to update eOS/BridgeOS as well. The ill-fated eOS (a.k.a. BridgeOS 1.1, used briefly in the 2016 touchbar macs) are updated through EmbeddedOSFirmware.pkg. The BridgeOS update files are provided in BridgeOSUpdateCustomer.pkg, but in order to update them BridgeOSBrain.pkg must be deployed first. The "Brain" is the com.apple.MobileSoftwareUpdate.UpdateBrainService.xpc XPC service (discussed in the next section), and its package also contains a .Trustcache file (as it is adhoc signed). Another important component are the BridgeOS personalization tickets for various firmware components, which are stored in their respective locations under PersonalizedManifests/. These are .im4m (IMG4) files, and (as discussed later), are uniquely tied to the BridgeOS processor's Exclusive Chip ID (ECID), thus making them "personalized" and specific to the downloading Mac.

*OS Software Images

The *OS Software images are provided as .ipsw (presumably, iPhone SoftWare) files. These are naturally available to download directly from Apple, but the iPhone Wiki Firmware Page^[3] and ipsw.me are often easier to access instead. The latter is especially useful thanks to its easy GUI and, as an added bonus, tracking of image signing status. BridgeOS updates are an exception to the rule as they are tracked by neither of the former sites, and thus require direct downloading from Apple^[3].

Base .ipsw contents

The .ipsw are .zip archives, commonly over 3GB in size. It helps, however, that the zips allow for random access, which Apple supports through the private StreamingZip.framework. This makes it easy to download just selected files in the .ipsw, by first obtaining the directory and then using HTTP(S) partial GETs to retrieve the selected files. An open source implementation of that can be found in Planetbeing's partial zip^[4] project.

The .ipsw contents follow a well defined structure and content naming convention. The exact naming of files in the .ipsw is dependent on the device type (iphone/ipad), board name (d22, etc), and (for graphics) the resolution (2x,3x,2436, etc). This becomes useful in cases where the same .ipsw is used for multiple devices. Files which are not images (i.e. the Device Tree and executable boot component) have a matching property list, containing a four character (32-bit) 'tag', and hashes. In some cases, a single IMG4 container may contain two images - one for the standard boot and the other for the recovery. Table A-9 shows the files common to all iOS images:

The .ipsw always contains three^* .dmg files, but their names are numeric sequences (e.g. 058-xxxxx-xxx.dmg). The first and second numbers are apparently tied to the *OS variant type, and the third is an incrementing build number. The exact semantics of the first two numbers are unknown (at least to this author). What is certain is that the largest of the .dmgs is the root filesystem, and the two smaller ones are the recovery and update images. The two are often too close in size to reliably detect which is which, although their are no longer encrypted and can therefore simply be mounted on MacOS, or elsewhere using fsleuth.

Experiment: Working with DMGs from .ipsw A-xxx

The .dmg files found in .ipsws are wrapped by a thin IMG4 container, which prevents them from being used as disk images. Inspecting them reveals they're raw HFS+ images, and miss the koly trailer commonly found in .dmg files.

$ hexdump -C ~/Downloads/058-97541-138.dmg 
00000000  30 84 04 b9 be 15 16 04  49 4d 34 50 16 04 72 64  |0.......IM4P..rd|
00000010  73 6b 16 01 30 04 84 04  b9 be 00 00 00 00 00 00  |sk..0...........|
00000020  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000410  00 00 00 00 00 00 00 00  00 00 00 48 58 00 05 80  |...........HX...|
00000420  00 01 00 31 30 2e 30 00  00 00 00 d6 f6 d8 e2 d6  |...10.0.........|
..
04b9bb80  0f 40 00 00 09 60 00 00  00 00 f4 00 00 00 f6 00  |.@...`..........|
04b9bb90  00 00 f4 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
04b9bba0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
04b9be10  00 00 00 00 00 00 00 00  00 00 00                 |...........|

Looking at the above output, note the "HX" (HFSX magic). This magic value is expected to be at the beginning of the superblock, which is commonly at offset 1024. But this one is at 0x41b - 27 bytes after where it should be. This makes sense, considering the IMG4 header is to be accounted for. The image can be made mountable if those 27 bytes are removed:

$ dd if=058-97541-138.dmg of=/tmp/a.dmg bs=0x1b skip=1
...
$ hdiutil attach -imagekey diskimage-class=CRawDiskImage /tmp/a.dmg
/dev/disk2                    /Volumes/Emet15E302.arm64CustomerRamDisk
morpheus@Zephyr (/)$ ls -F  /Volumes/Emet15E302.arm64CustomerRamDisk
System/  bin/    dev/    etc@    mnt1/   mnt2/   mnt3/   mnt4/   mnt5/   mnt6/   mnt7/    
private/ sbin/   usr/    var@
morpheus@Zephyr (/)$ ls -F /Volumes/Emet*/System/Library/LaunchDaemons
com.apple.PurpleReverseProxy.ramdisk.plist  com.apple.restored_external.plist
com.apple.ReportCrash.restored.plist        com.apple.syslogd.plist

As an alternative to extraction (or in cases where your home system is not MacOS), the fsleuth tool (discussed in II/8) can be used to explore the images. The tool will automatically locate the filesystem signature, enabling operations on the image while still encapsulated by the IMG4 container.

If you try this experiment on both Update and Recovery ramdisks, you will see that they are virtually identical. One of the key differences is the choice of restored variant that is executed. In the Update ramdisk, it is /usr/local/bin/restored_update, whereas in the Recovery ("Customer") ramdisk it is /usr/local/bin/restored_external.

Additional firmware images

In addition to the iOS image files, *OS software images also contain files used for the other firmware components, such as the AOP, baseband, and other embedded processors. The number of these steadily increases with the complexity of the SoC, as shown in Table A-12:

Many of the firmware images of the SoC components are Mach-O, embedded in the IM4P. Apple brought back the MH_PRELOAD file type (0x5) from oblivion, which makes sense in this case as all the images are effectively statically linked. The LC_UNIXTHREAD directs execution in the images to address 0x0, which in turn branches to the ARMv8 reset handler. The images all refer to "RTKit", which is Apple's Real Time Operating System framework for embedded components. More discussion on the dedicated processors, including some aspects of their software images, can be found in II/1.

OTA Updates

The *OS software update mechanism is somewhat different than that of MacOS. Up until iOS 5, updates could only be initiated over iTunes, by putting the i-Device in restore or update mode. It was only in iOS 5 that Apple introduced the notion of Over-The-Air (OTA) updates, which necessitated an on device daemon to handle the download and installation. Since then, jailbreakers everywhere have grown to loathe the incessant OTA update nag notifications - as they appear in the most inopportune times and often seem intent on wearing down (or tricking) the user to update - an irreversible action due to Apple's draconian iOS-signing policy.

*OS: softwareupdated

The iOS and TvOS daemon is also called softwareupdated, runs from within the private MobileSoftwareUpdate.framework's Support/ folder (In WatchOS, the daemon is nanosubridged, from SoftwareUpdateBridge.framework). Although the daemon is similarly named, its behavior is quite different. Not only does it operate on its own (and often when least expected to), but it is actually detached from its main logic - what Apple calls the "UpdateBrain". The brain is an application asset, fetched from Apple's servers by mobileassetd. It is downloaded to /var/MobileAsset/Assets/com_apple_MobileAsset_MobileSoftwareUpdate_UpdateBrain in a UUID.asset folder (similar to other assets), and the AssetData subfolder contains the com.apple.Mobile SoftwareUpdate.UpdateBrainService.xpc service.

The actual software updates are auto-downloaded into /var/MobileAsset/Assets as well - specifically, com_apple_MobileAsset_SoftwareUpdate for the OTA updates (discussed next), and com_apple_MobileAsset_SoftwareUpdateDocumentation for the user-facing documentation about each update - i.e. localized assets containing the License.html and ReadMe[Summary].html files which the Preferences.app displays to the user.

On jailbroken devices, it's a good idea to revoke the directory permissions of softwareupdated's asset folders. Doing so will prevent any updates from being fetched by the system, and disable the nag pop-ups Apple uses to coerce updating iOS - and thereby potentially losing the jailbreak. Another method is to set mesu.apple.com's IP Address to a non-reachable one (e.g. 127.0.0.1) via /etc/hosts. Side effects, however, may include failure to download "removable" apps, as mesu.apple.com is used as their download source. Yet another option is to subscribe to an incompatible *OS seed (e.g. iOS to TvOS, or vice versa).

OTA Updates are provided as .zip archives, with a particular directory structure, generalized as in Table A-13. 'Generalized', because there are subtle model differences, which affect graphic resolution and sometimes merit additional firmware images for coprocessors. Also, less consequential files (like additional plists accompanying .im4ps) have been omitted for brevity.

With the exception of the payload and patch file(s), OTA updates are virtually identical to the IPSWs. And it is regarding the payload files that this author, made a simple but powerful observation: Because payloads were handled post iOS boot, they could not - by design - be encrypted. As discussed in II/@BOOT, access to the firmware decryption key (also known as the "GID Key") is lost once iBoot transfers control to the kernelcache. It follows, therefore, that if one got an OTA and had a previously jailbroken device - and thus access to an unencrypted file system image - it would be a simple matter to reconstruct the updated iOS filesystem by manually applying the patch files to the older files, even outside the device. In a series of articles ("The OTA Trilogy" and its followups^[5]), both the initial packaging format - pbzx - and the proprietary cpio(1)-like formatting used post (XZ) decompression were reversed, and open source tools (compilable across all UNIX systems) - pbzx and ota(j) - provided for them.

At this point, another powerful observation was made - even though at the time .ipsw images were encrypted, for whatever reason Apple provided full OTA updates in some cases. Unlike the partial updates, which were mostly patch-based with only a few full files (usually for newly added or heavily modified ones), the non-differential ones had no base system to apply to. This meant that unpacking the payload would provide a full filesystem image, unencrypted and outside the i-Device, relieving the need for a base iOS filesystem from a jailbroken device.

With iOS 10 and its counterparts, Apple seems to have given up on filesystem encryption. Still, there are devices - like the Watch and the "HomePod" smart speaker, for which there are no images save OTA ones. The tools you'll find on the companion website are thus still highly useful. The OTA trilogy was extended further, and the tools have evolved, most recently to support PBZX file modifications and to allow search functionality inside the payload archive. This is shown in the following experiment.

Experiment: OTAs as a tool for *OS reversing

Whenever Apple releases an OTA - be it for a system update or a *OS beta version - it is usually a matter of hours, if not less, before the links show up on the iPhone Wiki (https://www.theiphonewiki.com/wiki/OTA_Updates). It is then a simple matter to use a browser, or the command line (wget/curl) to obtain the image^*.

Unpacking the OTA used to be a three step process - first recovering the pbzx chunks, then XZ-ing, and finally unpacking the format. The author's pbzx tool has since progressed, and is now capable of applying XZ inline, as well as correctly putting together both chunk types (compressed and uncompressed). Using pbzx is therefore sufficient to end up with an ota archive, on which the author's ota can be applied:

morpheus@Chimera(.../13.0b2)$ unzip 00d8558a7fe0be74dee2bf2d66c357dd728d2d14.zip
..
 extracting: AssetData/payloadv2/links.txt  
 extracting: AssetData/payloadv2/payload.000  
 extracting: AssetData/payloadv2/payload.000.ecc  
..
#
# Move to payloadv2 directory, and expand the payload.0?? files (ignoring .ecc) using pbzx
#
morpheus@Chimera(.../13.0b2)$ cd AssetData/payloadv2
morpheus@Chinera (../AssetData/payloadv2)$ for i in payload.0??; do pbzx $i > $i.ota; done
Flags: 0x800000
Chunk #1 (flags: 46f2fe, length: 247876 bytes) 
OK!  (4644864 bytes)
.
#
# Extract (for example, all files) with ota tool
#
morpheus@Chimera (../AssetData/payloadv2)$ mkdir /tmp/fs
morpheus@Chimera (../AssetData/payloadv2)$ cd /tmp/fs
morpheus@Chimera (/tmp/fs)$ for i in ../*.ota; do ota -e '*' $i; done
morpheus@Chimera (/tmp/fs)$ ls -F
Applications/  bin/  etc*  Library/  private/  sbin/  System/  tmp*  usr/

The most common function of ota used to be extraction (-e), which can still be used to unpack the entire filesystem (-e '*', as demonstrated above), although it is often easier to just mount the DMG from an IPSW directly or (on Linux) using FSleuth.

The search function, however, still deserves special mention - as it enables a quick method by means of which a reverse engineer can find occurrences of a string or pattern across the (packed) filesystem - immediately pinpointing which file in the OTA contains it. The method is far superior to its alternative (searching each individual file), and is exceptionally useful in figuring out entitlements, symbol dependencies and more.

#
# Example: Search for all occurrences of "filecoordination". -s searches any string match, 
# -S forces null terminated strings only. -v displays areas around match
#
morpheus@Chimera (../AssetData/payloadv2)$ for i in *.ota; do ota -v -s filecoordination $i; done
System/Library/Caches/com.apple.dyld/dyld_shared_cache_arm64e, @0x152ec9ef(rel)/0x152eca4a(abs): 
 .filecoordinationd waiting for response..
..
usr/sbin/filecoordinationd, @0xcc3e(rel)/0x9ba167e(abs): ..com.apple.filecoordinationd
System/Library/LaunchDaemons/com.apple.FileCoordination.plist, @0x102(rel)/0xff027(abs): 
 imit..._../usr/sbin/filecoordinationd[Interactive......._.%com.apple.FileCoordination...

Note, that in-OTA searching is just a much faster, but still context-insensitive search. It therefore makes sense to be as specific and case-sensitive as possible, to reduce partial matches which could end up in false positives.

iOS Beta Program

Apple releases iOS betas as early as June every year, to coincide with the announcements made at the WWDC. These were originally developer-only, but (as with MacOS) Apple has extended the program to die-hard enthusiasts who want to test out the latest and greatest features.

Subscribing to a beta program is similar to MacOS, but due to iOS's strict lockdown can only be performed with an Apple-signed configuration profile. This is a .mobileconfig file, normally used for Mobile Device Management (MDM). The file format is an ASN1 encoded, pkcs7-signedData property list, with a MIME type of application/x-apple-aspen-config. When opened, it has the same effects as running the MacOS defaults(1) command in order to set the defaults of the domains listed within, which points the URLs of the MobileAsset domains to the iOSversionDeveloperSeed, rather than the regular update paths:

PayloadContent[0]:
  PayloadIdentifier: com.apple.applebetasoftware
  PayloadType: com.apple.defaults.managed
  PayloadUUID: 6FFD7346-0FAE-47A4-85CD-9E87A43C063F
  PayloadVersion: 1
  PayloadContent[0]:
    DefaultsDomainName: .GlobalPreferences
    DefaultsData:
      SeedGroup: PublicBeta
  PayloadContent[1]:
    DefaultsDomainName: com.apple.seeding
    DefaultsData:  
      SeedProgram: DeveloperSeed
  PayloadContent[2]:
    DefaultsDomainName: com.apple.MobileAsset
    DefaultsData:
      MobileAssetSUAllowOSVersionChange: false
      MobileAssetSUAllowSameVersionFullReplacement: false
      MobileAssetServerURL-com.apple.MobileAsset.SoftwareUpdateDocumentation: 
	https://mesu.apple.com/assets/iOS13DeveloperSeed
      MobileAssetServerURL-com.apple.MobileAsset.SoftwareUpdate: 
	https://mesu.apple.com/assets/iOS13DeveloperSeed
      MobileAssetServerURL-com.apple.MobileAsset.MobileSoftwareUpdate.UpdateBrain: 
	https://mesu.apple.com/assets/iOS13DeveloperSeed
      MobileAssetAssetAudience: D8AB8A45-EE39-4229-891E-9D3CA78A87CA
PayloadDescription: Configures your iOS/iPadOS device for use with the Apple Beta Software Program.
PayloadDisplayName: iOS 13 & iPadOS 13 Beta Software Profile Beta Software Profile
PayloadIdentifier: com.apple.applebetasoftware
PayloadOrganization: Apple Inc.
PayloadType: Configuration
PayloadUUID: 22C1C514-0036-44E9-990D-CFC27173F27B
PayloadVersion: 1
ConsentText:
  default: iOS & iPadOS Beta are pre-release software.  Your use is subject to and licensed ..
	   ..  about restoring your device, or visit: https://support.apple.com/en-us/HT203282
RemovalDate: 2020-09-30T00:00:00Z
PayloadRebootSuggested: true
TargetDeviceType: 1

The APTicket

In the early versions of iOS, Apple seemed content to rely on the powerful encryption and authentication of boot components. The string of iOS 4.x jailbreaks, however, changed this view. One can imagine the powers that be at Cupertino riled by jailbreaks defacing the cherished Apple logo and replacing it with (*gasp*) a pineapple. Additionally, people made use of iTunes' undocumented ability to restore to any image when the alt (option) key was held down.

It was in iOS 5, therefore, that Apple introduced a formidable defense mechanism in its APTicket. The APTicket is personalized for each device, in that it contains device unique features - notably, the ECID (Exclusive Chip ID), and a boot nonce hash (BNCH) generated by iBoot. Generating a ticket further requires a protocol negotiation with Apple's Tatsu Signing Server (TSS). This makes it time sensitive, and reinforces the concept of a signing window, which is the span of time in which Apple's server will return a valid reply, rather than simply refusing the request. This is one of the ways Apple can ensure Tim Cook can always go on stage in every Apple event and boast the latest iOS version's amazing adoption rates - one reason being, Apple leaves no option but updating/recovering to the latest version.

The APTicket is thus generated by a request to Apple's server, with a property list containing all the attributes of the installation image. A sample of the request is shown in Listing A-16:

POST https://gs.apple.com:443/TSS/controller?action=2 HTTP/1.1
Host: gs.apple.com:443
User-Agent: ..
Content-Length: ...
Accept: */*
Content-Type: text/xml; charset="utf-8"
Accept-Encoding: gzip, deflate

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
	<key>@ApImg4Ticket</key>
	<true/>
	<key>@BBTicket</key>
	<true/>
	<key>ApBoardID</key>
	<integer>28</integer>
	<key>ApChipID</key>
	<integer>32800</integer>
	<key>ApECID</key>
	<integer>...ECID...</integer>
	<key>ApNonce</key>
	<data> ..base 64 encoded BNCH..</data>
	<key>ApProductionMode</key>
	<true/>
	<key>ApSecurityDomain</key>
	<integer>1</integer>
	<key>ApSecurityMode</key>
	<true/>
	<key>AppleLogo</key>
	<dict>
		<key>Digest</key>
		<data>
		kzUWqBBoaWDr5c14I6+ksmpzRoD9e+mZtw9PTXV99KjyT1H88q5RoP438xMLKuao
		</data>
		<key>EPRO</key>
		<true/>
		<key>ESEC</key>
		<true/>
		<key>Trusted</key>
		<true/>
	</dict>
 ... additional values for other IPSW components: 
  AOP, [Restore]SEP, LLB, iBEC, iBSS, iBoot, Battery*, [Restore]KernelCache, 
 [Restore]DeviceTree, RestoreRamDisk, Multitouch, RecoveryMode, StaticTrustCache, 
 and RestoreLogo. In newer devices: ANE, AVE, ISP, GFX and SIO   
</plist>

The resulting ticket is returned as the base64 payload of a an ApImg4Ticket key in a property list, and contains hashes for all IPSW components (identified by their four character tags, from Table A-9), the boot nonce hash, and Apple's signature. The filesystem copy of the APTicket can be found in /System/Library/Caches/apticket.der^*. Since DER is a standard format, the openssl utility (present by default in MacOS) can be used to view its contents, as shown in Output A-17:

mobile@iPhonoclast (~)$ openssl asn1parse  -inform der -in /S*/L*/Cach*/apticket.der
    0:d=0  hl=4 l=5473 cons: SEQUENCE          
    4:d=1  hl=2 l=   4 prim: IA5STRING         :IM4M
   10:d=1  hl=2 l=   1 prim: INTEGER           :00
   13:d=1  hl=4 l=2946 cons: SET               
   17:d=2  hl=9 l=2937 cons: priv [ 1296125506 ] 
   26:d=3  hl=4 l=2933 cons: SEQUENCE          
   30:d=4  hl=2 l=   4 prim: IA5STRING         :MANB
   36:d=4  hl=4 l=2923 cons: SET               
   40:d=5  hl=9 l= 256 cons: priv [ 1296125520 ] 
   49:d=6  hl=3 l= 253 cons: SEQUENCE          
   52:d=7  hl=2 l=   4 prim: IA5STRING         :MANP
   58:d=7  hl=3 l= 244 cons: SET               
   61:d=8  hl=7 l=  30 cons: priv [ 1112425288 ] 
   68:d=9  hl=2 l=  28 cons: SEQUENCE          
   70:d=10 hl=2 l=   4 prim: IA5STRING         :BNCH   # Boot nonce hash
   76:d=10 hl=2 l=  20 prim: OCTET STRING      [HEX DUMP]:24A9F4E46E810BC910C3E7FF41445DB38F9D4845
..
  196:d=9  hl=2 l=  15 cons: SEQUENCE          
  198:d=10 hl=2 l=   4 prim: IA5STRING         :ECID            # Exclusive Chip ID
  204:d=10 hl=2 l=   7 prim: INTEGER           :12543604BA0226
... # followed by digests of tagged components (e.g, kernel)
 1742:d=7  hl=2 l=   4 prim: IA5STRING         :krnl            # Kernelcache
 1748:d=7  hl=2 l=  91 cons: SET               
 1750:d=8  hl=7 l=  30 cons: priv [ 1145525076 ] 
 1757:d=9  hl=2 l=  28 cons: SEQUENCE          
 1759:d=10 hl=2 l=   4 prim: IA5STRING         :DGST
 1765:d=10 hl=2 l=  20 prim: OCTET STRING      [HEX DUMP]:098BA63BD200D173A203A580BB2A91D5D723FD9A
... # followed by Apple signature
 3336:d=6  hl=2 l=  20 cons: SEQUENCE          
 3338:d=7  hl=2 l=   3 prim: OBJECT            :commonName
 3343:d=7  hl=2 l=  13 prim: PRINTABLESTRING   :Apple Root CA
...

Exercise extreme caution when handling the apticket.der file. Accidentally modifying it or removing it will brick *OS devices, forcing them into the familiar (and odious) "Connect to iTunes" mode, wherein - due to Apple's signing window policy - they will only be updateable to the latest, greatest, and probably not-yet jailbreakable version.

The boot nonce hash (generated on every installation/recovery by iBoot), coupled with the online transaction, is explicity designed to prevent downgrades. There is, however, a way around this: As it so happens, Apple left behind a way to fix the boot nonce hash to a known value. This can be done by setting the com.apple.System.boot-nonce NVRAM variable, as is needed during OTA updates. This was figured out by jailbreakers, who created the "Prometheus" tools, which enable pinning the nonce using "nonceenabler", and using "futurerestore". This made Apple not only protect NVRam variables through entitlements, but further try to restrict their writability by mapping them in memory. Clever solutions like Viktor Oreshkin's^[7], however, work around this.

As of the A12 and later, Apple further doubles down against downgrade "attacks" by enabling nonce entanglement. This defeats existing methods to pin the nonce by first generating a key (using the device's UID key) from magic bytes (56 82 41 65 65 51 E0 CD  F5 6F F8 4C C1 1A 79 EF, present also in the ROM and the kernelcache), and using the value to encrypt the boot nonce, so as to ensure it had to have been created on the specific i-device. There is no presently known way to work around this method.

References

1. NewOSXBook.com - seedutil reversed source -
   http://NewOSXBook.com/src.jl?tree=listings&file=seedutil.m
2. NewOSXBook.com - pbzx unpacker -
   http://NewOSXBook.com/src.jl?tree=listings&file=pbzx.c
3. The iPhone Wiki - iPhone Wiki Firmware Page
4. Bridge OS Update XML - https://mesu.apple.com/assets/bridgeos/
   com_apple_bridgeOSIPSW/com_apple_bridgeOSIPSW.xml
5. PlanetBeing - Partial Zip - https://github.com/planetbeing/partial-zip
6. NewOSXBook.com - The OTA Saga - http://NewOSXBook.com/articles/OTA.html
7. Viktor Oreshkin - "What does NVRAM lock/unlock actually mean" - https://stek29.rocks/2018/06/26/nvram.html

Post Scriptum

The first (technically, second) volume of the series draws to its end, and with it we leave user mode behind. The next volume will deal almost exclusively with the kernel, but also find room for discussing the hardware of the Mac and various i-Devices, as well as one intentional omission - the network stack.

As with Volume III, which (at the time I'm updating this) has been out for two years in which it had seen six major revisions (i.e. additional chapters, four in the first year and two in the second), I hope to keep this volume updated as well - This version has already been revised with minor additions for Darwin 18, 19 (betas) and an entirely new chapter (0x10 - on user mode aspects of networking), and the Appendix. Unlike Volume III, however, wherein the updates are governed primarily by new jailbreaks, I expect this one to be updated more slowly. You can track the current update status of the series at http://NewOSXBook.com/ChangeLog.html.

As with what I've started with "Android Internals", and Volume III - if you find any factual error in this book, (assuming it pertains to the versions covered by the book, and not later Darwin, wherein Apple might have changed the API, or *OS/MacOS subtleties), I will gladly reward you with 0.01BTC, and immortalize you in the book's change log. This is significantly down from the 0.1BTC I offered only about year ago, but then BTC went up from $700 to just breaking $12,000(!) as I finished v1.0, and $19,000(!!) two weeks later with v1.0.1, though it's been relatively stable at around $6,500 or so recently. Fortunately, nobody has caught any such errors, so my stash of 2BTC I've allocated for "damages" is safe :-). Although from the looks of things in mid June 2019 BTC is back around $9,000 - so if you do find anything, better let me know and cash in $90! I hope Eddie (and later, Peter) managed to catch all the typos, but from experience, there's always more. If you find them, please let me know through the book's forum - I'd appreciate that.

Remember, that so much advanced functionality is not without its pitfalls: If you've read this book with a security oriented mindset (as I assume a fair portion of my reader-base is), you'll no doubt notice some potential security vulnerabilities^*. Unlike the trivial coding errors which result in memory corruption, a few of these are "baked into" the design - but Apple will likely fix these as they have most others to date - by slapping an entitlement onto them. Unfortunately, the more Apple does so - and they very well should - the more they lock out legitimate (albeit uncommon) use cases, primarily those of my various tools. The com.apple.private.* entitlements won't be given to anyone outside Cupertino, and it's a shame - as Apple's command line tools fall far short of their promise.

I hope to see you again in Volume II - which is hot off the presses and concludes this magnum opus of a trilogy... Or maybe even see you in person, in one of the trainings offered by Technologeeks.com - both the 5-day "OSX/iOS for Reverse Engineers" and the shorter (3-day) "Applied *OS Security". If you've really read this far - mention this when you book a training, and maybe they'll cut you a 5% discount if you bring the book. Heck, I could even sign your copy, if you want ;-)