jtool - Taking the O out of otool(1), and so much more
What is this?
The jtool utility started as a companion utility to the 1st edition of MacOS internals, because I wanted to demonstrate Mach-O format intrinstics, and was annoyed with XCode's otool(1). Along the way, jtool absorbed additional Mach-O commands such as atos(1), dyldinfo(1), nm(1), segedit(1), pagestuff(1), strings(1) , and even codesign(1) and the informal ldid. Most importantly, it can be run on a variety of platforms - OS X, iOS, and even Linux, where Apple's tools don't exist.
But that's not all. jtool provides many many novel features:
in-binary search functionality
built-in disassembler functionality with (limited but constantly improving) emulation capabilities, which already outdo fancy commercial GUI disassemblers.
Color terminal output, enabled by JCOLOR=1
As the code got more and more complex, I decided to rewrite jtool from scratch, bringing you jtool2 - and effectively deprecating the v1 binary. New features in jtool2 include:
--analyze to automatically analyze any Mach-O, generating a companion file.
kernelcache symbolication (what I formerly provided via joker) - which has become even more important since the advent of monolithic ("1469") kernelcaches, with no more symbols. jtool2 finds syscalls, Mach traps, MIG tables, interesting (for me, at least) functions, and IOKit objects - thousands of objects in all.
Panic log symbolication: *OS panic logs are JSON and have little to no symbols - but --symbolicate (with a companion file prebuilt by --analyze) will rectify that.
jtool and jtool2 ENTIRELY FREE for use of any type (AISE), and the latest version can always be found right here. For the legacy v1 download, click here, which I'm leaving here because I still am not finished with Objective-C support in v2.
Important: Jtool has been thoroughly tested, but I still rely on your bug reports to tackle some of the more exotic animals in the Mach-O menagerie. If you have an interesting binary which JTool is struggling with or if it (*gulp*) crashes, PLEASE LET ME KNOW. I can't fix bugs I don't know about.. and ranting on Twitter that you got a segfault is useless because a) I don't check Twitter much b) jtool2 was not meant to consider malicious malformed Mach-Os - it is targeting AAPL's own binaries and c) you're just denigrating a powerful, useful and FREE tool. If you think you can do better, write your own.
So let's go over options, one by one:
There are so many useful features to jtool that I can't put them all in the man page anymore. Truthfully, nobody reads the man anyway (since you have to move jtool.1 to /usr/share/man/man1). So I finally put up a decent HTML document. That, too, is growing out of proportion, so even here not everything is covered, but here's a hyperlinked ToC to what is:
When I started work on jtool, I thought I would make it a drop-in replacement for otool, so some of its options are in fact identical. Those include -L (to show library dependencies, similar to ldd), and -f (for FAT headers). Others are compatible, but deviate. For example: -h for the Mach-O header, will print the details about the header not on a single line, but on multiple ones:
The reason I deviated is to make jtool more grep(1) friendly, so you could isolate the line containing the attribute you want. In the case of the most common option (-l), however, it actually makes sense to combine the output from otool's multiple lines into less lines, in order to make it more friendly. Case in point:
otool's output is simply terrible, in that it is thinly spread over multiple lines. jtool condenses it so you can clearly see what matters, and take less space (or scrolling through pages and pages of more(1) or less(1)). Another benefit is using grep:
Notice in the above use of -v, which adds verbosity - in the case of LC_SEGMENT commands, the file mappings.
For universal (fat) files, jtool provides the same -arch switch to single out a particular architecture. There are several differences from otool:
jtool will refuse to touch a fat binary unless you specify the architecture
You can specify the architecture with -arch, or specify ARCH= as an environment variable. This makes it useful to set a default.
You can specify an architecture by number
The last option there seems unusual - after all, how many architectures can there be in a fat file? Yeah, well, I thought so too, until I encountered TaiG's 8.4 Jailbreak, which not only uses fat binaries of some 26-27 architectures, but also uses duplicate entries - confusing otool since it only matches the first one - but jtool can single out a particular slice this way. (q.v my writeup on that jailbreak here for an example).
dyldinfo(1) compatible options
The dyldinfo(1) utility is really useful to explore the inner workings of Apple's dynamic loader. But it's not available for iOS. jtool can match most of dyldinfo's options easily, and can in fact emulate it busybox-style, if you symbolically link dyldinfo to it and call it:
Since jtool basically mimics dyldinfo 1:1, refer to the latter's man(1) page for more detail. I use -function_starts, -bind and -lazy_bind most often.
You may be familiar with the pagestuff(1) command, which dumps the layout of a Mach-O binary. Syntax is odd, placing the filename as the first argument and a bit crude:
jtool --pages provides (what I believe to be) nicer output, like so:
Sometimes you're just interested in a quick lookup of an offset to an address, without having to sift through pagestuff(1) or jtool --pages output. That's where -a comes in:
With -o doing the offset to address mapping.
-S does the same thing that nm(1) does.
The option is simply -S (uppercase) and -v for full info (as in, which dylib the symbol is in). For nm, that -v is -m. Unlike dyldinfo, the options aren't compatible.
What's really useful here is the quick scriptability, like if you're looking for a particular symbol across many binaries:
Well, not quick, maybe (it grows on you), but certainly efficient.
(Luca, you can finally ln -s ......jtool2 /usr/bin/nm :-)
Find a string in a file. -v will also show you the string, highlighted if part of a larger string:
Kernel Caches (v2)
The joker(j) tool is effectively deprecated. It can now all be done with --analyze:
The companion file is intentionally designed to be readable and editable, unlike dSyms or databases which may get corrupted. When you next get them symbols from Lumina(TM), remember where they must have been sourced from..
--symbolicate with crash log supplied, assuming companion file from --analyze was created. You know. Like all some high priced productz do (often, poorly), only free.
DYLD Shared Caches
Apple prelinks most dylibs and plugins into a "Shared Library Cache". The SLC is located in /var/db/dyld (OS X) and /System/Library/Caches/com.apple.dyld (iOS). The OS X cache also has a "map", but the iOS one doesn't have a map.
In iOS, there are no free floating dylibs - it's all in the cache. This makes caches very important, and there are quite a few "decaching" tools. JTool doesn't offer comprehensive support, only the stuff I find useful. In particular:
Jtool can easily decache files - simply use -e dylib on a cache file to get your dylib. Note that the extracted dylib will be big - circa 50MB - because JTool does not deconstruct the merged __LINKEDIT segment.
But why extract??? A key feature of Jtool that other decachers do not have, is its ability to work on a dylib while still in the cache! This not only saves you disk space, but also allows you to see how cached dylibs interact with eachother (e.g. cross dylib calls). To use this feature, simply specify ":" as a delimiter between the cache and the dylib name. All the standard features (e.g. -l, -S, etc), work, but the really useful feature is -d. For example:
jtool now recognizes objective-C. Use "-d objc" to get a list of classes:
Using -v here will actually reverse the classes, a la classdump-Z, though with a bit more detail, and - as usual - color :-). You can also specify a classname as the argument to -d, and jtool is smart enough to figure out you want to reverse it:
(Yes, I know that the protocols aren't demangled yet - I'm working on that)
AND you can then use -d to get to a particular method. To get buy the horrid +/-/[/spaces, you can use C++ like syntax:
Note that this is done by reconstructing the classes without linking with Apple's Objective-C library so it also works on the Linux version! However, because I do the class walking by myself it might be A) still a bit slow (Springboard is a prime example, with countless classes) and B) possibly buggy. You can use NOOBJC=1 to disable objective-c, but please let me know if you encounter bugs here.
Darn Cool Options
MIG Detection (v0.98.9999 02/02/2016)
No special switches for this one - If jtool detects you're dumping a __DATA.__const segment of a MIG enabled binary (e.g. the kernel or one of the many daemons of /usr/libexec), it will automatically ferret out the MIG dispatch tables. In full color, too :-)
You can try this on the OS X kernel for fun (that's /System/Library/Kernels/kernel), but doing so on the iOS kernel is even more useful. That said, there's also Joker for that.
Misc. Features (Jan 2017)
Significant speedup when companion files are detected
More robust OBJ-C handling - won't crash as much. And please see note below on reporting crashes!
New option: -D: to decompile - like -d, but will only show "; " lines, i.e. lines that Jtool decompiled. You could always do that with grep(1), but color got confusing.
Can your IDA do this:
(err... maybe some IDAPython can do it. Dunno. But jtool does it for free :-)
Forget extracting from the Shared Library Cache! jtool can now resolve all symbols in the cache, even when they are in a different dylib! This is super useful because AAPL's reconstructed libraries (in ~/Library/Developer/Xcode/*DeviceSupport) lose external symbols and mess up some __DATA refs. Note that SLC files still have private symbols <redacted>. Working on it.
Emulating the stack for functions is something I've been putting off for a long time. But recently I've decided it's time.. And by only following STR/STP I've managed to go a reasonably long way: jtool can now detect blocks created on the fly, and tell you what the function they contain is. For example, dispatch_[a]sync or xpc_connection_set_event_handler:
Better still, it can follow mach_msg construction on the fly. This is really super useful - One of my @TODOs for MOXiI Vol.1 is to document all of SpringBoard, Backboard, and the other daemon's MIGs. You can see how much time it saves me to just do this:
Functions can now be defined with any argument, and jtool will easily decompile them for you! This has actually been around for a few months now, but I never documented it... And I can't even begin to extol how useful this is when reversing:
So you can now define your functions in the companion file with parentheses and one letter per argument, denoting type:
p - pointer
s - char *
c - char
i - uint32_t
@ - CFString
x - hex
B - block (^)
l - uint64_t
jtool has long been able to work on dylibs inside the shared cache - and actually has the ability to symbolicate references which would otherwise be lost when extracting - no matter how sophisticated your "decache" script is, as soon as the file is outside the cache any references to other files still in the cache are effectively lost. AAPL keeps modifying the various ways calls can be made - either by resolved stubs, direct calls to the function or (as of DYLD-5xx) "branch islands". But jtool supports them all.
Code Signing Options
Code signing options are, IMHO, the 2nd most useful feature of JTool. With iOS security revolving around code signatures and entitlements, it's important to have a way to quickly determine what given entitlements a binary possesses and how it is signed. OS X has codesign(1), but I find it crude (at best) - and what more there's no port to iOS, where it's really necessary.
The --sig option allows you to validate or display a code signature. With no more arguments, it will display the signature blobs, and perform silent validation. jtool2 presents cleaner output than its predecessor, and suppresses the hashes (since there are potentially hundreds) even when using -v:
If you really must display the hashes, use -vv.
If you want to examine the entitlements claimed by a binary, all it takes it --ent. Same as codesign -d --ent, but more accurate, because the latter also presents the blob header (as junk <FA><DE>qq^@^@)^S):
jtool2 also support SimPLISTic format, if you use --ent with -v. This is really useful for heavily entitled binaries - again, taking SpringBoard as an example:
Once again, this is designed with shell scripting and grep(1) in mind, so you can do a for file in /usr/libexec/*; do jtool --ent -v $file; done or other pattern, to get all those undocumented entitlements used by Apple all over the place (That's how I created the Entitlement database.
The --sign option is an option I've had for a long time internally, and finally released. It is meant to self-sign binaries much in the same way as Saurik's ldid. As with ldid, you can specify an entitlements file to be inserted (using --ent filename). The result is designed to be indistinguishable, once you get past the (temporary) warning/disclaimer:
Rather than using codesign(1) or codesign_allocate(1) internally, jtool performs the entire sequence step by step, so you can see for yourself how code signing ensues - using JDEBUG=1:
Code signing using Security.framework APIs (MacOS)
jtool (on MacOS) now uses security framework APIs if you specify --sident (not to be confused with --ident, for the Code Directory Identity) , and specify a partial name or Apple ID or Team ID.
There's much more, but I'll pick this up later. If you use jtool and find bugs, don't just complain silently - LET ME KNOW and I'll be glad to fix. Likewise if you have any suggestions for improvement. Remember - I build the functionality around my own use cases - and I'll be happy to adjust if you have any others.
And I don't care if you use IDA or IDA can do this (if they can't, they'll adopt this in their next version) or there's IDAPython, or whatever. This is the tool I use, I find it useful, and I'm encouraging others to use it (freely, unlike IDA/Hopper and their ilk) and suggest improvements.
More to come:
Full extraction of dylibs from cache
improved code signing
Frequently Asked Questions
Is this open source? No. Neither is my other reversing tool, Dextra (for Android). Free, yes. But not open source.
*Why* is this not open source? Is Hopper? Is IDA? So I'm in good company :-) jtool started as a simple otool -l, but things spiraled out of control when I started doing the disassembly. There's just too much original code in there for me to just share with whatever license, that others can rip and not even mention credit in passing. Sorry. I speak from bitter experience here. It's still FREE though, which is more than what most tools (save, of course, otool) can say.
Can I write plugins? better. You can script it with shell commands (it's designed to be grep(1) and other filter-friendly), and you can write entire programs to use jtool's logic - machlib and disarm - as dylibs.
What's up with color? Why is it disabled when I pipe, and messy when I JCOLOR=1 and pipe? Because color is done via curses. It's the default when you DON't pipe (i.e. jtool detects the stdout is a tty), and when you do it's disabled unless you insist (JCOLOR=1 or export...). But if you do that, use less -R so less (which is more) can handle the sequences. Or try --html for colorized, hyperlinked HTML output you can save to a file.
What's the deal with ARM32/Thumb? Though the disassembler originally was for 32, when it became clear it's a choice between my sanity and support ARM32/Thumb, I opted for the former. This way I can write you other books and tools. jtool actually is pretty good still with ARM, but it's not officially supported. ARM64 is, though, and is fairly comprehensive (sans FP/SIMD, which only SpringBoard and a few other daemons use. I'll get to that at some point, if I get stranded for hours with absolutely no Internet and nothing else to do). Update: ARMv7k is kind of back
Why not use foundstone or some other disassembly framework? Because it wouldn't be Jtool then, now, would it? :-) The code is 100% original and mine, sharing no sources what-so-ever with any other code, open or closed (save for obvious include files)
How do we contribute? By suggesting improvements, asking for features, or reporting bugs on the OS X Book Forum
What can we do that will be COUNTER productive, and not in any way contribute? You can bitch on Twitter whenever you encounter a crash:
and yes, you can use the pathetic otool which is rock solid, because it's as functional as a rock. And, you can NOT read the manual page which distinctly says that jtool does crash in some cases. I wish you could use otool instead, but AAPL ruined what was good in it by rewriting it to use objdump , dumping it down, indeed.
Seriously, IF JTOOL CRASHES FOR YOU, LET ME KNOW AND I'LL FIX IT. COMPLAINING ACHIEVES NOTHING, AND THERE IS AN INSANE AMOUNT OF LOGIC IN JTOOL, SO YEAH, IT AIN'T PERFECT. But hey - it leaves otool in ashes, and it's FREE. I don't expect thanks, but just slamming the tool and not even reporting a bug is being a douche.