JTool II: Testers wanted

Used for discussing the various tools in the book as well as encouraging members to share tools

JTool II: Testers wanted

Postby morpheus » Sat Sep 22, 2018 6:45 pm

Hello all,

JTool2 is a radical rewrite of jtool to make it more featureful and extensible, and less buggy.

From the ChangeLog (WhatsNew)

- Matched JTool features:

- -F (find string)
- -a
- -l
- -S, -v
- --pages (automatically in quick)
- -h (for shared cache)

- Everything is faster. Especially disassembly, and in some cases by an order of magnitude:
- When using -q jtool2 is produces almost same output (but faster than) otool/objdump
- Without -q jtool2 is still on par with otool, often faster, AND provides strings + basic decompilation
- When testing on SpringBoard (THE pathological case), -q -d finishes in under 2s! jtool v1 would get take minutes (even with -q)
- Caching in particular is way faster.
- Jtool v1 had an unnecessary cache lookup when adding, which slowed it considerably. No more.
- Sequential address lookups in cache now use cursor

- Decompilation is smarter:

- JTool detects the number of arguments for a given function automatically. Two corner cases are:
- First function call in a given function (won't know how many arguments)
- Functions with no arguments (void) may still show up with their first argument
- Arguments are also autodetected by type! No need to specify the ([ic@b...) etc (although this
will be supported again in a future release, for those cases where the arguments need to be refined)


- New Architectures: arm64e (A12 chipset and later), ARM64_32 (0x2000000c/0x1, for Apple Watch 4)

- ARMv8.1 instructions (CASP, PAN so far)

- ARMv8.3 instructions (so you don't have to wait for IDA plugins from people who won't ever deliver)
B[L]RA[A/B][Z], RETA[A/B], LDRA[A/B] - still need ERETA[A/B] and XPACD, XPACI, XPACLRI (C6-1002)
AUTIA/PACIA/PACGAA Appear to be unused as yet by AAPL

- Can now work directly on *COMPRESSED* kernelcaches! Plan is to integrate joker fully into jtool2 - and soon

- Fix for weird functions like zinit, which seems to ignore (i.e. not get) some arguments (case in point, X2 is not
set, while X3 is! Weird)

- The '-a' and '-o' switch are now much more precise when in shared caches


- Dyld-625 support: BIND_SUBOPCODE_THREADED_SET_BIND_ORDINAL_TABLE_SIZE_ULEB and BIND_SUBOPCODE_THREADED_APPLY
Shows opcodes in -opcodes
Successfully reconstruct the bindings for dyldinfo -bind (even though it's no longer the real table..)


Now, I'm still not caught up on all the features of jtool v1, notably code signing and such, but that's in the works, and when it's done, I'll retire jtool v1 for good.


PLEASE HELP TESTING by looking at WhatsNew.txt and trying out features! If you see something (crash) say something (to me) over this forum. You will help improve the tool, and I'll mention your found bugs in the ChangeLog :-) Thank you!

IMPORTANT NOTE: JTOOL v1 crashes hideously on arm64e (A12+) binaries. This is resolved in jtool2. Going forward, I will not fix jtoolv1 - just use jtoolv2

Also, feature requests are more than requested. I'm currently working on fixing the shared cache extraction so it creates perfect dylibs, as well as in-kernelcache processing, so as to meld joker into jtool2.

Latest build always available at http://NewOSXBook.com/tools/jtool2.tgz, for ARM64, x86_64, and Linux.
morpheus
Site Admin
 
Posts: 716
Joined: Thu Apr 11, 2013 6:24 pm

Re: JTool II: Testers wanted

Postby zchee » Sun Sep 23, 2018 12:38 pm

Hi, morpheus.
I'm zchee. I was sent twitter mentions and posted http://newosxbook.com/forum/viewtopic.php?f=3&t=16800. I hope you still remember me.

Thank for everything develop great tools :D

I was short testing jtool2 2.0 (alpha 1, Paris), and I found some nits bugs.

    1. Run "jtool2 --help", will display the jtool [options] _filename_.
      But it's jtool2.

    2. In the jtool2 --help, dyldinfo Compatible Options section, display the
    Code: Select all
    dyldinfo Compatible Options:
       --bind        print addresses dyld will set based on symbolic lookups
       --lazy_bind   print addresses dyld will lazily set on first use
       --opcodes     print opcodes used to generate the rebase and binding information
       --function_starts   print table of function start addresses

      But it will show the "--[flag] - Unknown option".
      Actually, the dyldinfo Compatible Options flags are available the single hyphen(-)

    3. In the jtool2 --help, not shown the "--pages" flag. Maybe adding Newer (JTool 2) Options section...?

Other than that, it works perfectly in my environment so far. Thank you for a nice tool !!


BTW, as before, I wrote a zsh completion script for jtool2. I am happy if morpheus and everyone who uses jtool2 can help.
The script code is below. Or I pushed to my zchee/zsh-completions(https://github.com/zchee/zsh-completions) github repository.
https://github.com/zchee/zsh-completions/blob/master/src/zsh/_jtool2

Code: Select all
#compdef jtool2

# -----------------------------------------------------------------------------
# Copyright (c) 2018, Jonathan Levin (@Morpheus______ / http://newosxbook.com)
# All rights reserved.
# -----------------------------------------------------------------------------
#
# jtool2
#   http://newosxbook.com/tools/jtool2.tgz
#
# version: 2.0 (alpha 1, Paris) compiled on Sep 22 2018 17:55:31
#
# -----------------------------------------------------------------------------
#
# Usage: jtool2 [options] _filename_
#
# OTool Compatible Options:
#    -h            Dump Mach-O (or DYLD Shared Cache) header
#    -l            List sections/commands in binary
#    -L            print shared libraries used
#
# JTool options:
#    -S            List Symbols (like NM)
#    -v            Toggle verbosity
#
# New Options:
#    -q            Quick operation - do not process any symbols in the Mach-O
#    -F            find all occurrences of _string_ in binary
#    -a            Find offset/segment corresponding to virtual address _addr_
#    -o            Find address corresponding to offset _offset_
#    -d            Dump (smart dump, will disassemble text and dump data by autodetecting)
#
# Newer (JTool 2) Options:
#    --analyze     Analyze file and create a companion file
#
# dyldinfo Compatible Options:
#    -bind        print addresses dyld will set based on symbolic lookups
#    -lazy_bind   print addresses dyld will lazily set on first use
#    -opcodes     print opcodes used to generate the rebase and binding information
#    -function_starts   print table of function start addresses
#
# Environment Variables:
#    ARCH                   Select architecture slice. Set to arm64, arm64e, arm64_32, armv7, armv7k, x86_64 or (not for long) i386
#    JDEBUG                 Enhanced debug output. May be very verbose
#    JCOLOR                 ANSI Colors. Note you'll need 'less -R' if piping output
#    JTOOLDIR               path to search for companion jtool files (default: $PWD).
#                           Use this to force create a file, if one does not exist
#    NOPSUP                 Suppress NOPs in disassembly
#
# -----------------------------------------------------------------------------

function _jtool2() {
  local context curcontext=$curcontext state line ret=1
  declare -A opt_args

  _arguments -C \
    '-h[dump Mach-O (or DYLD Shared Cache) header]' \
    '-l[list sections/commands in binary]' \
    '-L[print shared libraries used]' \
    '-S[list Symbols (like NM)]' \
    '-v[toggle verbosity]' \
    '-q[quick operation - do not process any symbols in the Mach-O]' \
    '-F[find all occurrences of _string_ in binary]:__string__' \
    '-a[find offset/segment corresponding to virtual address _addr_]:virtual address (0x...)' \
    '-o[find address corresponding to offset _offset_]:offset (0x...)' \
    '-d[dump \(smart dump, will disassemble text and dump data by autodetecting\)]' \
    '--analyze[analyze file and create a companion file]' \
    '--pages[show pages]' \
    '-bind[print addresses dyld will set based on symbolic lookups]' \
    '-lazy_bind[print addresses dyld will lazily set on first use]' \
    '-opcodes[print opcodes used to generate the rebase and binding information]' \
    '-function_starts[print table of function start addresses]' \
    '*:filename:_files' \
    && ret=0

  return ret
}

_jtool2 "$*"

# vim:ft=zsh:et:sts=2:sw=2


For morpheus, I wrote
Copyright (c) 2018, Jonathan Levin (@Morpheus______ / http://newosxbook.com) All rights reserved.
Please let me know if there is a problem with the license.

Also, I pushed filemon, joker, jtool, jtool2, kdv, and procexp zsh completion script to github. Likewise, if it is a problem I will delete it so please let me know.
https://github.com/zchee/zsh-completions/blob/master/src/zsh/

J Says: Not only is it no problem, it is great to have completion scripts!!! As for --arguments , you are right - I will fix, thank you! As for "jtool2", I still call the tool jtool internally because very soon jtool (1) is going to go away, and I want people to still use the same name in scripts and so on
- zchee
zchee
 
Posts: 6
Joined: Tue Dec 15, 2015 3:39 am

Re: JTool II: Testers wanted

Postby jonios » Wed Oct 17, 2018 11:43 pm

The -L argument doesn't seem to work. Sorry if thats known, its just the first thing I tried.

Code: Select all
./jtool2 -L myapp
Note: 13701 symbols detected in this file! This will take a little - generate a companion file or use '-q' for quick mode..

Where myapp is a swift arm64 macho. jtool1 spits out the dynamically linked libraries
Code: Select all
$ jtool -L myapp
...
/System/Library/Frameworks/UIKit.framework/UIKit
@rpath/libswiftCore.dylib
@rpath/libswiftCoreFoundation.dylib
@rpath/libswiftCoreGraphics.dylib
...


BTW, just noticed jtool2 is missing a word in the output: "this will take a little" (while is missing)

J says: Fixed, and typo fixed too. I had forgotten to implement -L.... http://NewOSXBook.com/tools/jtool2
jonios
 
Posts: 4
Joined: Tue May 01, 2018 10:11 pm

Re: JTool II: Testers wanted

Postby morpheus » Mon Oct 22, 2018 10:26 pm

Alpha 3: joker integration , especially for stripped 1469 caches - this will bring back almost 1,000 symbols

http://NewOSXBook.com/tools/jtool2.tgz
Attachments
Screen Shot 2018-10-22 at 6.05.31 PM.png
Screen Shot 2018-10-22 at 6.05.31 PM.png (138.71 KiB) Viewed 3404 times
morpheus
Site Admin
 
Posts: 716
Joined: Thu Apr 11, 2013 6:24 pm

Re: JTool II: Testers wanted

Postby morpheus » Wed Nov 07, 2018 2:22 pm

01/11/2018 (Maui!)
------------------

- jtool -l now does 32-bit again..
- Supports MH_PRELOAD. Yes, I know XNU doesn't. But iPhone11,*'s Petra does! So you can now disassemble those images (can your IDA do that?)

- Code signature improvements: --sig does 0x20500 (Thanks to Jeremy Agostino!)
Finally, requirement parsing works (Again, thanks to Jeremy Agostino asking for it)
Colorized output :-)
- (VERY) Initial support for notarization tickets


I'll be bringing back --sign (to self sign) very soon, and probably some rudimentary ARM32/Thumb disassembly again. But not in this version



http://NeWOSXBook.com/tools/jtool2.tgz
morpheus
Site Admin
 
Posts: 716
Joined: Thu Apr 11, 2013 6:24 pm

Re: JTool II: Testers wanted

Postby morpheus » Wed Nov 21, 2018 7:36 pm

- Bug fixes for -S
- New feature "--tbd" - works directly on dyld_shared_cache_arm64:framework_name and spits a TBDv3 file to stdout. Tested this on a couple of frameworks and it's valid, so you can link with it.

Code: Select all
mkdir -p /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS.sdk/System/Library/PrivateFrameworks/NetworkStatistics.framework

./jtool2 --tbd /Volumes/PeaceB16B92.N102OS/System/Library/Caches/com.apple.dyld/dyld_shared_cache_arm64:NetworkStatistics  > /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS.sdk/System/Library/PrivateFrameworks/NetworkStatistics.framework/NetworkStatistics.tbd


Latest build always available at http://NewOSXBook.com/tools/jtool2.tgz, for ARM64, x86_64, and Linux.
morpheus
Site Admin
 
Posts: 716
Joined: Thu Apr 11, 2013 6:24 pm

Re: JTool II: Testers wanted

Postby pwm » Fri Dec 07, 2018 7:24 pm

I think the ARM64 slice of the FAT binary is the wrong version. If I run the Linux or macOS versions, it shows

Code: Select all
$ jtool2
Welcome to JTool 2.0 (alpha 5, still Maui) compiled on Nov 21 2018 14:12:38. Try "--help" for help


whilst if I run on iOS it shows

Code: Select all
$ jtool2
Welcome to JTool 2.0 (alpha 3, Belize City) compiled on Oct 29 2018 15:46:54. Try "--help" for help


Doing "strings" on the FAT binary shows both Maui and Belize City strings rather than two Maui strings as I would expect.

In addition the iOS version seems to be missing the --ent and --sig commands.
pwm
 
Posts: 1
Joined: Sun Aug 14, 2016 5:19 am

Re: JTool II: Testers wanted

Postby morpheus » Sat Dec 08, 2018 10:11 pm

Oops. Fixed. Also updated to nightly build (Columbia) http://NewOSXBook.com/tools/jtool2.tgz
morpheus
Site Admin
 
Posts: 716
Joined: Thu Apr 11, 2013 6:24 pm


Return to Tools

Who is online

Users browsing this forum: No registered users and 0 guests