DYLD Detailed

1. About

While maintaining and adding more functionality to JTool, I found myself deeply bogged down in implementing support for Mach-O's LINKEDIT sections, LC_SYMTAB, and other arcane and relatively undocumented corners of DYLD. Add to that, DYLD has been relatively skimmed in my book *, and not much in that of my predecessor. Scouring the Internet with Google finds only one decent reference1, though it's woefully incomplete and basically just rehashes stuff from the book. Needless to say Apple makes no effort to provide documentation outside its "Mach-O Programming Topics"2 document, which is by now very dated. What better way, then, to right a wrong and shed some light on it, than an article?

Why should you care? (Target Audience)

I said so in the book, and I'll state it again - There is no knowledge that is not power, and in the case of linking - we're talking about a lot of power. Virtually every binary run in OS X or iOS is dynamically linked, and being able to intervene in the linking process bestows significant capabilities - function interception, auditing and hooking, being the most important ones. Reverse engineers, security-oriented developers (i.e. Anti-Malware) and hackers will hopefully find this information very useful. It should be noted that dyld allows for hooking and interception via environment variables - most notably DYLD_INSERT_LIBRARIES (akin to ld's LD_PRELOAD) and DYLD_LIBRARY_PATH (like ld's LD_LIBRARY_PATH), and its function interposing mechanism. These are covered in the book (somewhere in Chapter 4, with a demo on this website3), and are therefore not discussed in this document.

Prerequisite: About Linking

• Code reuse: commonly used code can be extracted to a library, which is then shared by many clients
• Easy updating: code residing in a library can easily be updated, and the library replaced, so long as the symbols are by and large the same. A classic example of this can be seen in Windows' "CreateWindow", which creates totally different-looking windows for the same application throughout Windows versions (think Win95 vs. XP vs. 7-8). The developer merely says "CreateWindow", not knowing how the window gets created. The OS does the rest, and different versions of the OS may do so differently.
• Reducing disk usage: as commonly used code now has only one copy, as opposed to having to include the code in every single binary which uses it.
• Reducing RAM usage: is by far, the most important advantage: A single copy of the library may be mmap(2)-ed into all processes, thereby only getting hit by the library's RAM usage once. The library code is usually marked r-x (read only, executable), so the same physical copy is implicitly shared by many consumers. This is crucial and saves immense amounts of memory, especially in RAM-challenged systems like Android.

Nomenclature

Throughout this article, the following terms are used:

• dylib: A dynamic library. Akin to a UN*X shared object. A Mach-O object of type MH_DYLIB (0x6), loaded into other executables by the LC_LOAD_DYLIB (0xc) Mach-O command or the dlopen(3) API. For the record, it's worth noting that OS X also supports the concept of a fixed library (A Mach-o object of type MH_FVMLIB (0x3) loaded into other executables by the LC_LOADFVMLIB (0x6) command. Fixed libraries, however, are virtually extinct.
• symbol: A variable or function in a Mach-O file which may or may not be visible outside that file.
• binding: Connecting a symbol reference to its address in memory. Binding may be load-time, lazy (deferred) or (missing/overridable). These can be controlled at compile time: ld's -bind_at_load specifies load-time binding, and __attribute((weak_import)) for weak symbols. There is also an option to prebind libraries to fixed addresses (-prebind switch of ld)

Tools:

In the outputs shown, I've color coded: white is what you should type. yellow is for my own annotations. Everything else is verbatim the output of the commands.

Calling external functions

If you disassemble any Mach-O dynamically linked binary, you will no doubt see, sooner or later, a call to an external function, supplied by some library (commonly, libSystem.B.dylib). These calls are implemented as calls to the Mach-O's symbol stub section. Consider the following example, from OS X's /bin/ls:

morpheus@Zephyr (~)$ otool -tV /bin/ls | grep stub

0000000100000ab5        jmpq    0x100003fea ## symbol stub for: _strcoll
..
0000000100000e49        callq   0x100003fd8 ## symbol stub for: _setlocale
0000000100000e59        callq   0x100003f84 ## symbol stub for: _isatty
0000000100000e73        callq   0x100003f4e ## symbol stub for: _getenv
0000000100000e85        callq   0x100003ef4 ## symbol stub for: _atoi
0000000100000e9c        callq   0x100003f7e ## symbol stub for: _ioctl
0000000100000eca        callq   0x100003f4e ## symbol stub for: _getenv
0000000100000ed7        callq   0x100003ef4 ## symbol stub for: _atoi
0000000100000ee8        callq   0x100003f6c ## symbol stub for: _getuid
0000000100000f40        callq   0x100003f5a ## symbol stub for: _getopt
0000000100001073        callq   0x100003efa ## symbol stub for: _compat_mode
00000001000010b1        callq   0x100003fd2 ## symbol stub for: _setenv
..
0000000100003e6c	callq	0x100003f42 ## symbol stub for: _fwrite
0000000100003e76	callq	0x100003f06 ## symbol stub for: _exit

morpheus@Zephyr (~)$ jtool -l -v /bin/ls | grep stubs
   Mem: 0x100003e7c-0x10000403e	File: 0x00003e7c-0x0000403e		__TEXT.__stubs             	(Symbol Stubs)

Following on the experiment from page 116**, If you have gdb or lldb (as of Xcode 5), you can use either to examine the contents of this "stub" section:

morpheus@Zephyr (~) $ /Developer/usr/bin/lldb /bin/ls 
Current executable set to '/bin/ls' (x86_64).
(lldb) x/i 0x100003fea
0x100003fea:  ff 25 30 12 00 00  jmpq   *4656(%rip)
(lldb) b 0x100003fea
Breakpoint 1: address = 0x0000000100003fea
(lldb) r  # run the process, to hit the breakpoint
Process 1671 launched: '/bin/ls' (x86_64)
Process 1671 stopped
* thread #1: tid = 0x18105, 0x0000000100003fea ls`strcoll, queue = 'com.apple.main-thread, stop reason = breakpoint 1.1
    frame #0: 0x0000000100003fea ls`strcoll
ls`symbol stub for: strcoll:
-> 0x100003fea:  jmpq   *4656(%rip)               ; (void *)0x00000001000042b2

# Ok.. let's see what lies in 42b2...
(lldb) x/2i 0x00000001000042b2
0x1000042b2:  68 60 04 00 00  pushq  $1120
0x1000042b7:  e9 84 fd ff ff  jmpq   0x100004040

DYLD_INFO and LINKEDIT

$  jtool -l -v /bin/ls
...

LC 03: LC_SEGMENT_64          Mem: 0x100006000-0x100009000      File: 0x6000-0x87a0     __LINKEDIT
LC 04: LC_DYLD_INFO_ONLY     
           Rebase info: 24    bytes at offset 24576 (0x6000-0x6018)
           Bind info:   104   bytes at offset 24600 (0x6018-0x6080)
           Lazy info:   1352  bytes at offset 24704 (0x6080-0x65c8)
        No Weak info
           Export info: 32    bytes at offset 26056 (0x65c8-0x65e8)
LC 05: LC_SYMTAB                Symbol table is at offset 0x66c4, with 83 entries
...

$  jtool --pages /bin/ls
bash-3.2# jtool --pages /bin/ls
0x0-0x0	__PAGEZERO
0x0-0x5000	__TEXT
0x5000-0x6000	__DATA
0x6000-0x87a0	__LINKEDIT
	0x6000-0x6018	Rebase Info
	0x6018-0x6080	Binding Info
	0x6080-0x65c8	Lazy Bind Info
	0x65c8-0x65e8	Exports
	0x65e8-0x6620	Function Starts
	0x6620-0x66c4	Code Signature DRS
	0x66c4-0x6bf4	Symbol Table
	0x6bf4-0x6e68	Indirect Symbol Table
	0x6e68-0x7230	String Table
	0x7230-0x87a0	Code signature

DYLD makes extensive use of the ULEB128 encoding, which is (in the author's humble opinion) a crude and stingy encoding method. Low level implementors would be wide to familiarize themselves with the encoding, which is also used in DWARF and other binary-related formats.

DYLD OpCodes

Binding opcodes

bind information:
segment section          address        type    addend dylib            symbol

lazy binding information (from lazy_bind part of dyld info):
segment section          address    index  dylib       

0x0000 BIND_OPCODE_SET_DYLIB_ORDINAL_IMM(3)                                   # Sets DYLIB to #3 (3rd LC_LOAD_DYLIB, in this case libSystem)
0x0001 BIND_OPCODE_SET_SYMBOL_TRAILING_FLAGS_IMM(0x00, __DefaultRuneLocale)   # Sets symbol name to __DefaultRuneLocale
0x0016 BIND_OPCODE_SET_TYPE_IMM(1)                                            # Sets type to pointer
0x0017 BIND_OPCODE_SET_SEGMENT_AND_OFFSET_ULEB(0x02, 0x00000000)              # Sets segment to segment #2 (__DATA)
0x0019 BIND_OPCODE_DO_BIND()                                                  # Row done

#
# The second row will inherit all the values from the first, but override symbol name:
#

0x001A BIND_OPCODE_SET_SYMBOL_TRAILING_FLAGS_IMM(0x00, ___stack_chk_guard)    # Sets symbol name to __stack_chk_guard
0x002E BIND_OPCODE_DO_BIND()

#
# Again, third row inherits all the values from second, save symbol name:

0x002F BIND_OPCODE_SET_SYMBOL_TRAILING_FLAGS_IMM(0x00, ___stderrp)
0x003B BIND_OPCODE_DO_BIND()
..

Symbol Tables

The Symbol Table (LC_SYMTAB)

struct symtab_command {
        uint32_t        cmd;            /* LC_SYMTAB */
        uint32_t        cmdsize;        /* sizeof(struct symtab_command) */
        uint32_t        symoff;         /* symbol table offset */
        uint32_t        nsyms;          /* number of symbol table entries */
        uint32_t        stroff;         /* string table offset */
        uint32_t        strsize;        /* string table size in bytes */
};

LC 05: LC_SYMTAB                Symbol table is at offset 0x66c4, with 83 entries
                                String table is at offset 6e68, 968 bytes

..
LC 06: LC_DYSYMTAB           
          1 local symbols at index     0
          1 external symbols at index  1
         81 undefined symbols at index 2
            No TOC
            No modtab
        157 Indirect symbols at offset 0x6bf4
..

morpheus@Zephyr (~)$ jtool -do 0x6bf8 /bin/ls
Dumping from offset 0x6bf8 (address 0x100006bf8, Segment: __LINKEDIT)
Offset of address specified (100006bf8) falls within Indirect Symbol Table - dumping from beginning
Entry 1: 0000002c       _humanize_number
Entry 2: 00000047       _tgetent
Entry 3: 00000048       _tgetstr
Entry 4: 00000049       _tgoto
Entry 5: 0000004b       _tputs
Entry 6: 00000003       ___assert_rtn
Entry 7: 00000004       ___error
Entry 8: 00000005       ___maskrune
...

/*
 * For the two types of symbol pointers sections and the symbol stubs section
 * they have indirect symbol table entries.  For each of the entries in the
 * section the indirect symbol table entries, in corresponding order in the
 * indirect symbol table, start at the index stored in the reserved1 field
 * of the section structure.  Since the indirect symbol table entries
 * correspond to the entries in the section the number of indirect symbol table
 * entries is inferred from the size of the section divided by the size of the
 * entries in the section.  For symbol pointers sections the size of the entries
 * in the section is 4 bytes and for symbol stubs sections the byte size of the
 * stubs is stored in the reserved2 field of the section structure.
 */
#define S_NON_LAZY_SYMBOL_POINTERS      0x6     /* section with only non-lazy
                                                   symbol pointers */
#define S_LAZY_SYMBOL_POINTERS          0x7     /* section with only lazy symbol
                                                   pointers */
#define S_SYMBOL_STUBS                  0x8     /* section with only symbol
                                                   stubs, byte size of stub in
                                                   the reserved2 field */

morpheus@Zephyr(~)$ jtool -dA __TEXT.__stub_helper  ~/Documents/RE/ScreenShotr  # note -dA, to disassebmle ARM, not Thumb
Disassembling from file offset 0x18d4, Address 0x28d4
28d4    e52dc004        PUSH   IP               ; STR IP, [ SP, #-4 ]!           # PUSHes R12 onto the stack
28d8    e59fc010        LDR    IP, [PC, 16]     ; R12 = *(28f0) = 0x7f8          # Load 
28dc    e08fc00c        ADD    IP, PC, IP       ; R12 = 0x30dc                   # Correct R12 for PC-relative addressing
28e0    e52dc004        PUSH   IP               ; STR IP, [ SP, #-4 ]!           # PUSHes R12 (0x30dc) onto the stack

28e4    e59fc008        LDR    IP, [PC, 8]      ; R12 = *(28f4) = 0x7e8          # Load
28e8    e08fc00c        ADD    IP, PC, IP       ; R12 = 0x30d8                   # Correct R12 for PC-relative addressing
28ec    e59cf000        LDR    PC, [IP, 0]      ; R15 = *(30d8) dyld_stub_binder # goto dyld_stub_binder

28f0    7f8             DCD    0x7f8            # Offset of 0x30dc, PC-relative
28f4    7e8             DCD    0x7e8            # Offset of dyld_stub_binder, PC-relative
---------------------------------------------
28f8    e59fc000        LDR    IP, [PC, 0]      ; R12 = *(2900) = 0x0  # Lazy binding opcode@0x0 (_IOSurfaceCreate)
28fc    eafffff4        B      0xffffffd0       ; 0x28d4               # Jump to stub_handler
2900    0               DCD    0                    
---------------------------------------------
2904    e59fc000        LDR    IP, [PC, 0]      ; R12 = *(290c) = 0x17 # Lazy binding opcode@0x17 (_IOSurfaceGetBaseAddress)
2908    eafffff1        B      0xffffffc4       ; 0x28d4               # Jump to stub_handler
290c    17              DCD    0x17            
---------------------------------------------
...

morpheus@Zephyr(~)$ jtool -opcodes  ~/Documents/RE/ScreenShotr  # Can also use dyldinfo -opcodes, same output
..
lazy binding opcodes:
0x0000 BIND_OPCODE_SET_SEGMENT_AND_OFFSET_ULEB(0x02, 0x00000000)
0x0002 BIND_OPCODE_SET_DYLIB_ORDINAL_IMM(2)
0x0003 BIND_OPCODE_SET_SYMBOL_TRAILING_FLAGS_IMM(0x00, _IOSurfaceCreate)
0x0015 BIND_OPCODE_DO_BIND()
0x0016 BIND_OPCODE_DONE
0x0017 BIND_OPCODE_SET_SEGMENT_AND_OFFSET_ULEB(0x02, 0x00000004)
0x0019 BIND_OPCODE_SET_DYLIB_ORDINAL_IMM(2)
0x001A BIND_OPCODE_SET_SYMBOL_TRAILING_FLAGS_IMM(0x00, _IOSurfaceGetBaseAddress)
0x0034 BIND_OPCODE_DO_BIND()
0x0035 BIND_OPCODE_DONE
0x0036 BIND_OPCODE_SET_SEGMENT_AND_OFFSET_ULEB(0x02, 0x00000008)
0x0038 BIND_OPCODE_SET_DYLIB_ORDINAL_IMM(2)
0x0039 BIND_OPCODE_SET_SYMBOL_TRAILING_FLAGS_IMM(0x00, _IOSurfaceLock)
0x0049 BIND_OPCODE_DO_BIND()
0x004A BIND_OPCODE_DONE
0x004B BIND_OPCODE_SET_SEGMENT_AND_OFFSET_ULEB(0x02, 0x0000000C)
0x004D BIND_OPCODE_SET_DYLIB_ORDINAL_IMM(2)
0x004E BIND_OPCODE_SET_SYMBOL_TRAILING_FLAGS_IMM(0x00, _IOSurfaceUnlock)
0x0060 BIND_OPCODE_DO_BIND()

morpheus@Zephyr(~)$ ARCH=armv7s jtool -dA dyld_stub_binder /Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS7.0.sdk/usr/lib/system/libdyld.dylib
dyld_stub_binder:
# coming into this function, we have two arguments on the stack: 
# *SP     = Offset into bind information
# *(SP+4) = 0x30c4 - Address of image loader cache
10cc    e92d408f        PUSH   {R0,R1,R2,R3,R7,LR} ; SP -= 24           # save registers    
10d0    e28d7010        ADD    R7, SP, #0x10       ; R7 = SP + 0x10 (point to previous R7)
10d4    e59d0018        LDR    R0, [SP, 24]        ; R0 = *(SP + 0x18) = *(Initial_SP)
10d8    e59d101c        LDR    R1, [SP, 28]        ; R1 = *(SP + 0x1c) = *(Initial_SP + 4) 
10dc    fa0001ef        BLX    0x7bc               ; 0x18a0 __Z21_dyld_fast_stub_entryPvl
10e0    e1a0c000        MOV    IP, R0	 	   ;                      
; IP = dyld_fast_sub_entry (void *, long)
10e4    e8bd408f        POP    {R0,R1,R2,R3,R7,LR}                      # restore registers     
10e8    e28dd008        ADD    SP, SP, #0x8        ; SP  = 0x8          # Clear stack
10ec    e12fff1c        BX     IP                                       # Jump to bound symbol

* thread #1: tid = 0x185f7, 0x0000000100004040 ls, queue = 'com.apple.main-thread, stop reason = breakpoint 1.1
    frame #0: 0x0000000100004040 ls
# stack already contains the offset into the LINKEDIT bind information, which is different per symbol.
# When we get here, this is common code, and we further push the address of the cache:
-> 0x100004040:  leaq   4073(%rip), %r11          ; (void *)0x0000000000000000
   0x100004047:  pushq  %r11
   0x100004049:  jmpq   *4057(%rip)               ; (void *)0x00007fff8c80e878: dyld_stub_binder
   0x10000404f:  nop    

References:

1. MikeAsh.com, article on DYLD by G. Raskind
2. Apple Developer - Mach-O Programming Topics
3. Source code of DYLD Interpose example from the book
4. Apple Developer - OS X Assembler Reference
5. http://opensource.apple.com/source/dyld/dyld-210.2.3/src/dyld_stub_binder.s - Source of DYLD's stub_binder, for both x86_64 and ARM

Footnotes