procexp - Going over the top(1)

Process Explorer (procexp) is a utility I've started writing as a simple tool to demonstrate the vast swaths of information accessible by proc_info (a.k.a system call #336, my favorite). Since then, however, and like its sibling - jtool - the tool took a life of its own, as more and more features have been added to it, making it not just a full replacement to the default (and crummy) top(1) utility, but also providing features available nowhere else. It is slowly living up to the golden standard - its Windows namesake - but (deliberately) through a terminal interface (which makes it perfect over SSH as well).

The tool, however, (like most of mine), was built around my own use cases and habits, so - while it's natural for me to use, many people remain unaware of its powerful yet nonintuitive features. This (and the man page, which nobody reads.. :-) aims to rectify that. There's soooo many options to cover, though, I'll just cover the latest ones.


Download Latest Version

Here

Examples

# Run with -h for descriptive error message
#  procexp -h
Usage: ./procexp [pid|all] [core|fds|ports|net|tree|threads|regions|all]
           or
       ./procexp vmmon

This is J's Process Explorer v1-e, compiled on Jan 26 2016, 12:55:05

Running with a pid or all argument will produce grep(1)-friendly output. Running with no arguments will enter interactive mode:

You can resize the terminal window to get more/less output, remove and rearrnage columns, sort, filter.. All the things top(1) still can't do after 15 years.. '?' is your friend

War Walking

In interactive mode, will turn your MacBook or iOS device into a signal detector for the current WiFi. Useful :-)

Dumping core

Does not affect the process in any harmful way, and - Couldn't be simpler. Run with a pid and "core" argument, or go to process details screen and hit "c". Full core will be generated in /tmp. Yes, it works on iOS. And AFAIK it's the only way to get an iOS core dump. Decryption of Apps is transparently thrown in as a bonus :-).

Listing Mach Ports!

Version 1-ε can list Mach ports and even name them(!) using some nice undocumented functions I've found (Until AAPL ruins this with an entitlement, too..). The "arrows" indicate the unidirectionality (-> send to, <- receive from).
Edit: As of 02/27 procexp shows names in quotes ("") so you can also grep all named ports easily (not all are com.apple).


# isolate specific named ports (e.g. com.apple) in SpringBoard
SpringBoard:38198:0x2db57	"com.apple.softwareupdateservicesd"	->softwareupdatese:188:0x1807 
Pademonium-II:~ root# procexp 38198 ports | grep \"       
SpringBoard:38198:0x927	"com.apple.SystemConfiguration.NetworkInformation"	->configd:39:0x4b0f 
SpringBoard:38198:0xe03	"com.apple.system.notification_center"	->notifyd:83:0xe03 
SpringBoard:38198:0x1703	"com.apple.cfprefsd.daemon"	->cfprefsd:84:0x1403 
SpringBoard:38198:0x1e07	"com.apple.searchd"	->searchd:169:0x600b 
SpringBoard:38198:0x1f03	"com.apple.backboard.system-app-server"	->backboardd:38197:0x2a707 
SpringBoard:38198:0x2107	"com.apple.backboard.display.services"	->backboardd:38197:0x2a807 
SpringBoard:38198:0x250b	"com.apple.distributed_notifications@1v3"	->distnoted:86:0x1703 
SpringBoard:38198:0x2fbb	"com.apple.Preferences.gsEvents"	->Preferences:38283:0x2703 
SpringBoard:38198:0x3107	"com.apple.frontboard.systemappservices"	<-launchd:1:0x8e03 <-mediaserverd:29:0x2d03 <-mediaremoted:36:0x588b <-configd:39:0x16907 <-atc:41:0xaf03 <-installd:51:0x1c81b <-assetsd:57:0x5303 <-BTServer:66:0x4b07 <-imagent:67:0x9f33 <-assertiond:68:0x2f07 <-CommCenter:82:0x16103 <-lsd:88:0xc003 <-coreduetd:99:0xbb13 <-nsurlsessiond:119:0x3b03 <-callservicesd:127:0x7c03 <-homed:136:0x2f07 <-useractivityd:145:0x3803 <-calaccessd:167:0xf17 <-replayd:273:0x1e03 <-CacheDeleteDaily:288:0x2b03 <-aggregated:38152:0x2f03 <-backboardd:38197:0x387ab <-com.apple.access:38199:0x3e03 <-MobileMail:38203:0x4903 <-CalendarWidget:38273:0x3603 <-Preferences:38283:0x3703 <-Skype:38295:0x3603 <-SafariViewServic:38339:0x3a0b <-TheMesh:41049:0x3a03 
SpringBoard:38198:0x3507	"com.apple.iohideventsystem"	->backboardd:38197:0x1ea07 
SpringBoard:38198:0x3a07	"com.apple.assertiond.processinfoservice"	->assertiond:68:0x1c07 
SpringBoard:38198:0x5707	"com.apple.UIKit.statusbarserver"	<-launchd:1:0x7d03 <-com.apple.access:38199:0x4213 <-MobileMail:38203:0x5603 <-CalendarWidget:38273:0x3c07 <-Preferences:38283:0x4603 <-Skype:38295:0x4503 <-SafariViewServic:38339:0x4d03 <-TheMesh:41049:0x4703 
...
..
SpringBoard:38198:0x12a4b	"com.apple.diagnosticd"	->diagnosticd:85:0x1903 
SpringBoard:38198:0x12c17	"com.apple.springboard.blockableservices"	<-launchd:1:0x7c0b 
SpringBoard:38198:0x12d17	"com.apple.springboard.services"	<-launchd:1:0x950b <-CommCenter:82:0x14417 <-gamed:137:0xc25b <-itunescloudd:141:0x5f0b <-duetexpertd:146:0x7577 <-MobileMail:38203:0x7a17 
SpringBoard:38198:0x12f07	"com.apple.springboard.watchdogserver"	<-launchd:1:0x8b03 
SpringBoard:38198:0x13303	"com.apple.springboard.backgroundappservices"	<-launchd:1:0x8a0b <-mediaserverd:29:0x1769b <-configd:39:0x16c17 <-symptomsd:98:0x3c13 <-searchd:169:0x7113 <-kbd:185:0x2907 <-aggregated:38152:0x6d07 <-MobileMail:38203:0xd153 <-Preferences:38283:0x6017 <-TheMesh:41049:0x6a0b 
SpringBoard:38198:0x13703	"com.apple.springboard.icongeneration"	<-launchd:1:0x8d03 
...
SpringBoard:38198:0x2bb03	"com.apple.locationd.synchronous"	->locationd:65:0x290b 
SpringBoard:38198:0x2ce13	"PurpleSystemEventPort"	->backboardd:38197:0x4703 
SpringBoard:38198:0x2db57	"com.apple.softwareupdateservicesd"	->softwareupdatese:188:0x1807 
# isolate channels by name
Pademonium-II:~ root# procexp all ports | grep com.apple.iohideventsystem
SafariViewServic:38339:0x2503	"com.apple.iohideventsystem"	->backboardd:38197:0x1ea07 
Skype:38295:0x2403	"com.apple.iohideventsystem"	->backboardd:38197:0x1ea07 
CalendarWidget:38273:0x1f03	"com.apple.iohideventsystem"	->backboardd:38197:0x1ea07 
MobileMail:38203:0x2403	"com.apple.iohideventsystem"	->backboardd:38197:0x1ea07 
com.apple.access:38199:0x2d03	"com.apple.iohideventsystem"	->backboardd:38197:0x1ea07 
backboardd:38197:0x1ea07	"com.apple.iohideventsystem"	<-launchd:1:0x3415f <-locationd:65:0x6c07 <-UserEventAgent:72:0x770b <-biometrickitd:118:0x210f <-aggregated:38152:0x4e07 <-SpringBoard:38198:0x3507 <-com.apple.access:38199:0x2d03 <-MobileMail:38203:0x2403 <-CalendarWidget:38273:0x1f03 <-Preferences:38283:0x1f03 <-Skype:38295:0x2403 <-SafariViewServic:38339:0x2503 <-TheMesh:41049:0x2403 
Preferences:38283:0x1f03	"com.apple.iohideventsystem"	->backboardd:38197:0x1ea07 
aggregated:38152:0x4e07	"com.apple.iohideventsystem"	->backboardd:38197:0x1ea07 
SpringBoard:38198:0x3507	"com.apple.iohideventsystem"	->backboardd:38197:0x1ea07 
TheMesh:41049:0x2403	"com.apple.iohideventsystem"	->backboardd:38197:0x1ea07 
root@Pademonium-II (~) # procexp $$ ports                             
zsh:422:0x10b	->launchd:1:0xe03 
zsh:422:0x207	
zsh:422:0x30b	
zsh:422:0x407	
zsh:422:0x50b	(thread)
zsh:422:0x607	
zsh:422:0x707	(task)
zsh:422:0x803	(clock)
zsh:422:0x903	(semaphore)
zsh:422:0xa07	(voucher)
zsh:422:0xb03	(host)
zsh:422:0xc03	
zsh:422:0xd03	(voucher)
zsh:422:0xe03	"com.apple.system.notification_center"	->notifyd:81:0xe03

02/27/2016: And, of course, JCOLOR, which makes it easier to sift out busy ports:

12/10/2016: Stack snapshot works again. This means you can use "procexp all threads" (even with SIP enabled on MacOS :-) to get stack traces of every thread in system. This is especially useful as it now retrieves thread names (helps with kernel threads!). Also tested stable on iOS 10.x. 

root@Chimera (~/Documents/Work/ProcExp) # procexp 0 threads | tail -30                                                                                     
		TID: 780812 State: Waiting Uninterruptible  PRI: 82/82 Flags: 0x2 
		Thread Name: ifnet_start_en5
		CPU Times: User: 0.000000 secs, System: 0.006748 secs
				Kernel Stack Trace:
		LR: 0xffffff8000401bbe, SP: 0xffffff9225dfbd20
		LR: 0xffffff800030c91b, SP: 0xffffff9225dfbda0
		LR: 0xffffff800030b73e, SP: 0xffffff9225dfbde0
		LR: 0xffffff8000301344, SP: 0xffffff9225dfbe20
		LR: 0xffffff800078c063, SP: 0xffffff9225dfbea0
		LR: 0xffffff800078c522, SP: 0xffffff9225dfbee0
		LR: 0xffffff800058ac64, SP: 0xffffff9225dfbfb0
		LR: 0xffffff80002a2af7, SP: 0x0000000000000000

	
	
		TID: 780830 State: Waiting Uninterruptible  PRI: 93/93 Flags: (none) 
		Continuation: 0xffffff800032b000 (no kernel stack)
		CPU Times: User: 0.000000 secs, System: 16.862980 secs
	
	
		TID: 780831 State: Waiting Uninterruptible  PRI: 93/93 Flags: (none) 
		Continuation: 0xffffff800032b000 (no kernel stack)
		CPU Times: User: 0.000000 secs, System: 16.744660 secs

Process Explorer - PRO!

All the fabulous features of Process Explorer, With GUI, AND remote monitoring - Coming soon!

Q & A

  • Is this open source? No.
  • Will it ever be open source? No. YES. If AAPL decides to throw out top(1) and wants my code - I would be immensely honored, and happily open source procexp! That would also enable it to work perfectly under SIP, since it could legally obtain entitlements..
  • Why is it not open source? Because I've put too much work into it to see people plagiarize my hard efforts outright, close the source, and sell it as a commercial product. The tool is free, though, so you're welcome.
  • But I really want to know how you do this stuff!!! You want to know how it does what it does, there's MOXiI and the TechnoloGeeks training for that
  • How are you getting all this info?! Process Explorer was started as a simple demo of proc_info, my favorite system call (#336). Undocumented by AAPL, of course, but wrapped by libproc (q.v <libproc.h>). It has since gotten functionality from stack_snapshot (#365, my second favorite :-), miscellaneous sysctl(2) MIBs, sandbox APIs, and the symbolication functions of my jtool.

  • Will this work on 10.11? Mostly, yes, but - Alas, System Integrity Protection (SIP, a.k.a rootless) impairs some advanced functions (notably those requiring Mach task ports). It's ridiculous that Apple's pathetic top(1) gets com.apple.system-task-ports and I can't. (To my readers in 17.x.x.x - I'd gladly give you my code if you promise to replace top :-)
  • Is there a GUI version of this? you mean outside of Activity monitor? :-P The whole point of procexp is to be runnable without GUI , over SSH. Plus I hate hate hate GUI programming
  • Is there a process explorer agent? There will be one, coming very, very soon, to enable you to remotely monitor OS X and (jailbroken) iOS. I call it "Process Explorer Pro". And, *sigh* it will have GUI. Stay tuned :-)
  • When is version 1.0? Soon (for a sufficiently small ε). I plan to release it with MOXiI 2. So that's very soon, but give me time here. I'm just one man deeply entrenched in two vast ecosystems.
  • Seriously, no plans for open source? NO.
  • "I'd be willing to pay $10 as end-user for the source (with license prohibiting redistrib) to procexp so I can fix some bugs." - Geez, thanks. $10 for all this work. I always wanted an exit!!! How's about a deal - you keep your $10, and I keep my dignity?
  • But.. but.. what if you die? (actual question!) Then procexp is the least of my concerns, and you can keep on using whatever version is stable at the time. I do hope to outlive my tools, though.

    • ChangeLog

    You can find a change log in the downloaded tar file, in "WhatsNew.txt".

    
v0.2

- (Some) color
- Scroll list
- Show power details
- Show memory compression (Mavericks/iOS 7 only)

v0.2.1
 - Process filtering
 - Accurate time statistics (user, system, total) - verified against top(1)
 - (More) color
 - Process hierarchy (output needs to be improved, but is correct)
 - Fixed power output for VMs (wherein power sources API doesn't work)

v0.2.2
 - Also showing swap statistics (from sysctl vm.swapusage)
 - Also showing local IP communication (IPv4, for now) endpoints (i.e. who's on the other side of that ::1 or 127.0.0.1)
 - CPU utilization! Also supporting Linux Top's "1" (for each CPU). "I"rix mode coming soon

v0.2.3

 - ProcExp is now multi (well, dual) threaded! Separate thread implements vm_pressure_monitor (#296) to trace VM pressure
 - Mavericks/iOS7: shows memory status
 - Mavericks/iOS7: shows Disk I/O stats (proc_pid_rusage)
 - Shows task modification statistics (still need to improve and work around pid_for_task(), though)
 - Fixed process information to display pipes. Still need to show other side of pipe, though
 - Added handler to catch curses library failures
 - Added update delay ('U')
 - Socket information now shows TCP state (A la netstat)

v0.2.4 - the birthday edition

  - stack_snapshot integrated into ProcExp! (basic, but going strong). Shows last RA for thread in 64-bit. 32-bit still choppy
  - added #FDs to column display (useful!)
  - handle revoke(2)d FDs as well
  - preliminary support for deltas - shows colors on increase/decrease of RSIZE, #Threads, increase of time


v0.2.5          - 12/03/13

  - sysctl KERN_PROCARGS2 now displays full name and args of processes in detailed view
  - internal refactoring into separate C files for subsystems
  - Columns now OS-version-sensitive (e.g. disk I/O will only show in 10.9, iOS7 and later)

v0.2.6          - 01/19/14
 
  - New options "regions" - e.g. "./procexp.universal $$ regions" - will show memory regions.
     Will have that in the full screen option as well (as soon as I figure out curses and scrolling..)
  
v0.2.7          - 01/19/14

  - Systemwide Network packets (RX/TX) now available; Per-process statistics coming soon

v0.2.8          - 01/23/14

   - Made network statistics in bytes/sec. 
   - Prompt mode fixed, now case sensitive
   - New commands: K/R/T: KILL/Resume/Stop highlighted process (note uppercase)

v0.2.9		- 02/22/14

   - Workqueue information now displayed
   - FD output improved in CLI mode:

      bash-3.2# ./procexp.universal all fds | grep tcp
      WebProcess       11117 FD 23u  socket tcp4   10.0.1.107:52246->    23.199.72.44:443   ESTABLISHED
      ssh              11066 FD  3u  socket tcp4   10.0.1.107:51869->  207.36.228.144:443   ESTABLISHED

v0.3  		- 03/20/14 

   - Adapts display to screen width

v0.3.2		- 06/13/14

   - Works on Yosemite, DP1
   - Stack Snapshot fixed, AND auto symbolicates kernel! e.g.

	#Threads:   2/1   (Process has no workqueues)

	TID             USER            KERNEL
	0x4da4f         0x7fff9198c8ae  _stack_snapshot2 + 0x21a
			0x10000318b     _unix_syscall64 + 0x262
			0x100003ab4     _hndl_unix_scall64 + 0x16
			0x10000a41b
			0x10000bc4a
			0x10000cd77
			0x100001464
			0x1
	0x4da59         0x7fff9198c8ae  _machine_switch_context + 0x16f
			0x10000c191     _preemption_enabled + 0xbaa
			0x7fff91cb4844  _thread_block_reason + 0xaf
			0x7fff91cb47c1  _mach_vm_pressure_monitor + 0x4b
			0x7fff91cb2a11  _vm_pressure_monitor + 0x2e
					_unix_syscall64 + 0x262
					_hndl_unix_scall64 + 0x16


v0.3.4	- 11/25/14

     - Integrates with iOS's MobileGestalt (displays iOS name and version. Also - try 'G' from main screen)
     - Updated system event sockets to display OS X 10.10/iOS 8 KEV_* classes and subclasses

v0.4 - 3/30/15 - It's been a while
 
     - Added 'F' to freeze screen updates. This is especially useful if you're sorting by CPU or state, which would cause re-sorting frequently. Simply press 'F' to freeze table updates, and 'F' to unfreeze.

     - Now works with OS X 10.10/iOS 8 (shows compressed memory. Still working on ledgers)

v0.5 - 4/13/15 - London - WiFi!

     - Used direct ioctl()s to get WiFi details with RSSI - so you now have a built-in WiFi Signal detector :-) (on iOS, this requires the wlan.authentication entitlement, which I self-signed)

v0.55 - 4/25/15 - Delhi - MemoryStatus and some other changes

     - ' ' and 'b' now flip through process list faster (forward - space, backward - b. Just like less(1))
     - procexp now only shows columns whose details require root access if you're running it as root.
     - uses csops() (#169) to show code signing status of a process. Great to find unsigned/malware-suspect processes
     - Uses the memorystatus_control() syscall (#440) to obtain memorystatus snapshot and display memory status priority of each process in 'MS' column.  In OS X, it's rather boring (most procs are 18 or 0). In iOS, the gamut of processes is more populated. Also in iOS, will only be displayed if you're root.

     - fixed a bug with 8 processors (*blush*) - I never tested on a MacBook Pro .. 

v0.56  - 6/27/15 

   Fixed thread dumps on iOS. Also have thread symbolication for user mode! (well, at least what AAPL hasn't redacted...)
   Initial support for xnu-32xx (iOS 9/OS X 10.11)
   Still @TODO: Thermal info

v0.6 - 8/1/15
 
   - Bug fixes
   - Can now dump core of a selected process (press 'C' when inspecting the process)


v0.62   - 9/20/15 Bug Fixes 

   - MOVE COLUMNS! Press SHIFT + Left or right to move an entire column!!!
        - Note: requires TERM to be set correctly (works great with TERM=xterm, misinterprets as '1' key if TERM=screen)
 
   - Separated DISK to "DSK R" and "DSK W".
   - Move/sort now correctly only reaches to edge of screen. As you readjust screen width, you can move further/less

   - Loads of fixes:
           - Fixed abort which was caused by rusage
           - CPU usage now shows per-cpu without messing up display
           - CPU %-age is accurate (apparently more so than top!)
           - Colorless mode (toggle with 'c') is now truly colorless and dull. Just like the real top :-)
           - Selecting processes now works well with active filters
           - Fixed memory leak
           - Curses bugs resolved

v0.99   - Special Edition for MacSysAdmin.se. A couple more features, and I'll get to 1.0.

         (hey, if Linus Torvalds can arbitrarily declare version numbers, so can I! :-)

           - More accurate statistics (up to two decimal points when in MB, GB)
           - Fixed bug due to com.apple.network.statistics change in 10.10 that eluded me..
           - Perspectives 
           - Detecting power assertions 

v0.999 - lots of tiny bug fixes, used my own task_for_pid workaround

        - The Mickel Bug: UIDs on AD machines (where UID > 65536!) no longer crash ProcExp
        - Print out warning if can't get task ports (Thanks for #$%#$%ing processor_set_tasks, AAPL..)
        - Displays Containers (for iOS and OS X App Store Apps)

v1.0.0-ε - Displaying Mach Ports! 


	02/27/16 - fixed some ports which were erroneously resolved because they were "U" and not "M" in launchd output.
		 - Output port names with quotes, so as to procexp ... ports | grep \"

		 - "all" now includes ports

v1.0.0-ε2
	03/16/16
		 - thread stack trace now shows offset from nearest symbol also in user mode
		 - regions shows SHM , just like vmmap (this one's for you, elist)

	04/02/16
		 - WAY more responsive: pressing  or any other unassigned key will autorefresh


	07/31/16

		 - Default sort now CPU
		 - Core dumps now correctly mark protection flags
		 - new "Binary" option (to dump decrypted/protected)


	09/20/16
		 - Disabled stack_snapshot on Sierra/iOS10 until I can get ...config working

v1.0.0-ε3

	12/05/16
		- Fixed double free bug
		- stack_snapshot no longer needed - have micro_stackshot working and integrated 

	1/24/17 (Harbin)
	
  - Stop signal to processes on 'T' from main menu
  - monitor now responds to knotes and will auto freeze processes matching criteria
  - Got all KEV codes in

  	5/14/17 (Sydney)

  - now SIP-compliant without entitlements :-) - uses task_name_for_pid whenever possible. AAPL - if you are reading this, Will you PLEASE enable obtaining mach_port_names from a task already, so "procexp ports" works with SIP?
	

  10/25/17 (Punta Cana - It's been a while, but this is the 1.0!)

        - A few bug fixes
        - Kqueue information is now the extended variant, telling you what kqueue blocks on and
          what event filter mask

        - Procexp now accepts process name as argument. Geez. How did I not have this till now?
        - procexp ports works again on iOS 11 and MacOS 13.
        - procexp ports also recognizes all the host special ports! 
		You can grep HSP and '<-' in procexp all to find owners





-----------