MOXiI 2nd Edition Status Update Page
10/10/2019 - Ultimate update, in more than one way :-)
TL;DR - I'm done
Slightly longer version: Exactly a year after schedule, *OS Internals, Volume II, is going to print at last. Yes, it was delayed, but - the wait is over, the book covers XNU-6153 (Darwin 19), and this time I didn't need any open sources from AAPL.
You can get book direct only for now (I've pulled it from AMZN since I'm sick of them shaving 20% off every time!) Order details at firstname.lastname@example.org. And it's the preferred method if you're domestic: $75, and shipping is on me. If you're not, it's (alas) PYPL, with +$50/book extra ($45 shipping and $5 their fees). And you can get it in BTC, too,. 0.02BTC includes int'l priority shipping. More details on main page of site.
Also, if you're reading this (or saw my Tweet) this marks the end of the trilogy - so
- That's it , no more updates: While I was working on the trilogy I kept updating and maintaining a detailed changelog. These updates grew very big - Vol III got six more chapters, and Vol I got two more - amounting to what could have easily been a third edition. But now, as Vol II is done, so is the trilogy.
- Deluxe Boxed Hardcover, Color Set will ship by XMAS: It won't fit in your sock, it's heavy (over 1,600+ pages, 7lbs, almost 4kg!) , but it is A-W-E-S-O-M-E. To get an idea, Check out the Korean Translation of Vol III, which is in full color as well. Only 100 sets of these will be printed, each signed & numbered by yours truly (*groan* that's a lot of signatures). $500/set, shipping free in US (likely ~$150 for int'l..). True believers only (and really, the books cost over $120+ to print in color, so I'm not just overcharging.
- I'm officially retiring from Darwin for the foreseeable future: My Technologeeks Trainings this Dec are my last. It's been swell. But hitting the glass ceiling every year (only to have AAPL raise it a bit with a new version) #%$#%$#% hurts. And Einstein once said that repeating an experiment and expecting different results defines insanity. I'll do one important update to all my set of tools, so maybe expect a nice parting gift, too :-)
08/23/2019 - Penultimate update
TL;DR - At the latest, 9/30/2019, book will be published
Slightly longer version: Apple may be pulling their "Designed by Apple in California" book, first published when I started MOXiI2, from print, but I'm happy to report that my books are here to stay.. and - that Vol II is basically done. I've already got the world's best minds helping me review chapters, I'm up to date with the latest betas (which seem to have stabilized around 6153.x.x.x), and I have 94-96% of the content written (485 out of a guesstimated 505-515 pages!). At this point I just want to make sure that AAPL doesn't throw a nasty September surprise at me by changing something again in the final release (and if they do, that's why I'm giving myself up to 2 more weeks to either cover it or at least mention it and deem it out of scope). Honestly, I'd release it sooner, but I bet readers would be disgruntled if when D19 comes out I release an update already.
- And this is not a guess/wishful thinking/etc as before - this is f-i-n-a-l. I'll submit the copy to be printed there and then (or earlier), and books should ship 1-2 weeks later. So only a year behind schedule - but, hey, you get up-to-date XNU-6153 stuff!
Part of me is also hoping that my readers in Cupertino (you know who you are - there's quite a few of you, too ;-) will favor me and maybe, just maybe, release the sources MacOS 13 style (read: three days after) and not MacOS 14 style (i.e. three months after). You know. slip of the finger on them push scripts? :-) That said, SOURCES ARE NOT A PREREQUISITE FOR RELEASE as they were last year.
With or without updated 6153 sources, this is coming out. When sources *ARE* out, whenever, I reserve the right to perform minor updates - namely, adapt listings and maybe plug in names of new constants, for a v1.1, which I will also document in the changelog page. Otherwise, the v1.0 book should be considered final, barring the usual typos my astute readers point out. When those are squashed, I'll get on the hardcover, full color edition.
More details on the hardcover edition:
- ONLY available as a full set of all three books
- Full color illustrations, and (hopefully) glossy paper: To see how cool that looks, peek at The Korean translation, which the folks at Acorn.co.kr did a fabulous job with.
- Limited to a run of exactly 100 sets: I claim 10 for myself and family/close friends. No more will be printed.
- Expected weight of each volume will be close to or over 3lbs(!): This has ramifications for shipping (over 2lbs shipping rates shoot up int'l - shipping 2.1 lbs, for example, already costs $31-39 on average.
- Price for set will be $500 because printing costs alone are guesstimated at $300. Yes, you heard right. Small, prestigious and expensive to print. Domestic shipping: On me. Int'l shipping: I'm guessing $100. Yes, seriously. Not my price - USPS's.
- WILL NOT BE OFFERED VIA AMZN: Since they take 20% cut ($%#$%# exorbitant!). Sales will be direct - Apple Pay or BTC (price TBD) only. If you're reading this and want the standard edition print - PLEASE PLEASE PLEASE consider Apple Pay as well.
In other news, I am thinking of putting in Darwin 19 changes and Ned Williamson's amazing Sockpuppet exploit into a v1.7 of Volume III. Yes, I said I wouldn't update it anymore.. but.. such an awesome exploit... So if you want to buy Volume III , maybe wait a week or two.
And, for those enthusiasts of the other operating system - Android Internals Volume II will be the only book left at that point. And Volume I will get a signficant update to cover all the delta between Nougat and Q/10.
08/04/2019 - Can't keep up with them betas
q.v. https://twitter.com/Morpheus______/status/1158021049985974272 - I'm going to wait till Cupertino-folk make up their minds about critical structures before committing them to the printed volume II - wait for Darwin 19's golden master, at the very least. Incidentally, you're looking at a book of 500 pages.
05/03/2019 (= May 3rd) - *Still* Wrapping things up (but..)
Long story short, and no excuses, it's six months after where I wanted to be - both the other volumes were largely on time, this one isn't. I hoped it would be ready for my British sojourn. It wasn't. Though it's mostly done, but here's what remains:
Review of existing chapters by some of the best minds in the industry
Another chapter I thought I'd avoid (networking stack). This was the chapter I added from the user mode perspective (free, in the Volume I v1.2 update), but there's a lot to cover from the kernel perspective, as well. To be honest, not too much has changed from the first edition's Chapter 17, hence I thought I'd avoid it, like I avoid HFS+ by referring to the 1st edition. Unlike HFS+, however, which is effectively obsolete, networking will remain relevant forever, I want to cover it fully, and this will be a complete rewrite.
The book is 430+ pages already, and I'm aiming for 470 (with said chapter).
And here's the complication - I'm working (in my real job as part of Technologeeks) for the next couple of weeks, spreading the gospel of MOXiI and Android around the world. My bandwidth for the book is fairly constrainted. Meaning by the time I get to finally publish this, it will be the last week of May. And you know what that means, right? That Darwin 19 is due a week and a half later. If I do print v1.0 by end of May (which I can), I feel some readers will feel cheated that Darwin 19 might introduce features/updates/changes (or might not - I don't know) which will merit a v1.1
So my dilemma was, should I get this out, or should I wait just a bit more. I KNOW you've been waiting for this. I also know you know that elucidating the dark reaches of XNU is far more complicated than Volume I (User Mode) was. I therefore, at the moment, think I'll stretch a bit longer, just to see what Darwin 19 beta 1 brings, and possibly incorporate as much as I can into MOXiI. I am giving myself till the third week of June at this point (i.e. up to two weeks after AAPL releases the beta). BEAR IN MIND ALL DARWIN 19 CONTENT will be reversed only - since sources won't be available for a long time. If the best AAPL can muster is greyscale mode, I'll release earlier. If there're as many under the hood features as there were in '18, I'll take the full two weeks.
The silver lining in all this? Though I repeatedly stated I intent to quit this endeavor before Darwin 19 emerges, I'll have to stick around for a bit longer. If iOS13/MacOS15 merit significant changes to Volume I, I'll publish a v1.3 of that as well. As it stands, though, this will signal EOL for Volume I, and finalize Volume II.
I still maintain this will be worth the wait. Going over this stuff in depth, I realized two things - one , how much has changed since Singh's seminal tome. Two, how much Singh actually had to skim. It takes a LOT to get all this covered, and it's excruciating grunt work with -273.15° help from AAPL (unlike "Windows Internals" had from MSFT). The book is ridiculously detailed. Have a look at the detailed ToC to see what I mean. Many topics are seeing the light of day (outside Cupertino) for the first time in history
So, my apologies for a(nother, seriously, the last) delay, but better wait a bit longer than frustrate people who will buy Volume II v1.0 only to be hit with a v1.1 a week or two later. And yes, it should have been out by now, but , no, I seriously had no idea there would be THIS MUCH MORE to write, especially compared to the 1st Edition. And I know I proimsed it sooner in my previous two updates, but, real life and circumstances beyond my control, people. I hope you appreciate the update. If you want to comment on this, please let me know via the forum.
And while I have you here:
- The final Table of contents can be found here
- Once Volume II is out and has no more typos (I'm guessing, a few weeks later), I'll turn to the limited edition boxed set - hardcover + color). I haven't decided how many copies there will be, but there will be a LIMITED number of them, each signed and numbered by the author (=me) in a special inner cover page. Only available as a set, and likely $500. For true believers only (printing costs of this will apparently be over $100/book alone!). I shudder to think what overseas shipping costs will be, since the soft cover books are 1.1kg each (and close to $40 shipping per book!) and the hardcovers will likely be closer to 2kb.. The only payment methods for this will be either Apple Pay or BTC.
- When the hardcover edition comes out, this will signal END OF ALL UPDATES for MOXiI. If somebody wants to do a third edition in the future, I'd be happy to pass the baton.
- I remain committed, once this is done, to deliver Android Internals Volume II AND update Volume I for Quiche - AND release shortly after Quiche is out.
12/15/2018 - Wrapping things up (but..)
Sources are (finally) out. That's great as I'll include the Zone Cache, Turnstiles, Priority Queues and other goodies from the sources (as well others which were redacted by AAPL..). That said, there's holidays ahead and a much needed vacation I had hoped would be also to celebrate the completion of the books. So right now expect Volume II mid-late January 2019. Yeah, I know you're waiting (And I appreciate that) - but please understand there's this little thing called real life, too :-) Volume II is absolutely going to be worth the wait, and I expect it to be the definitive XNU kernel reference for many years to come.
11/21/2018 - Waiting on AAPL
So, it's been a while. To be honest I thought I'd be keeping with tradtion and releasing late October/putting the book out on Black Friday (like last year). Thanks to AAPL shuffling their feet with XNU sources, this won't happen. More than before, I need the latest XNU source since, after all, this IS the volume dealing with the kernel.. I have a ton of illustrations showing the exact kernel structures and their offsets - and so I have to make sure I'm up to date and have the right field names (which are lost during compilation). Assuming AAPL holds to their informal promise of "the week after Thanksgiving", expect the book by Christmas..
In other news, I realized that some stuff I've been pushing to Volume II - notably, networking - has so much user mode stuff in it that it would have been better in Volume I. The network stack implementation still will be in II, but the user mode syscalls,
BPF, Firewalling/ALF.kext and such are now in their own chapter in Volume I. Yes, that means an update to Volume I, but keeping with another tradition of publishing free chapters, expect that chapter (Volume I's 16th) to be available as a PDF. Keep an eye out on the Change Log for when it comes out.
Buying the books in BTC hasn't looked so attractive in years.. But I'm not sure I want to risk it myself anymore..
08/24/2018 - I'm alive and well, Volume II is still on track, Oct/Nov 2018
LOTS of additions to original ToC, describing under-the-hood Darwin 18 changes - but still on track.
New BTC price for every one of the trilogy: 0.025BTC. Get 'em while you can.
04/28/2018 - Volume II is being worked on - ETA Oct/Nov 2018
Not to break an informal tradition - expect this to be released Oct-Nov this year, subject to AAPL being nice and releasing their Darwin 18 sources early like last time (and please, with ARM64 again ;-) . Why? Because it doesn't make sense to publish a book about the kernel, which is the most important component of the system and the one most affected by Darwin versioning, only to have a new OS come out a few months later.
Watch this space for more news. Volume II is still in its infancy, but is shaping up to be the biggest and best-est of the trilogy! Topics covered include:
- Extensive discussion of ARM64 with source code examples (now that it's open source)
- Detailed illustrations of EFI, iBoot *and* whatever you want to call what drives SEP.
- Lots of XNU stuff which was never covered in the 1st ed since it was added way later - thread level QoS, more schedulers, compressed memory, ledgers, vouchers, etc.
- Specific coverage of VFS including a basic filesystem implementation
- Everything you could ever possibly want to know about kernel memory management - vm_map, pmap and zones (there's no "heap" in kernel mode, folks).
- Detailed coverage of pexpert, IOKit, selected reversed kexts
- APFS internals (no thanks to AAPL for not releasing the spec TWO YEARS LATER...)
Loads of examples will be given using Xnoop, my kernel inspection tool (which will also be released officially by @Technologeeks , hopefully sooner than the book is published
It's not too late to submit requests via the website forum!
Oh, and - When Vol II will be out, I will also be doing the limited edition COLOR HARDCOVER edition. Stay tuned for that.
11/22/2017 - Volume I goes on sale - just in time for Black Friday!
Yes, I know, longer than expected. I have a life and other things to take care of. But - at least when I say a book is coming you know it'll be there within a 1-3 month standard deviation, eh? :-)
- The book goes on sale effective NOW, through the link that's on the main page of the website. For the lazy, that will be: THIS LINK RIGHT HERE. The first 50 copies will ship early december. The printing press I use is "swamped for the holidays". So when/if you order I'll acknowledge it via AMZN, and then send you the book a short while later, when I get the books myself.
- Why the link? Because AMZN has this referral program, and if I refer to my own book, they will refund me some of their really exorbitant fees. A lot of people asked how they can make sure I get more money for my work (and I thank them for it) - This is really the only way, since I banned Paypal...
- The price of BTC is skyrocketing, so I doubt anyone would like to pay the 0.05BTC (reduced from the initial 0.15BTC), because that amounts to $409. Still, if you got into the BTC craze early, and don't mind, I would totally appreciate it :-)
- I'm experimenting with Apple Pay as a method of payment . Mac@Hisown.com , if you feel like trying. Just let me know first
- Thank you so much for being patient with this. Volume I was even harder than Volume III, because it didn't have such clear boundaries. Ended up blowing past 500 pages, even though I was really aiming for 400..
09/28/2017 - Volume I Status Update
Apple is mutating 10.13 in between betas (and people are already bugging me for "if I buy it, will I get updates") that I have decided to wait till End of September for the official release of Volume I - Just like I did last year with Volume III.
Procrastination is the devil's gift - and I got another gift from AAPL today. For whatever reason, which baffles my mortal mind, Apple put out the sources of XNU-4570, libdispatch, dyld, libpthread, Security.framework and a bunch of other goodies - early. What's really amazing is that I was getting ready to submit the very first batch of MOXiI Volume I for next week!
Well, it ain't happening
I now ABSOLUTELY MUST push back by a few weeks, to incorporate refs to the sources (esp. DYLD) and specific constant names that I had reversed but didn't know the "official" name of until now. This is a necessary evil, because otherwise I'd be putting out a v1.1 a week later.
NO, THIS IS NOT AN EXCUSE FOR AN INDEFINITE PUSHBACK. NEW DATE is 2nd or 3rd week of October. I may be slow, but I try to be thorough. And when I say a book is coming out, it's coming out (within a reasonable margin of error - Android notwithstanding). So - MOXiI Vol 1 is coming out, at 450+ pages, a few weeks later, yeah, but it's going to be even better than I had hoped. This does screw my plans to officially launch the book at MacSysAdmin, which is a conference I really like, and bring their amazing organizer a signed copy :-( Sorry Tycho.. (next year? ;-)
Other good news:
- I'm pulling a long night to add a few other minor tweaks to Volume III (which is going to print tomorrow in a new batch for v1.4.2). Mostly just pointing from my analysis of KPP to the sources, Code signature 0x20400 and a couple of constants, and a discussion of KTRR (iPhone 7 and later AMCC).
- This will expedite Volume II (kernel) since I was bottlenecked on the XNU sources, which I now have.
- This will bring EVEN MORE DETAIL to Volume II, because AAPL [decided to/messed up and accidentally] release the sources for ARM64 - not just
CONFIG_EMBEDDED, which they hid as of 2050 - but the whole package, including the pexpert, KPP (MONITOR), KTRR (iPhone 7+) - some 30k lines of code with a lot of amazing stuff.
- The big surprise of Volume II was supposed to be my re-sourcing of XNU's arm64/ directories. That won't be necessary. You can get that straight from the source now. The day is nigh upon us when XNU will be runnable in even the lowliest of emulators or BeagleBoards..
- I'll be updating my ioS binpack soon for Darwin 17
- I'll be putting out a couple of other important open source packages Apple [forgot to/didn't mess up and accidentally] release today. (@launchderp, don't pull a Wilson Fisk on me ;-)
- Unrelated, but good - Android Volume II has a release date for January 2018, only two years behind schedule :-)
So stay tuned. And yes, it's coming out. And a mighty mighty thanks to Apple for this, even if I can't begin to fathom the reasoning behind it
I'm finally in the process of getting Volume I finalized and ready for printing! It's just as hefty as Volume III was (i.e. ~400 pages).
- wen eta
jailbreak b00k? 9/27/2017. 10/14 or 10/21/2017. Would make for interesting Yom Kippur reading :-)
- What?! Why not NOW NOW? Because this way I don't risk Apple changing something in between Tigris betas and the final. And see above.
- So how will we get the book? Direct ordering via this website will be available when it's ready - BTC only or Pay only - but you can also get it via Amazon. Price will be $75 - just like last one.
- BTC? Pay? Seriously? Why not Paypal? Because I avoid Paypal like the plague, seeing as they are an oppressive monopoly who could not care less about their users, subjecting them to ridiculous fees, holds, and other whimsical constraints without considering that their users are human beings (not to mention paying customers).
- Will there be a PDF edition? ABSOLUTELY NOT. Every time I reconsider this, one or another idiot comes up on Twitter and asked for a pirated copy, not noticing he's CCing me. It gets worse when said idiot extolls the virtues of pirating books, not realizing it devalues the insane work I put into the book. People have no respect.
- But $75 is a lot of money! True, it's not cheap. But then, considering that it costs about $20 to print, and AMZN takes their pound of flesh ($14 to be exact), that doesn't leave me with much.
- But I need to try before I buy! If the tons of free articles and tools I periodically provide can't convince you that the content is worth buying, don't buy it.
- If I buy this, will I get updates? Nope. It's a book, not a subscription. I do make the guarantee that, when you do buy it, you will get the most updated version available. That's more than any other publisher.
- But I want updates! Tough. I couldn't handle the logistics of subscriptions even they made sense.
- Can I see what's in the book? sure. In a nutshell, everything related to user mode - what my first edition covered in seven chapters, I now cover in a full book. Unlike the first edition, I delve into Apple's private frameworks, libraries, and offer FAR more detail than I did in the first edition.
- No, really, like detailed what's in the book? - Sure. The detailed ToC is right here.
- What about the hardcover, color edition? My printer gave me so much pain with the proof copy I decided that will wait for all three volumes. There's still the logistics of getting 1,500 glossy A4 pages into a hard cover, but if I can work that out, it will be T-O-T-A-L-L-Y worth it :-)
- Where can I comment/flame/ask questions? - Same place as usual - The NewOSXBook forum. I welcome all suggestions and queries in The Book's Forum, anytime. My only request is that you post there (anonymously is fine), rather than email or twitter, please.
- When's the next instance of your MOXiI training? - That's The Tg Training.
But I won't be doing any training at the very least until November. Or maybe not till 2018 . Another training is set for December 11th, 2017.
09/27/2016 - Volume III is OUT
Volume III is out - you can get it directly here when orders open next week, or just preorder and buy through Amazon
Why was this delayed? you promised Late August! Yeah, well, what can I do. I was really hoping to get my hands on a sample of Pegasus/Trident so I could also add it to the book. All entities in possession of this sample were very promising, but didn't really deliver. And I can't hold back anymore now that Mac OS 12/iOS 10 is out!
So how do we get the book? Direct ordering via this website will be available soon, but you can also get preorders via Amazon
Why not direct orders through this website now???
Because I still need to print stock. I'll open this up when I have more - just so you get the idea I have preorders in the (low) hundreds. I'm doing my best. I'm just one guy here!
What if I preordered through the website already? Then you get preference, of course. I'll be reaching out to you to ask you if you haven't changed your mind, and if so ask for a humble payment.
Can I see what's in the book? sure. In a nutshell, everything related to security in Apple's OSes - both MacOS and the *OS variants, of course. Part I deals with the mechanisms (Authentication, Authorization, Auditing, MACF, Code Signing, Gatekeeper, AMFI, SIP, Sandboxing, Privacy and Data Protection), and Part II deals with all modern jailbreaks from evasi0n 6 to Pangu's 9.1, as well as classic vulnerablities for [10.]10.x and [10.]11.x, some of which are explained in detail for the very first time! Oh, yeah, and there's an appendix with a MacOS Hardening checklist, too
No, really, like detailed what's in the book? - Sure. The detailed ToC is right here.
Where can I comment/flame/ask questions? - Same place as usual - The NewOSXBook forum. I welcome all suggestions and queries in The Book's Forum, anytime. My only request is that you post there (anonymously is fine), rather than email or twitter, please.
Wait, what's this "print" thing - What, no e-Book?!?!?! Nope. Not for a while. Sorry
When's Volume I and II coming? Soon. Volume III started as one chapter in Volume I.. just so you get the level of detail we're talking about here. Volume I (user space) will be next, then Volume II (kernel/hardware). But I also owe a debt to Android enthusiasts first, which I intend to repay with interest.
When's the next instance of your MOXiI training? - That's The Tg Training in NYC this 10/31!.