I tried to boot an iOS 12 kernelcache in QEMU: I managed to get as far as IOKit startup before receiving a kernel panic. I learned a lot about how iOS boots up with this project.

Introduction

This is Part 1 of a series on the iOS boot process. Part 2 is here. Sign up with your email to be the first to read new posts.

Let’s get the obvious out of the way first: this is completely useless. If you want to run iOS, you should ask @CorelliumHQ instead, or just buy an iPhone.

I wanted to learn how iOS starts up, but modern iOS devices can only be jailbroken after they’ve already booted. Corellium built a service that simulates an entire virtual iPhone, bootup process and all, but it’s almost impossible to get an invite.

I thought: how hard can it be to boot a tiny bit of iOS in an emulator? After all, Corellium proves that it’s possible. I decided to try it myself.

My goal: to boot enough of iOS to receive a kernel panic (a crash log).

After three days of work, I got iOS to print a crash out of my virtual iPhone’s virtual serial port:

iBoot version: 

corecrypto_kext_start called
FIPSPOST_KEXT [57909750] fipspost_post:156: PASSED: (4 ms) - fipspost_post_integrity
FIPSPOST_KEXT [58102375] fipspost_post:162: PASSED: (1 ms) - fipspost_post_hmac
FIPSPOST_KEXT [58198312] fipspost_post:163: PASSED: (0 ms) - fipspost_post_aes_ecb
FIPSPOST_KEXT [58296812] fipspost_post:164: PASSED: (0 ms) - fipspost_post_aes_cbc
FIPSPOST_KEXT [64344625] fipspost_post:165: PASSED: (95 ms) - fipspost_post_rsa_sig
FIPSPOST_KEXT [68161937] fipspost_post:166: PASSED: (57 ms) - fipspost_post_ecdsa
FIPSPOST_KEXT [69025687] fipspost_post:167: PASSED: (12 ms) - fipspost_post_ecdh
FIPSPOST_KEXT [69226375] fipspost_post:168: PASSED: (0 ms) - fipspost_post_drbg_ctr
FIPSPOST_KEXT [69469125] fipspost_post:169: PASSED: (2 ms) - fipspost_post_aes_ccm
FIPSPOST_KEXT [69593562] fipspost_post:171: PASSED: (1 ms) - fipspost_post_aes_gcm
FIPSPOST_KEXT [69702000] fipspost_post:172: PASSED: (0 ms) - fipspost_post_aes_xts
FIPSPOST_KEXT [69835250] fipspost_post:173: PASSED: (1 ms) - fipspost_post_tdes_cbc
FIPSPOST_KEXT [69960062] fipspost_post:174: PASSED: (1 ms) - fipspost_post_drbg_hmac
FIPSPOST_KEXT [70015312] fipspost_post:197: all tests PASSED (198 ms)
panic(cpu 0 caller 0xfffffff0072052b8): Kernel data abort. (saved state: 0xffffffe0313035d0)

	  x0: 0xffffffe0373b8000  x1:  0x0000000000000000  x2:  0x0000000000008000  x3:  0xffffffe0373b8000

	  x4: 0x0000000000000000  x5:  0x0000000000000000  x6:  0x000000002b1f476d  x7:  0x0000000000000770

	  x8: 0x0000000000000000  x9:  0x00000000004a0000  x10: 0x0000000000000025  x11: 0x00000000ffdfffff

	  x12: 0xfffffff0076ae440 x13: 0xffffffe000296600  x14: 0x0000000000d400d5  x15: 0x000000005feecd89

	  x16: 0x000000002b36e61b x17: 0x0000000029116d52  x18: 0xfffffff0070dd000  x19: 0xfffffff0076db000

	  x20: 0xfffffff00766a000 x21: 0xffffffe0004ef300  x22: 0x0000000000008000  x23: 0xfffffff0076db000

	  x24: 0x0000000000008000 x25: 0xffffffe0373b8000  x26: 0x0000000000000096  x27: 0xfffffff0076db000

	  x28: 0xfffffff0076dbd08 fp:  0xffffffe031303920  lr:  0xfffffff00756be30  sp:  0xffffffe031303920

	  pc:  0xfffffff0070d56dc cpsr: 0x200002c4         esr: 0x9600004f          far: 0xffffffe0373b8000

That’s real iOS 12 beta 2 code printing that crash. With a bit more work, I’m confident I can get it to boot further.

Initial research

From research, I already know how an iPhone starts up:

  • the bootrom, burned into the CPU chip, loads iBoot, the bootloader
  • iBoot loads the kernelcache, a file bundling XNU - iOS’s kernel (the core of the operating system) together with all of the kernel’s device drivers
  • the kernel then loads the rest of the operating system components

I decided to boot the XNU kernel directly in an emulator, bypassing iBoot. Previous iOS emulation efforts such as iEmu (by the team that later made Corellium) tries to emulate iBoot because it’s simpler to understand than the iOS kernel at the time. However, iBoot is closed source, but XNU is now open source. In addition, to obtain iBoot, one needs a jailbroken device, but the kernel itself is unencrypted and can be obtained from an update IPSW file. Therefore, it’s now actually easier to obtain and understand the kernel itself.

I chose to modify QEMU for this experiment, since it’s the standard open source virtual machine, and has great support for ARM64 processors.

Finally, I decided to only look at Apple’s own open source code if possible. While there are other open source bootloaders that can start XNU, like Chameleon, GRUB, and winocm’s GenericBooter, I didn’t look at them, since I wanted to get information first hand instead of relying on other people’s research. (It’s more fun this way.)

Loading kernel into QEMU

First, I needed a copy of the iOS kernel. I downloaded the iOS 12 beta 2 update for the iPhone X. There’s plenty of tutorials online (like this one) on extracting a kernel from an IPSW file, so I followed one, and got a Mach-O executable file.

To learn how to load a Mach-O file into memory, I consulted Apple’s Boot-132. It turns out loading a kernel is very simple: for each segment, convert the virtual address to a physical address by masking out the top bits, then copy the data into memory at the physical address. I’m already familiar with Mach-O files, so it’s easy to port the logic over.

QEMU already supports loading Linux kernels directly. It does this by loading the kernel into a buffer, then registering the buffer as a ROM in the emulation, so that the buffer is copied to the correct address when the virtual device boots. All I had to do was to load the Mach-O file’s segments into a buffer, and register it, just like the Linux boot code.

Once I implemented kernel loading, I tried starting QEMU with the kernelcache:

aarch64-softmmu/qemu-system-aarch64 -M virt \
-cpu max -kernel kcache_out.bin -dtb devicetree.dtb \
-monitor stdio -m 2G -s -S -d unimp,int \
-serial file:/dev/stdout -serial file:/dev/stdout \
-serial file:/dev/stdout -append "debug=0x8 kextlog=0xfff"

Note that I passed in -d unimp,int to print out unimplemented CPU special registers and every processor exception, so I can find where the kernel is crashing. I also passed in -s -S to enable GDB support and to pause at startup so I can attach to it.

I attached an ARM64 GDB (from DevkitPro) and started execution:

(gdb) target remote :1234
Remote debugging using :1234
warning: No executable has been specified and target does not support
determining executable automatically.  Try using the "file" command.
0x0000000040000000 in ?? ()
(gdb) c

… and QEMU promptly exited with the error:

write access to unsupported AArch64 system register op0:3 op1:4 crn:15 crm:2 op2:1

Good news: By single stepping through GDB, I can see that this happens after we already entered the kernel’s entry point. So we ran a tiny bit of iOS already!

Tweaking CPU emulation, part 1

QEMU emulates a Cortex-A57 CPU, which doesn’t have all the control registers of the iPhone X’s custom Mistral CPU. To make QEMU ignore the invalid register write, I searched for the error message, and commented out the error.

After fixing this, the kernel now crashes with a Data Abort exception (as printed by QEMU’s console) when it tries to read the boot arguments. Well, duh: I didn’t provide any yet, so it’s trying to load from a null pointer.

Providing boot args

XNU needs boot arguments to find where the kernel is loaded and how much memory the system has.

The boot_args struct is well documented by Apple. I created and populated a boot_args structure, registered it as a QEMU ROM (like the kernel) so that QEMU copies it into the emulated device’s memory, and passed its address into x0 as requested during startup.

I got a bit confused on where to put the boot arguments in memory. I originally put it just after the kernel, but it got overwritten by the boot code. By examining the early boot code, it turns out that the kernel allocates initial page tables directly after the end of the kernel, overwriting my boot arguments. Another file showed me how to fix this: I had to extend the topOfKernelData address in the boot_args to include any extra data such as boot arguments so that they do not get overwritten.

With this fixed, the kernel actually booted into C code before it crashed with yet another null dereference exception. By examining QEMU’s log of the data abort exception, and cross referencing using Hopper, I found that it crashed in pe_identify_machine. That function reads from the device tree, which I didn’t provide yet.

Loading a device tree

iOS uses a device tree, a data structure containing a list of devices and their addresses so that the kernel knows how to access the devices in an SoC. (The concept was later adopted by Linux on ARM platforms.)

The device tree is also contained in the IPSW update file. I initially loaded the device tree as is, but the kernel crashed immediately. By placing a breakpoint on panic in GDB, I saw that the kernel expected iBoot to populate some fields in the device tree. Because I’m bypassing iBoot, I had to write a script to populate the timer frequency and early random seed in the device tree.

After loading the modified device tree into memory (as yet another QEMU ROM), the kernel now dies with an Invalid Instruction exception.

Tweaking CPU emulation, part 2

The offending instruction turned out to be a stadd instruction, introduced in ARMv8.1. QEMU, in full system emulation mode, only emulates a Cortex-A57, which supports ARMv8 only; however, the ARMv8.1 instructions are enabled in user mode emulation mode. So I modified QEMU’s cpu selection code to enable those features for full system emulation mode as well.

Now the kernel crashes with a bunch of repeated data aborts. By placing a breakpoint on panic, it became clear that panic itself was crashing. Ignoring the later panic, I found that the first panic happens when the kernel’s Kext loading code tries to bzero a newly allocated buffer.

That’s good: it seems that the kernel is already trying to load kexts, which means it’s gotten quite far in the boot process, and should be able to output information through the serial port.

Getting serial port output

Up to this point, my only outputs from the virtual device were QEMU’s log messages and GDB breakpoints. However, my goal was to get the iOS kernel to print errors out the serial port. Now that the device tree is loaded and the serial port initialization code in iOS works, all I need to do is to emulate a serial port at the correct address.

Obviously, QEMU doesn’t have support for the iPhone X, so I’ve been booting the kernel using QEMU’s virt machine type. While the peripherals are completely different and incompatible, for early boot, only two devices need to work properly: the timer and the serial port. Timers are now standardized across all ARMv8 cpus, so only the serial port must be implemented for debug output.

It turns out QEMU already supports emulating the iPhone’s serial port, thanks to a quirk of history. Modern iPhones still use a serial port design compatible with the very first iPhones, which used Samsung CPUs. (Why change what works?) It’s the same serial port design Samsung used in all their CPUs starting in 2004 all the way to the latest Exynos CPUs. QEMU has support for emulating an Exynos4210, so I simply added an Exynos serial port to the Virt machine, and I got output from the serial port.

What’s next

To diagnose why the kernel crashes with a Data Abort exception, I need to understand how the memory is mapped by iOS. To do that, I will need to add a command in QEMU to dump the CPU’s pagetables. Once that’s done, I’ll be able to figure out why the memory allocation fails, and get the kernel to boot a bit further.

My next goal is to start launchd on a virtual iPhone. That’s probably going to take much longer than three days, but I’ll definitely learn even more about iOS, ARM, and QEMU.

What I learned

  • how iOS boots
  • how to modify QEMU to load code directly into memory
  • the value of a debugger for board bringup
    • in the emulator, I can single step, examine registers, set breakpoints, and get output even when serial port isn’t working. Meanwhile, when I tried doing low-level bringup on my Nexus 6P, the only way I can check that my code is actually running is by adding a reboot command. No wonder that most developer boards include JTAG support to support the same level of debugging as emulators.