PS3 Heap Overflow Exploit Explained

As of very recently the PS3′s security has been finally been compromised!

Commercially, a company is selling a USB device called PSJailBreak (for a steep $130) which takes advantage of this exploit but it was reverse engineered and released as an open source project, yesterday, called PSGroove. The open source release is targeted towards homebrew developers who want to develop their own things on the PS3.

The guys over at lan.st have a really nice wiki post about the reverse engineering they did on PSJailBreak. Be sure to check out the original article for a more in depth explanation. I’ve summarized the exploit below. I take no credit for any of these findings. All the credit goes to the awesome PS3 hackers out there (GeoHot)!

Summarizing the Exploit

The exploit takes advantage of a heap overflow by making the PS3 think the single USB device that’s plugged in is a 6 port USB hub. The single USB then tells the PS3 to allocate/unallocate large chunks of heap space (4K) (using large device descriptors) by faking that devices are being plugged and unplugged from the USB hub.

One of the fake USB devices thats plugged in manages to overwrite another USB device’s heap space when allocating it’s configuration data. The key point here is that when it overwrites this data, it overwrites malloc’s pointer to where the next free memory block is. That means when malloc gets called again, it allocates space to wherever that pointer is pointing. The exploit overwrites it with a pointer to a function that it knows the PS3 will need to call. The function has something to do with freeing memory when a USB device is unplugged. (You might see where this is going)

All PS3′s have special built in functionality that allows an authorized USB “Jig” device to be plugged into the PS3 which turns it into development/debug mode. Sony uses this for repairing PS3′s and likely distributes it to game developers.

The exploit takes advantage of this fact and tells the PS3 that a “Jig” device has been plugged in. The PS3 sends an authentication challenge to the device to verify it’s authenticity. A real device with a key from Sony would be able to compute the response, respond, and continue on it’s merry way but the exploit takes advantage of the fact that the PS3 has to call malloc to allocate space for the challenge response. Malloc’s pointer to the next free space in memory was overwritten earlier. That means malloc will allocate space in the wrong place, overwriting a function the PS3 calls when a USB device gets unplugged. That means we can send a function of our choice (the shellcode) as the challenge response and the PS3 will overwrite another function with our function!

This is where the exploit gets invoked. The exploit explicitly unplugs one of it’s fake USB devices causing the PS3 to call the function related to freeing device memory which was overwritten with our challenge response data. The PS3 happily starts executing our code. From here we can now start patching memory to tell the PS3 that it’s in debug mode!

So what can you do in debug mode? You can start writing code and developing on the PS3!

Sony is very likely to patch this ASAP. In the meantime, you better hurry and get yourself a PS3. I know I’m thinking about it!