The blog continues at suszter.com/ReversingOnWindows

May 13, 2012

About termination on heap corruption feature

The following line enables the feature to terminate the process if heap corruption is detected by the Windows heap manager.
BOOL bResult = HeapSetInformation(NULL, HeapEnableTerminationOnCorruption, NULL, 0);
I created a simple application calling the above function, and loaded it in Windbg to trace how the feature is enabled.

At line 77396d11 there is a check if HeapEnableTerminationOnCorruption, that is 1, is specified.
77396d11 837d0c01        cmp dword ptr [ebp+0Ch],1
77396d15 7507            jne ntdll!RtlSetHeapInformation+0x59 (77396d1e)
77396d17 83259800447700  and dword ptr [ntdll!RtlpDisableBreakOnFailureCookie (77440098)],0
77396d1e 33c0            xor eax,eax
As it can be seen, if HeapEnableTerminationOnCorruption is specified, [RtlpDisableBreakOnFailureCookie] is set to zero.

As the next step, a simple application that intentionally corrupts the heap was created. I placed a breakpoint to see what happens when [RtlpDisableBreakOnFailureCookie] is accessed.

When the breakpoint hit, I got the code snippet below. The caller function was RtlpReportHeapFailure implying heap failure, so I have reason to think the code snippet is not reached unless heap corruption is encountered.

RtlpGetModifiedProcessCookie returns with a cookie value, and I observed that value was the same to the initial value of [RtlpDisableBreakOnFailureCookie].
7740f5ad e8e67cf7ff      call ntdll!RtlpGetModifiedProcessCookie (77387298)
7740f5b2 390598004477    cmp dword ptr [ntdll!RtlpDisableBreakOnFailureCookie (77440098)],eax
7740f5b8 7411            je ntdll!RtlpReportHeapFailure+0x23 (7740f5cb)
7740f5ba 6868424477      push offset ntdll!RtlpHeapFailureInfo (77444268)
7740f5bf 68740300c0      push 0C0000374h
7740f5c4 e8a3f0ffff      call ntdll!RtlReportCriticalFailure (7740e66c)
7740f5c9 eb1a            jmp ntdll!RtlpReportHeapFailure+0x3d (7740f5e5)
7740f5cb 8b4508          mov eax,dword ptr [ebp+8]
7740f5ce 3b0540674477    cmp eax,dword ptr [ntdll!RtlpHeapErrorHandlerThreshold (77446740)]
7740f5d4 7f0f            jg ntdll!RtlpReportHeapFailure+0x3d (7740f5e5)
7740f5d6 e864f1ffff      call ntdll!RtlpPrintErrorInformation (7740e73f)
7740f5db e840f0ffff      call ntdll!RtlIsAnyDebuggerPresent (7740e620)
7740f5e0 84c0            test al,al
7740f5e2 7401            je ntdll!RtlpReportHeapFailure+0x3d (7740f5e5)
7740f5e4 cc              int 3
7740f5e5 5d              pop ebp
7740f5e6 c20400          ret 4
At line 7740f5b2 the cookie value [RtlpDisableBreakOnFailureCookie] is being compared against the return cookie value of RtlpGetModifiedProcessCookie.

I observed the following situations are possible.
  • HeapEnableTerminationOnCorruption is unset, and the heap is corrupted. In that case at line 7740f5b8 the jump is taken and the process might not be immediately terminated.
  • HeapEnableTerminationOnCorruption is set, and the heap is corrupted. In that case at line 7740f5b8 the jump is not taken and the process is to be immediately terminated.
  This blog is written and maintained by Attila Suszter. Read in Feed Reader.