Home‎ > ‎Articles and Publications‎ > ‎Articles (ENG)‎ > ‎

Intercepting System Calls

by Antonino Calderone - Computer Programming - n. 170 - Luglio/Agosto 2007 Any programmer at least once in life needed to intercept a system call for modifying or extending the related semantics. Perhaps we are exaggerating, but we cannot deny it is at worst an interesting exercise. Indeed if we try to do such thing for a program without any chance to access or modify the source code, e.g. having just the binary, it is also not so easy. Unless we use a library like Detours ([1]).

This library is made by a prankster hacker, but it has been created by Microsoft Research Labs (https://www.microsoft.com/en-us/research/project/detours/)

According to Microsoft, Detours is is a library for instrumenting arbitrary Win32 functions Windows-compatible processors. Detours intercepts Win32 functions by re-writing the in-memory code for target functions. The Detours package also contains utilities to attach arbitrary DLLs and data segments (called payloads) to any Win32 binary. Detours is used widely within Microsoft and within the industry.

So, Detours is a research tool a programmer can use for software analysis and debugging purposes especially in consideration that  in today’s world systems researchers seldom have access to all relevant source code. 

This means, when the solution is not easy as modifying existing code, Detours can help us.

Installing Detours

At time of writing the released library version is 2.1 which supports both IA-32 and IA-64 architectures.  Installation package includes source code, examples and comple user guide. Anything is typically installed  into Microsoft Research\Detours Express 2.1 sub-directory of program folder.

The library can be built using nmake from installation directory. Updated information, included the licence one, can be found at https://www.microsoft.com/en-us/research/project/detours/.

Inside Detours

Detours provides an API for intercepting functions by re-writing the in-memory code for target functions, without needs to modify the binaries on the file system.

It lets you intercept a function placing a jmp instruction in the specified address creating a trampoline to your code (let's call it DetourFunction).

The "jmp DetourFunction" code will replace the original function code, which must be moved (and preserved) at beginning of the trampoline function.

After the DetourFunction execution completes, it will jump to the TrampolineFunction, which first will executed the preserved instructions and then will jump to the original code at an offset that will skip the "jmp DetourFunction", as shown in the following example:

List 1 - Pseudo code how Detours modifies a program

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
TargetFunction:
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;; push ebp     ;;; Original code overwritten
;;; mov ebp,esp  ;;; by 
;;; push ebx     ;;; JMP DetourFunction
;;; push esi     ;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
jmp DetourFunction

;;; Here the original code of TargetFunction continues... 

TargetFunction+5: ;;; <- Trampoline jumps here
push edi 
; ...
; ... 
ret


;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
TrampolineFunction:
;;; Original assembly code is copied
;;; into trampoline function

push ebp
mov ebp,esp
push ebx
push esi

;;; Then next instruction is a jmp
;;; to original target code
;;;
jmp TargetFunction+5

More in details Detours does the following steps:

  • Create TrampolineFunction preserving original instructions overwritten by JMP instruction.
  • Modify target process memory access permission to allow to overwrite the existing code
  • Modify target function code
  • Resetting back the memory access permission in order to make the code executable
  • Flush the instruction cache

Original assembly instructions are copied preserving their integrity. To do that Detours uses a kind of disassembler which maps each instruction and the related length. In fact it is not possible assuming which is the number or the length in bytes of overwritten instructions because of x86 instruction variable length.

At this point the obvious question is if a different way to obtain such result. 

An alternative solution could be to modify the import table entries but it works fine for DLL only.

Another way is to use the breakpoints via the debugging exceptions. This approach typically has two drawbacks: one is that requires an additional process which handles the debug exceptions, and second the performance can be deeply impacted.

Using Detours

While Detours internals could be not easy to be fully understood, how to use the library should be simpler.

Just let us analyse in detail one of the examples provided with the library, named simple which shows how to intercept the Win32 function Sleep.

The actors are a console application named sleep5 (sleep5.cpp) a the DLL simple.dll (simple.cpp).

Binaries are installed in a directory named bin at same level of samples directory.

sleep5 just calls Sleep syscall.

Without any modification, the program shows the following output:

sleep5.exe: Starting.
sleep5.exe: Done sleeping.

The function Sleep is executed in between the two lines above, suspending the process for 5 seconds.

For injecting the simple.dll library detouring Sleep is possible to use an utility named withdll. For example running the following command 

withdll /d:"simple.dll" sleep5.exe

it will show the following output:

withdll.exe: Starting: `sleep5.exe'
withdll.exe: with `simple.dll'
withdll.exe: marked by `detoured.dll'
simple.dll: Starting.
simple.dll: Detoured Sleep().
sleep5.exe: Starting.
sleep5.exe: Done sleeping.
simple.dll: Removed Sleep() (result=0), slept 5000 ticks.

List 2 - Fragment of simple.dll source code

//////////////////////////////////////////////////////////////////////////////
//
// Detours Test Program (simple.cpp of simple.dll)
//
// Microsoft Research Detours Package, Version 2.1.
//
// Copyright (c) Microsoft Corporation. All rights reserved.
//
// This DLL will detour the Windows Sleep API so that TimedSleep function
// gets called instead. TimedSleep records the before and after times, and
// calls the real Sleep API through the TrueSleep function pointer.
//
...
static VOID (WINAPI * TrueSleep)(DWORD dwMilliseconds) = Sleep;
VOID WINAPI TimedSleep(DWORD dwMilliseconds) { ... }
BOOL WINAPI DllMain(HINSTANCE hinst, DWORD dwReason, LPVOID reserved)
{
  ...
  if (dwReason == DLL_PROCESS_ATTACH) {
    DetourRestoreAfterWith();
    DetourTransactionBegin();
    DetourUpdateThread(GetCurrentThread());
    DetourAttach(&(PVOID&)TrueSleep, TimedSleep);
    error = DetourTransactionCommit();

  if (error == NO_ERROR) …
    … 
  }
  else if (dwReason == DLL_PROCESS_DETACH) {
    DetourTransactionBegin();
    DetourUpdateThread(GetCurrentThread());
    DetourDetach(&(PVOID&)TrueSleep, TimedSleep);
    error = DetourTransactionCommit();
    printf("simple.dll: Removed Sleep() (result=%d), slept %d ticks.\n",error, dwSlept);
    fflush(stdout);  
  }
}



 
Fig1 - sequence diagram of Sleep and the detoured version of Sleep which executes TimedSleep 


Sleep function is intercepted when the DLL is connected to the process sleep5 via the function DllMain which uses DLL_PROCESS_ATTACH and DLL_PROCESS_DETACH parameters to attach and detach the target function.

The function DetourAttach prepares the trampoline within a detour transaction context (managed via DetourTransactionBegin and DetourTransactionCommit).

The code is modified during the transaction commit.

The attach function has the following prototype:

LONG DetourAttach( PVOID * ppPointer, PVOID pDetour );

First parameter refers the intercepting function, the second the detoured function.

Once intercepted, in order to call the original function the trampoline must be used.

The function DetourRestoreAfterWith is called before the transaction in order to reset the import table modified by DetourCreateProcessWithDll used by withdll tool for injecting the simple.dll into sleep5 process.


List 3 - TimedSleep  function implemented in simple.dll,

static LONG dwSlept = 0;


VOID WINAPI TimedSleep(DWORD dwMilliseconds)
{
  DWORD dwBeg = GetTickCount();
  TrueSleep(dwMilliseconds);
  DWORD dwEnd = GetTickCount();
  InterlockedExchangeAdd(&dwSlept, dwEnd-dwBeg);
}

The original Sleep is called via the Trampoline function pointer hold by TrueSleep.

Another preparatory operation is performed by DetourUpdateThread which can inform Detours about execution threads.

Such threads must be suspended before performing any memory patch.

In case if the transaction is executed within DllMain (as shown in the example) it is guarantied that no other threads are running.

When the the library simple.ddl is detached, the DetourDetach function is called to reset the original process state.

Adding payloads and modifying import table

Even if the main feature of Detours is intercepting syscalls, it has additional interesting features, as the feature which allows to link data segments (payloads) to a process or to modify the DLL import table with the possibility to reverse back the original content.

Detours can modify the Portable Executable (PE) of binary executable. PE format derives from Unix Common Object File Format (COFF) and it is used to describe EXEs, OBJs, DLLs and device drivers (.SYS).

Such format is used in Windows with extensions for .NET PE, 64 bit architecture, etc. (see https://msdn.microsoft.com/library/windows/desktop/ms680547(v=vs.85).aspx?id=19509 for more details).

PE provides to O/S information for loading a program in a process, and contains the DLL references, import and export address tables, resource data and TLS data.

A program in memory is divided in sections, such as .text which contains the executable code, .data for initialised data; a debug section for debug symbols if present, export tables (for DLL), etc.

Sections follows the DOS/PE headers and they are optional depending on binary target (EXE, DLL, SYS, ...)

To modify a PE Detours creates a new section named .detours before any debug symbols section (which is always in the last section). Such new section is used to store the original PE header or any additional payload (as shown in Fig.2).

Fig. 2 - Win32 program modified by Detours Regardless the reason for modifying a PE, Detours is extremely effective in doing that. Specific API is available such as DetourBinaryEditImports or DetourBinaryEnumeratePayloads, plus additional "minor" features.

Conclusion

We have described a powerful and simple tool, rich of interesting features (and we have presented here just the main ones). Often the best documentation comes from the source ode, and Detours source code does not represent an exception to such rule.

If you are curious we suggest to read the reference article and Microsoft research documentation and, of course we suggest to download, install and use Detours.


References

[1] G. Hunt, D. Brubacher - "Detours: Binary Interception of Win32 Functions", Microsoft Research

Comments