21.03.2013 Views

Problem - Kevin Tafuro

Problem - Kevin Tafuro

Problem - Kevin Tafuro

SHOW MORE
SHOW LESS

Create successful ePaper yourself

Turn your PDF publications into a flip-book with our unique Google optimized e-Paper software.

12.12 Detecting Debuggers<br />

<strong>Problem</strong><br />

Software protection crackers frequently rely on debuggers to observe the runtime<br />

behavior of an application and to test binary patches that remove or bypass a protection.<br />

You would like to prevent casual analysis of your application by including antidebugger<br />

code.<br />

Solution<br />

The Intel x86 instruction set uses the int3 opcode (0xCC) as a one-byte embedded<br />

breakpoint. Key addresses in the program—such as the first address in a function—<br />

can be checked to see whether they have been replaced with an int3 opcode.<br />

Discussion<br />

General debugger detection is difficult to perform successfully because of the limited<br />

number of techniques available and the ease with which they may be defeated. We<br />

advise you to attempt to detect specific debuggers in addition to using these general<br />

methods (see Recipes 12.13, 12.14, and 12.15).<br />

The two macros defined below can be used to mark locations in the source where<br />

you might expect an int3 to be placed by someone trying to debug your program.<br />

The names used with these macros can then be passed as an argument to spc_check_<br />

int3( ) to test for the existence of the breakpoint instruction.<br />

#define SPC_DEFINE_DBG_SYM(name) asm(#name ": \n")<br />

#define SPC_USE_DBG_SYM(name) extern void name(void)<br />

inline int spc_check_int3(void *address) {<br />

return (*(volatile unsigned char *)address = = 0xCC);<br />

}<br />

The SPC_DEFINE_DBG_SYM macro can be used to label an arbitrary code address, which<br />

can then be made available with the SPC_USE_DBG_SYM macro and passed to spc_<br />

check_int3( ):<br />

#include <br />

void my_func(void) {<br />

int x;<br />

SPC_DEFINE_DBG_SYM(myfunc_nodebug);<br />

for (x = 0; x < 10; x++) printf("X!\n");<br />

}<br />

SPC_USE_DBG_SYM(myfunc_nodebug);<br />

This is the Title of the Book, eMatter Edition<br />

Copyright © 2007 O’Reilly & Associates, Inc. All rights reserved.<br />

Detecting Debuggers | 681

Hooray! Your file is uploaded and ready to be published.

Saved successfully!

Ooh no, something went wrong!