28.11.2012 Views

Back-tracing in MIPS-based Linux Systems

Back-tracing in MIPS-based Linux Systems

Back-tracing in MIPS-based Linux Systems

SHOW MORE
SHOW LESS

You also want an ePaper? Increase the reach of your titles

YUMPU automatically turns print PDFs into web optimized ePapers that Google loves.

<strong>Back</strong>-<strong>trac<strong>in</strong>g</strong> <strong>in</strong> <strong>MIPS</strong>-<strong>based</strong> L<strong>in</strong>ux <strong>Systems</strong><br />

Kim, Jong-Sung (jsungkim@lge.com)<br />

LG Electronics<br />

1<br />

Great Company Great People


Agenda<br />

� <strong>Back</strong>grounds<br />

� <strong>MIPS</strong> stack-frame structure<br />

� <strong>Back</strong>-<strong>trac<strong>in</strong>g</strong> <strong>in</strong> <strong>MIPS</strong> systems<br />

� <strong>Back</strong>-<strong>trac<strong>in</strong>g</strong> from the signal context<br />

� Sample applications<br />

� Summary<br />

� References<br />

� Appendix: Crash Report System applied to LGE products<br />

2<br />

Great Company Great People


<strong>Back</strong>grounds<br />

3<br />

Great Company Great People


The <strong>MIPS</strong> Core<br />

� Brief history<br />

� In 1981, a team led by John L. Hennessy at Stanford University started work<strong>in</strong>g on what<br />

would become the 1st <strong>MIPS</strong> processor<br />

� In 1984, Hennessy left Stanford to form <strong>MIPS</strong> Computer <strong>Systems</strong><br />

� In 1992, SGI bought the company to guarantee the design would not be lost<br />

� The company became known as <strong>MIPS</strong> Technologies<br />

� Key concepts<br />

� Deep <strong>in</strong>struction pipel<strong>in</strong>es<br />

� One cycle for one <strong>in</strong>struction (elim<strong>in</strong>at<strong>in</strong>g <strong>in</strong>terlocks)<br />

� Core design licens<strong>in</strong>g<br />

� Broadcom (SiByte), IDT, LSI Logic, NEC, Philips, Toshiba, …<br />

� Very popular <strong>in</strong> develop<strong>in</strong>g CE products (BDP, DTV, PDA, STB, …)<br />

� Known as roll<strong>in</strong>g back stack-frames is not possible<br />

4<br />

Great Company Great People


<strong>Back</strong>-<strong>trac<strong>in</strong>g</strong><br />

� In many cases, it’s very hard and takes long time to reproduce an error<br />

� Just-<strong>in</strong>-time debug <strong>in</strong>formation is very useful<br />

� Process/thread ID<br />

� Register dumps<br />

� Variable dumps<br />

� Programm<strong>in</strong>g language-level call-stack<br />

� Et cetera<br />

� <strong>Back</strong>-<strong>trac<strong>in</strong>g</strong>: extract<strong>in</strong>g the function call-stack<br />

5<br />

Great Company Great People


Related Works<br />

� __built<strong>in</strong>_return_address function/macro <strong>in</strong>side GCC<br />

� Written by Richard Henderson (rth@redhat.com)<br />

� Several just-<strong>in</strong>-time debug features <strong>in</strong>side Glibc<br />

� Includ<strong>in</strong>g:<br />

● backtrace(3), backtrace_symbols(3), …<br />

● catchsegv(1), libSegFault.so<br />

� Written by Ulrich Drepper (drepper@redhat.com)<br />

� However, they’re not available for <strong>MIPS</strong> systems<br />

6<br />

Great Company Great People


<strong>MIPS</strong> Stack-frame Structure<br />

7<br />

Great Company Great People


<strong>MIPS</strong> Stack-frame Structure from ABI<br />

� Conceptual structure of a <strong>MIPS</strong> stack-frame<br />

8<br />

Great Company Great People


Real-world <strong>MIPS</strong> Stack-frame Structure<br />

#<strong>in</strong>clude <br />

#<strong>in</strong>clude <br />

� Sample C function<br />

…<br />

� Nested function<br />

static <strong>in</strong>t shared_local(void)<br />

{<br />

void *dl_obj;<br />

<strong>in</strong>t (*dl_fcn)(void);<br />

� Two automatic variables<br />

pr<strong>in</strong>tf(“%s\n”, __FUNCTION__);<br />

dl_obj = dlopen(“libdynamic.so”, RTLD_NOW);<br />

dl_fcn = (<strong>in</strong>t (*)(void))dlsym(dl_obj, “dynamic_global”);<br />

return dl_fcn();<br />

}<br />

� Stack-frame structure<br />

� Reserved region for arguments<br />

� Old stack-frame po<strong>in</strong>ter<br />

� Return address<br />

� Hmm.. what’s the problem?<br />

� Variable offsets from the top of stack<br />

� This figure is not always true<br />

9<br />

Great Company Great People


<strong>Back</strong>-<strong>trac<strong>in</strong>g</strong> <strong>in</strong> <strong>MIPS</strong> <strong>Systems</strong><br />

10<br />

Great Company Great People


B<strong>in</strong>ary Code Scann<strong>in</strong>g<br />

� The stack-frame is not enough for back-<strong>trac<strong>in</strong>g</strong><br />

� Previous stack-frame po<strong>in</strong>ter<br />

● Offset from $sp is variable<br />

● Sometimes not saved<br />

� Return address<br />

● Offset from $sp is variable<br />

● Sometimes not saved (but, don’t care <strong>in</strong> this section)<br />

� So, b<strong>in</strong>ary code scann<strong>in</strong>g is required to acquire:<br />

� Current stack-frame size<br />

� Offset of stack-stored return address<br />

11<br />

Great Company Great People


Function Prologue & Epilogue<br />

� Prologue for a nested function<br />

� Context register setup<br />

� Current stack-frame allocation<br />

� Return address sav<strong>in</strong>g<br />

� Epilogue for a nested function<br />

� Return address load<strong>in</strong>g<br />

� Current stack-frame de-allocation<br />

� Function return<br />

12<br />

Great Company Great People


<strong>Back</strong>-<strong>trac<strong>in</strong>g</strong> Procedure<br />

� Initialization<br />

� Registers latch<strong>in</strong>g (ra � $ra, sp � $sp)<br />

� Code scann<strong>in</strong>g for current stack-frame size<br />

� Adjust sp to previous stack-frame (sp � sp + stack_size)<br />

� Repeat until maximum depth reached or ra is zero<br />

� Save ra <strong>in</strong> return address buffer<br />

� Code scann<strong>in</strong>g for current stack-frame size and offset of saved return address<br />

� Load return address to ra (ra � sp[ra_offset])<br />

� Adjust sp to previous stack-frame (sp � sp + stack_size)<br />

� Return the count of the return addresses found<br />

13<br />

Great Company Great People


Instruction Formats<br />

31 26 25 21 20 16 15 0<br />

ADDIU<br />

rs rt immediate<br />

001001<br />

6 5 5 16<br />

Format: ADDIU rt, rs, immediate<br />

Description: GPR[rt] GPR[rs] + immediate<br />

31 26 25 21 20 16 15 0<br />

LW<br />

base rt offset<br />

100011<br />

6 5 5 16<br />

Format: LW rt, offset(base)<br />

Description: GPR[rt] memory[GPR[base] + offset]<br />

31 26 25 21 20 16 15 0<br />

SW<br />

base rt offset<br />

101011<br />

6 5 5 16<br />

Format: SW rt, offset(base)<br />

Description: memory[GPR[base] + offset] GPR[rt]<br />

14<br />

Great Company Great People


acktrace_mips32 Function<br />

� Work<strong>in</strong>g source code of backtrace_mips32<br />

15<br />

Great Company Great People


<strong>Back</strong>-<strong>trac<strong>in</strong>g</strong> from The Signal Context<br />

16<br />

Great Company Great People


Signal Handler Context<br />

� backtrace_mips32 can’t handle stack-frames from signal contexts<br />

� In the signal handler context:<br />

� $ra po<strong>in</strong>ts to the code block (by kernel) <strong>in</strong> the stack<br />

� backtrace_mips32 can’t handle this non-function code block<br />

� To back-trace from signal contexts:<br />

� Skip the kernel-<strong>in</strong>serted code/data block by referenc<strong>in</strong>g the signal context structure<br />

(ucontext_t) given to the signal handler<br />

� Handle the possible leaf function at the top of the call-stack<br />

● No saved return address<br />

● No stack-frame<br />

17<br />

Great Company Great People


<strong>Back</strong>-<strong>trac<strong>in</strong>g</strong> from The Signal Context<br />

� Initialization<br />

� F<strong>in</strong>d $pc, $ra, and $sp from the signal context structure<br />

(pc � mcontext_t::pc, ra � mcontext_t::gregs[31], sp � mcontext_t::gregs[29])<br />

� Save pc <strong>in</strong> return address buffer<br />

� Code scann<strong>in</strong>g from pc to f<strong>in</strong>d stack-frame size and stored ra offset<br />

� If return address was stored, load it to ra (ra � sp[ra_offset])<br />

� Adjust sp to previous stack-frame (sp � sp + stack_size)<br />

� Repeat until maximum depth reached or ra is zero<br />

� Save ra <strong>in</strong> return address buffer<br />

� Code scann<strong>in</strong>g for current stack-frame size and offset of saved return address<br />

� Load return address to ra (ra � sp[ra_offset])<br />

� Adjust sp to previous stack-frame (sp � sp + stack_size)<br />

� Return the count of found return addresses<br />

18<br />

Great Company Great People


sigbacktrace_mips32 Function<br />

� Work<strong>in</strong>g source code of sigbacktrace_mips32<br />

__out_of_loop:<br />

#def<strong>in</strong>e abs(s) ((s) < 0 ? -(s) : (s))<br />

if(ra_offset)<br />

ra = *(unsigned long **)((unsigned long)sp + ra_offset);<br />

if(stack_size)<br />

sp = (unsigned long *)((unsigned long)sp + stack_size);<br />

<strong>in</strong>t sigbacktrace_mips32(void **buffer, <strong>in</strong>t size, ucontext_t const *uc)<br />

{<br />

unsigned long *addr;<br />

unsigned long *pc, *ra, *sp;<br />

size_t ra_offset, stack_size;<br />

<strong>in</strong>t depth;<br />

// repeat backward scann<strong>in</strong>g<br />

for(depth = 1; depth < size && ra; ++depth)<br />

{<br />

buffer[depth] = ra;<br />

ra_offset = stack_size = 0;<br />

if(size == 0)<br />

return 0;<br />

if(!buffer || size < 0 || !uc)<br />

return -EINVAL;<br />

for(addr = ra; !ra_offset || !stack_size; --addr)<br />

{<br />

switch(*addr & 0xffff0000)<br />

{<br />

case 0x27bd0000:<br />

stack_size = abs((short)(*addr & 0xffff));<br />

break;<br />

// get current $pc, $ra and $sp<br />

pc = (unsigned long *)(unsigned long)uc->uc_mcontext.pc;<br />

ra = (unsigned long *)(unsigned long)uc->uc_mcontext.gregs[31];<br />

sp = (unsigned long *)(unsigned long)uc->uc_mcontext.gregs[29];<br />

buffer[0] = pc;<br />

if(size == 1)<br />

return 1;<br />

case 0xafbf0000:<br />

ra_offset = (short)(*addr & 0xffff);<br />

break;<br />

// scann<strong>in</strong>g to f<strong>in</strong>d the size of the current stack-frame<br />

ra_offset = stack_size = 0;<br />

case 0x3c1c0000:<br />

return depth + 1;<br />

default:<br />

break;<br />

}<br />

}<br />

for(addr = pc; !ra_offset || !stack_size; --addr)<br />

{<br />

switch(*addr & 0xffff0000)<br />

{<br />

case 0x27bd0000:<br />

stack_size = abs((short)(*addr & 0xffff));<br />

break;<br />

ra = *(unsigned long **)((unsigned long)sp + ra_offset);<br />

sp = (unsigned long *)((unsigned long)sp + stack_size);<br />

}<br />

case 0xafbf0000:<br />

ra_offset = (short)(*addr & 0xffff);<br />

break;<br />

return depth;<br />

}<br />

case 0x3c1c0000:<br />

goto __out_of_loop;<br />

default:<br />

break;<br />

}<br />

}<br />

19<br />

Great Company Great People


More Considerations for The Safer <strong>Back</strong>-<strong>trac<strong>in</strong>g</strong><br />

� Leaf functions<br />

� Leaf functions usually don’t save registers<br />

� Leaf functions can run with zero-size stack-frame<br />

� Assembly-coded or hard-optimized functions<br />

� These functions may not save registers<br />

� These functions may run with zero-size stack-frame<br />

� These functions may not have normal function prologue and/or epilogue<br />

� If a function without normal function prologue is located at the first place of a<br />

loaded object, sigbacktrace will dereference illegal addresses<br />

� Therefore, back-<strong>trac<strong>in</strong>g</strong> needs hands of the loaded object/symbol table<br />

20<br />

Great Company Great People


Sample Applications<br />

21<br />

Great Company Great People


Build & Runn<strong>in</strong>g Environment<br />

� Processor: Broadcom BCM7440P 266MHz<br />

� L<strong>in</strong>ux kernel: 2.6.12<br />

� C library: uClibc 0.9.28<br />

� GCC version: 3.4.6<br />

� CFLAGS: -O –W –Wall –export-dynamic –fPIC –fno-optimize-sibl<strong>in</strong>g-calls –g<br />

22<br />

Great Company Great People


Sample Application #1<br />

� Simple application to test backtrace_mips32<br />

� Us<strong>in</strong>g static/shared/dynamic-loaded libraries<br />

� All functions pr<strong>in</strong>t its name<br />

� dynamic_local dumps the call-stack us<strong>in</strong>g backtrace_mips32<br />

23<br />

Great Company Great People


Outputs from The Application<br />

24<br />

Great Company Great People


Outputs from The Application (Stripped B<strong>in</strong>aries)<br />

25<br />

Great Company Great People


Outputs from The Application (Optimized B<strong>in</strong>aries by –O2 or –O3)<br />

26<br />

Great Company Great People


Sample Application #2<br />

� Same with sample application #1, except:<br />

� dynamic_local tries null-po<strong>in</strong>ter assignment<br />

� sigbacktrace_mips32 is called from the (SIGSEGV handl<strong>in</strong>g) signal context<br />

27<br />

Great Company Great People


Outputs from The Application<br />

28<br />

Great Company Great People


Accompanied to objdump Utility<br />

� If we have b<strong>in</strong>aries compiled with “-g” option…<br />

29<br />

Great Company Great People


Wrap-up<br />

30<br />

Great Company Great People


Summary<br />

� <strong>Back</strong>-<strong>trac<strong>in</strong>g</strong> <strong>in</strong> the <strong>MIPS</strong> needs some code <strong>in</strong>spections<br />

� <strong>Back</strong>-<strong>trac<strong>in</strong>g</strong> from the signal context needs some more handl<strong>in</strong>gs<br />

� Work<strong>in</strong>g backtrace/sigbacktrace functions are presented<br />

� Now I’m work<strong>in</strong>g on mak<strong>in</strong>g these functions as an open-source library or <strong>in</strong>side <strong>MIPS</strong>ports<br />

of C libraries<br />

31<br />

Great Company Great People


References<br />

� Documents<br />

� <strong>MIPS</strong>32® Architecture For Programmers – Volume I: Introduction to the <strong>MIPS</strong>32®<br />

Architecture<br />

� <strong>MIPS</strong>32® Architecture For Programmers – Volume II: The <strong>MIPS</strong>32® Instruction Set<br />

� System V Application B<strong>in</strong>ary Interface – <strong>MIPS</strong>® RISC Processor Supplement, 3 rd Edition<br />

� Us<strong>in</strong>g the GNU Compiler Collection<br />

� Internet resources<br />

� <strong>MIPS</strong> Architecture – History<br />

32<br />

Great Company Great People


Appendix: Crash Report System Applied to LGE Products<br />

33<br />

Great Company Great People


Crash Report System<br />

� Purpose<br />

� Guarantee not to lose <strong>in</strong>-time <strong>in</strong>formation of system crashes<br />

� Easy extraction of <strong>in</strong>-time <strong>in</strong>formation<br />

● /proc filesystem entry<br />

● Extractable to a USB drive<br />

� With Crash Report System…<br />

� All console output is stored on a circular log buffer<br />

� On watchdog expiration, the captured log is stored to an NVRAM<br />

� Developers can extract the stored log later<br />

� The stored log <strong>in</strong>cludes the just-<strong>in</strong>-time debug <strong>in</strong>formation<br />

34<br />

Great Company Great People


Block Diagram<br />

Transmit<br />

Store log<br />

Expiration<br />

Refresh<br />

Mesg<br />

COP<br />

35<br />

Great Company Great People


In-time Debug Information<br />

� In-time debug <strong>in</strong>formation by the sample application<br />

36<br />

Great Company Great People

Hooray! Your file is uploaded and ready to be published.

Saved successfully!

Ooh no, something went wrong!