Quick introduction to reverse engineering for beginners

1.20 x86-64
It’s a 64-bit extension to x86-architecture.
From the reverse engineer’s perspective, most important differences are:
∙ Almost all registers (except FPU and SIMD) are extended to 64 bits and got r- prefix. 8 additional
registers added. Now general purpose registers are: rax, rbx, rcx, rdx, rbp, rsp, rsi, rdi, r8, r9,
r10, r11, r12, r13, r14, r15.
It’s still possible to access to older register parts as usual. For example, it’s possible to access lower
32-bit part of RAX using EAX.
New r8-r15 registers also has its lower parts: r8d-r15d (lower 32-bit parts), r8w-r15w (lower 16-bit
parts), r8b-r15b (lower 8-bit parts).
SIMD-registers number are doubled: from 8 to 16: XMM0-XMM15.
∙ In Win64, function calling convention is slightly different, somewhat resembling fastcall 2.5.3. First 4
arguments stored in RCX, RDX, R8, R9 registers, others — in stack. Caller function should also allocate
32 bytes so the callee may save there 4 first arguments and use these registers for own needs. Short
functions may use arguments just from registers, but larger may save their values into stack.
See also section about calling conventions 2.5.
∙ C int type is still 32-bit for compatibility.
∙ All pointers are 64-bit now.
Since now registers number are doubled, compilers has more space now for maneuvering calling register
allocation 79 . What it meanings for us, emitted code will contain less local variables.
For example, function calculating first S-box of DES encryption algorithm, it processing 32/64/128/256
values at once (depending on DES_type type (uint32, uint64, SSE2 or AVX)) using bitslice DES method
(read more about this method here 1.19):
/*
* Generated S-box files.
*
* This software may be modified, redistributed, and used for any purpose,
* so long as its origin is acknowledged.
*
* Produced by Matthew Kwan - March 1998
*/
#ifdef _WIN64
#define DES_type unsigned __int64
#else
#define DES_type unsigned int
#endif
void
s1 (
DES_type a1,
DES_type a2,
DES_type a3,
DES_type a4,
DES_type a5,
DES_type a6,
79 assigning variables to registers
103