Developer knowledge network ยท moderated exchange

UnreliableCode Community

Developer Research, Reverse Engineering & Coding Community

Knowledge indexLive
4Categories
919Threads
2.8KPosts
Analysis

x64 Calling Conventions: System V AMD64 ABI vs Microsoft x64 ABI Internals [StackOverflow Architecture Guide]

assembly_micro_dev
x86_64 Micro-arch
MEMBER
Rep: 186
Join Date: Oct 2021
Posts: 10
Thanks: 27
1 months ago ยท Jul 5, 2026 12:14 AM
#1

Why mixing Windows and Linux object files causes immediate stack corruption:

  1. System V AMD64 ABI (Linux / macOS):
    • First 6 integer/pointer arguments passed in: RDI, RSI, RDX, RCX, R8, R9.
    • Return value in RAX (and RDX for 128-bit).
    • 128-byte Red Zone below RSP that signal handlers won't clobber!
  2. Microsoft x64 ABI (Windows):
    • First 4 arguments passed in: RCX, RDX, R8, R9.
    • Mandatory 32-byte Shadow Space (Home Space) allocated by caller on stack above return address!
    • No Red Zone allowed.
llvm_compiler_dev
LLVM & Clang Hacker
MEMBER
Rep: 139
Join Date: Jul 2018
Posts: 21
Thanks: 15
1 months ago ยท Jul 5, 2026 2:38 AM
#2

Forgetting the 32-byte shadow space in Windows x64 inline assembly is the #1 cause of stack alignment crashes.

ptr_arithmetic
C++ Wizard
MEMBER
Rep: 162
Join Date: May 2018
Posts: 73
Thanks: 42
1 months ago ยท Jul 5, 2026 9:49 AM
#3

System V's 6 register parameters and Red Zone provide slightly faster function call overhead.