Why is RCX not used for passing parameters to system calls, being replaced with R10? [duplicate]

X86-64 system calls use syscall instruction. This instruction saves return address to rcx, and after that it loads rip from IA32_LSTAR MSR. I.e. rcx is immediately destroyed by syscall. This is the reason why rcx had to be replaced for system call ABI.

This same syscall instruction also saves rflags into r11, and then masks rflags using IA32_FMASK MSR. This is why r11 isn’t saved by the kernel.

So, these changes reflect how the syscall mechanism works. This is why the kernel is forced to declare rcx and r11 as not saved and even can’t use them for parameter passing.

Reference: Intel’s Instruction Set Reference, look for SYSCALL.

Leave a Comment