The Function Signature Is a Lie

ABI series | Part 1 of 9 | May 29, 2026

Modern ABIs, hidden arguments, and the binary protocols beneath ordinary-looking function calls.

The interface between two programs consists of the set of assumptions that each programmer needs to make about the other program…

A function signature is one of those assumptions. It is not all of them.

You write this:

struct Big make_big(long long seed);

You call it like this:

struct Big b = make_big(100);

One source argument. One return value. A perfectly ordinary function signature.

Now cross the ABI boundary.

On common 64-bit targets, if the return type is not ABI-register-returnable, the caller allocates storage for the result and passes a pointer to that storage into the callee. Size is the easiest way to trigger that path, which is why this example uses a deliberately large struct. But the real rule is ABI classification: layout and C++ object semantics can force an indirect return even when a casual size-only rule would mislead you. That pointer is not in your source signature. It is not the seed. It is not a local variable you named. But at the ABI level, it may be the first thing the callee receives.

For this deliberately large example:

struct Big {
    long long data[8];
};

struct Big make_big(long long seed);

Across common 64-bit targets, that hidden field lands differently:

• Windows x64: result pointer in RCX, returned again in RAX; visible seed shifts to RDX.
• System V AMD64: result pointer in RDI, returned again in RAX; visible seed shifts to RSI.
• AAPCS64: indirect result location in x8; visible seed stays in x0.

The first source argument may not be the first ABI argument.^{1 2 3}

Open the hidden result-storage lab at the System V AMD64 trace. It highlights mov rax, rdi in missing_sret.sysv.s.txt, then explains what that line shows and where the evidence stops. The ABI standard remains the authority.

The small excerpts below keep the one-argument make_big shape from this section. The lab receipts use the two-argument make_sret_record(seed, tag) variant introduced later so that argument shifting is visible for more than one source parameter.

; Windows x64, abbreviated one-argument shape
make_big:
    movq    %rcx, %rax      ; return the hidden result pointer
    movq    %rdx, (%rcx)    ; seed was shifted to RDX

; System V AMD64, abbreviated one-argument shape
make_big:
    mov     rax, rdi        ; return the hidden result pointer
    mov     qword ptr [rdi], rsi

; AArch64 Linux, abbreviated one-argument shape
make_big:
    str     x0, [x8]        ; seed in x0, result storage in x8

If a JIT stub, hand-written trampoline, or FFI bridge forgets the hidden result channel, the failure depends on the target ABI. On Windows x64 and System V AMD64, omitting the hidden result channel can put the first visible value where the callee expects a writable result address. On AAPCS64, the visible arguments can still be in x0 and x1; the danger is that x8 is stale, unset, or otherwise not a valid result-storage address. The source call looked like ordinary visible arguments. The machine-level call also needed an address.

That is the lie.

Not that the function signature is wrong. The source signature is faithful as a source-level API. It tells the programmer what type is returned, what type is accepted, and what expression is legal to write.

The lie is that the signature is complete.

A source-level function signature is a programmer-facing projection. The actual cross-boundary call is a target-specific protocol involving data layout, argument classification, hidden parameters, register assignment, stack obligations, saved state, and unwind metadata.⁴

The function call you write is the public API. The ABI call is the private wire format.

“Private” does not mean secret. Many ABIs are public documents, and if you write compilers, JITs, foreign-function interfaces, profilers, hooks, loaders, crash dump tools, or hand-written assembly, those documents are your operating manual. Private means beneath the source API. It is the protocol the caller and callee speak after the pretty syntax has been compiled away.

The wire is usually registers, stack, and metadata, not a byte stream.

The source call is not the whole call

A source signature is written for humans and type checkers. It says:

struct Big make_big(long long seed);

The ABI has a different job. It must answer operational questions:

Where does the return value go? Which register carries the first integer-like value? Which stack slots must exist before the call? Which registers can the callee destroy? Which registers must survive? Is an aggregate returned in registers, passed by reference, or classified into pieces? If the stack walker arrives later, how does it recover the caller’s frame?

Those questions are not decorations around the signature. They are the binary contract.

The compiler can hide that contract when both sides are under its control. It can inline the function. It can eliminate the call. It can use a private internal convention for a local helper. The System V AMD64 psABI even says its standard calling-sequence requirements apply to global functions, while local functions unreachable from other compilation units may use different conventions.⁵

But once a call crosses a boundary that independently compiled code must agree on, the contract stops being optional. A dynamic library export, a callback, a plugin interface, a JIT stub, a language FFI declaration, a virtual dispatch edge, a hand-written trampoline: these are not just jumps. They are protocol handoffs.

The rest of this series is about what happens when we forget that.

Exhibit 1: the return value that arrives as an argument

The opening article example uses a small return as the control case and a deliberately large return as the interesting case:

struct Pair make_pair(int x, int y);
struct Big make_big(long long seed);

The small Pair gives you the control case. The deliberately large Big forces the interesting path: caller-provided result storage. Small values often return in registers; large or otherwise indirectly returned aggregates can change the shape of the call. On Windows x64 and System V AMD64, the hidden result pointer consumes the first integer-argument register and shifts visible arguments. On AAPCS64, the indirect result location uses x8, so the visible seed stays in x0.^{1 2 3}

That distinction is why the hosted ABI lab exists. The lab uses a two-argument variant, make_sret_record(seed, tag), so the argument shift is visible for more than one source parameter. Start with the hidden result-storage trace, then inspect the generated missing_sret.*.s.txt receipt for the target you care about.

For the lab variant, the System V AMD64 receipt shows the hidden result pointer in RDI, with seed and tag shifted to RSI and RDX:

make_sret_record:
    mov     rax, rdi
    mov     qword ptr [rdi], rsi
    mov     qword ptr [rdi + 8], rdx

The exercise is not to memorize one compiler listing. It is to build the habit: source shape, ABI packet, artifact, boundary. The ABI document is the authority, but the compiler listing is the microscope. Use both. The source types may be beautiful while the binary protocol is still wrong.

Now write a call that looks even more familiar:

counter.add(5);

At the source level, this is a method call. It reads like the receiver is outside the argument list and 5 is the argument.

At the ABI level, the callee still needs the object.

A non-static member function has a hidden object parameter. Old 32-bit Microsoft C++ made this especially visible: under __thiscall, ordinary arguments are pushed on the stack and the this pointer is passed in ECX.⁶ Modern 64-bit conventions are more register-heavy, but the conceptual point survives: the object pointer is part of the call protocol even when the source call expression does not spell it as an argument.

A first approximation is:

counter.add(5);

// Conceptual ABI intuition, not valid C++ syntax:
Counter_add(&counter, 5);

That approximation is useful. It is also incomplete.

C++ object calls can involve more than “pass this as the first pointer.” With inheritance and virtual dispatch, the pointer passed to the final code may be a pointer to a base subobject, not the address you would casually call “the object.” The Itanium C++ ABI describes virtual function entry points that may expect an adjusted this pointer, and it describes adjusting entry points that convert one subobject pointer to the one expected by the non-adjusting implementation before transferring control.⁷

That is the thunk teaser. The source call still says counter.add(5), but the ABI path may choose a subobject view, load a table entry, adjust the hidden object pointer, and then enter the implementation.

You do not need the whole C++ ABI to understand the lesson. You only need to lose the source illusion that the visible argument list is the call. The object is data. The subobject adjustment is data. The call is a protocol over all of it.

Composition is where this stops being trivia. A member function that returns a large aggregate can carry both a hidden object pointer and a hidden result-storage channel, and the ordering is ABI-specific. A bridge that remembers one hidden field but forgets the other is still building the wrong packet.

That is the appeal of a plain C-shaped plugin boundary. The boring boundary is a feature. A table of function pointers taking void *ctx is not elegant, but it makes the hidden object explicit. It turns some private ABI machinery back into a public contract the plugin author can reason about. It also avoids exporting a vendor-specific C++ object model as the plugin ABI: name mangling, vtables, thunks, RTTI, exception machinery, and member-function pointer representations are not one universal contract.

Exhibit 3: varargs and the missing type list

Now consider the most famous function signature in C:

int printf(const char *fmt, ...);

A fixed-arity function gives the compiler a typed list of parameters. A variadic function intentionally stops doing that. The callee receives the fixed part of the signature plus whatever the caller supplied after it. The rest of the type information lives in conventions: default promotions, va_start, va_arg, and, for printf, a runtime format string.

That missing type list matters more on register-heavy ABIs than it did in the old mental model where everything was simply pushed on the stack.

On Windows x64, vararg and unprototyped calls require floating-point values to be duplicated into the corresponding general-purpose register as well as the floating-point register, for the relevant argument positions.⁸ On System V AMD64, calls that may reach variadic functions use AL as a hidden upper-bound marker for vector-register usage, and the va_list machinery includes offsets into a register save area and an overflow area for stack-passed arguments.⁹ That AL byte is not a type tag and not necessarily an exact count; it is a bounded upper count of vector registers used by the call.

So this source line:

printf("x=%f\n", x);

is not just “put a string somewhere, put a double somewhere, call printf.” The ABI has to make the unnamed argument recoverable by a callee whose declared parameter list does not name it.

This is why variadic FFI is treacherous. A wrapper that can call ordinary fixed signatures may still be wrong for printf. It may put the double in the right floating-point register for a fixed call but fail to duplicate it for a Windows x64 variadic call. It may forget the System V AMD64 AL state. It may treat stack overflow arguments as if the first four or six registers did not exist.

A bridge that works for printf("%d", i) can still be wrong for printf("%f", x).

Again, the source can look plausible while the protocol is wrong. The current Bug Zoo listings show the difference in miniature:

; System V AMD64, from varargs_probe.sysv.s
mov     al, 1

; Windows x64, from varargs_probe.win64.s
movdqa  %xmm0, %xmm1
movq    %xmm0, %rdx

Do not overread those two lines. They are deliberately small signposts. The useful habit is to ask what extra call-state the variadic ABI requires before treating a fixed-signature FFI path as general.

The Bug Zoo’s varargs_probe.c keeps this example intentionally small. One wrapper calls a variadic integer sum. Another calls printf with a double. Treat those listings as companion evidence, not law. The law is in the ABI.

Calling convention is not the whole ABI

At this point it is tempting to say the post is about calling conventions. That is too small.

Calling convention usually means the register-and-stack rules for a call: which arguments go in RCX or RDI, how the stack is aligned, who cleans it up, which registers are volatile, where return values appear.

An ABI is broader.

It includes type size and alignment. It includes aggregate classification. It includes which registers are preserved across calls. It includes stack layout and red zones or shadow space. It includes object-code conventions such as name mangling, vtables, thunks, and exception interfaces. It includes unwind metadata, because returning correctly is not enough if the debugger, profiler, exception runtime, or crash dumper cannot walk through the frame.⁴

Windows x64 makes this last point concrete. Nonleaf functions are described by unwind data, stored through pdata and xdata, so an exception handler or debugger can recover nonvolatile registers and caller state. Microsoft’s x64 exception-handling documentation says dynamically generated functions must provide function-table information to the operating system, and failure to do so makes exception handling and debugging unreliable.¹⁰

That is an ABI fact, even though it is not visible in the source signature.

A function that returns the right integer but breaks unwinding is still a bad citizen at the boundary. A trampoline that preserves the visible arguments but clobbers a nonvolatile register is still wrong. A JIT stub that works in a microbenchmark but lacks unwind metadata may fail only when a profiler, exception, sanitizer, or crash dump enters the story.

ABI bugs are often delayed-action bugs. You violate the protocol at the call. The visible failure appears somewhere else.

The boundary is where the truth matters

Inside a single compiler’s optimization bubble, the source function may never become an ABI call at all. The compiler can inline it, specialize it, vectorize it, split it, tail-call it, or erase it. No programmer should look at every source-level call and imagine a literal platform ABI ceremony happening at runtime.

The ceremony matters where independent pieces of code must meet.

That includes ordinary native boundaries: object files, shared libraries, static libraries, callbacks, system APIs. It also includes modern boundaries that do not look like old assembly work: Rust calling C, Python loading native extensions, Java entering JNI, WebAssembly components lowering values across language boundaries, eBPF helper calls, JIT compilers registering generated code, mixed-architecture processes, instrumentation hooks, profilers, and security monitors.

The common problem is not nostalgia for cdecl and stdcall. Those are still useful words, especially when debugging 32-bit Windows or reading old code. But the modern lesson is larger:

A call boundary serializes source-level intent into a machine-level protocol.

The protocol might be mostly registers. It might include stack slots the callee assumes exist. It might include hidden storage for a return value. It might include a hidden object pointer. It might include a hidden vector-register-use marker. It might include frame records and unwind tables. It might include platform-reserved registers that portable assembly must not treat as scratch. Future posts will cover security enforcement at call boundaries.

The signature is the part you meant.

The ABI is the part the other side can actually receive.

What to remember

The safest way to think about a function call is as two different artifacts that happen to share a name.

The source-level call is the public API:

struct Big b = make_big(100);

The ABI-level call is the private wire format:

allocate result storage
place hidden result pointer where this ABI expects it
place visible arguments after whatever hidden state the ABI inserted
satisfy stack, register, and unwind obligations
transfer control
recover the result from the ABI-defined channel

Most programmers do not need to recite the register table every day. But experienced programmers do need the abstraction boundary in their heads. Without it, ABI behavior feels like trivia: RCX here, RDI there, x8 somewhere else, shadow space on one platform, red zone on another.

With the boundary in your head, the trivia becomes a protocol.

The next post will use the same source-level function and watch it become three different binary realities. After that, the series turns adversarial: what breaks when you omit the hidden result pointer, use the wrong register order, mishandle varargs, borrow a platform-reserved register, or write code that returns correctly but cannot be unwound.

The function signature is not useless. It is not false. It is the view from one layer up.

The lie is believing that layer is the call.

• labHidden result-storage lab