On the PostgreSQL extension ABI issue in the latest patch release (17.1, 16.5, ...). PostgreSQL extension C code includes header files from PostgreSQL itself. When the extension is compiled, functions from the headers are represented as abstract symbols in the binary. The

4 min read Original article ↗

On the PostgreSQL extension ABI issue in the latest patch release (17.1, 16.5, ...). PostgreSQL extension C code includes header files from PostgreSQL itself. When the extension is compiled, functions from the headers are represented as abstract symbols in the binary. The symbols are linked to the actual implementations of the functions when the extension is loaded based on the function names. That way, an extension compiled against PostgreSQL 17.0 can usually still be loaded into PostgreSQL 17.1, as long as the function names and signatures from headers do not change (i.e. the application binary interface or "ABI" is stable). The header files also declare structs that are passed to functions (as pointers). Strictly speaking, the struct definitions are also part of the ABI, but there is more subtlety around that. After compilation, structs are mostly defined by their size and offsets of fields, so for instance a name change does not affect ABI (though does affect API). A size change does affect ABI, a little. Most of the time, PostgreSQL allocates structs on the heap using a macro that looks the compile-time size of the struct ("makeNode") and initializes the bytes to 0. The discrepancy that arose in 17.1 is that a new boolean was added to the ResultRelInfo struct, which increased its size. What happens next depends on who calls makeNode. If it's PostgreSQL 17.1 code, then it uses the new size. If it's an extension compiled against 17.0, then it uses the old size. When it calls a PostgreSQL function with a pointer to a block allocated using the old size, the PostgreSQL function still assumes the new size and may write past the allocated block. That is in general quite problematic. It could lead to bytes being written into an unrelated section of memory, or the program crashing. When running tests, PostgreSQL has internal checks (asserts) to detect that situation and throw warnings. However, PostgreSQL uses its own allocator which always rounds up the number of bytes in its allocations to a power of 2. The ResultRelInfo struct was 376 bytes (on my laptop) so it would round up to 512, and this is still the case after the change (384 bytes on my laptop). So, in general this particular change in the struct does not actually affect the allocation size. There may be uninitialized bytes, but that is usually resolved by calling InitResultRelInfo. The issue primarily causes warnings in tests / assert-enabled builds for extensions that allocate ResultRelInfo, though only when running those tests using the new PostgreSQL version with an extension binary that was compiled against the old PostgreSQL versions. Unfortunately, that's not the end of the story. TimescaleDB is a heavy user of ResultRelInfo and does some things that do suffer from the size change. For instance, in one of its code paths, it needs to find the index of a ResultRelInfo pointer in an array, and to do so it does pointer math. This array was allocated by PostgreSQL (384 bytes), but the timescale binary assumes 376 bytes and the result is a nonsense number which then hits an assert failure or segmentation fault. github.com/timescale/time… The code here is not really at fault, but the contract with PostgreSQL was not quite as assumed. That's an interesting lesson for all of us. It's quite possible that there are other issues like this in other extensions, though there not many extensions as advanced as Timescale. Another advanced extension is Citus, but I've done some validation and Citus looks safe to me. It does exhibit the assert warnings. Some caution is advised. The safest thing to do is to make sure extensions are compiled using the headers from the PostgreSQL version that you are running.