proposal: cmd/cgo: cgo without a C toolchain

matloob · GitHub ·

13 min read Original article ↗

Background

Cgo provides a mechanism to incorporate C code into Go programs. Currently, it requires a C toolchain to be present on the host system whenever building code with import "C". While this is natural when interacting with C, it creates significant friction for Go developers and users:

  • End-user distribution: It requires the end user to have a C toolchain installed in order to build or use the packages.
  • Cross-compilation: Cross-compiling pure Go programs is seamless, but cross-compiling cgo packages requires setting up C cross-compilers, as well as the build environment (sysroots, header files, etc.).

In many common use cases, a Go package does not contain C function definitions; rather, it only interfaces with an existing pre-compiled shared library (.so) or static library (.a) using C declarations. For these packages, there is an opportunity to build them without using a C toolchain.

Even in these cases, currently the build still requires a C toolchain. One main reason is that during the build, the cgo command will process the C preamble and generate glue code, some of which is in C and needs to be compiled with a C compiler. This proposal will tackle this issue.

Proposal

We propose a mechanism in the Go toolchain to build cgo packages that interface with pre-built C libraries without requiring a C toolchain to be installed.

Goals

  • No C toolchain at build time: For a cgo program that only references pre-compiled C libraries (without C source definitions), go build can proceed without requiring a C compiler installed.
  • Type safety: Preserve Go type safety at the C boundary. Calls to C functions and access to C types, structs, and variables are statically type-checked by the Go compiler.
  • Compatibility: Retain compatibility with standard cgo (import "C", C.symbol_name syntax) so that Go source code does not need to be rewritten.
  • Gradual & per-platform adoption: Allow package authors to migrate to the new mechanism for specific target platforms while continuing to use standard cgo on other platforms.

Design overview

Currently, when building a cgo package, the C toolchain is used in two places: processing the C preamble, and compiling the generated C glue code. We can separate them into two steps: the first step is to process the C preamble, and understand the C types, variables, and function signatures referenced from Go. The second step is to generate and build the code gluing the two ABIs.

For the first step, what if we split it out of the build process, and instead have some way to describe the C interface? The main idea is to use go syntax to describe the interface. The binding file can be written by hand, or conveniently generated (see below), and stored along with the source code.

Here we make a distinction between the package author and the end user: the end user should be able to just go get and build the package without using a C toolchain, whereas the package author could use, although not strictly required, a little help from a C toolchain to generate the bindings. We are targeting stable C libraries where the expectation is that the representations of the structs and interfaces of the functions do not change over time. Once the bindings are written, the rest of the build does not require a C toolchain.

For the second step, currently this requires a C toolchain as some of the glue code is in C. With a better handling of the C ABI, we can avoid C code, and instead, use Go and Go assembly code for the glue.

In short,

  • In the development phase: the package author writes or generates the bindings, optionally using a C toolchain.
  • At build time: the cgo command processes files with bindings, and generates the glue code in Go and assembly, which is then built with the Go toolchain, without a C compiler.

Bindings

We do not want the Go toolchain itself to understand C declarations and header files directly. C declarations can be arbitrarily complex—involving nested header inclusions, #define macros, conditional compilation (#if, #ifdef), platform-specific type definitions, etc. Building a full C preprocessor and C parser into the Go toolchain would be overly complex and difficult to maintain.

Instead, we use Go syntax to represent the bindings, with annotations to clarify that they are C bindings. This captures the necessary type definitions, global variables, and function signatures, acting as an explicit, high-level specification of the C interface for the Go toolchain.

The Go files for the bindings satisfies the following requirements:

  • Stable and extensible. The file format should be stable, as we expect package developers to check in the binding files. Therefore the format should be well-documented and maintained. It would also be good to be extensible to accommodate future additions.
  • Human-readable, editable, and writable. While we will provide a mechanism to generate the binding file (see below), the files can be edited by a human, e.g. to extend to other platforms where they know the binding is the same. Because they are go files that programmers are used to working with they will also be easy to read and write.
  • Easy to produce and parse. The file will be produced by cmd/cgo, and read by the compiler and we can reuse the support for reading and writing go files.
  • Multiple platform support. Supporting different binding definitions on different platforms should be straightforward, as the binding may vary from platform to platform (consider C #if and #define).

The go file describes the C types, variables, and function declarations (but no definitions (i.e. function bodies)). The C symbols are distinguished from Go symbols by adding annotations. We have considered using a standard serialization format like JSON, which is easy to generate and parse, but the human readability is not as good, and the writeability is worse. We also have considered using a special go-like binding file, but using Go itself means there are fewer parts to keep track of.

The //cgo:binding annotation takes as an argument the C symbol being bound to, prefixed with "C.". It will introduce the binding C.<Symbol> as well and binding to the go declaration that is annotated.

An example of a hand-written binding file:

//go:build darwin

package mypkg

import "C"

// type definition
//cgo:binding C.point
type Point struct {
	x, y float32
}

// global variable
//cgo:binding C.global
var Global Point

// function declaration
//cgo:binding C.compute
func Compute(p0 Point) float32 // note that Point can also be used as C.point.

Binding generation

As mentioned above, the bindings could be written by hand or generated. We will provide a convenient tool to do the generation: when a C toolchain is present, one can run go tool cgo -gen-binding to generate the binding file.

In this mode, the cgo command invokes the C compiler to compile the C preamble, and extracts type layouts and symbols from the DWARF information. This is similar to what the cgo command does in regular cgo, but instead of emitting a C wrapper, it emits the bindings in a generated Go file.

Note that using the cgo command and the C toolchain to generate the binding file is not required. One could choose to write the binding file by hand, which would avoid using a C toolchain at all. One could also modify the generated binding file to, e.g. extend to more platforms, without using a C cross compiler.

The bindings generated by cgo -gen-binding will all be named with underscores, so that they don't create a binding that the user didn't intend, so by default they will have to be used by their same C. name that would have been generated by cgo unless a user adds a binding.

So from the following traditional Cgo file,

package mypkg

/*
struct point {
	float x;
	float y;
};

point global;

float compute(point p0);
*/
import "C"

we'd generate the following:

//go:build darwin

package mypkg

import "C"

// type definition
//cgo:binding C.struct_point
type _point struct {
	x, y float32
}

// global variable
//cgo:binding C_global
var _global _point

// function declaration
//cgo:binding C.compute
func _compute(p0 _point) float32 // note that _point can also be used as C.Point.

C ABI handling

At go build time, cmd/cgo will process the binding file, and compute the C calling convention for each declared function according to the platform C ABI. It will then generate Go and assembly trampolines that pass arguments and results between the Go ABI and the C ABI, with runtime calls for stack switching and bookkeeping. As the generated glue code is Go and Go assembly, it can be compiled with the Go toolchain.

The cgo command will need to understand the C calling convention for all supported platforms, which introduces non-trivial complexity. But it is still manageable and maintainable (as the C ABI does not change over time).

The runtime/cgo package

Currently, the runtime/cgo package contains C code, which still requires a C toolchain to compile. In the new mechanism, the runtime/cgo package will be rewritten to use Go and assembly. So the user can build the whole program without using a C toolchain.

There are limitations: the C compiler flags provided in CGO_CFLAGS would have no effect on the runtime/cgo package, as there is no C code to be compiled. As a consequence, it may not interact well with C sanitizers.

Dynamic loading

In some use cases, a C shared library is loaded dynamically at runtime. In the C world, this is done with dlopen and dlsym. However, although the library itself is loaded dynamically, the function names and signatures are often known statically at development time.

The binding mechanism will also support dynamically loaded functions, by generating type-safe Go wrappers and trampolines for dynamically resolved functions. This would be an opportunity to provide a type-safe API for calling C functions in a dynamically loaded library, rather than using e.g. reflect calls.

We are still working on the details of how this will work and will share more in a separate document.

Alternatives

Separate binding files

This is the primary alternative being considered and we'd actively like feedback on whether we should go in this direction. In this alternative, instead of adding annotations to Go declarations, there would be interface definition files that exist separately from go files. An example would be the following:

//go:build darwin

package mypkg

// type definition
type C.Point struct {
	x, y float32
}

// global variable
var C.Global C.Point

// function declaration
func C.Compute(p0 C.Point) float32

This would only generate bindings in the "C" namespace separate from those Go identifiers are in. The cgo generator would produce these files when run in gen-binding mode, but they could also be hand-written.

One advantage of going this route is that we won't add a new way of declaring Cgo identifiers and we won't have to deal with each cgo entity having two identifiers: one in the "C" namespace and one in the package's namespace. gen-binding mode gen-binding mode will continue to work closer to how it currently does.

Directly generate (or hand-write) ABI assembly trampolines

One possibility is that the package author could write (or use a tool to generate) low-level assembly stubs that call the C functions. This is similar to the syscall stubs used in the runtime or the syscall package.

The drawbacks of this approach are:

  • Unsafe, and relies on internal runtime and toolchain details.
  • Highly platform-specific, requiring separate assembly files for every target architecture.
  • Brittle across Go releases if internal runtime calling conventions change.
  • Significantly worse developer experience compared to standard cgo syntax.

Platform specific binding description

The binding file can be very platform specific, or relatively generic. The proposal above chooses a relatively generic approach. A binding description can work across platforms as long as the underlying C definitions are the same, e.g. not defined differently on different platforms with #if, etc.

Another possibility would be to make the binding file very platform specific, where it includes all platform specific details, e.g. which registers are used for which arguments, etc. This description could be generated by inspecting the DWARF information of a sample C compilation (similar to what cmd/cgo does in regular cgo). It could also be written by hand, if the author is familiar with the platform ABI.

With this approach, the Go toolchain would not need to understand the C ABI at all. It is all described in the binding file. However, to write or generate the binding, it needs either a C toolchain for each platform, or human expertise for the platform ABI. Instead of putting this burden on the package authors, it makes sense for the Go toolchain to be the central place to handle the C ABI.

Handle the C ABI in the Go compiler

Instead of handling the C calling convention in the cgo command, we could handle it directly in the Go compiler. This would eliminate the need for separate assembly trampolines. While the compiler could emit calls to C functions with open-coded argument passing, there is little performance benefit, as calling a C function still requires runtime calls for stack switching and the runtime's own bookkeeping.

Handling the C ABI in the cgo command isolates this platform-specific complexity from the compiler.

Open questions

When to opt in?

At the package level, the presence of any bindings (//cgo:binding) and the absence of .c source files allows the go command to automatically enable the new mechanism for that package. This choice can be made on a per-package basis: some packages can use the new mechanism, while other packages use regular cgo. These packages can be linked together in the same build. However, a C toolchain would still be required to build the whole program if any package uses traditional cgo.

What about the runtime/cgo package? We should make our best effort to use the new mechanism uniformly. Ideally we can do that but we'll need to determine if there are projects that depend on it to see if there needs to be a fallback.

  • One possibility, if we need to have a fallback, is that if explicit C compilation flags are specified (e.g. non-empty CGO_CFLAGS or in sanitizer mode), the standard C-based runtime/cgo will be used, and if no C flag is specified, we can use the runtime/cgo implementation with Go and assembly. The difference shouldn't matter?
  • Another possibility for users that depend on runtime/cgo being built with CGO_CFLAGS, is that we could add a cmd/go flag that would cause the compiler to build runtime/cgo with traditional cgo.
  • We should avoid dynamically selecting whether we use traditional cgo for runtime/cgo by checking if any of the packages use traditional cgo: in that case, if any package used traditional cgo, runtime/cgo, one of that package's dependencies would need to be built differently. That breaks a fundamental assumption of the go command and its caching system, which is that in a build, packages only depend on their dependencies.

How to ensure the binding file is consistent with the C code?

Because binding files can be checked in or edited by hand, they could potentially drift from the underlying C API. We'd want to keep them in sync.

One possible idea is that when a C toolchain is present, go test will verify that the checked-in bindings match the C signatures, e.g. by letting the cgo command regenerate the bindings and check their consistency.