extending Go forward compatibility · golang go · Discussion #55092

This discussion is about forward compatibility, meaning old versions of Go compiling newer Go code. For the problem of new versions of Go compiling older Go code, see this other discussion about backward compatibility.

It seems strange to me that you can edit go.mod to update all the code your program depends on except Go itself. Suppose go.mod says go 1.17. Why shouldn't changing that to go 1.18 make the build use Go 1.18? I think probably it should. Here is how that might work.

To start, let's get the machinery of an old go version downloading and running another out of the way. Assume that we publish all binary Go distributions as module zip files (in addition to the usual zip files and other installers). Then if the go command notices it needs a different distribution, it can download it from the module proxy and authenticate it using the checksum database. (We don't need to invent a separate authenticated download channel, nor a separate caching mechanism.) After unpacking the distribution somewhere appropriate (perhaps the module cache) and restoring the execute bits on the binaries, the old go command can reinvoke the newly downloaded go command. On a future invocation, the old go version finds the cached copy directly and runs it, no download required.

Having gotten the “is it possible?” out of the way, let's turn to what should cause that to happen. I am thinking about something along these lines:

By default, if the go line in go.mod in the current module or workspace lists a newer version of Go than the current one being run, the current one goes and gets the new one and reinvokes it. Supposing the change happens in Go 1.90, if you are running Go 1.90 and change your go.mod line to go 1.92, any go command like go build or go test will download and reinvoke Go 1.92.

On the other hand, if you change the line to go 1.88, Go 1.90 will keep running and will do its best to emulate Go 1.88. (See also this discussion about backward compatibility.)
If you want more fine-grained control, a new go.mod line toolchain, which is only respected in the current module's go.mod (just like replace) sets a specific toolchain to use. For example you could do
```
go 1.91
toolchain go1.92.4
```
(with Go 1.90 still installed) to stay on Go 1.91 semantics but update to the Go 1.92.4 toolchain. Similarly, you can use toolchain to “downgrade”, as in:

(with Go 1.90 still installed).

The toolchain line overrides the version selection logic in step 1.

To ensure accurate compilation of code written for newer Go versions, the go command would update the toolchain line when adding a dependency with a newer version of Go to maintain the invariant that the toolchain (or the implied toolchain from the go line) is at least as new as the go version required by any dependent module. Continuing the examples above, if you go get a new or updated dependency that has a go.mod that says go 1.100, then the current module’s go.mod toolchain line would be updated to toolchain go1.100, with a message to standard error about the update.
For command-line control, setting GOTOOLCHAIN overrides any go.mod toolchain line, like:
```
GOTOOLCHAIN=go1.92.3 go test
```
Using GOTOOLCHAIN=local would use the local toolchain, whatever its version (Go 1.90 in this running example).

And GOTOOLCHAIN=auto would be an explicit name for what happens when there's no toolchain line, so that you could have the go.mod above and run

to get Go 1.91.

No matter how the effective toolchain is specified, it would become a build error to attempt to use an old toolchain with new code. (Today the build proceeds and hopes for the best; in case of a compilation error, the go command prints a final note about the version mismatch as a possible explanation of the compilation error.)

This mechanism would make it a lot easier to try out new Go releases, as well as betas and release candidates: just edit your go.mod files and commit them, and everything building your code knows to use the new release. This is analogous to when you edit your go.mod to update a dependency version and everything building your code knows to use that newer version. (In contrast, what happens today with Go toolchain selection is very much like the old days of GOPATH, when each different build machine injects its own local state into the build.)

This mechanism would also help a lot with Go packagers that don't always keep up with Go releases, such as some Linux distributions and cloud providers that I wont name.

If we ever moved toward issuing more frequent ephemeral Go releases (like monthly alpha releases when the tree is open), this mechanism would make those easier to consume.

Finally, I started a separate discussion about a Go toolchain being able to emulate an older one’s runtime semantics based on the go.mod go line. And Go toolchains already emulate an older one’s language semantics based on each package’s go.mod’s go line. The mechanism discussed above is a nice complement to those, allowing a Go toolchain to emulate a newer one based on the go.mod go line.

This is a discussion, not a proposal. I haven't implemented all of this nor even worked out all the implications. I'm curious what people think and what concerns they have. Thanks!