Data races and the memory model in Go

We write a value in one goroutine and read it in another. Nothing crashes, and the value is there. It looks like our code works.

We actually got lucky. Go does not guarantee that one goroutine will see a write made by another unless the program explicitly coordinates their operations. This post explains what happens on a real machine, why it happens, and what we should use instead.

1. The program that works

Suppose we want to pass a value from one goroutine to another without using a channel. A simple approach is to store the value in one variable and use a boolean to report when the write is complete. The reader waits for that boolean before accessing the value:

func main() {
	var done bool
	var msg string

	go func() {
		msg = "hello"
		done = true
	}()

	for !done {
	}
	fmt.Println(msg)
}

You can run this example in the Go Playground. The Playground prints hello and exits, so this execution produces exactly the result we expected.

But when we run this snippet with go run -race, the race detector reports one race involving done and another involving msg:

==================
WARNING: DATA RACE
Write at 0x00c0000121cf by goroutine 7:
  main.main.func1()
      main.go:11 +0x68

Previous read at 0x00c0000121cf by main goroutine:
  main.main()
      main.go:14 +0x110
==================
==================
WARNING: DATA RACE
Read at 0x00c000014040 by main goroutine:
  main.main()
      main.go:16 +0x128

Previous write at 0x00c000014040 by goroutine 7:
  main.main.func1()
      main.go:10 +0x30
==================
hello
Found 2 data race(s)
exit status 66

So is this snippet safe and valid because we use a for loop to check the done flag? Let’s consult the Go memory model.

2. The Go memory model

The race detector does not care about the output. It checks whether 2 goroutines access the same memory concurrently without synchronization and at least 1 access is a write.

The Go memory model answers the next question: “Which write must each read use?” It tells us which behaviors Go guarantees across all runs. I know this is not obvious, so let’s diagnose the 2 reported races.

Race 1: main may not read true

Let’s put the snippet here so we don’t lose context:

// goroutine A
go func() {
	msg = "hello"
	done = true
}()

for !done {
}
fmt.Println(msg)

The first warning is for done:

WARNING: DATA RACE
Write at 0x00c0000121cf by goroutine 7:
  main.main.func1()
      main.go:11 +0x68

Previous read at 0x00c0000121cf by main goroutine:
  main.main()
      main.go:14 +0x110

done = true is a non-atomic write, and every evaluation of !done contains a non-atomic read of the same variable. The program does not require the write to happen before any of those reads.

Since 2 goroutines access the same variable and one access is a write, done has a read-write data race.

The Go memory model does not guarantee that a write in 1 goroutine becomes visible to another goroutine by itself. This snippet does not synchronize the write to done with the reads of done in main, so the loop may continue reading false. That may sound strange because the order looks clear in the Go source code.

In theory, the program may behave as if the generated code reused the value from its first read:

Of course, the code on the right is only for explanation. The compiler does not generate that form for this example. The important point is that the Go source code and generated assembly do not need a one-to-one relationship.

There is no guarantee that a write made by the new goroutine will become visible to main, so the compiler may reuse a loaded value in a register or a temporary, or arrange instructions in another order, as long as the optimization stays within the Go memory model.

Another question is what happens if the writer goroutine updates done while the main goroutine is reading it. Can the main goroutine receive a partially written value?

The answer for this specific case is no.

• On arm64, a bool uses one byte,
• The writer stores that entire byte with one MOVB (move byte) instruction,
• main loads the entire byte with one MOVBU (move byte unsigned) instruction.

Since each instruction accesses the complete one-byte bool, the access is indivisible: main cannot receive half of its value.

But the same reasoning does not apply to a whole struct, array, or other value made from multiple parts.

Go may read or write a struct one field at a time, an array one element at a time, and a complex number one component at a time. A value larger than one machine word can combine parts from separate writes. Strings, slices, and interfaces commonly use multiword internal representations, so a race can create an inconsistent value and may corrupt memory.

For example, consider a 24-byte struct made from three uint64 fields:

type State struct {
	A uint64
	B uint64
	C uint64
}

var state State

go func() {
	state = State{A: 2, B: 2, C: 2}
}()

snapshot := state

The assignment is 1 statement in the source code, but Go may write the 3 fields separately and in any order.

One valid execution writes C first. The racing reader can then read the old values of A and B together with the new value of C:

The resulting snapshot is {A: 0, B: 0, C: 2}. On the 64-bit machine used for this example, each field contains a complete uint64 value, but the struct as a whole matches neither the initial {A: 0, B: 0, C: 0} nor the value {A: 2, B: 2, C: 2} assigned by the writer.

Race 2: done == true does not guarantee msg == "hello"

Assume that the loop reads true and exits, exactly as it does in the Playground. Race 2 asks a separate question: “does done == true also guarantee that fmt.Println reads "hello" from msg?”

Let’s put the relevant snippet here so we can follow Race 2 without scrolling back:

var done bool
var msg string

go func() {
	msg = "hello"
	done = true
}()

for !done {
}
fmt.Println(msg)

The second warning points to msg:

WARNING: DATA RACE
Read at 0x00c000014040 by main goroutine:
  main.main()
      main.go:16 +0x128

Previous write at 0x00c000014040 by goroutine 7:
  main.main.func1()
      main.go:10 +0x30

Since two goroutines access the same variable and one access is a write, msg has a second data race.

The Go source code gives us one order inside each goroutine. The new goroutine writes msg before it writes done. main reads done before it leaves the loop, then reads msg for fmt.Println:

If we read the code from top to bottom, it may seem obvious that msg must contain "hello" when main leaves the loop. The new goroutine writes msg before setting done to true, and main reads msg only after reading true from done.

But from Go’s point of view, the read of done answers only one question: which write supplied the value returned by this read? The value true came from done = true, but msg is a separate memory location with a separate read.

Under the Go memory model, nothing guarantees that when main leaves the loop, msg contains the value written by the other goroutine.

Go therefore allows this result:

read done    true
read msg     ""

The Playground prints "hello" in this example, so this run does not show us what can go wrong. Let’s use another snippet where the same missing cross-goroutine order produces a result that we can reproduce on real hardware.

Two goroutines start at the same time. Each one writes to its own variable, then reads the other one:

var x, y int
var r1, r2 int

go func() { // goroutine A
	x = 1
	r1 = y
}()

go func() { // goroutine B
	y = 1
	r2 = x
}()

Now let’s think about the possible results:

• If goroutine A finishes before goroutine B starts, then r2 is 1.
• If goroutine B finishes first, then r1 is 1.
• If they interleave, at least one goroutine sees the other’s write, so either r1 or r2 is 1, or both are.

But whatever order we imagine, it should be impossible for both r1 and r2 to be 0, because that would require each read to happen before the other goroutine’s write.

Running that experiment 200,000 times on an Apple M-series machine produced:

both goroutines read 0: 2 out of 200000 rounds (0.0010%)

The program produced this “impossible” result twice out of 200,000 rounds, which is 0.0010%. And nothing is wrong with the hardware. You can run the same experiment in the Playground. If it times out, reduce the number of iterations.

But this result needs 2 separate explanations.

First, Go guarantees the result required by the source inside one goroutine. See this snippet:

a must contain 1. The compiler may combine the 2 statements, replace them with other instructions, or arrange those instructions differently. But any optimization must still preserve the dependency from b to a and produce the correct result.

But in our case, the 2 statements in goroutine A are independent:

• r1 = y does not use x,
• x = 1 does not use y.

The same is true for goroutine B. The compiler does not have to preserve their textual order in the binary as long as the generated program still follows the Go memory model.

This kind of compiler reordering could explain the result described in Race 2. But it did not happen in the experiment above. The compiler kept the 2 memory instructions in source order in the generated arm64 binary.

That leaves a second explanation: how CPU cores make writes visible to each other.

Even when the machine instructions keep the source order, 1 core does not have to make its write available to the other core before its next read finishes. No memory barrier enforces that order in this binary.

Core A can read the old value of y while core B reads the old value of x, so r1 and r2 can both be 0.

This is also why the rate in our run is 0.0010% and not 50%. The exact rate depends on goroutine scheduling, core placement, processor memory behavior, and other runtime conditions, so a test may see the result only occasionally. This kind of flaky and annoying bug is often the hardest to reproduce.

3. How synchronization makes earlier writes visible

For this snippet to be correct, it needs 1 guarantee: the writer must write "hello" to msg before main reads msg. The smallest change that provides this guarantee is to replace the plain done bool flag with done atomic.Bool:

var msg string
var done atomic.Bool

go func() {
	msg = "hello"
	done.Store(true)
}()

for !done.Load() {}
fmt.Println(msg)

Why does atomic.Bool create this order?

atomic.Bool reads and writes its value through operations defined by sync/atomic. Go defines the following rule for atomic operations:

If the effect of an atomic operation A is observed by atomic operation B, then A "synchronizes before" B.

In our example:

1. done.Store(true) is operation A.
2. The done.Load() call that returns true is operation B.
3. Operation B reads the value written by operation A, so A synchronizes before B.

In other words, every write sequenced before done.Store(true) in the writer goroutine is guaranteed to be visible to main after done.Load() reads that true. This includes msg = "hello", so the later read of msg in fmt.Println must see "hello".

If you read the Go runtime and compiler source, you will see names such as StoreRelease and LoadAcquire.

• Release describes the store’s guarantee for writes completed before it.
• Acquire describes the load’s guarantee for reads that run after it.

A mutex provides exclusion and visibility

You are probably familiar with sync.Mutex and its main job: allowing only one goroutine at a time to access protected state.

type Counter struct {
	mu sync.Mutex
	n  int
}

func (c *Counter) Add() {
	c.mu.Lock()
	defer c.mu.Unlock()
	c.n++
}

func (c *Counter) Value() int {
	c.mu.Lock()
	defer c.mu.Unlock()
	return c.n
}

But a mutex also makes writes from one lock holder visible to the next lock holder. This Counter uses both guarantees.

Go guarantees that a call to Unlock synchronizes before a later call to Lock returns. In other words, after Add writes n and unlocks mu, a Value call that locks mu later is guaranteed to see that write.

WaitGroup waits for task completion

The original snippet only needs main to wait for one task. sync.WaitGroup provides that relationship without a busy loop:

func main() {
	var msg string
	var tasks sync.WaitGroup

	tasks.Go(func() {
		msg = "hello"
	})

	tasks.Wait()
	fmt.Println(msg)
}

tasks.Go starts the function and tracks the task. tasks.Wait() does not return until the function has completed. Go also guarantees that writes made by the function before it returns are visible after Wait() returns. fmt.Println therefore reads "hello".

Unlike the earlier for !done loop, main blocks inside Wait() instead of repeatedly checking a value and using CPU while the task is still running.

Channel close can signal completion

The same program can use a channel when one goroutine needs to announce an event:

func main() {
	var msg string
	ready := make(chan struct{})

	go func() {
		msg = "hello"
		close(ready)
	}()

	<-ready
	fmt.Println(msg)
}

Of course, calling an arbitrary function such as abc() does not by itself create a guarantee between goroutines. The function would need to use a synchronization operation internally. close(ready) provides such an operation because Go connects it to a receive that completes after ready is closed.

So Go guarantees that closing a channel synchronizes before a receive that returns because the channel is closed. The goroutine writes msg before close(ready), and main reads msg after <-ready, so fmt.Println is guaranteed to read "hello".

sync.Once makes initialization visible

Sometimes many goroutines need the same value, but the code that initializes that value must run only once. sync.Once provides that guarantee:

var once sync.Once
var message string

func getMessage() string {
	once.Do(func() {
		message = "hello"
	})
	return message
}

Only one call to Do runs the function. Other calls wait for that function to return. Go guarantees that the function’s return synchronizes before every Do call returns, so every caller can safely read message after once.Do returns.

Atomic operations follow one global order

The earlier x and y experiment can use atomic integers:

var x, y atomic.Int32
var r1, r2 int32
var tasks sync.WaitGroup

tasks.Go(func() {
	x.Store(1)
	r1 = y.Load()
})

tasks.Go(func() {
	y.Store(1)
	r2 = x.Load()
})

tasks.Wait()

The atomic version produces no rounds in which both reads return 0:

with sync/atomic, both read 0: 0 out of 200000 rounds

Go requires all atomic operations to behave as if they ran in one global order. In our example, x.Store, y.Load, y.Store, and x.Load must all belong to that same order. Both goroutines use this order when deciding which value each Load returns. Another execution may use a different order, but the two goroutines still cannot use separate orders.

In other words, only two cases are possible when we compare the two stores:

At least one load must therefore return 1. The result r1 == 0 and r2 == 0 is no longer possible here.

Atomic operations are useful when one shared value can be updated independently, such as a counter or a ready flag. They cannot combine several updates into one operation. For example, if balance and version must always change together, another goroutine could read between two atomic stores and see the new balance with the old version. A mutex can protect both fields while they are updated and read.

4. Testing the synchronization

The question in every example above is not whether the program returned the expected value. The question is whether Go guarantees that the reader sees the writer’s work.

The race detector can report executions that lack this guarantee:

The detector tracks memory accesses made by the running application. It reports a race when concurrent goroutines access the same location, at least one access is a write, and no valid synchronization connects those accesses.

But this is a dynamic check. An unsafe function that never runs during a test cannot produce a report. Even a function that does run may require a particular execution path or schedule before both conflicting accesses occur. A clean result should therefore be read as “no race was reported in these executions,” not “the program contains no races.”

The source still needs a clear explanation of why each shared read is safe. In the examples above, that explanation comes from atomics, mutexes, task completion, channels, or one-time initialization.

Source references

• The Go memory model
• sync package documentation
• sync/atomic package documentation
• src/sync/mutex.go
• The race detector
• Memory access ordering in the Arm architecture