The Message Arrived. Did the Operation Succeed?

On End-to-End Arguments in System Design

July 17, 2026

Prelude

Every few years, I revisit one of my favorite papers END-TO-END ARGUMENTS IN SYSTEM DESIGN and this time I decided I will write an exposition of it to maybe inspire others to read it but also serve as a set of notes for my future self.

Introduction

Imagine we have an application that holds bank accounts for Alice and Bob. Alice wants to transfer $50 from her account to Bob’s.

This is the current state:

How would you implement the transfer operation? (Think about this for a bit. No, really, think about it to internalize the problem).

Let’s start with a simple implementation:

At first glance, it appears to work. Alice now has $50, Bob has $100, and their combined balance is still $150.

But what exactly does “Transfer complete” mean?

Before examining the implementation, let’s define the result we expect from a correct transfer:

• Alice’s balance is $50.
• Bob’s balance is $100.
• Their combined balance remains $150.
• Either both balances change, or neither does.
• The final state represents one $50 transfer - not zero transfers or two.

The state shown by the interactive example above is consistent with those conditions. But it demonstrates only the simplest successful case. It hides everything that happened between clicking Transfer $50 and seeing the updated balances.

To decide whether we can trust the words “Transfer complete”, we need to open the box and examine the operation step by step.

The first step is straightforward: Alice’s request must reach the application.

Suppose the application confirms that it received the request. Is that enough to conclude that the transfer happened?

Well, “Request received” tells Alice only that her request reached the application. It does not tell her whether the application performed the transfer. The application could fail after receiving the request but before changing either balance. Acknowledging delivery is not the same as acknowledging completion.

So when is the application justified in saying “Transfer complete”? Let’s look inside the application and follow the transfer step by step.

Now we can give “Transfer complete” a more precise meaning: the application has produced the state required by the transfer. Alice has $50, Bob has $100, and their combined balance remains $150. The application knows that the transfer completed but Alice does not observe the balances directly; she learns the result through the application’s response.

What happens if the transfer completes, but that response never reaches Alice?

The transfer has succeeded. Alice now has $50, Bob has $100, and the application knows that the operation completed. Alice, however, received no response.

From her perspective, several things might have happened:

• Her request may never have reached the application.
• The application may still be processing it.
• The transfer may have completed, but the response may have been lost.

Silence does not allow Alice to distinguish between these possibilities. The actual state of the application and Alice’s knowledge of that state are two different things.

What should Alice do when she cannot tell whether the transfer happened?

Retrying the transfer

A natural response is to try again. If the original request was lost, retrying may be exactly what is needed. But what if the original transfer succeeded and only the response was lost?

Alice intended to transfer $50 once. The application received two requests and applied both of them.

From the application’s point of view, the second request looked like a new request. Nothing told it that Alice was repeating an earlier attempt whose outcome she did not know.

Retrying solved one problem and introduced another:

• If the first request was lost, retrying allows the operation to happen.
• If the first request succeeded, retrying may cause the operation to happen twice.

The network may be able to detect when it has delivered the same packet twice. But these were not necessarily the same packets. Alice created a new request because she did not receive a response.

The two requests were different messages, but they represented the same intended transfer.

How can the application distinguish two transfers from two attempts to perform one transfer?

Identifying the operation

Alice can assign an identifier to the transfer and reuse it whenever she retries the same operation.

For example:

Transfer ID: transfer-7F3A
From: Alice
To: Bob
Amount: $50

The identifier belongs to the logical transfer, not to an individual attempt to send it.

When the first request arrives, the application records transfer-7F3A and performs the transfer. If Alice retries using the same identifier, the application can recognize that the transfer has already been applied. Instead of moving another $50, it returns the result of the existing operation.

The final balances remain:

A lower layer can suppress duplicates that it creates or recognizes. It cannot generally know that two separate requests express one piece of user intent. Only the application understands that both attempts refer to the same logical transfer.

This does not magically guarantee that every operation executes exactly once under every possible failure, but it gives the application enough information to recognize this retry and avoid applying the transfer again.

We have dealt with repeated requests. What about related requests whose order matters?

Delivered in order

Suppose Alice submits the transfer and then, before learning its result, asks the application to cancel it.

For this simplified application, we will use the following rule:

If the cancellation is accepted before the transfer completes, the transfer must not take effect.

The network delivers the two messages in this order:

1. Transfer $50
2. Cancel the transfer

Is that enough to guarantee that their effects occur in the required order?

The network can preserve the order in which messages are presented to the application. It cannot guarantee how the application schedules, coordinates, or completes the resulting work.

These are three different orders:

1. The order in which Alice sends the commands
2. The order in which the application receives them
3. The order in which their effects occur

Ordered delivery establishes the second. The application cares about the third.

Only the application knows what relationship must hold between a transfer and its cancellation. It must enforce that relationship where the meaning of those operations is understood.

Suppose delivery, retries, duplicate handling, and ordering now work as intended. Does that prove that the final result is correct?

When every message succeeds

Imagine that every observation so far reports success:

• The request was delivered.
• The transport acknowledged it.
• The message checksum passed.
• No transport duplicate was delivered.
• The messages were delivered in send order.
• The application finished processing.

And yet the final balances are:

Every report may have been accurate within its own narrow scope. The request may have arrived intact and in the expected order.

The final application state is still wrong.

The combined balance is now $140. Ten dollars have disappeared.

To detect this, the application must validate the result using information that belongs to the application:

• Alice should have been debited by $50.
• Bob should have been credited by $50.
• Their combined balance should still be $150.
• The recorded result should correspond to the intended transfer.

A transport checksum cannot check these conditions. It can determine whether bytes changed while being transmitted. It does not know what Alice’s and Bob’s balances should be.

This has the same shape as the careful file-transfer example in the original paper.

A network may deliver every packet correctly, but the final file can still be corrupted by memory, software, storage, or another failure along the path. The receiving application must therefore check the final stored file against what the sender intended to transfer.

Correct delivery of the parts does not prove correctness of the final result.

Protecting the transfer

Security follows the same pattern.

Suppose we encrypt the request while it travels across the network.

Encryption prevents an observer on that protected network path from reading the request. That is valuable, but it does not satisfy the application’s complete security requirement.

The endpoint may still:

• Allow the wrong user to initiate the transfer.
• Expose the request in logs.
• Store sensitive information without adequate protection.
• Disclose the data to another application.
• Use the wrong keys or trust the wrong party.

The network can protect data while it crosses a particular boundary but it cannot control what an endpoint does before encryption or after decryption. Only the endpoints know the identities, permissions, data, and trust relationships that determine whether the operation is secure. Network encryption remains useful. It is simply not the complete end-to-end security guarantee.

Are lower-layer guarantees pointless?

We have repeatedly seen that a lower layer cannot establish the application’s complete notion of success.

That does not mean lower-layer mechanisms are useless.

Retransmission can recover from a lost packet without requiring Alice to repeat the entire operation.

Checksums can detect corruption early.

Ordered delivery can save the application from reconstructing a useful message sequence.

Duplicate filtering can remove common duplicates before they reach the application.

Encryption can protect traffic across untrusted networks.

These mechanisms reduce the frequency and cost of failures. They make the common case faster and more reliable.

What they don’t do, however, is remove the need for the final application-level check. Even with every lower-layer mechanism enabled, the application must still determine whether:

• The intended transfer occurred.
• The correct accounts changed.
• The correct amount moved.
• The application’s invariants still hold.
• A retry refers to an existing logical transfer.
• Related operations produced effects in an acceptable order.

Lower layers help perform the work but they do not define what successful work means.

The end-to-end argument

We can now state the idea that has been hiding inside the example.

When only the endpoints possess enough information to implement a function completely and correctly, the complete guarantee must be implemented there.

The relevant endpoints are determined by the function we are trying to guarantee. In this example, they are the parts of the application that know Alice’s intent and can validate the resulting balances.

For our transfer:

• The network can know that it delivered a request.
• A checksum can know that the transmitted bytes were unchanged.
• An ordering mechanism can know which message it presented first.
• A duplicate filter can recognize certain repeated messages.
• Encryption can know that data was protected across part of its path.

Only the application can know that one intended $50 transfer moved money from Alice to Bob while preserving the application’s required invariants. This does not mean that every function should exist only at the highest layer. A partial implementation in a lower layer may still be worthwhile when it improves performance, handles common failures cheaply, or provides a useful shared service.

The important distinction is between:

• Implementing part of a function as an optimization.
• Relying on that partial implementation as the complete correctness guarantee.

The former may be good engineering. The latter mistakes a partial guarantee for a complete one.

What “Transfer complete” means

We can now return to the phrase that was present in the introduction:

Transfer complete

The intended logical transfer has been identified. Alice has been debited by $50. Bob has been credited by $50. Their combined balance remains $150. A retry has not created a second transfer. The application’s ordering and security requirements have been satisfied, and its final state has been validated.

No acknowledgement from one intermediate layer can establish all of that. The endpoint that understands the operation must make the final judgment.

That is the end-to-end argument:

When correctness depends on application knowledge, the complete guarantee must be established at the relevant endpoints. Lower layers may duplicate parts of the function when doing so improves performance or reduces common failures.

References

• J. H. Saltzer, D. P. Reed, and D. D. Clark. “End-to-End Arguments in System Design.” ACM Transactions on Computer Systems 2, no. 4 (1984): 277–288.