Inter-thread communication in C++ : Futures/Promises Vs. Sockets

In this blog post I walk through two approaches I considered to solve a thread communication problem I ran into, and discuss some tradeoffs. Trying out these implementations helped me better understand how to build lightweight background services, and I hope to share what I learned with the reader. The code samples here will be in C++, but the concepts are language-agnostic.

Adding a parallel service

First things first — what exactly was I looking to build? Expressed verbally, I wanted a way to define a small service to run in parallel to my main application logic, and be able to communicate with this service asynchronously from the main code. Furthermore, I expected the lifetime of this service to be long (often as long as the lifetime of the main application), and that I would not need to have many services like this (~2–4 running alongside the main). A notional use case I started out with looked like this:

Deciding how to implement

Looking at this, I quickly realized that we would need a multi-threaded or multi-process solution to satisfy the requirement that the service needs to run in parallel to the main logic. While multi-process would certainly scale better, since we would be able to host the service on a separate machine as it required more resources, I opted for multi-threaded for the ease of implementation, and because I did not expect my service to ever grow very resource-hungry.

Following this choice, my first instinct for a solution was “an HTTP server in a separate thread”. I noted this as a solid, straightforward approach — I could leverage some framework to spin up the server, and the use of a protocol to communicate with my service would immediately decouple it from my main application. The latter allows me to easily convert the service thread into a service process later on, if my initial scalability assumption ended up being wrong. However, this decoupling comes at a price: I have to serialize/deserialize (serdes) my messages to communicate. If there are many messages or complex messages to send/receive, the serdes step becomes a performance bottleneck, especially if I don’t expect my service to do much with the messages it gets. Additionally, given my specific choice of putting my service in a separate thread, I am not using the fact that threads share an application space to my advantage if I go with HTTP.

This got me thinking — is there a way that I can reliably share state between my main application thread and my service thread that would mimic messaging? At this point I remembered reading that, for C++ specifically, there exists a standard library (STL) data structure called a std::promise, for holding a value that can be later acquired asynchronously via an aptly-named std::future object (the future delivery of the promised value). This data structure exists in many languages (Ex. Promise and CompletableFuture in Java). I realized that I can have my main thread set the value of a std::promise object on my service thread to communicate, and the service thread can reset this std::promise after consuming the promise’s corresponding available std::future, thereby allowing the main thread to set it to something else again. Effectively messaging! Furthermore, because std::promise/std::future are templated, my “values that look like messages” could be arbitrary data structures, which means no need for potentially performance-draining serdes here. On the other hand, shared state in multi-threaded applications leads to compexities involving locks/mutexes to combat race conditions.

Comparing some implementations

So, to compare my two aforementioned approaches, I decided to implement a minimal common “background service” interface using both, and see how they would fare. I defined this interface as follows:

Use of the interface given my two proposed implementations (PFBackgroundService for promise/future and SocketBackgroundService for messaging over a transport) is shown below:

Note how I instantiated the implementers of the IBackgroundService with a lambda that does something with a string message. This is so I can have the background service threads do something in response to messages.

For the SocketBackgroundService, I used ZeroMQ as a lightweight wrapper around many of the network transports (Ex. TCP) we have, just to write less code. The salient parts of the implementation are shown in its header below:

The SocketBackgroundService class uses a member pointer to a ZeroMQ socket (m_manager) to communicate over TCP with a pair socket that is instantiated at the beginning of the logic within m_thread. m_thread is started in the constructor of SocketBackgroundService, creating the pair socket, and then running a while(true) loop, inside of which the pair socket listens for messages from m_manager. Messages are then sent to the m_thread via processMessage by simply forwarding them to m_manager.

In a similar vein to SocketBackgroundService, the most important parts of PFBackgroundService are also shown in its header, reproduced below:

This implementation has a nested, header-only templated class called PromiseAndFuture, which is how the aforementioned “messaging via quasi-shared state” works. An instance of PromiseAndFuture is placed into PFBackgroundService via m_PF, and utilized within the logic for m_thread, which otherwise works identically to the way it does in SocketBackgroundService. processMessage functions by calling m_PF’s set function with the message string. The code within m_thread’s while(true) loop polls for new messages via m_PF’s get function.

For all of the source code (there’s not that much more in the actual cpp files) and a working demo of the previous main I showed, see https://gitlab.com/comrades-code/cpp-thread-communication.

Outcome of testing

To me, neither implementation for this example was particularly more complex than the other. I will save a performance comparison for a future post (or for the reader themselves), since that requires a benchmark discussion to test against. However, in finishing my little coding project, I reaffirmed my previously stated tradeoffs. It would be impossible to factor out the background thread logic of an IBackgroundService implemented as a PFBackgroundService into a separate process without changing the messaging code. However,an IBackgroundService implemented as a SocketBackgroundService can survive this refactor by doing messaging over TCP. Additionally, SocketBackgroundService requires that messages are strings, while PFBackgroundService’s m_PF can be templated to any class we include.

Possible applications

Hopefully this helps in some specific ventures. Applications of such long-lived thread communication could include a custom GUI framework (via an event-driven reactor), or an asynchronous publish/subscribe network that can start out entirely in process, but possibly scale to a distributed environment.

The Author: Ilya Krasnovsky is a software engineer at Innovative Defense Technologies, a small defense contractor disrupting the industry with automated test solutions for mission-critical DoD systems. He enjoys being a tech generalist, dabbling in all layers of the stack. If you would like to connect, ping him here.