The Bridge Between Sync and Async World

Call Python async coroutines from synchronous code with AsyncioThreadLoop.

Asyncio, a Python built-in library, provides advantages in heavy IO-bound and complex-concurrency-state use cases, but on the other hand, spreads incompatibility across the synchronous code base like weeds. This post will give you a quick overview of the AsyncIO library and will demonstrate a utility class AsyncioThreadLoop that can isolate the async method calls from your synchronous code.

Python provides at least 3 “ways” of executing code within a single process: Synchronous, Threaded, and AsyncIO. We won’t consider multiprocessing now. These represent different styles of multitasking. In multithreaded execution, the operating system switches between threads for us, and we have to lock resources explicitly. In contrast, in AsyncIO we have everything locked by default and release locks by waiting or sleeping, and the OS only calls us back.

Both multithreaded and AsyncIO multitasking clock-in at the same speed. AsyncIO in Python often slightly underperforms but can save memory and potentially power. In highly interconnected-state applications, AsyncIO can be simpler than multithreading. People use this in front end frameworks.

Multithreading

In Python, multithreading allows at-the-same-time (parallel) execution of multiple external calls: an external library code or I/O operation. Multithreading is thus useful for outside of Python compute heavy operations, but works well for I/O as well. There is an inherent issue in Python threading due to global interpreter lock, which disallows parallel Python code execution, which on the other hand makes most Python’s data structures thread-safe.

Threads themselves don’t have a truly special syntax. Multithreaded code is compatible with single threaded code, and even looks the same. To prevent parallel access to a resource, we use locks to delineate synchronous-only areas. A lock (mutex) is a common concurrency control concept. For example in a SQL database we may hold lock for time of a transaction.

Various other abstractions helps us to work with threads. Threads can communicate via thread-safe queues. Future object allows us to wait for results from another thread. Unfortunately, stopping threads is not trivial in Python, as you can see in below code with ThreadPoolExecutor.

# Custom stopping mechanism to work around lack of thread stopping implementation.
stop_threads_flag = False
def sleep_stoppable(time_secs: float):
    if stop_threads_flag:
        raise CancelledError()
    time.sleep(time_secs)
def loop_function():
    while True:
        print("hello world!")
        sleep_stoppable(1)
def get_task_result_function():
    return 1
if __name__ == "__main__":
    with ThreadPoolExecutor(max_workers=2) as pool:
        # prints: "hello world!" 3 times.
        pool.submit(loop_function)
        # prints: "1"
        print(pool.submit(get_task_result_function).result())
        time.sleep(1)
        stop_threads_flag = True

AsyncIO

Asynchronous IO, which underlies Python’s AsyncIO, has a principle: “I don’t call you, you call me”. We avoid polling and blocking used in threading by waiting for the operation system (or the hardware) to call us instead. This conveys efficiency advantages, but also trade-offs.

AsyncIO, in its most basic usage, is more memory efficient for extreme concurrency input-output situations on the order of thousands of concurrent requests. AsyncIO task costs 5 times less memory (2kB) than a thread (10kB), but it has higher fixed memory cost. Since we don’t poll repeatedly, it is more power efficient in principle, but I am not sure in the case of Python or MicroPython.

Instead of weaving threads of execution, AsyncIO processes queued tasks in a single thread. Since we run a single thread, long running requests will increase the latency of all other requests, since there is no operating system to even resource allocation or leverage more CPU cores.

Thus AsyncIO is suboptimal for computation-bound situations and in general synchronous code cannot be called directly from async code. Sometimes we can work around this issue by yielding inside of async for loops with sleep, but the best is to spawn separate worker threads. In this sense, the code has to cooperate, that is why it is called cooperative multitasking. AsyncIO tasks can be stopped at points of sleeping or waiting, in contrast to threads.

Processing tasks single threaded from a single queue simplifies concurrent code. In front end development, we often deal with highly interconnected application state. That is why asynchronous IO is used in UI frameworks. For example ReactJS uses JavaScript’s event loop. On the other hand, often we can get simple code by just using threading abstractions like thread pool and Futures. The Web UI world now has threads as well in the form of Web Workers.

At the same time, most application code sits in the external database, locks may still be needed and deadlocks may still happen with AsyncIO depending on where we yield execution.

Python’s AsyncIO introduced custom keywords async and await beyond standard Python, which introduced incompatibility. We cannot directly call long running synchronous code from async methods and we cannot directly call async methods from sync code. Some code can become simpler, but there is added complexity of extra syntax and limitations.

The trade off here is thus in limitations and extra syntax, in exchange for memory savings in case of high request count of above 1000. Unfortunately, AsyncIO is not implemented in a compatible way with threads, even though it seems it would be possible, and so this makes AsyncIO trade off unfavourable most of the time due to incompatibility. At the same time, it may be simpler in some highly interconnected-state use cases.

The Bridge

But don’t sweat! Bridging both incompatible worlds AsyncIO and Threaded just got easier! You can use AsyncioThreadLoop similarly to a ThreadPoolExecutor to create synchronous versions of some asynchronous variants of your components. Below is a simplified version of the class that creates an AsyncIO loop inside of a new thread. You can call coroutines through AsyncioThreadLoop, while having all complications handled, including graceful exit. For example if a coroutine lives forever, that task will live within the new thread and will not block the main thread.

async def get_task_result_coroutine():
    return 1
async def loop_coroutine():
    while True:
        print("hello world!")
        await asyncio.sleep(1)
if __name__ == "__main__":
    with AsyncioThreadLoop() as loop:
        # prints: "hello world!" twice due to sleep and immediate task cancellation.
        loop.submit(loop_coroutine())
        # prints: "1"
        print(loop.submit(get_task_result_coroutine()).result())
        sleep(1)

Full version is available on GLAMI Github. Recently we also published multilingual multimodal dataset GLAMI-1M.

GLAMI is looking for someone to help us develop, build, and further improve our codebase. Apply here.

Thanks to:

• Roman Diba for help with the class implementation and discussion of the AsyncIO
• A. Jesse Jiryu Davis for this excellent post
• wiki on Asynchronous IO

Author: Vaclav Kosar