Client-side balancing in YDB

It is common for modern services to extend beyond physical machines. That means that service requires better characteristics than it is possible under physical limitations of a concrete machine. Scaling back-ends horizontally is one way to improve the performance and reliability of a service in comparison to its single instance version. Since the service is run on multiple nodes, there can be a division of functionality between backends, or a homogeneous copy of the backend may simply be used. Both cases require a solution to the problem of request routing and load balancing, which is necessary so that end users have a single entry point and do not need to think about how many backends are actually involved in processing requests.

YDB is a distributed SQL database, eliminating single points of failure while offering built-in sharding and much more. Thats why we need to solve the balancing problem. Clients have to interact with multiple YDB nodes (endpoints) and distribute requests across them based on a variety of criteria.

What are the generally accepted approaches to the implementation of balancing?

Server-side balancing makes things easier for the clients, representing a single entry point (L3 or L7 balancer) that redirects requests to destination nodes.

Here are the main advantages of server-side balancing:

1. Scalability. Server-side balancing allows databases to be spread across multiple machines, allowing greater capacity and flexibility to quickly scale up or down as necessary.
2. Availability. Database servers are more resilient when they are spread across multiple machines, since a single server failure doesn’t affect the full system.
3. Performance. When a single database query is distributed across multiple servers, it can reduce overall processing time and improve response times for end-users.

But at the same time, server-side balancing has its disadvantages:

1. Low throughput. All requests go through the load balancer, so the maximum throughput is limited by the load balancer’s throughput.
2. The complexity of managing the balancer. The balancer is a separate service (software) that also requires maintenance, observability, and administration team responsible for the availability of the service.
3. The complexity of the point routing of individual requests to the execution nodes. Common server-side proxy software solves common cases. Database-specific queries require a smart balancing algorithm specified for concrete databases.

Client-side balancing does not have these limitations, but has other advantages and disadvantages. High throughput over client-side balancing requires an implementation of balancing algorithms inside driver of database. If a database supports N drivers for different programming languages, the database team has to implement balancing algorithms in each supported language.

An alternative approach is to have a sidecar-application next to the client application. In this case, the client application transitions to a sidecar-application, which in turn implements a client-side balancing algorithm.
In YDB, we use client-side balancing. It’s more efficient when a lot of traffic reaches the database from multiple client applications, though it requires equal public APIs on each node in the cluster. Client-side balancing is enabled by default during the system design process.

We implement client-side balancing algorithms in each of the officially supported drivers for YDB.

How client-balancing works in YDB?

During initialization, the YDB driver (SDK) sends a Discovery/ListEndpoints request to an initial endpoint (entry point). The response contains information about all the database nodes in the cluster. Then, the driver establishes gRPC connections with all nodes initiating the connection pool. At this point, connections would be lazy rather than real. All subsequent requests are proxied to the client-balancer, an algorithm included in the driver/SDK, which selects a specific connection for specific requests.

We have a variety of balancing algorithms, including round robin, random choice, as well as some with simple filtering logic (preferred DC, preferred node, only the nearest nodes), and also combinations of them. Each one has its pros and cons that should be considered when deciding which one to use.

Balancing algorithms in YDB SDKs

Balancing algorithms have been around for some time. We do not propose any fundamental differences with common balancing algorithms for the purposes of this article.

The round robin algorithm is the simplest of the three, simply assigning each read or write request to the next node in a pre-determined order. This ensures that each node in the cluster receives an equal number of requests. That having been said, it can lead to an uneven distribution of requests if some nodes are significantly faster than others. Let’s imagine a cluster with four nodes where the client uses the round robin algorithm. They create four queries, three of which are lightweight, while the other is heavy. In this case, one of the four nodes will always be overloaded by the heavier query.

The random choice algorithm randomly selects a target node for each read or write request, ensuring that each request is given an equal chance of being assigned to any node in the cluster, and reducing the uneven distribution inherent in the round robin algorithm.

Configuring of preferred connections in YDB

The second part of configuring the client-side balancer is using an option to prefer connections.

By default, YDB driver interacts with all YDB nodes, so the load from clients is evenly distributed across the cluster nodes.

However, some YDB clients may require better query response latencies. This can be achieved by reducing network delays. The SDK implements special algorithms for determining the nearest data center with a previously unknown network topology between user applications and YDB nodes. If the network topology is known in advance, then there are other ways in the sdk to specify preferred YDB nodes (beyond the scope of the arcticle, see documentation for details).

Determining the nearest data center on the client side is a challenge. For clients, they’re better off checking round-trip time, or RTT, for all YDB nodes. The lower the RTT, the closer the data center housing the node, generally speaking. It could also mean that the route to some data centers features better network equipment than others or that some YDB nodes are less loaded than others. But that doesn’t matter since the client is getting the best times out of it regardless.

The YDB driver lets you set a balancing algorithm with preferred the nearest data center. Given the latter’s short path to the nodes in the YDB cluster, it can provide the best query timing. A YDB cluster is usually hosted in several data centers. Client applications can be distributed in one or more of them, meaning some may be closer to the client application than others. While some clients go with prefer the nearest DC option enabled to boost timings, that can lead to a load skew on the YDB nodes in the cluster’s data centers. The possible cases are illustrated on the picture below.

YDB driver provides different settings for YDB node preferences. We do, however, recommend the default, which is random node selection from any data center, to maximize reliability and availability for the database and client applications.

Conclusion

This article is for informational purposes and reveals some of the principles on the basis on which client balancing in YDB is built. The code for configuring the YDB driver can be found in our documentation: Working with YDB SDK / Code Recipes / Balancing.

If you have any difficulties or questions, please contact us:
• GitHub: https://github.com/ydb-platform
• Telegram: https://t.me/ydb_en