Building a new execution platform for Redis customers

over the past decade, Redis It has become a popular and widely used cache in memory. Unlike most distributing cachesit provides a compelling set of inline data structures that enhance its capabilities beyond just storing serialized objects.

Created in 2009 As a single server repository to help design a web server, Redis provides applications with many useful options for organizing and retrieving stored data, including collections, lists, hashes, and sorted collections. A few years later, she added later block support To scale throughput and add data replication for high availability. However, the implementation of the Redis cluster introduced specialized concepts, such as master/symmetric fragmentation, that system administrators must understand and manage. Along with Redis’ use of Ultimate instead of full consistencythese concepts created complexity that makes group management more difficult while reducing flexibility and reliability.

in contrast, ScaleOut StateServer┬«an enterprise-focused distributed cache for serialized objects It was first released in 2005Designed from the ground up to run on a server family with specific design goals of high performance and reliable data storage while maximizing ease of use. It automates load balancing, data redundancy and retrieval, and data storage with complete consistency (eg, sequential consistency) across replicas. It also executes customer orders using all available processing centers to maximize productivity. These features greatly simplify group management, improve flexibility, and lower total cost of ownership, especially for enterprise users. For example, unlike Redis, ScaleOut server clusters can grow seamlessly from one server to multiple servers, and system administrators don’t need to manage hashing or core/symmetric partitions.

Challenge: Integrate Redis command processing into a new assembly architecture

ScaleOut Software realized that running Redis commands on a ScaleOut StateServer assembly would provide Redis users with the best of both worlds: familiar, rich data structures combined with significantly simpler management and complete data consistency. However, an ideal implementation would need to use the actual open source Redis code to execute the Redis commands so that the client commands behave similarly to the open source Redis collections. The challenge then was to integrate Redis code into the ScaleOut StateServer implementation platform and take advantage of highly automated compilation features while eliminating Redis’ single-threading limitations. event loop Architectural Engineering.

The company recently completed this integration, and in version 5.11 of the ScaleOut StateServer, Community preview released It offers support for the most common Redis data structures (strings, sets, lists, hashes, and ordered sets) as well as publish/subscribe, transactions, and various utility commands (such as FLUSHDB and EXPIRE) on both Windows and Linux. Uses Open Source Redis Version 6.2.5 code inside servers for processing Redis commands.

Automated block operations, scalable productivity, full data consistency

Through this integration, Redis clients can connect to a ScaleOut StateServer group to one or more servers using the standard Redis RESP protocol. Redis client libraries obtain hash mapping for servers by internally calling CLUSTER SLOTS or CLUSTER NODES commands and then directing Redis access requests to the appropriate ScaleOut server. When cluster membership changes, Redis clients automatically use the RESP protocol to redirect commands to the correct servers.

To maximize throughput, each ScaleOut server processes incoming Redis commands on multiple threads using all available processor cores. There is no need to deploy multiple fragments on each server to measure performance or provide data replication; These functions are performed in a single process on each server.

With the growing need for increased productivity, system administrators can simply add new servers to the cluster. ScaleOut StateServer automatically rebalances the hash across the cluster as servers are added or removed. It also delays the execution of Redis commands during load balancing (and recovery) to maintain a consistent image of the hash slot situation and avoid client exceptions. After the entire hash is migrated to a remote server, client requests return a -MOVED flag so they can be forwarded to the new server.

In addition, ScaleOut StateServer automatically replicates all hashes. There is no need to manually create core and replica parts or move them from server to server during membership changes and recovery. ScaleOut StateServer automatically places the replica hash on different servers from the corresponding initial hash and relays it as necessary during membership changes to ensure optimal load balancing. If the server fails or there is a network outage, the ScaleOut StateServer “self-heals” by upgrading replicas to primaries and creating new replicas as necessary.

Redis uses a final consistency model to update replicas. Although this model is fast, it is prone to presenting stale data to clients in the event of a server failure or a network outage. To avoid this problem, ScaleOut StateServer implements fully consistent updates of stored objects using the patented quorum algorithm. When the server receives a Redis command, it completes the command on the quorum containing the initial hash and its replicas before returning to the client. Transactions are processed in the same way.

Summary: New possibilities for Redis users

ScaleOut Software’s innovative integration of open source Redis code into its cluster architecture will bring powerful new capabilities to Redis users while ensuring native Redis behavior for client applications. Enterprise users now have a simpler way to manage Redis clusters with automated cluster management and integrated transport scaling, and benefit from fully consistent updates of stored data executed by Redis commands. They also have access to ScaleOut StateServer’s native APIs, which include advanced features not available in open source Redis clusters, including parallel data computing, streaming analytics, and large-scale data replication.

Editor’s Note: This article is related to ScaleOut.

Tags: RedisAnd the state server domain

Leave a Comment