The Redis architecture is designed to ensure speed, reliability, and scalability while remaining simple and elegant. Its internal structure revolves around the following core principles and components:
Core Components of Redis
Redis Server
The server is the core of Redis, handling requests, managing data, and ensuring persistence:
- Single-Threaded Model: Redis processes all commands sequentially using a single thread. While this limits Redis to one CPU core for command execution, it avoids the complexity and overhead of multi-threaded synchronization mechanisms.
- Event Loop: The server uses an event-driven architecture powered by epoll (Linux), kqueue (macOS), or select (Windows). This non-blocking I/O event loop ensures that the server can handle thousands of concurrent connections without delay.
Redis Data Structures
Redis supports several highly-optimized data structures:
- Strings: Basic values, often used for caching and counters.
- Lists: Doubly linked lists, supporting fast insertions and deletions at both ends.
- Hashes: Collections of key-value pairs, ideal for representing objects.
- Sets: Unordered collections of unique values, implemented using hash tables.
- Sorted Sets: Like sets but with scores attached, allowing for sorted access.
- Streams: Append-only log structures for real-time data streaming.
Each structure is optimized for specific use cases, making Redis highly flexible.
Memory Management in Redis
Redis stores all data in RAM, leveraging its fast access speeds. However, managing RAM efficiently is crucial for performance:
Encodings for Space Efficiency
Redis dynamically chooses memory encodings based on data size and type:
- SDS (Simple Dynamic Strings): An improvement over C strings, used for efficient string storage.
- Ziplist: A compact, contiguous memory representation for small lists or hashes.
- Intset: An optimized structure for small sets of integers.
- Quicklist: A hybrid of linked lists and ziplists, used for large lists.
Eviction Policies
When the memory limit is reached, Redis employs eviction strategies to free space:
- Noeviction: Rejects new writes.
- Volatile-lru: Removes least recently used keys with expiration times.
- Allkeys-lru: Removes least recently used keys, regardless of expiration.
- Volatile-ttl: Removes keys closest to expiration.
- Allkeys-random: Removes random keys.
Memory Fragmentation
Redis uses the Jemalloc memory allocator by default to manage memory efficiently and minimize fragmentation.
Clustering and Replication
Master-Slave Replication
Replication is the foundation of Redis’s high availability:
- Master: Handles writes and propagates changes.
- Slaves: Serve as read-only replicas and provide failover support.
- Replication is asynchronous, but slaves can periodically send ACKs to the master.
Redis Cluster
Redis Cluster provides horizontal scalability by partitioning data across nodes:
- Hash Slots: Keys are distributed into 16384 slots using a hash function.
- Data Distribution: Each node manages a subset of slots.
- Failover: If a master fails, one of its replicas is promoted to master.
Redis Data Persistence
Redis persistence mechanisms are designed to balance durability and performance. These mechanisms ensure that in-memory data is not lost during server restarts or crashes.
RDB Persistence
How It Works
- Redis saves the in-memory dataset as a snapshot (RDB file) at regular intervals.
- Snapshots are created using a forked process, where:
- The parent process continues serving clients.
- The child process writes data to disk.
Trade-offs
- Pros: Compact files, minimal impact on performance.
- Cons: Risk of data loss between snapshots.
AOF Persistence
How It Works
- AOF (Append-Only File) logs every write operation sequentially.
- Commands are appended to the log file and replayed during recovery.
Modes
- Always: Syncs every command (most durable but slowest).
- Everysec: Syncs once per second (balanced approach).
- No: Lets the OS handle syncing (fast but less durable).
Rewriting
To prevent AOF files from growing too large, Redis periodically rewrites them:
- A new compacted file is created with only the latest state.
- Rewriting is non-blocking to avoid impacting performance.
Trade-offs
- Pros: Near-complete durability, command-level recovery.
- Cons: Slower than RDB, larger file size.
Hybrid Persistence (RDB + AOF)
Redis 4.0 introduced hybrid persistence:
- Combines the compactness of RDB with the durability of AOF.
- During recovery, Redis loads the RDB snapshot first and replays recent AOF logs.
Recovery Process
When Redis restarts:
- If both RDB and AOF files are present, AOF is prioritized.
- Redis reads the file(s) to reconstruct the in-memory dataset.
- Hybrid mode speeds up recovery by using the snapshot and replaying fewer commands.
Data Expiration and Lazy Deletion
Redis allows keys to have time-to-live (TTL) settings:
- Expiration: Keys are removed after a specified time.
- Lazy Deletion: Keys are deleted when accessed, ensuring minimal CPU overhead.
- Active Expiration: A background process periodically scans and removes expired keys.
Advanced Data Persistence Features
6.1. Diskless Replication
Instead of creating an RDB file for replicas, Redis streams the dataset directly to reduce latency.
Backup and Restore
- Backups can be created by copying RDB files.
- Restoring requires placing the RDB file in the Redis data directory.
Data Compression
Redis compresses snapshots to save disk space, especially for large datasets.
Redis’s architecture and data persistence mechanisms provide a powerful combination of speed, scalability, and reliability. Its in-depth design principles make it a cornerstone technology for modern applications requiring high-performance data operations.
Happy Reading!!!..
Lochan R