Data Replication: Cost vs Performance

Data replication ensures data availability, reliability, and faster access across servers or regions. However, it comes with trade-offs between cost and performance. Here's what you need to know:

• Synchronous replication: Guarantees no data loss and strong consistency but increases latency and requires high-speed networks. Best for critical systems like finance.
• Asynchronous replication: Faster and cheaper but risks temporary data inconsistencies. Ideal for read-heavy workloads like analytics or social media.
• Storage costs: High-performance storage (e.g., AWS EBS) is expensive, while long-term storage (e.g., S3 Glacier) is cheaper but slower.
• Network costs: Cross-zone and cross-region transfers can significantly add up, often hidden in billing categories.
• Compute overheads: Processes like Change Data Capture (CDC) and compression can strain CPU resources.

Key takeaway: Balancing cost and performance involves selecting the right replication strategy, optimising storage tiers, and controlling network traffic. For tailored solutions, experts like Hokstad Consulting can help reduce costs while maintaining system performance.

P99 CONF 2023 | Writing Low Latency Database Applications Even If Your Code Sucks by Glauber Costa

The Main Cost Drivers in Data Replication

Understanding the factors that influence costs is crucial for managing replication performance while keeping expenses under control.

Storage Costs

Storage is often the most obvious expense when it comes to replication, but the total cost depends heavily on the type and location of the storage used. For example, high-performance block storage like AWS EBS gp3 is ideal for Write-Ahead Logging (WAL) but comes at a higher price. On the other hand, long-term replica retention is better suited to object storage solutions such as Amazon S3, where costs decrease significantly as you move to lower storage tiers.

To put this into perspective, S3 Standard costs around £0.023 per GB per month, whereas S3 Glacier Deep Archive costs approximately £0.00099 per GB. That’s about 23 times cheaper. However, storage classes like Standard-IA and Glacier Instant Retrieval come with conditions: a minimum billable object size of 128 KB and mandatory storage durations of 30 to 180 days. These requirements can unexpectedly inflate costs for workloads with small or short-lived objects[2].

The gap between the right and wrong storage class choice is enormous, and most teams don't realise it until the bill arrives. - Cloud Burn[2]

Another factor to consider is the replication factor multiplier. For instance, a replication factor of 3 - common in systems like Apache Kafka - triples the storage volume and doubles the network replication traffic. This means that every 1 GB of incoming data generates 2 GB of cross-Availability Zone (AZ) replication traffic[3].

While storage is a major cost, network data transfers can also significantly impact replication expenses.

Network and Data Transfer Costs

Network costs are often overlooked but can quickly add up. On AWS, cross-AZ transfer charges are frequently buried under generic categories like EC2-Other, making it difficult to pinpoint their source[3].

Cross-AZ traffic is charged at £0.01 per GB in and £0.01 per GB out, resulting in an effective rate of £0.02 per GB[3]. While this might seem small, the costs scale rapidly. For example, a three-node Kafka cluster operating at 100 MiB/s throughput can incur between £14,000 and £24,000 per month in cross-AZ traffic costs alone[3]. In some cases, such as replication setups with short local retention periods, cross-AZ data transfer can account for 90–95% of the combined storage and networking costs[1].

You spend more on invisible traffic than on visible servers. - AutoMQ Team[3]

Cross-region transfers are similarly priced, typically around £0.02 per GB outbound. However, in certain regions like Singapore, rates can climb as high as £0.09 per GB[4][5]. Internet egress costs can range from £0.08 to £0.12 per GB, depending on the data volume[5][6].

Compute and administrative overheads also contribute to the overall cost equation.

Compute and Management Overheads

Replication isn’t just about storage and networking; it also involves compute costs, which are easy to underestimate. For instance, logical replication and Change Data Capture (CDC) processes can place significant strain on CPU resources, particularly in complex environments. A CPU utilisation rate above 70% is often a clear sign that an upgrade is necessary[7].

Adding read replicas can help offload analytical queries, but this approach introduces additional management challenges. Each replica requires adjustments to connection strings, routing logic, and ongoing monitoring. Horizontal scaling is cost-effective only when reads make up more than 80% of the workload[7]. Otherwise, vertical scaling tends to be the better choice.

Compression can reduce replication data by 70–90%, but it increases CPU demand[4]. Similarly, enabling parallel or multi-threaded replication can enhance throughput by 2x to 10x, though it requires sufficient compute capacity to handle the load[8]. Striking the right balance between these trade-offs is essential when designing a replication strategy.

Performance Factors in Data Replication

When discussing replication strategies, it's not just about managing costs. Performance plays an equally critical role, and finding the right balance between speed and expenses can be tricky. Focusing solely on one aspect without considering the other can lead to inefficiencies and, ultimately, higher costs.

Replication Lag and Latency

Replication lag refers to the delay between a write operation on the primary node and when it appears on a replica. In synchronous replication, every write must wait for at least one replica to acknowledge receipt before the client gets a response. This process inherently involves a full network round trip, adding to the delay.

Physical distance is a major factor in latency. For instance, the round-trip time (RTT) between US-East (Virginia) and EU-West (Dublin) is approximately 90 ms [12]. A synchronous setup between these regions can add 160 ms or more to each write operation [13]. On high-latency networks, this can result in performance drops of over 50% [8].

Strong consistency across regions requires synchronous replication, which forces every write to pay the cross-region latency penalty. - System Design with Sage [12]

Lag has a direct impact on your Recovery Point Objective (RPO), which defines the maximum amount of data loss you can tolerate. Synchronous replication ensures an RPO of zero, as no write is confirmed until it is safely replicated. In contrast, asynchronous replication ties your RPO to the current lag, increasing the risk of data loss if a failover occurs unexpectedly.

These latency considerations don't just affect system responsiveness - they also set the stage for throughput challenges.

Throughput and Scalability

Latency is only one piece of the puzzle. Throughput - the number of transactions a system can handle - is another critical factor, especially when considering scalability. Here, asynchronous replication has a clear edge. Since writes don’t wait for acknowledgements, the primary node isn't slowed down by the cluster's slowest replica [11].

Multi-threaded replication can further boost throughput by factors of 2–10, depending on how parallel the workload is [8]. For systems where reads significantly outnumber writes (e.g., 10:1 or higher), asynchronous read replicas are a practical solution. They offload the primary node without compromising write speeds [10]. Additionally, optimising batch sizes - typically 200–500 transactions per batch - helps maintain a balance between memory usage and latency [8].

Consistency Models

Consistency models define how data appears to users and significantly influence both performance and reliability. These models also directly affect replication performance. The table below provides a comparison of key consistency models:

Strong consistency ensures all nodes have identical data at the same time, making it ideal for financial systems or environments with strict regulations. However, this comes at the cost of higher write latency due to cross-region delays [12].

Eventual consistency, on the other hand, prioritises lower write latency by allowing replicas to temporarily serve slightly outdated data. Netflix, for example, leverages Cassandra's asynchronous, multi-directional replication across AWS regions. This active-active architecture allows them to handle the loss of an entire region without disrupting user experience [13].

For most production environments, semi-synchronous replication strikes a balance. The primary node waits for confirmation from just one replica, rather than all of them, before proceeding. This approach reduces the risk of data loss while keeping performance degradation manageable. For instance, MySQL's semi-synchronous replication typically results in only a 5–15% performance hit [8]. Discord’s 2023 migration to ScyllaDB highlights the benefits of choosing the right consistency model, with P99 read latencies dropping from 40–125 ms to 15 ms and P99 write latencies reaching just 5 ms [14].

Cost vs Performance: A Comparison of Replication Strategies

::: @figure {Data Replication Strategies: Cost vs Performance Trade-offs} :::

Understanding consistency models is just one piece of the puzzle. The real challenge lies in selecting the right replication strategy for your specific workload. Below, we break down key replication strategies and examine their cost-performance trade-offs.

Synchronous Database Replication

Synchronous replication is often seen as the safest choice because every write operation is only confirmed after at least one replica acknowledges it. This ensures an RPO (Recovery Point Objective) of zero, meaning no data loss. However, this level of safety comes with a latency cost, as it requires round-trip communication. For instance, Oracle Data Guard reports a 3–12% reduction in throughput when operating in synchronous mode. As its documentation explains:

Greater success with synchronous transport when round trip network latency is less than 5ms. - Oracle Data Guard Documentation [8]

Once the network latency exceeds 5ms, or the distance between nodes surpasses roughly 100 km, synchronous replication becomes impractical due to physical network limitations [15]. This method is ideal for financial systems or regulated industries where data loss is unacceptable, but for most other workloads, the associated costs often outweigh the benefits. Asynchronous replication is a better fit for scenarios where immediate consistency isn't critical.

Asynchronous Replication with Read Replicas

Asynchronous replication offers a more flexible and cost-efficient alternative, especially for write-heavy workloads. Here, changes are written locally first and then propagated to replicas later. This approach minimises write latency and works well across long distances or wide-area networks. The trade-off is a variable RPO - data loss can occur if a failure happens during replication lag.

This strategy shines in read-heavy environments. For example, Shopify's MySQL 8 infrastructure handled a 10:1 read-to-write ratio during Black Friday 2026, processing approximately 3 million writes per second and generating £11.5 billion in sales over four days. By offloading reads to replicas, the primary node remained dedicated to writes, maintaining high throughput through horizontal sharding and optimised replication [8].

Replication lag in asynchronous systems can range from milliseconds to several minutes. Most production setups set alert thresholds between 10 and 120 seconds; if lag exceeds these thresholds, primary nodes may throttle writes to prevent replicas from falling too far behind [10].

Log-Based and Snapshot Replication

Log-based replication splits into two main categories: physical (streaming) replication and logical replication.

• Physical replication copies raw write-ahead log (WAL) bytes to a standby. Since WAL is already generated for crash recovery, the performance impact is minimal - essentially just the cost of transmitting data over the network [16]. However, this method requires the replica to be an exact copy of the primary, leading to higher storage costs.

• Logical replication decodes WAL into specific change events (like INSERT, UPDATE, DELETE) and replicates them selectively. This reduces storage and network demands but increases CPU and memory usage for decoding [16]. Logical replication is particularly useful for analytics, ETL pipelines, and zero-downtime upgrades, as it supports cross-version and cross-platform replication.

Snapshot replication, on the other hand, captures a point-in-time state of the database. While it's not designed for live replication, it’s ideal for initial data seeding or periodic backups.

Aerospike's Cross Datacenter Replication (XDR) is a prime example of efficient log-based replication. It achieves less than a 3% increase in CPU usage while scaling from three to six clusters and can recover 10 million records in just 20–50 seconds [17]. Whether you choose physical, logical, or snapshot replication depends on your system's specific needs and priorities, allowing for tailored performance and cost adjustments.

How to Tune Replication for Cost and Performance

Fine-tuning replication is key to balancing system efficiency and costs. Once you've chosen a replication strategy, optimising its parameters can have a noticeable impact on both your expenses and your system's responsiveness.

Selective Replication and Data Classification

Not all data is created equal, so prioritisation is essential. Start by cataloguing your data stores and ranking datasets based on factors like business importance, access frequency, and return on investment (ROI) [19]. With this understanding, segment your data:

• High-demand, frequently updated data: Store this on fast, high-performance systems with tight replication schedules.
• Archival or infrequently accessed data: Place this on more economical storage tiers with relaxed replication intervals.

The format of your data also makes a difference. For example, Avro is well-suited for write-heavy operations, while Parquet or ORC work better for read-heavy, columnar query workloads. These formats help reduce unnecessary input/output operations [19]. Additionally, applying deduplication and compression to cold data can lower storage costs without requiring changes to your replication setup.

Once data is classified, focus on network settings to improve both performance and cost efficiency.

Consistency and Network Tuning

Network expenses can quickly add up, especially with inter-AZ (Availability Zone) transfers. For example, AWS charges £0.01 per GB for inter-AZ data transfer in one direction, or £0.02 per GB round-trip. Add to this NAT Gateway processing costs of £0.045 per GB, and the bill can grow fast [20].

To address this, consider the following:

• Use VPC Gateway Endpoints: For services like S3 and DynamoDB, these endpoints bypass the NAT Gateway entirely, eliminating extra costs.
• Enable Kubernetes topology-aware routing: By using the annotation service.kubernetes.io/topology-mode: Auto, you can ensure traffic stays within the same Availability Zone, potentially reducing inter-AZ traffic by 60–80% [20].
• Compress data before transfer: Applying gzip or zstd compression to text-based payloads such as logs, JSON exports, or database backups can shrink billable data volumes by 60–90% [20].
• Disable unnecessary cross-region replication: For non-critical data like temporary caches or ephemeral logs, it’s often best to turn off cross-region replication entirely.

Finally, integrating storage tiering into your replication strategy can help achieve further cost savings.

Storage Tiering

For workloads that don’t demand real-time updates, dynamic synchronisation offers a flexible solution. Instead of sticking to a fixed replication schedule, methods like Sync-Opt adjust intervals based on current cloud load. This reduces WAN transport costs during peak periods while maintaining acceptable data freshness [18].

Pair this with tiered storage to align costs with data value. Keep frequently accessed data on high-performance systems, while migrating older or less critical data to more affordable object storage. This layered approach ensures you're not overpaying for storage that doesn’t match the data's usage patterns.

Conclusion: Getting the Balance Right in Data Replication

Making decisions about data replication is rarely straightforward. Each choice - whether it’s between synchronous and asynchronous methods, or decisions around storage tiers and network routing - has direct implications for both costs and performance. For instance, transferring data across regions on major cloud platforms typically costs around £0.016 per GB[9]. At scale, these costs add up fast. On top of that, unexpected expenses like API request fees can surprise teams if they haven’t fully accounted for their replication footprint.

However, it’s possible to strike a balance between cost and performance. The strategies discussed earlier, such as selective replication, consistency modelling, and tiered storage, show how to achieve reliability and low latency without breaking the bank. As Aerospike points out:

Most high-performance consumer applications cannot tolerate the high write latencies required for synchronizing writes across far-apart sites… asynchronous replication… becomes the only practical option. [17]

If you’re unsure where to begin, conducting a cloud cost audit can be a practical first step. Hokstad Consulting offers expertise in cutting cloud infrastructure costs by 30–50%. They focus on optimising DevOps pipelines, replication setups, and storage configurations, with a no savings, no fee model to minimise risk.

Maintaining the right balance in data replication requires ongoing effort. Regularly reviewing data classifications, keeping a close eye on transfer costs, and adjusting replication intervals as workloads evolve are all key. By continuously fine-tuning strategies like selective replication and tiered storage, you can adapt to changing needs while keeping both costs and performance in check.

FAQs