Database Sharding

Sharding là gì?

Database sharding = chia data theo chiều ngang (horizontal) thành nhiều database nhỏ hơn gọi là shards.

Mỗi shard chứa một phần data, không phải bản sao.

Partitioning = chia trong một database instance.

Sharding ≠ Replication

Vì sao cần Sharding?

Khi nào database cần sharding?

• Data quá lớn (TB+)
• Read/Write quá cao
• Single DB trở thành bottleneck
• Vertical scaling không còn hiệu quả

Khi index + cache + read replica không đủ → nghĩ tới sharding.

Sharding vs Replication

Sharding Key

Shard Key là gì?

• Field dùng để quyết định record nằm ở shard nào

• High cardinality
• Even distribution
• Frequently used in queries

• Time-based
• Low cardinality
• Causes hot shard

Sharding Strategies

1) Range-based Sharding

user_id1–1M → shardA
user_id1M–2M → shardB

Pros

• Query theo range dễ
• Predictable

• Hot shard
• Rebalancing khó

2) Hash-based Sharding (phổ biến)

hash(user_id) % N

Pros

• Even distribution
• Tránh hot shard

• Range query khó
• Resharding phức tạp

3) Geo-based Sharding

VN → shard VN
US → shard US

Pros

• Low latency theo region
• Compliance

• Cross-region query khó

4) Directory-based Sharding

• Shard mapping stored in lookup table

• Flexible
• Move shard dễ

• Extra lookup
• SPOF nếu lookup fail

Querying in Sharded DB

Single-shard query

• Fast
• Preferred

• Fan-out → merge results
• Expensive
• Avoid if possible

Query by shard key first.

Sharding Architecture Patterns

Application-level Sharding (common)

• App logic quyết định shard
• App kết nối đúng DB

• Full control
• DB đơn giản

• App phức tạp

Middleware Sharding

• Proxy layer (Vitess, Citus)

• Transparent to app

• Extra infra
• New SPOF

Database-native Sharding

• MongoDB
• CockroachDB
• Yugabyte

• Easier setup

• Vendor lock-in
• Less control

Re-Sharding (Hard Problem)

Khi cần re-shard?

• Data growth
• Hot shard
• More capacity needed

• Data migration
• Consistency
• Downtime

• Consistent hashing
• Dual write
• Background migration

Hot Shard Problem

Một shard nhận traffic vượt trội.

Causes

• Poor shard key
• Time-based keys

• Composite shard key
• Randomized suffix
• Split hot shard

Transactions in Sharded DB

Single-shard transaction

• OK

• 2PC
• Saga
• Eventual consistency

Avoid cross-shard transactions if possible.

Sharding + Indexes

• Each shard có index riêng
• Global index = expensive
• Duplicate small lookup tables on all shards

Sharding Use Cases

When to use

• User-based data
• Tenant-based SaaS
• Massive write throughput

• Complex joins
• Heavy analytics
• Small datasets

Cheat Sheet

Sharding Pros

✅ Horizontal write scaling

✅ Unlimited growth

✅ Fault isolation

Sharding Cons

❌ High complexity

❌ Hard queries

❌ Operational overhead