Caching Strategies for Scalable Systems

Caching is the most effective performance tool most systems have, and also one of the easiest ways to introduce bugs that only appear under load. A cache can cut database load dramatically and bring response times down by an order of magnitude. It can also serve stale prices, collapse your database the moment it expires, or hide a slow query until the day the cache is flushed.

This post covers the caching strategies that come up in system design, their trade-offs, and how to implement them in .NET with in-memory caching, Redis, and HybridCache.

Why and where to cache

A cache trades freshness and memory for speed and reduced load. It works best when data is read far more often than it is written, when computing or fetching it is expensive, and when slightly stale values are acceptable.

Caching can happen at several layers, and a request may pass through many of them:

• Client and browser: HTTP caching with Cache-Control and ETag.
• CDN or edge: static assets and cacheable API responses close to users.
• Application in-memory: fastest, but per instance and lost on restart.
• Distributed cache: shared across instances, typically Redis.
• Database: buffer pools, query plan caches, and materialized views.

The closer to the user, the cheaper the hit, but the harder it is to invalidate. A value cached in thousands of browsers cannot be recalled; a value in Redis can be deleted with one command.

Core caching patterns

Cache-aside

The application checks the cache first. On a miss, it loads from the database, writes the result to the cache, and returns it. This is the most common pattern because it is simple, and the cache only holds data that is actually requested.

public async Task<Product?> GetProductAsync(int id, CancellationToken ct)
{
    string key = $"product:v1:{id}";
    string? cached = await _cache.GetStringAsync(key, ct);
    if (cached is not null)
        return JsonSerializer.Deserialize<Product>(cached);

    Product? product = await _db.Products.AsNoTracking().FirstOrDefaultAsync(p => p.Id == id, ct);
    if (product is not null)
    {
        await _cache.SetStringAsync(key, JsonSerializer.Serialize(product),
            new DistributedCacheEntryOptions { AbsoluteExpirationRelativeToNow = TimeSpan.FromMinutes(10) }, ct);
    }
    return product;
}

The downside is that the first request after expiry pays the full cost, and concurrent misses can all hit the database at once.

Read-through

The cache itself loads missing data through a configured loader, so callers only talk to the cache. In .NET, HybridCache.GetOrCreateAsync gives you read-through semantics in application code.

Write-through

Every write goes to the cache and the database synchronously. Reads after writes are consistent and the cache stays warm, but writes are slower, and you cache data that may never be read.

Write-behind

Writes go to the cache and are persisted to the database asynchronously. This absorbs write spikes and reduces latency, but a cache failure before the flush loses data. It suits counters, analytics, and other data that tolerates loss, not orders or payments.

Refresh-ahead

Popular entries are refreshed in the background before they expire, so users rarely see a miss. It costs extra load for entries that might not be requested again.

Expiration and eviction

Every cache entry needs a lifetime. Two expiration styles are common:

• Absolute expiration removes an entry at a fixed time. It bounds staleness.
• Sliding expiration extends the lifetime on each access. It keeps hot entries alive but can keep a stale value forever if it is popular. Pair it with an absolute cap.

When memory fills up, the cache evicts entries using a policy such as LRU (least recently used) or LFU (least frequently used). Redis supports several maxmemory-policy options; pick one deliberately rather than relying on defaults, and always set a TTL on keys so the cache cannot fill with data nobody reads.

Add jitter to TTLs. If you cache ten thousand items at startup with the same ten-minute TTL, they all expire in the same second.

var ttl = TimeSpan.FromMinutes(10) + TimeSpan.FromSeconds(Random.Shared.Next(0, 120));

Cache invalidation

Invalidation is where most caching bugs live. The main approaches:

• TTL only: accept staleness up to the TTL. Simplest, and correct for many read models.
• Explicit delete on write: after updating the database, delete the cache key. Delete rather than update, because concurrent writers updating the cache can leave it with an older value.
• Event-driven: publish a change event and let subscribers invalidate their caches. Necessary when multiple services or in-memory caches hold the same data.
• Versioned keys: include a version in the key, such as product:v2:42. Changing the serialization format or schema then never reads incompatible old entries.

There is a classic race with cache-aside: a reader misses, loads an old value from the database, a writer updates the database and deletes the key, then the reader writes its old value into the cache. Short TTLs bound the damage. For stricter needs, use a delayed second delete or version checks.

Cache stampedes

When a hot key expires, every concurrent request misses and hits the database simultaneously. This is a cache stampede, sometimes called a thundering herd, and it can take down the database that the cache was protecting.

Mitigations:

• Request coalescing: only one caller per key loads the value; others wait for its result.
• Distributed locks: coordinate the refresh across instances, at the cost of complexity.
• Early or probabilistic refresh: refresh before expiry so the entry never actually disappears.
• Serve stale while revalidating: return the expired value while one request refreshes it.

HybridCache in .NET

HybridCache from Microsoft.Extensions.Caching.Hybrid combines an in-process L1 cache with an optional distributed L2 cache, and it coalesces concurrent requests for the same key. It covers cache-aside, read-through, and stampede protection in one API.

dotnet add package Microsoft.Extensions.Caching.Hybrid
dotnet add package Microsoft.Extensions.Caching.StackExchangeRedis

builder.Services.AddStackExchangeRedisCache(options =>
{
    options.Configuration = builder.Configuration.GetConnectionString("Redis");
    options.InstanceName = "shop:";
});

builder.Services.AddHybridCache(options =>
{
    options.DefaultEntryOptions = new HybridCacheEntryOptions
    {
        Expiration = TimeSpan.FromMinutes(10),        // L2 (distributed)
        LocalCacheExpiration = TimeSpan.FromMinutes(1) // L1 (in-process)
    };
});

When an IDistributedCache is registered, HybridCache uses it as L2 automatically. Usage:

public sealed class ProductService(HybridCache cache, ShopDbContext db)
{
    public ValueTask<Product?> GetAsync(int id, CancellationToken ct) =>
        cache.GetOrCreateAsync(
            $"product:v1:{id}",
            async token => await db.Products.AsNoTracking().FirstOrDefaultAsync(p => p.Id == id, token),
            tags: ["products"],
            cancellationToken: ct);

    public async Task UpdateAsync(Product product, CancellationToken ct)
    {
        db.Products.Update(product);
        await db.SaveChangesAsync(ct);
        await cache.RemoveAsync($"product:v1:{product.Id}", ct);
    }
}

Keep the L1 expiration short. Removing a key clears it in the current instance and in Redis, but other instances keep their in-memory copy until their local expiration, so the L1 TTL is your staleness bound across the fleet.

Designing good cache keys and values

• Namespace keys by entity and version: order:v3:{tenantId}:{orderId}.
• Include every input that affects the result, such as tenant, culture, or user role. Missing a dimension leaks data between users.
• Never cache per-user sensitive data under a shared key.
• Cache DTOs, not EF Core entities with navigation properties and tracking state.
• Keep values small. Large values increase serialization cost and network time, which erodes the benefit.
• Consider caching negative results (not found) with a short TTL to stop repeated lookups for missing IDs.

Operating Redis in production

Redis is the default distributed cache in most .NET stacks. On Azure, check the current guidance on managed Redis offerings, as Microsoft has been moving new deployments toward Azure Managed Redis. A few practices apply regardless of hosting:

# Check memory usage and eviction behavior
redis-cli INFO memory
redis-cli CONFIG GET maxmemory-policy

# Find large keys without blocking the server
redis-cli --bigkeys

• Reuse a single ConnectionMultiplexer; the Redis cache extensions do this for you.
• Avoid KEYS * in production; use SCAN if you must enumerate.
• Design for the cache being unavailable. A cache outage should degrade performance, not correctness, which means the database must survive at least a partial cold-cache load.
• Use Entra ID authentication instead of access keys where your Redis offering supports it.

Measuring cache effectiveness

A cache you do not measure is a guess. Track:

• Hit ratio per key prefix, not only globally; a high global ratio can hide a useless cache for one entity.
• Latency of hits versus misses.
• Evictions, which signal memory pressure or TTLs that are too long.
• Database load before and after, which is the number that actually matters.

If the hit ratio for an entity is low, the cache adds latency on every miss and consumes memory. Remove it.

Conclusion

Caching is a set of trade-offs between freshness, latency, cost, and complexity. Start with cache-aside and TTLs, add jitter, protect hot keys against stampedes, and invalidate by deleting keys rather than updating them. In .NET, HybridCache handles much of this for you, combining a fast local cache with Redis and built-in request coalescing. Above all, design the system to be correct without the cache and measure whether the cache is actually earning its place.