⚡ Caching Strategy Architect

Caching Strategy: B2B Project Management SaaS

Suitability Check

Cache the slow + hot: dashboard data, timeline data, integrations directory, search results, static content.

Don't cache (skip the cache for these):

• Real-time presence — already 5-second staleness, just hit Redis directly as a presence store, not a cache
• Search results — your scale doesn't justify; native DB indexes should handle
• Anything 1-time queried (one-off admin operations)

Highest-leverage: caching the dashboard + timeline endpoints alone will cut p95 latency 50-70%.

Per-Data-Type Layer Mapping

Layer rationale

App-memory LRU (in Node process):

• Best for tiny, hot, semi-static data: integrations directory, current user's profile during their session
• LRU eviction at 200 entries (avoids unbounded growth)
• Per-Lambda; doesn't share across instances (acceptable — 5K users / 4 instances = ~1.2K cache entries each, low hit-rate but cheap at this scale)
• Library: lru-cache npm package

Redis (Upstash):

• Best for shared per-user data, session-like data, larger-than-app-memory items
• TTL-based expiry
• Pub/sub for invalidation (if needed)

Cloudflare CDN (edge):

• Best for public static content + GET-only data
• Set Cache-Control headers; CF respects

Invalidation Strategy

User profile (Redis cache)

// Write pattern
async function updateUserProfile(userId: string, updates: ProfileUpdates) {
  // Atomic: write to DB + invalidate cache in transaction
  await db.transaction(async (tx) => {
    await tx.users.update({ id: userId }, updates);
    await redis.del(`user:${userId}`);  // invalidate Redis
  });
  // Invalidate app-memory caches via pub/sub
  await redis.publish('cache.invalidate.user', userId);
}

// Read pattern with stale-while-revalidate
async function getUserProfile(userId: string): Promise<Profile> {
  // Try app-memory first
  const memCached = appCache.get(`user:${userId}`);
  if (memCached) return memCached;
  
  // Try Redis
  const cached = await redis.get(`user:${userId}`);
  if (cached) {
    const profile = JSON.parse(cached);
    appCache.set(`user:${userId}`, profile, { ttl: 5 * 60 * 1000 });
    return profile;
  }
  
  // Fetch from DB (singleton-fetch — see Stampede section)
  const profile = await singletonFetch(`user:${userId}`, async () => {
    return await db.users.findById(userId);
  });
  
  // Populate caches
  await redis.set(`user:${userId}`, JSON.stringify(profile), { EX: 15 * 60 });
  appCache.set(`user:${userId}`, profile, { ttl: 5 * 60 * 1000 });
  return profile;
}

Project list (Redis cache)

// Write pattern
async function createProject(userId: string, projectData: ProjectCreate) {
  await db.transaction(async (tx) => {
    await tx.projects.create({ ...projectData, userId });
    await redis.del(`projects:${userId}`);  // invalidate user's project list
  });
}

async function deleteProject(userId: string, projectId: string) {
  await db.transaction(async (tx) => {
    await tx.projects.delete({ id: projectId });
    await redis.del(`projects:${userId}`);
    await redis.del(`timeline:${projectId}`);  // invalidate timeline too
  });
}

Integrations directory (rarely-invalidated)

• Updated only when admin adds new integration
• Manually trigger cache invalidation: redis.del('integrations:directory')
• Or auto-invalidate via DB trigger on integrations table writes

Stale-while-revalidate pattern (advanced, for high-traffic)

async function getProjectsWithSWR(userId: string) {
  const key = `projects:${userId}`;
  const cached = await redis.get(key);
  
  if (cached) {
    const { data, fetchedAt } = JSON.parse(cached);
    const ageMs = Date.now() - fetchedAt;
    
    if (ageMs < 60 * 1000) {
      // Fresh — return immediately
      return data;
    } else {
      // Stale — return data + trigger background revalidation
      revalidateInBackground(userId);
      return data;
    }
  }
  
  // Cold cache
  const data = await fetchProjects(userId);
  await redis.set(key, JSON.stringify({ data, fetchedAt: Date.now() }), { EX: 5 * 60 });
  return data;
}

Benefit: user never waits for cache fetch. Cache stays "sufficiently fresh" via background revalidation.

Cache Key Strategy

Format: <namespace>:<id>:<version>

Examples:
  user:user_456:v2
  projects:user_456:v1
  timeline:proj_789:v1
  integrations:directory:v1

Version bump strategy:
  - Schema changes (e.g., add field to User) → bump v2 → v3
  - During migration: write to new key, eventual transition
  - Old keys naturally expire via TTL

Why versioned keys: when you change the cached object's shape, you don't have to invalidate the cache cluster manually. The new version uses a new key; old version expires naturally.

Avoid: non-prefixed keys (456), reused keys across services (456 collides between user-service and other-service).

Cache Stampede Prevention

// Singleton fetch: when N concurrent requests miss the same key,
// only ONE fetches; others wait for that result.

const inflight = new Map<string, Promise<any>>();

async function singletonFetch<T>(key: string, fetcher: () => Promise<T>): Promise<T> {
  if (inflight.has(key)) {
    return inflight.get(key)!;
  }
  
  const promise = (async () => {
    try {
      return await fetcher();
    } finally {
      inflight.delete(key);  // Cleanup whether success or fail
    }
  })();
  
  inflight.set(key, promise);
  return promise;
}

This is per-Lambda. Across instances, you'd use Redis-based locking (SETNX cache:lock:user_456 1 EX 30). But for your scale (Lambda, 4-8 instances), per-instance is sufficient — at most 8 stampede attempts simultaneously instead of 200.

For higher-scale fix: Redis-based singleton with SET NX + waiting clients poll briefly.

Hot Key Mitigation

Detection: Redis SLOWLOG + Datadog metrics show hot keys.

For your scale + data shapes, hot keys unlikely. The integrations:directory key would be hot if it weren't tiny + cached at app-memory layer too.

Mitigation if needed (future):

1. Replicate hot key across N variants (`integrations:directory:0`, `integrations:directory:1`, ...). Pick one randomly per request.

2. Move hot key to app-memory only (skip Redis).

3. Use Redis Cluster with consistent hashing if you outgrow single Redis.

Cache Warming

For scheduled rebuilds (e.g., after a deploy):

// Run on deploy or scheduled
async function warmIntegrationsCache() {
  const directory = await db.integrations.findAll();
  await redis.set('integrations:directory:v1', JSON.stringify(directory), { EX: 24 * 60 * 60 });
}

Warm: integrations directory after admin updates.

Don't warm: per-user data (you'd warm 80K user profiles after every deploy — expensive).

Cold cache failure modes: without warming, the first request after deploy is slow. Use stale-while-revalidate to mask this — old cached value (if Redis didn't restart) served, refresh in background.

Observability

Track per cache layer:

• Hit rate: target >80% for hot data, >50% for medium-warm. Below 50% means cache isn't earning its keep.
• Miss rate: every miss adds DB load. Watch trend.
• Eviction rate: if Redis is evicting > 1% of entries before TTL, you need more memory or shorter retention.
• Stampede events: when singleton-fetch is invoked. Track via metric.
• Latency: cache fetch latency p95. Should be <5ms for Redis, <0.5ms for app-memory.

Datadog dashboard panel:

• App-memory cache: hit rate, size, eviction count
• Redis: hit rate per key namespace, evictions, memory used
• CDN: hit rate (Cloudflare provides natively)

Alerts:

• Hit rate drops >20% over 1 hour → likely a deployment introduced cache key change
• Eviction rate >5% → memory under-provisioned
• Redis memory >80% used → scale up

Implementation Skeleton

/lib/cache/
  app-memory.ts        (LRU cache wrapper)
  redis.ts             (Redis client + helpers)
  index.ts             (high-level get/set with all layers)
  invalidate.ts        (pub/sub-based cross-instance invalidation)
  singleton-fetch.ts   (stampede prevention)

/lib/cache/strategies/
  user-profile.ts      (per-data-type cache logic)
  projects.ts
  timeline.ts
  integrations.ts

/lib/cache/observability.ts (metric instrumentation)

Pattern: per-data-type cache modules.

// /lib/cache/strategies/user-profile.ts
import { appCache, redis, singletonFetch } from '../index';

export const UserProfileCache = {
  KEY_PREFIX: 'user',
  VERSION: 'v2',
  TTL_REDIS: 15 * 60, // 15 min
  TTL_MEMORY: 5 * 60 * 1000, // 5 min
  
  key(userId: string) {
    return `${this.KEY_PREFIX}:${userId}:${this.VERSION}`;
  },
  
  async get(userId: string): Promise<Profile | null> {
    const memCached = appCache.get(this.key(userId));
    if (memCached) {
      metrics.increment('cache.hit', { layer: 'memory', type: 'user' });
      return memCached;
    }
    
    const redisCached = await redis.get(this.key(userId));
    if (redisCached) {
      const profile = JSON.parse(redisCached);
      appCache.set(this.key(userId), profile, { ttl: this.TTL_MEMORY });
      metrics.increment('cache.hit', { layer: 'redis', type: 'user' });
      return profile;
    }
    
    metrics.increment('cache.miss', { type: 'user' });
    return null;
  },
  
  async set(userId: string, profile: Profile) {
    appCache.set(this.key(userId), profile, { ttl: this.TTL_MEMORY });
    await redis.set(this.key(userId), JSON.stringify(profile), { EX: this.TTL_REDIS });
  },
  
  async invalidate(userId: string) {
    appCache.delete(this.key(userId));
    await redis.del(this.key(userId));
    await redis.publish('cache.invalidate', JSON.stringify({ type: 'user', id: userId }));
  },
};

Use like:

async function getUser(userId: string) {
  const cached = await UserProfileCache.get(userId);
  if (cached) return cached;
  
  const profile = await singletonFetch(
    UserProfileCache.key(userId),
    () => db.users.findById(userId)
  );
  await UserProfileCache.set(userId, profile);
  return profile;
}

What This Strategy Won't Solve

• Won't replace fast queries. If your DB query is 50ms, caching adds 5ms overhead — minimal benefit.
• Won't handle truly real-time data. Presence at 5-second staleness should NOT be cached; treat Redis as the authoritative store.
• Won't fix architectural slowness. If your dashboard does 25 sequential queries, caching helps but the real fix is parallelizing queries.
• Won't compress beyond data shape. Caching 50-100KB timeline data is fine; caching 5MB is questionable.
• Won't auto-handle TTL drift. If your invalidation logic has bugs, stale data will be served. Tests + monitoring required.

Migration from Current State

Week 1: Foundation

• Add app-memory cache wrapper + LRU library
• Set up Redis connection
• Build the per-data-type cache modules (UserProfileCache, ProjectsCache, etc.)
• Add metric instrumentation

Week 2: Cache the slow endpoints

• Wrap getUserProfile with UserProfileCache
• Wrap getProjectList with ProjectsCache
• Wrap getProjectTimeline with TimelineCache
• Test hit rates in staging

Week 3: Invalidation correctness

• Wire up write-pattern invalidation for each cached data type
• Test scenarios: update profile → cache invalidated → next read returns fresh
• Spot-check cross-instance invalidation via pub/sub

Week 4: Optimize

• CDN setup for static content (Cloudflare cache headers)
• Stale-while-revalidate pattern for high-traffic endpoints
• Cache warming for integrations directory

Week 5: Monitor + tune

• Verify hit rates >80% on intended caches
• Tune TTLs based on observed usage patterns
• Address any cache bugs (stampede events, hot keys)

Maintenance Cadence

Weekly (eng team):

• Review hit rate dashboard. Anything trending down?
• Check Redis memory usage. Approaching limits?

Monthly:

• Review TTL settings. Any data type where TTL is too long (causing stale bugs) or too short (causing high miss rate)?
• Audit cache key versions. Any unused old versions?

Quarterly:

• Cache strategy review: still hitting the right slow endpoints?
• Cost review: Redis memory + CDN spend
• Test invalidation correctness end-to-end with synthetic test

Key Takeaways

• Cache the slow + hot. Dashboard, timeline, user profiles. Skip cold + fast queries.
• 3-layer strategy: app-memory (fastest, smallest), Redis (shared, medium), CDN (edge, public). Each data type fits one layer best.
• Invalidation is a write pattern. Atomic write+invalidate prevents staleness windows.
• Singleton-fetch on cache miss prevents stampede. Per-Lambda is sufficient at your scale.
• Stale-while-revalidate for high-traffic endpoints. User never waits; background revalidation keeps data fresh.
• Versioned cache keys enable schema evolution without manual cache flushes. New version = new key; old version expires naturally.