Scaling Reads: Reducing API Latency from 4s to 40ms

Our newsfeed was taking 4 seconds to load a feed. We added a cache. Now it takes 40 milliseconds.

Here's what happened.

The Problem

We ran a load test with just 50 concurrent users:

1. Feed latency (P95): ~4.12s 🐢
2. Post detail pages: ~2.93s 🐢

The worst part?

Every user action triggered heavy DB aggregations that we were computing repeatedly.

We were recalculating the exact same feed over and over. User opens the app, we hit the database. They refresh, we hit it again. Nothing changed, but we did all the work anyway.

The Solution: A Simple Cache

We built an in-memory LRU cache with a strategy pattern for eviction:

• Time-based eviction: Data expires after a set TTL.
• Least Recently Used-only eviction: Hot data stays cached indefinitely as long as it's accessed frequently. Only gets evicted when the cache fills up and it becomes the least recently used.

How it works:

• First request: Fetch from database → save to cache → return
• Next requests: Return from cache instantly
• Eviction: Happens either by time or by LRU, depending on the strategy

The Results

Same load test, same 20 users:

1. Feed (P95): 4,120ms -> 40ms
2. Post (P95) 2,930ms- > 35ms
3. 100x faster. The app feels instant now.

Core Web Vitals Impact (Measured via Lighthouse)

📉 Speed Index (Biggest Win)

Speed Index measures how quickly the page becomes visually complete.

Before caching~11.4 s

After caching~3.1 s

Improvement~72% faster

PS: Where the Caching Strategy Came From?

A big part of this optimisation was surprisingly rooted in algorithm practice.

While grinding LeetCode, I spent a lot of time implementing:

• LRU Cache (hash map + doubly linked list)
• Time-based eviction patterns
• Constant-time read/write guarantees under concurrency

Those same principles became the backbone of Unisala’s caching layer:

• O(1) cache access for hot reads
• LRU eviction to cap memory usage safely
• TTL-based expiry to balance freshness and correctness
• Predictable performance under load

#Engineering #Performance #Backend #NodeJS #TechThread