We Cut API Latency by 67% By Fixing One Simple Thing | Metrics Bus |

We cut our API latency by 67% and boosted throughput by 31%. We didn't change a single line of business logic. We didn't upgrade our servers. We fixed one stupidly simple thing.

Stop making your users wait for your logs.

This is a thread on async logging.

The Problem:

We sprinkle logging throughout our applications for

1. debugging,
2. observability
3. audit trails.

As developers, we log frequently across the entire request journey.

Logging is the silent tax on every request.

This creates a latency. Meaning we make our user wait unnecessary, for logging to be done before we could fulfill that request.

HOW IT HAPPENS

In average, a single request might hit somewhere from 1-5 log statements.

Request received → log
Auth check → log
Database query → log
Business logic → log
Response sent → log

Each log = I/O operation.

Each I/O = blocking wait.

It adds up fast.

THE HIDDEN COST

logger.info('Processing request', { userId });
// Your code waits here while this writes

logger.debug('Query result', { data });
// blocks again

logger.info('Request complete', { duration });
// And again

The Curiosity ?

I was doing some load testing around of api. We had our average latency around 800-900 millisecond. The average throughput was around 11 request per second.

 Request → Log (wait 10ms) → Log (wait 10ms) → Response
             ↓                       ↓
        I/O blocking          I/O blocking

 Request → Log (wait) → Continue

Our Innovation:

We used async metrics bus that takes the info but only logs it after the request has been sent to the client.

 Request → Buffer (instant) → Continue
             ↓
      [Background process flushes every 5s]
             ↓
      Logger | Sentry | PostHog (parallel)

 Logging happens after the response is sent.

 THE ARCHITECTURE

 ┌─────────┐
 │ Request │
 └────┬────┘
      │
      ▼
 ┌──────────────┐
 │ Metrics Bus  │ ← In-memory buffer (0.1ms)
 │ (Buffer)     │
 └──────┬───────┘
        │ Every 5s or 1000 logs
        ▼
 ┌─────────────────┐
 │ Orchestrator    │ ← Batch processor
 │ (Flush handler) │
 └────────┬────────┘
          │ Parallel writes
          ▼
     ┌────┴───┬─────┐
     ▼        ▼     ▼
  Logger  Sentry  PostHog

  THE IMPLEMENTATION

 Step 1: Zero-latency emission

 emit(eventType: string, data: any): void {
  this.bus.info(logObj);
  // Returns immediately - no I/O
 }

 Step 2: Batched flush

 private readonly MAX_BUFFER_SIZE = 1000;
 private readonly PROCESS_INTERVAL_MS = 5000;

 setInterval(() => {
  this.flushLogs(); // Process batch async
 }, 5000);

 ---

 Step 3: Parallel writes using composite pattern

 private async flushLogs(): Promise<void> {
  const batch = this.logBuffer.splice(0);

  // Write to all destinations in parallel
  const writePromises = batch.map(log =>
   this.compositeWriter.write(log)
  );

  await Promise.allSettled(writePromises);
  // Individual failures don't fail the batch
 }

THE RESULTS

Before : Synchronous Logging

1. Throughput: 12 req/s
2. P95 Latency (http_req): 858 ms

 After (Using the Metrics Bus):

1. Throughput: 16 req/s (+31% )
2. P95 Latency (http_req): 247 ms (−67% )