Error handling in microservices: retries and circuit breakers

In a single application, a function call either works or throws. Between microservices, a call can also be slow, time out, succeed without the reply arriving, or fail for a second and then recover. With twenty services, some part of the system is always in one of those states.

The goal is not to prevent failures. It is to stop one small failure from becoming an outage. Five practices do most of that work.

1. Put a timeout on every call

A request with no timeout can wait forever, holding a connection and a thread. Enough of those and a healthy service stops responding because it is waiting on a sick one.

const res = await fetch(url, { signal: AbortSignal.timeout(2000) });

Choose timeouts from real latency data, a little above the normal slow case.

2. Retry, with care

Many failures are brief. A retry a moment later succeeds. Retries done badly, though, turn a small problem into a flood.

async function withRetry(fn, { attempts = 3, baseMs = 200 } = {}) {
  for (let attempt = 1; ; attempt++) {
    try {
      return await fn();
    } catch (err) {
      if (attempt >= attempts || !isTransient(err)) throw err;
      const delay = baseMs * 2 ** (attempt - 1) * (0.5 + Math.random());
      await new Promise((resolve) => setTimeout(resolve, delay));
    }
  }
}

The rules behind that code:

• Retry only transient errors: timeouts, connection resets, and responses such as 502, 503 and 504. A 400 will fail the same way every time.
• Back off exponentially and add jitter. The random factor stops a thousand clients from retrying at the same instant.
• Limit the attempts.
• Retry only what is safe to repeat. For a payment or an order, send an idempotency key so the server can recognize a repeated request.

3. Add circuit breakers

When a service is down, calling it again and again helps nobody. A circuit breaker watches the failure rate and stops the calls for a while.

The caller fails fast and the struggling service gets room to recover. Mature libraries exist for this in every major language, and a service mesh can apply it without code changes.

4. Have a fallback

Decide what the user sees when a dependency is unavailable:

• cached or slightly old data,
• a sensible default,
• the page without that one section,
• a clear message that the action was queued.

A product page without recommendations is still a product page.

5. Use one error format

When every service invents its own error shape, clients cannot react properly. Agree on one. The Problem Details format (RFC 9457) is a ready-made standard.

{
  "type": "https://api.example.com/errors/insufficient-funds",
  "title": "Insufficient funds",
  "status": 422,
  "detail": "The balance is lower than the transfer amount.",
  "traceId": "b7f3c2"
}

Include the trace ID so support can find the request.

How it plays out

A user transfers money. The transfer service asks the balance service, which is having a brief network problem. The call times out after two seconds and is retried with backoff. If the balance service stays down, the breaker opens and later requests fail at once with a clear "try again shortly" error, instead of piling up and taking the transfer service down too.

None of this works blind. You need to see the retries and open circuits as they happen, which is the job of microservice observability.