Resilient API Design: Retries and Timeouts

Your client sends POST /api/orders, waits, and gets a timeout. Did the order go through? You can't tell. The server may have created it and lost the response on the way back, or it may never have seen the request at all.

If you retry blindly, you risk a duplicate order or a double charge. If you never retry, a brief network blip becomes a failed checkout. This guide shows you how to retry safely using timeouts, backoff with jitter, retry limits and idempotency keys, with Node.js code you can run.

Quick Answer

Resilient API design means your API and its clients behave predictably when requests time out, connections drop or servers return temporary errors. Set a timeout on every request and retry only transient failures such as 429, 503 or a dropped connection. Space retries with exponential backoff plus jitter, and cap them with a retry limit or deadline. For any operation that must not run twice, like payments or orders, send an idempotency key so the server recognises a retry and returns the original result.

What Is Resilient API Design?

Resilient API design is the practice of building APIs and clients so that temporary failures do not cause wrong results, duplicate operations or extra load. It assumes that any request can fail midway through processing.

This includes cases where the request reaches the server but the response is lost, the server takes longer than the client's timeout, or several retries arrive for the same logical operation. The goal is not to eliminate failures. It is to make them safe.

The Problem

A timeout tells you the client stopped waiting. It does not tell you whether the server did the work. That gap is what turns ordinary network trouble into data problems.

Without protection, the retry in the diagram above creates a second order.

The Solution

You combine a small set of techniques. Each one covers a gap the others leave open.

The steps below build these up for an order endpoint. You need Node.js 18 or later, because the examples use the built-in fetch and AbortSignal.timeout().

Step 1: Put a timeout on every request

A client that waits forever ties up resources and leaves the user waiting. Pass an abort signal to fetch so the request fails after a fixed time:

const response = await fetch("https://api.example.com/orders", {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify(order),
  signal: AbortSignal.timeout(5000),
});

Pick the value from how long the endpoint and its dependencies normally take. Too short and you trigger needless retries; too long and you tie up resources.

Step 2: Decide which failures to retry

Not every error deserves a second attempt. These status codes often indicate a temporary condition:

408 Request Timeout
429 Too Many Requests
500 Internal Server Error
502 Bad Gateway
503 Service Unavailable
504 Gateway Timeout

Dropped connections and client-side timeouts are candidates for retries too. A 400, 401 or 403 is not: the same request will fail the same way until someone fixes the input or the credentials, so you fail fast instead.

Step 3: Add backoff, jitter, a retry limit and Retry-After

Retrying immediately piles more traffic onto a server that is already struggling. Exponential backoff doubles the wait after each attempt, and jitter randomises it so thousands of clients don't come back at the same moment. Full jitter picks a random wait between zero and the capped delay.

When the server sends a Retry-After header, you use it instead. The header contains either a number of seconds or an HTTP date.

Save this as retry.js:

const RETRYABLE_STATUSES = new Set([408, 429, 500, 502, 503, 504]);

const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));

function retryAfterMs(response) {
  const header = response.headers.get("retry-after");
  if (!header) return null;

  const seconds = Number(header);
  if (!Number.isNaN(seconds)) return seconds * 1000;

  const date = Date.parse(header);
  return Number.isNaN(date) ? null : Math.max(0, date - Date.now());
}

export async function requestWithRetry(
  request,
  { maxRetries = 3, baseDelay = 500, maxDelay = 5000 } = {}
) {
  for (let attempt = 0; ; attempt++) {
    let response;

    try {
      response = await request();
      if (!RETRYABLE_STATUSES.has(response.status) || attempt === maxRetries) {
        return response;
      }
    } catch (error) {
      // Network failure or timeout
      if (attempt === maxRetries) throw error;
    }

    // Full jitter: random wait between 0 and the capped exponential delay
    const backoff = Math.random() * Math.min(maxDelay, baseDelay * 2 ** attempt);
    const serverHint = response ? retryAfterMs(response) : null;

    await sleep(serverHint ?? backoff);
  }
}

With maxRetries = 3, the client makes the first request plus up to three retries, then gives up. You can swap the attempt count for a total deadline if your callers care more about time than tries.

Step 4: Send an idempotency key

Backoff makes retries polite. It does not make them safe. A retried POST /api/orders still creates a second order unless the server can tell it is the same request.

You fix that with an Idempotency-Key header. The client generates the key once per logical operation and sends the same key on every retry. This follows the IETF Idempotency-Key header draft.

import crypto from "node:crypto";
import { requestWithRetry } from "./retry.js";

export async function createOrder(order) {
  // Generated once, outside the retry loop
  const idempotencyKey = crypto.randomUUID();

  const response = await requestWithRetry(() =>
    fetch("https://api.example.com/orders", {
      method: "POST",
      headers: {
        "Content-Type": "application/json",
        "Idempotency-Key": idempotencyKey,
      },
      body: JSON.stringify(order),
      // A fresh timeout for each attempt
      signal: AbortSignal.timeout(5000),
    })
  );

  return response.json();
}

Two details matter here. The key lives outside the retry callback, because a new key tells the server this is a new order. The timeout lives inside it, because an AbortSignal that has already fired would abort every retry instantly.

Step 5: Store keys atomically on the server

The server records each key with the result it returned. A naive version checks for the key and then inserts it, which leaves a race:

Request A -> check key -> not found
Request B -> check key -> not found
Request A -> create order
Request B -> create order

Both requests pass the check and you get two orders. The fix is to let the database decide. A primary key on the idempotency column means only one insert can ever win.

Create the tables in Postgres:

create table orders (
  id         bigint generated always as identity primary key,
  payload    jsonb not null,
  created_at timestamptz not null default now()
);

create table idempotency_keys (
  idempotency_key text primary key,
  request_hash    text not null,
  status_code     int,
  response_body   jsonb,
  created_at      timestamptz not null default now(),
  expires_at      timestamptz not null default now() + interval '24 hours'
);

Then the Express handler claims the key before doing any work. Install the dependencies with npm install express pg and set the standard PG* environment variables:

import crypto from "node:crypto";
import express from "express";
import pg from "pg";

const pool = new pg.Pool();
const app = express();
app.use(express.json());

app.post("/api/orders", async (req, res) => {
  const key = req.get("Idempotency-Key");
  if (!key) {
    return res.status(400).json({ error: "Idempotency-Key is required" });
  }

  const requestHash = crypto
    .createHash("sha256")
    .update(JSON.stringify(req.body))
    .digest("hex");

  // Atomic claim: only one request can insert this key
  const claim = await pool.query(
    `insert into idempotency_keys (idempotency_key, request_hash)
     values ($1, $2) on conflict (idempotency_key) do nothing`,
    [key, requestHash]
  );

  if (claim.rowCount === 0) {
    const { rows } = await pool.query(
      `select request_hash, status_code, response_body
       from idempotency_keys where idempotency_key = $1`,
      [key]
    );
    const record = rows[0];

    if (record.request_hash !== requestHash) {
      return res.status(422).json({ error: "Key reused with a different body" });
    }
    if (record.status_code === null) {
      return res.status(409).json({ error: "Original request still in progress" });
    }
    return res.status(record.status_code).json(record.response_body);
  }

  try {
    // Create the order and store the result in one atomic statement
    const { rows } = await pool.query(
      `with new_order as (
         insert into orders (payload) values ($2) returning id
       )
       update idempotency_keys
       set status_code = 201,
           response_body = jsonb_build_object('orderId', (select id from new_order))
       where idempotency_key = $1
       returning response_body`,
      [key, req.body]
    );
    return res.status(201).json(rows[0].response_body);
  } catch (error) {
    // Release the claim so the client can retry
    await pool.query("delete from idempotency_keys where idempotency_key = $1", [key]);
    return res.status(500).json({ error: "Order could not be created" });
  }
});

app.listen(3000);

The request hash catches a client that reuses a key with a different body, and the 409 tells a concurrent retry that the first attempt is still running. These responses use the 422 and 409 status codes described for these cases in the IETF Idempotency-Key draft.

Step 6: Log every attempt

When retries go wrong, you need to reconstruct what happened. Log the request ID, idempotency key, endpoint, attempt number, status, latency and retry reason for each attempt:

request_id=req-789 idempotency_key=abc-123 endpoint="POST /api/orders" attempt=2 status=503 reason="Service Unavailable"

Never log authentication tokens, passwords or payment details. The idempotency key is enough to tie attempts together.

Trade-offs

Retries are extra traffic by design. Even with backoff and jitter, a large outage still means more requests hitting a failing service, and each retry adds latency for the user who is waiting. Keep retry limits low on anything user-facing.

Retries also multiply across layers. If your browser client, API gateway and backend service each retry three times, one failure can become dozens of calls downstream. Decide which single layer owns retries and turn them off everywhere else.

Idempotency is not free either. You pay for an extra table, an extra write on every request and a cleanup job for expired keys. Idempotency also does not mean exactly-once: it guarantees the same business result, not that every internal side effect, such as a sent email, runs only once.

Skip this approach where it adds cost without benefit. GET requests are already safe to repeat, so they need retries but no keys. Low-value writes that nobody would notice twice may not justify the storage. And for long-running work, a queue with its own delivery guarantees often fits better than synchronous retries.

Conclusion

On Monday, list the endpoints where a duplicate would hurt, usually orders, payments and anything that sends money or messages. Add a timeout to every client call, route retries through one helper with backoff, jitter and a limit, and put an idempotency key with a unique constraint in front of those critical writes. Then log attempts so you can see whether retries are rescuing requests or piling on during outages. If you want help applying this to your own stack, Contact us.