REST API Errors, Idempotency, Headers and OpenAPI Documentation

Q: What is an idempotency key in a REST API?

It is a unique value, usually sent in an Idempotency-Key header, that a client generates for one logical operation. The server stores the result under that key, so if the same request is retried after a timeout it returns the saved response instead of performing the action twice. It is mainly used to make POST requests such as payments and orders safe to retry.

Web DevelopmentLesson 4 of 47 min read

What you'll learn

Return consistent machine-readable errors using the problem+json format
Make POST safe to retry with an Idempotency-Key header
Use ETag, If-None-Match and If-Match for caching and to stop lost updates
Describe the API in OpenAPI 3.1 and verify real responses against the spec

On this page

Errors a program can read
Idempotency keys: retries without duplicates
Conditional requests: caching and lost updates
Headers worth knowing
Documenting the contract with OpenAPI, and checking it
Common mistakes and the symptoms they produce
How this shows up in real projects and interviews

A customer taps "Pay" on a concert-ticket page while on a train. The request reaches the server, the order is created, but the response is lost in a tunnel. The app shows a timeout, the customer taps "Pay" again, and the server dutifully creates a second order. Two charges, and the payment was not even wrong: the *request* was repeated, which networks make routine. A dependable API is designed for that world: errors a program can act on, retries that do not duplicate work, and updates that notice when someone else changed the data first.

Errors a program can read

Every failure should share one shape so that clients write one error handler. There is a standard for it, RFC 9457 "Problem Details for HTTP APIs", served as application/problem+json. Its core members are type (a URI identifying the kind of problem, about:blank when the status code says it all), title (short, stable summary), status, detail (specific to this occurrence) and optionally instance. You may add your own fields, such as the errors array from the previous lesson. Never put stack traces or internal ids in detail.

Idempotency keys: retries without duplicates

GET, PUT and DELETE are naturally idempotent. POST is not, and POST is exactly what creates orders and payments. The fix, popularised by payment APIs, is a client-generated `Idempotency-Key` header (the header name is a convention rather than a settled standard). The client creates one unique value per user intent, sends it with every attempt, and the server remembers the outcome: the first request does the work, and repeats with the same key get the saved response instead of repeating the work. If the same key arrives with a different body, that is a client bug, and the server refuses it.

tickets.py

import hashlib
import json
from flask import Flask, Response, jsonify, request

app = Flask(__name__)
app.json.sort_keys = False
app.json.compact = False

PRICE = {"amount": 1800, "currency": "INR"}   # per seat, minor units
orders = {}
idempotency = {}   # key -> (request fingerprint, status, body)


def problem(status, title, detail, type_="about:blank"):
    body = {"type": type_, "title": title, "status": status, "detail": detail}
    resp = jsonify(body)
    resp.status_code = status
    resp.headers["Content-Type"] = "application/problem+json"
    return resp


def etag_of(order):
    digest = hashlib.sha256(json.dumps(order, sort_keys=True).encode()).hexdigest()[:12]
    return f'"{digest}"'


@app.post("/orders")
def create_order():
    key = request.headers.get("Idempotency-Key")
    if not key:
        return problem(400, "Missing header", "Send an Idempotency-Key header with POST /orders.")
    body = request.get_json(silent=True) or {}
    fingerprint = json.dumps(body, sort_keys=True)

    if key in idempotency:
        saved_fingerprint, status, saved = idempotency[key]
        if saved_fingerprint != fingerprint:
            return problem(422, "Key reused", "This Idempotency-Key was used with a different request body.")
        resp = jsonify(saved)
        resp.status_code = status
        resp.headers["Idempotent-Replayed"] = "true"
        return resp

    quantity = body.get("quantity")
    if not isinstance(quantity, int) or not 1 <= quantity <= 6:
        return problem(422, "Invalid quantity", "quantity must be an integer from 1 to 6.")

    order_id = f"ord_{len(orders) + 1:04d}"
    orders[order_id] = {
        "id": order_id,
        "event": body.get("event", "jazz-night"),
        "quantity": quantity,
        "total": {"amount": PRICE["amount"] * quantity, "currency": "INR"},
        "status": "confirmed",
    }
    idempotency[key] = (fingerprint, 201, orders[order_id])
    resp = jsonify(orders[order_id])
    resp.status_code = 201
    resp.headers["Location"] = f"/orders/{order_id}"
    return resp


@app.get("/orders/<order_id>")
def get_order(order_id):
    order = orders.get(order_id)
    if order is None:
        return problem(404, "Order not found", f"There is no order {order_id}.")
    tag = etag_of(order)
    if request.headers.get("If-None-Match") == tag:
        resp = Response(status=304)
        resp.headers["ETag"] = tag
        return resp
    resp = jsonify(order)
    resp.headers["ETag"] = tag
    return resp


@app.patch("/orders/<order_id>")
def change_order(order_id):
    order = orders.get(order_id)
    if order is None:
        return problem(404, "Order not found", f"There is no order {order_id}.")
    match = request.headers.get("If-Match")
    if match is None:
        return problem(428, "Precondition required", "Send If-Match with the ETag you last read.")
    if match != etag_of(order):
        return problem(412, "Stale version", "The order changed since you read it. Fetch it again.")
    quantity = (request.get_json(silent=True) or {}).get("quantity")
    if not isinstance(quantity, int) or not 1 <= quantity <= 6:
        return problem(422, "Invalid quantity", "quantity must be an integer from 1 to 6.")
    order["quantity"] = quantity
    order["total"] = {"amount": PRICE["amount"] * quantity, "currency": "INR"}
    resp = jsonify(order)
    resp.headers["ETag"] = etag_of(order)
    return resp


if __name__ == "__main__":
    app.run(port=5104)

idem.sh

curl -s -i -X POST http://localhost:5104/orders \
  -H "Content-Type: application/json" -H "Idempotency-Key: 7c1e-checkout-81" \
  -d '{"event": "jazz-night", "quantity": 2}'

# the client timed out and retries with the same key
curl -s -i -X POST http://localhost:5104/orders \
  -H "Content-Type: application/json" -H "Idempotency-Key: 7c1e-checkout-81" \
  -d '{"event": "jazz-night", "quantity": 2}'

# same key, different body
curl -s -i -X POST http://localhost:5104/orders \
  -H "Content-Type: application/json" -H "Idempotency-Key: 7c1e-checkout-81" \
  -d '{"event": "jazz-night", "quantity": 5}'

Output

HTTP/1.1 201 CREATED
Content-Type: application/json
Content-Length: 150
Location: /orders/ord_0001

{
  "id": "ord_0001",
  "event": "jazz-night",
  "quantity": 2,
  "total": {
    "amount": 3600,
    "currency": "INR"
  },
  "status": "confirmed"
}
HTTP/1.1 201 CREATED
Content-Type: application/json
Content-Length: 150
Idempotent-Replayed: true

{
  "id": "ord_0001",
  "event": "jazz-night",
  "quantity": 2,
  "total": {
    "amount": 3600,
    "currency": "INR"
  },
  "status": "confirmed"
}
HTTP/1.1 422 UNPROCESSABLE ENTITY
Content-Type: application/problem+json
Content-Length: 146

{
  "type": "about:blank",
  "title": "Key reused",
  "status": 422,
  "detail": "This Idempotency-Key was used with a different request body."
}

The retry returned the same order id and status as the original, plus a custom Idempotent-Replayed: true header so the client (and your logs) can tell a replay from a first execution. The third request, same key but a different quantity, got a 422 in problem+json form.

Conditional requests: caching and lost updates

An ETag is an identifier for a specific version of a resource; here it is a hash of the order's JSON. Conditional headers use it in two directions. If-None-Match on a GET means "send the body only if it changed", and a matching tag earns a tiny 304 Not Modified. If-Match on a PATCH or PUT means "apply this only if the resource is still the version I read", which prevents the lost-update problem where two clients overwrite each other.

cond.sh

curl -s -I http://localhost:5104/orders/ord_0001

curl -s -i http://localhost:5104/orders/ord_0001 -H 'If-None-Match: "b0f35222114e"'

curl -s -i -X PATCH http://localhost:5104/orders/ord_0001 \
  -H "Content-Type: application/json" -d '{"quantity": 3}'

# headers only, to keep the output short
curl -s -o /dev/null -D - -X PATCH http://localhost:5104/orders/ord_0001 \
  -H "Content-Type: application/json" -H 'If-Match: "b0f35222114e"' -d '{"quantity": 3}'

# a second client still holds the old ETag
curl -s -i -X PATCH http://localhost:5104/orders/ord_0001 \
  -H "Content-Type: application/json" -H 'If-Match: "b0f35222114e"' -d '{"quantity": 4}'

Output

HTTP/1.1 200 OK
Content-Type: application/json
Content-Length: 150
ETag: "b0f35222114e"

HTTP/1.1 304 NOT MODIFIED
ETag: "b0f35222114e"

HTTP/1.1 428 PRECONDITION REQUIRED
Content-Type: application/problem+json
Content-Length: 139

{
  "type": "about:blank",
  "title": "Precondition required",
  "status": 428,
  "detail": "Send If-Match with the ETag you last read."
}
HTTP/1.1 200 OK
Content-Type: application/json
Content-Length: 150
ETag: "477a63443945"

HTTP/1.1 412 PRECONDITION FAILED
Content-Type: application/problem+json
Content-Length: 141

{
  "type": "about:blank",
  "title": "Stale version",
  "status": 412,
  "detail": "The order changed since you read it. Fetch it again."
}

Without If-Match the server demanded one with 428 Precondition Required. With the right tag the update succeeded and returned a new ETag. The second client, still holding the old tag, got 412 Precondition Failed and must refetch before trying again. That is optimistic concurrency, with no locks held between requests.

Headers worth knowing

Header	Direction	What it does
`Location`	Response	URL of a newly created resource (with 201)
`Allow`	Response	Methods a path accepts (with 405)
`Retry-After`	Response	Seconds or a date to wait; used with 429 and 503 (see Rate Limiting)
`ETag` / `If-None-Match`	Both	Cache validation, 304 responses
`If-Match`	Request	Optimistic concurrency, 412 on mismatch
`Cache-Control`	Response	How long and by whom a response may be cached
`Authorization`	Request	Credentials; failures return 401 (see JWT Authentication)
`Idempotency-Key`	Request	Makes POST safe to retry (a convention)

Documenting the contract with OpenAPI, and checking it

An OpenAPI document is a machine-readable description of your API: paths, methods, parameters, request and response schemas. From it, tools generate interactive documentation, client libraries and mock servers. It is only valuable if it stays true, so treat it as a contract and test against it. Below is a trimmed spec for the ticket service. The schemas are written once under components and referenced with $ref.

openapi.yaml

openapi: 3.1.0
info:
  title: Ticket Orders API
  version: 1.0.0
paths:
  /orders:
    post:
      summary: Create an order
      parameters:
        - name: Idempotency-Key
          in: header
          required: true
          schema: { type: string, minLength: 8 }
      requestBody:
        required: true
        content:
          application/json:
            schema: { $ref: "#/components/schemas/NewOrder" }
      responses:
        "201":
          description: Order created (or replayed for a repeated key)
          content:
            application/json:
              schema: { $ref: "#/components/schemas/Order" }
        "422":
          description: Invalid quantity, or the key was reused with another body
          content:
            application/problem+json:
              schema: { $ref: "#/components/schemas/Problem" }
  /orders/{orderId}:
    get:
      summary: Read an order
      parameters:
        - name: orderId
          in: path
          required: true
          schema: { type: string }
      responses:
        "200":
          description: The order, with an ETag header
          content:
            application/json:
              schema: { $ref: "#/components/schemas/Order" }
        "404":
          description: No such order
          content:
            application/problem+json:
              schema: { $ref: "#/components/schemas/Problem" }
components:
  schemas:
    NewOrder:
      type: object
      required: [quantity]
      properties:
        event: { type: string }
        quantity: { type: integer, minimum: 1, maximum: 6 }
    Money:
      type: object
      required: [amount, currency]
      properties:
        amount: { type: integer, description: "Minor units, e.g. paise" }
        currency: { type: string, pattern: "^[A-Z]{3}$" }
    Order:
      type: object
      required: [id, event, quantity, total, status]
      properties:
        id: { type: string }
        event: { type: string }
        quantity: { type: integer }
        total: { $ref: "#/components/schemas/Money" }
        status: { type: string, enum: [confirmed, cancelled] }
    Problem:
      type: object
      required: [title, status]
      properties:
        type: { type: string }
        title: { type: string }
        status: { type: integer }
        detail: { type: string }

OpenAPI 3.1 schema objects are JSON Schema, so the jsonschema package can validate real responses against them. This script makes real calls to the Flask app through its test client and checks the bodies against the Order and Problem schemas. It then removes a required field from a payload to show what drift looks like.

contract.py

import yaml
from jsonschema import Draft202012Validator
from tickets import app

spec = yaml.safe_load(open("openapi.yaml", encoding="utf-8"))
print("loaded", spec["info"]["title"], "with", len(spec["paths"]), "paths")


def check(response, schema_name):
    # Wrap the named schema so "#/components/..." references resolve.
    schema = {"$ref": f"#/components/schemas/{schema_name}", "components": spec["components"]}
    errors = sorted(Draft202012Validator(schema).iter_errors(response.get_json()), key=str)
    print(f"{response.status_code} against {schema_name}:", "OK" if not errors else errors[0].message)


client = app.test_client()
good = client.post("/orders", json={"quantity": 2}, headers={"Idempotency-Key": "contract-test-1"})
check(good, "Order")
check(client.get("/orders/ord_9999"), "Problem")

# Simulate drift: the server forgets the required "status" field.
drifted = good.get_json()
del drifted["status"]
errors = list(Draft202012Validator(
    {"$ref": "#/components/schemas/Order", "components": spec["components"]}).iter_errors(drifted))
print("drifted payload:", errors[0].message)

Output

loaded Ticket Orders API with 2 paths
201 against Order: OK
404 against Problem: OK
drifted payload: 'status' is a required property

Run something like this in CI and the document cannot silently rot. There are two workflows: design-first (write the spec, review it with consumers, then implement) and code-first (generate the spec from the code, as FastAPI does automatically). Either is fine; what is not fine is a hand-written spec that nobody checks.

Common mistakes and the symptoms they produce

Different error shapes per endpoint. Symptom: client code with a dozen parsers, and a mobile crash whenever a new endpoint returns an unexpected format.
Leaking internals in error bodies. Symptom: stack traces, SQL fragments or file paths visible in responses, which hands an attacker a map of your system. Log those server-side and return a generic detail with a request id.
Retrying non-idempotent requests without a key. Symptom: duplicate orders, double emails and double charges after timeouts.
Reusing one idempotency key for different operations. Symptom: a user's second purchase silently returns the first purchase's response. Generate a new key per user intent.
Ignoring `If-Match` on updates. Symptom: "last write wins" overwrites, such as a seat count that mysteriously reverts when two agents edit the same order.
Letting the OpenAPI file drift. Symptom: generated client libraries fail on real responses and consumers stop trusting the docs.

How this shows up in real projects and interviews

Payment and order APIs treat idempotency as mandatory, and "how do you stop a retry from charging twice?" is a standard interview question; a complete answer covers client-generated keys, atomic storage of the key with the result, what to do when the body differs, and expiry. Conditional requests come up in discussions of caching and of collaborative editing. OpenAPI is the common language between backend, frontend and QA teams, and knowing how to generate it, lint it and test against it is a practical skill listed in many job descriptions you will find on the jobs board. With this lesson you have the complete path from resource design to a documented, retry-safe API; next, protect it with JWTs.

Try it yourself

Add PATCH /orders/{orderId} to openapi.yaml, including the If-Match header parameter and the 200, 412 and 428 responses (both error ones using the Problem schema). Then extend contract.py with a conditional PATCH through the test client: read the order's ETag, patch it with that tag, and validate the response against Order. Finally, call PATCH without the header and validate the 428 body against Problem.

Show a hint

With Flask's test client, client.get("/orders/ord_0001").headers["ETag"] gives the tag, and you pass it with headers={"If-Match": tag}. Remember the order must exist, so create it first with an Idempotency-Key header.

Key takeaways

Use one error shape everywhere; application/problem+json (RFC 9457) is a standard to adopt, with type, title, status and detail.
Make POST retry-safe with a client-generated Idempotency-Key; replay the saved response for the same key and body, and refuse the same key with a different body.
Use ETag with If-None-Match (304 for caching) and If-Match (412 or 428 to prevent lost updates).
Store idempotency keys atomically in a real datastore, record the response with the work, and expire keys on a schedule.
Treat the OpenAPI document as a contract and test real responses against it in CI.

Test yourself

A trick from this lesson's ride: Media type recipe: application/ + what it is + "+json" to say how to parse it. Four more tricks like it wait along the route, each followed by a question.

Ride this lesson

Frequently asked questions

What is an idempotency key in a REST API?

It is a unique value, usually sent in an Idempotency-Key header, that a client generates for one logical operation. The server stores the result under that key, so if the same request is retried after a timeout it returns the saved response instead of performing the action twice. It is mainly used to make POST requests such as payments and orders safe to retry.

What is the standard format for API error responses?

RFC 9457, Problem Details for HTTP APIs, defines a JSON format served as application/problem+json with the members type, title, status, detail and instance. You can add extension members, such as a list of field errors. Using it means clients can handle every failure with one parser.

What is OpenAPI and why should I use it?

OpenAPI is a specification for describing HTTP APIs in YAML or JSON: paths, methods, parameters and request and response schemas. Tools can turn it into interactive documentation, client SDKs, mock servers and automated contract tests. It is most valuable when you verify the real server against it, so the documentation cannot drift from the behaviour.