```meta
title: Checkout API latency — where one request spends its time
subtitle: Trace of one p95 POST /checkout request from 11 Sep 2026, stage by stage, against the same stages one week earlier.
tag: INCIDENT
```

p95 latency of `POST /checkout` rose from 590 ms to 1,340 ms between 4 Sep and 11 Sep 2026. One stage explains the change: the inventory reservation, which took 96 ms last week and now takes 820 ms. Every other stage moved by less than 10 ms.

```callout
tone: note
title: Assumptions
body: "The trace below is one real p95 request captured on 11 Sep 2026 (6 cart lines, one shipping address). Last-week numbers are the p95 of the same stages from 4 Sep. The inventory service shipped release 2026.09.08 on 8 Sep, which replaced one batch reservation query with one row lock per cart line. We take that release as the cause. The inventory team did not confirm it yet."
```

## One request, stage by stage

The checkout handler runs its stages in series. Inventory holds the request for 820 of the 1,340 ms, and inside inventory six `SELECT … FOR UPDATE` statements run one after the other, about 125 ms each. Tax, payments, and the order write together take 440 ms and did not change.

```spans
id: checkout-trace
title: POST /checkout · trace 7f3a91 · 11 Sep 2026
unit: ms
spans:
  - checkout-api/checkout: POST /checkout · 0 · 1340
  - checkout-api/auth: verify session token · 2 · 9 · checkout
  - { id: cart, service: cart-cache, name: "GET cart:c_81f2", start: 12, duration: 4, parent: checkout, kind: cache }
  - { id: validate, service: checkout-api, name: validate cart lines, start: 17, duration: 6, parent: checkout, kind: internal }
  - { id: reserve, service: inventory, name: POST /reservations, start: 24, duration: 820, parent: checkout, kind: client, note: "6 cart lines, locked one at a time" }
  - { id: lock1, service: inventory-db, name: "SELECT stock FOR UPDATE · sku 1", start: 30, duration: 125, parent: reserve, kind: db }
  - { id: lock2, service: inventory-db, name: "SELECT stock FOR UPDATE · sku 2", start: 158, duration: 125, parent: reserve, kind: db }
  - { id: lock3, service: inventory-db, name: "SELECT stock FOR UPDATE · sku 3", start: 286, duration: 125, parent: reserve, kind: db }
  - { id: lock4, service: inventory-db, name: "SELECT stock FOR UPDATE · sku 4", start: 414, duration: 125, parent: reserve, kind: db }
  - { id: lock5, service: inventory-db, name: "SELECT stock FOR UPDATE · sku 5", start: 542, duration: 125, parent: reserve, kind: db }
  - { id: lock6, service: inventory-db, name: "SELECT stock FOR UPDATE · sku 6", start: 670, duration: 125, parent: reserve, kind: db }
  - { id: reserve-insert, service: inventory-db, name: INSERT reservations, start: 798, duration: 40, parent: reserve, kind: db }
  - { id: tax, service: tax, name: POST /tax/quote, start: 848, duration: 210, parent: checkout, kind: client }
  - { id: pay, service: payments, name: POST /authorizations, start: 1062, duration: 190, parent: checkout, kind: client }
  - { id: order-insert, service: orders-db, name: "INSERT orders, order_lines", start: 1256, duration: 38, parent: checkout, kind: db }
  - { id: publish, service: order-bus, name: publish order.placed, start: 1298, duration: 12, parent: checkout, kind: queue }
  - { id: respond, service: checkout-api, name: build 201 response, start: 1314, duration: 22, parent: checkout, kind: internal }
```

## Which calls wait on which

Nothing in the handler runs in parallel. The tax quote waits for the reservation, but it needs only the cart and the address. The payment authorization waits for the tax quote because it needs the final amount. The row locks inside inventory are the only loop in the request. The 4 ms cart-cache read is left out of the diagram.

```sequence
id: checkout-calls
endpoint: { method: POST, path: /checkout, status: 201 }
actors:
  - { id: Client, name: Web client, external: true }
  - { id: API, name: Checkout API }
  - { id: Inv, name: Inventory }
  - { id: InvDB, name: Inventory DB }
  - { id: Tax, name: Tax }
  - { id: Pay, name: Payments }
  - { id: OrdDB, name: Orders DB }
  - { id: Bus, name: Order bus }
messages:
  - Client -> +API: POST /checkout
  - API -> +Inv: POST /reservations
  - loop: per cart line
  - Inv -> InvDB: SELECT stock FOR UPDATE
  - InvDB --> Inv: row locked
  - end
  - Inv -> InvDB: INSERT reservations
  - Inv --> -API: 201 reservation
  - API -> Tax: POST /tax/quote
  - Tax --> API: quote
  - API -> Pay: POST /authorizations
  - Pay --> API: authorized
  - API -> OrdDB: INSERT orders, order_lines
  - API -> Bus: publish order.placed
  - API --> -Client: 201 Created
foot:
  - { label: Serial stages, value: "7" }
  - { label: Parallel stages, value: "0" }
```

## Each stage, last week and this week

```slopegraph
id: checkout-stage-shift
title: p95 per stage, 4 Sep vs 11 Sep 2026
left: 4 Sep
right: 11 Sep
unit: ms
items:
  - { label: Inventory reservation, from: 96, to: 820, accent: red }
  - { label: Tax quote, from: 205, to: 210 }
  - { label: Payment authorization, from: 180, to: 190 }
  - { label: Order write, from: 35, to: 38 }
  - { label: Event publish + response, from: 30, to: 34 }
  - { label: Auth + cart load, from: 14, to: 15 }
```

The reservation stage moved by 724 ms; all other stages together moved by 23 ms. Tax and payments were slow before last week and are still slow. They set the floor the API can reach after the inventory fix: about 590 ms at p95. Reverting the inventory release, or restoring the batch lock, removes the regression. Running the tax quote in parallel with the reservation would take a further 210 ms off the critical path, but that is a separate change.