Skip to content
chiltepin

Generated from: “Reads outnumber writes fifty to one and the product page query joins nine tables. Document the read-model approach we agreed on.

Read model for the product page

Written by an agent from the skill, validated by chiltepin check, rendered by the renderer — shown as generated, 13 September 2026.

DOCUMENTDECISION

Read model for the product page

Why the product page reads from a projected document instead of the nine-table join, and how a write reaches it.

The Catalog service serves the product page at about 6,000 reads per second. It takes about 120 writes per second, a ratio near fifty to one. Every page read runs one query that joins nine tables and costs 210 ms at p95 on the primary. The write side needs those nine tables normalized; the read side needs one document per product.

We agreed to split them. Writes stay on the normalized schema. A projector keeps one read document per product, and the page reads it with one lookup.

SECTION 01 · Note

Assumptions

Note
The write store is Postgres 16 with the nine tables listed below. Writes commit through the Catalog API only; no other service writes these tables. The read model lives in a second Postgres database, one row per product with a jsonb document. Events leave the write store through an outbox table and a relay into a Kafka topic named catalog.changes. The page tolerates a read that is up to 5 seconds stale, except for price, which must reach the page within 2 seconds. Numbers come from the production primary over the last 30 days.

The numbers that forced the split

SECTION 02 · Metrics

Product page today

50:1
Reads to writes
— 6,000 rps to 120 wps
9
Tables in the page query
210 ms
Page query p95 on the primary
▼ target 25 ms
38%
Primary CPU spent on page reads

The primary spends more than a third of its CPU on a query whose result changes a few times a day per product. A read replica would move the CPU but not the 210 ms, because the join itself is the cost. The read model removes the join from the read path entirely.

How a write reaches the page

SECTION 03 · Architecture
EVENT
Block diagram: 7 nodes, 7 connectionsCommand sideQuery sideCatalog APIwrites 9 tables +outboxPRODUCERCatalog DBPostgres 16DBOutbox relaypolls every 200 msSVCcatalog.changesKafka, key =product_idTOPICProduct projectorrebuilds one documentper eventCONSUMERProduct page storePostgres, jsonb perproductDBProduct pageserviceone lookup byproduct_idSVC1234567
1commit row + outbox in one transaction2read pending outbox rows3publish change event4consume by product_id5load the nine rows for that product6upsert product document7select document by product_id
LegendPRODUCERproducerDBdatabaseSVCserviceTOPICtopicCONSUMERconsumercallsasync / optionalentry point

The command side never knows the read model exists. The projector is the only writer of the product page store. It always rebuilds the whole document from the nine source rows rather than patching one field. That choice costs one join per write, which is about 120 joins per second instead of 6,000.

SECTION 04 · Sequence

A price change reaches the page

SEQUENCE
Sequence diagram: 14 messages between 8 actorsMerchant adminCatalog APICatalog DBOutbox relaycatalog.changesProduct projectorProduct page storeProduct pageserviceLOOP[every 200 ms]ALT[seq 917 > stored seq][stale or duplicate event]1PATCH /products/p-4821/price2BEGIN; update prices; insert outbox(product_id=p-4821, seq=917)3COMMIT4200 OK5select pending outbox rows6publish PriceChanged(p-4821, seq=917)7mark outbox row published8PriceChanged(p-4821, seq=917)9select the nine rows for p-482110upsert product document p-4821 (seq=917)11skip, commit offset12commit offset13select document where product_id = p-482114document (seq=917)
Legendcallresponsethe answer the caller getsfragment (alt / opt / loop)active
Commit to page p95: 1.4 sCommit to page p99: 3.8 s

Between the COMMIT and the upsert the page serves the previous document. The admin UI reads the write side directly after a save, so the merchant sees the new price at once. Shoppers see it within 2 seconds at p95 and 5 seconds at worst, which is inside the tolerance we set.

What the document holds and what refreshes it

SECTION 05 · Comparison

Nine source tables and the document sections they feed

Source tableDocument sectionRefresh triggerRows per product
productstitle · description · statusProductChanged1
brandsbrandBrandChanged1
product_variantsvariants[]VariantChanged1 to 40
pricesvariants[].price · variants[].compare_atPriceChanged1 per variant
inventory_levelsvariants[].in_stockInventoryChanged1 per variant per warehouse
categoriesbreadcrumbs[]CategoryChanged1 to 4
product_categoriesbreadcrumbs[]ProductCategorized1 to 4
media_assetsimages[]MediaChanged1 to 12
review_summariesrating · review_countReviewSummaryChanged1

A BrandChanged or CategoryChanged event fans out to every product under it; the projector enqueues one rebuild per affected product_id.

Brand and category edits are the fan-out risk. A rename of a category with 40,000 products creates 40,000 rebuilds. The projector drains them at about 800 per second, so the tail of that fan-out is 50 seconds stale. We accept that for breadcrumbs; price never fans out, because a price row belongs to one variant.

Approaches we rejected

SECTION 06 · Options

Three ways to take the join off the read path

Option 1Read replica plus Redis cache
Run the nine-table join on a replica and cache the result in Redis for 60 seconds.
  • No new write path
  • Two weeks of work
  • Every cache miss still pays the 210 ms join
  • A 60-second TTL breaks the 2-second price rule, so prices need a separate invalidation path
  • Cold start after a deploy sends the full miss rate to the replica
REJECTED — fails the price rule and keeps the join
Option 2Postgres materialized view
One materialized view of the join, refreshed concurrently every 2 seconds.
  • No new service
  • One lookup per read
  • REFRESH rebuilds all 2.1 million rows every cycle, about 9 minutes each
  • A refresh cannot finish inside the 2-second price window
  • Refresh load lands on the primary
REJECTED — the refresh is slower than the lag it must beat
Option 3Projected read model
An outbox, a relay, a projector, and a per-product jsonb document in a second database.
  • One lookup per read, 4 ms at p95 in the load test
  • Price reaches the page in 1.4 s at p95
  • The write schema stays normalized
  • The read store can move to another engine without a write-side change
  • One more service and one more topic to operate
  • Eventual consistency that every page caller must accept
  • A full rebuild takes about 45 minutes
CHOSEN — the only option that meets the price rule at fifty to one

Rules the projector must keep

SECTION 07 · Spec

Projector invariants

Source of truth
The nine tables in the Catalog DB. The product page store is a derived copy and can be dropped and rebuilt at any time.
Ordering
Events for one product_id share a Kafka partition. The projector stores the outbox seq with each document and skips any event whose seq is not greater.
Idempotency
Every rebuild reads the current nine rows, so replaying an event produces the same document.
Full rebuild
Pause the projectorTruncate product page storeRun the batch projector over all product_idsResume from the paused offset
Lag alert
Page on consumer lag above 5 seconds for 1 minute, or on any outbox row pending for more than 10 seconds.
Staleness bound
Price within 2 seconds at p95. Every other section within 5 seconds, except brand and category fan-outs, which may take up to 60 seconds.

The seq check is what makes the projector safe under retries and rebalances. Without it, a redelivered event after a newer one would overwrite the document with an older join result.

View the Markdown
```meta
title: Read model for the product page
subtitle: Why the product page reads from a projected document instead of the nine-table join, and how a write reaches it.
tag: DECISION
```

The Catalog service serves the product page at about 6,000 reads per second. It takes about 120 writes per second, a ratio near fifty to one. Every page read runs one query that joins nine tables and costs 210 ms at p95 on the primary. The write side needs those nine tables normalized; the read side needs one document per product.

We agreed to split them. Writes stay on the normalized schema. A projector keeps one read document per product, and the page reads it with one lookup.

```callout
tone: note
title: Assumptions
body: "The write store is Postgres 16 with the nine tables listed below. Writes commit through the Catalog API only; no other service writes these tables. The read model lives in a second Postgres database, one row per product with a jsonb document. Events leave the write store through an outbox table and a relay into a Kafka topic named catalog.changes. The page tolerates a read that is up to 5 seconds stale, except for price, which must reach the page within 2 seconds. Numbers come from the production primary over the last 30 days."
```

## The numbers that forced the split

```stats
title: Product page today
stats:
  - { value: "50:1", label: Reads to writes, delta: "6,000 rps to 120 wps", trend: flat }
  - { value: "9", label: Tables in the page query, trend: flat }
  - { value: "210 ms", label: Page query p95 on the primary, delta: "target 25 ms", trend: down }
  - { value: "38%", label: Primary CPU spent on page reads, trend: down }
```

The primary spends more than a third of its CPU on a query whose result changes a few times a day per product. A read replica would move the CPU but not the 210 ms, because the join itself is the cost. The read model removes the join from the read path entirely.

## How a write reaches the page

```block
id: topology
preset: event
dir: LR
groups:
  - { id: cmd, col: 1, row: 1, cols: 2, rows: 2, label: Command side }
  - { id: qry, col: 4, row: 1, cols: 2, rows: 2, label: Query side }
nodes:
  - { id: api, col: 1, row: 1, kind: producer, name: Catalog API, tech: writes 9 tables + outbox }
  - { id: pg, col: 2, row: 1, kind: store, name: Catalog DB, tech: Postgres 16 }
  - { id: relay, col: 2, row: 2, kind: service, name: Outbox relay, tech: polls every 200 ms }
  - { id: topic, col: 3, row: 2, kind: topic, name: catalog.changes, tech: "Kafka, key = product_id" }
  - { id: proj, col: 4, row: 2, kind: consumer, name: Product projector, tech: rebuilds one document per event }
  - { id: read, col: 4, row: 1, kind: store, name: Product page store, tech: "Postgres, jsonb per product" }
  - { id: page, col: 5, row: 1, kind: service, name: Product page service, tech: one lookup by product_id }
edges:
  - api -> pg: commit row + outbox in one transaction
  - relay -> pg: read pending outbox rows
  - relay -> topic: publish change event
  - topic --> proj: consume by product_id
  - proj -> pg: load the nine rows for that product
  - proj -> read: upsert product document
  - page -> read: select document by product_id
```

The command side never knows the read model exists. The projector is the only writer of the product page store. It always rebuilds the whole document from the nine source rows rather than patching one field. That choice costs one join per write, which is about 120 joins per second instead of 6,000.

```sequence
id: price-change
title: A price change reaches the page
actors:
  - { id: Admin, name: Merchant admin }
  - { id: API, name: Catalog API }
  - { id: DB, name: Catalog DB }
  - { id: Relay, name: Outbox relay }
  - { id: Topic, name: catalog.changes }
  - { id: Proj, name: Product projector }
  - { id: Read, name: Product page store }
  - { id: Page, name: Product page service }
messages:
  - Admin -> +API: PATCH /products/p-4821/price
  - API -> +DB: "BEGIN; update prices; insert outbox(product_id=p-4821, seq=917)"
  - DB --> -API: COMMIT
  - API --> -Admin: 200 OK
  - loop: every 200 ms
  - Relay -> DB: select pending outbox rows
  - Relay -> Topic: publish PriceChanged(p-4821, seq=917)
  - Relay -> DB: mark outbox row published
  - end
  - Topic --> +Proj: PriceChanged(p-4821, seq=917)
  - Proj -> DB: select the nine rows for p-4821
  - alt: seq 917 > stored seq
  - Proj -> Read: upsert product document p-4821 (seq=917)
  - else: stale or duplicate event
  - Proj -> Proj: skip, commit offset
  - end
  - Proj --> -Topic: commit offset
  - Page -> Read: select document where product_id = p-4821
  - Read --> Page: document (seq=917)
foot:
  - { label: Commit to page p95, value: 1.4 s }
  - { label: Commit to page p99, value: 3.8 s }
```

Between the COMMIT and the upsert the page serves the previous document. The admin UI reads the write side directly after a save, so the merchant sees the new price at once. Shoppers see it within 2 seconds at p95 and 5 seconds at worst, which is inside the tolerance we set.

## What the document holds and what refreshes it

```table
id: sources
title: Nine source tables and the document sections they feed
columns: [Source table, Document section, Refresh trigger, Rows per product]
rows:
  - [products, title · description · status, ProductChanged, "1"]
  - [brands, brand, BrandChanged, "1"]
  - [product_variants, "variants[]", VariantChanged, "1 to 40"]
  - [prices, "variants[].price · variants[].compare_at", PriceChanged, "1 per variant"]
  - [inventory_levels, "variants[].in_stock", InventoryChanged, "1 per variant per warehouse"]
  - [categories, "breadcrumbs[]", CategoryChanged, "1 to 4"]
  - [product_categories, "breadcrumbs[]", ProductCategorized, "1 to 4"]
  - [media_assets, "images[]", MediaChanged, "1 to 12"]
  - [review_summaries, rating · review_count, ReviewSummaryChanged, "1"]
note: A BrandChanged or CategoryChanged event fans out to every product under it; the projector enqueues one rebuild per affected product_id.
```

Brand and category edits are the fan-out risk. A rename of a category with 40,000 products creates 40,000 rebuilds. The projector drains them at about 800 per second, so the tail of that fan-out is 50 seconds stale. We accept that for breadcrumbs; price never fans out, because a price row belongs to one variant.

## Approaches we rejected

```options
title: Three ways to take the join off the read path
items:
  - { kicker: Option 1, title: Read replica plus Redis cache, how: "Run the nine-table join on a replica and cache the result in Redis for 60 seconds.", pros: ["No new write path", "Two weeks of work"], cons: ["Every cache miss still pays the 210 ms join", "A 60-second TTL breaks the 2-second price rule, so prices need a separate invalidation path", "Cold start after a deploy sends the full miss rate to the replica"], verdict: "REJECTED — fails the price rule and keeps the join", tone: rejected }
  - { kicker: Option 2, title: Postgres materialized view, how: "One materialized view of the join, refreshed concurrently every 2 seconds.", pros: ["No new service", "One lookup per read"], cons: ["REFRESH rebuilds all 2.1 million rows every cycle, about 9 minutes each", "A refresh cannot finish inside the 2-second price window", "Refresh load lands on the primary"], verdict: "REJECTED — the refresh is slower than the lag it must beat", tone: rejected }
  - { kicker: Option 3, title: Projected read model, how: "An outbox, a relay, a projector, and a per-product jsonb document in a second database.", pros: ["One lookup per read, 4 ms at p95 in the load test", "Price reaches the page in 1.4 s at p95", "The write schema stays normalized", "The read store can move to another engine without a write-side change"], cons: ["One more service and one more topic to operate", "Eventual consistency that every page caller must accept", "A full rebuild takes about 45 minutes"], verdict: "CHOSEN — the only option that meets the price rule at fifty to one", tone: chosen }
```

## Rules the projector must keep

```spec
title: Projector invariants
accent: teal
rows:
  - { label: Source of truth, value: "The nine tables in the Catalog DB. The product page store is a derived copy and can be dropped and rebuilt at any time." }
  - { label: Ordering, value: "Events for one product_id share a Kafka partition. The projector stores the outbox seq with each document and skips any event whose seq is not greater." }
  - { label: Idempotency, value: "Every rebuild reads the current nine rows, so replaying an event produces the same document." }
  - { label: Full rebuild, steps: [Pause the projector, Truncate product page store, Run the batch projector over all product_ids, Resume from the paused offset] }
  - { label: Lag alert, value: "Page on consumer lag above 5 seconds for 1 minute, or on any outbox row pending for more than 10 seconds." }
  - { label: Staleness bound, value: "Price within 2 seconds at p95. Every other section within 5 seconds, except brand and category fan-outs, which may take up to 60 seconds." }
```

The seq check is what makes the projector safe under retries and rebalances. Without it, a redelivered event after a newer one would overwrite the document with an older join result.