Ticket-routing model — handoff card

```meta
title: Ticket-routing model — handoff
subtitle: What ticket-router-v4 does, where it is measured to work, and the conditions under which another team can reuse it.
tag: HANDOFF
```

```callout
tone: note
title: Assumptions
body: "The model is ticket-router-v4, owned by Support Platform, serving the English and Spanish support inbox. The receiving team routes a different inbox with its own queue set. Numbers below come from the 2026-08 evaluation run. Replace any of these if your setup differs."
```

## The card

The model assigns one of 12 support queues to an inbound ticket from its subject and first message. It runs on CPU inside the `ticket-router` service and returns a queue plus a confidence score. Reuse means deploying the same weights against your inbox, not retraining; retraining is a separate project.

```modelcard
name: ticket-router-v4
version: 4.1.2
task: Text classification (support ticket to queue)
architecture: XLM-RoBERTa base fine-tune, 12 layers, classification head over 12 queues
params: 278M
owner: Support Platform
license: Internal
intendedUse:
  - Route an inbound ticket to one of 12 support queues at intake
  - Suggest a queue to a triage agent when confidence is below the auto-route threshold
outOfScope:
  - Any language other than English and Spanish
  - Tickets with an attachment as the only content
  - Priority or severity scoring
  - Any customer-facing decision (refund, closure, escalation)
trainingData:
  - 620k tickets from the support inbox, 2024-03 to 2026-05, PII scrubbed, queue label from the final human assignment
  - 14k held-out tickets from 2026-06 as the test split, same label source
metrics:
  - { name: Macro F1, value: 0.89, split: test }
  - { name: Accuracy, value: 0.93, split: test }
  - { name: Auto-route coverage, value: 71%, split: prod, note: "share of tickets at confidence >= 0.85" }
  - { name: Auto-route accuracy, value: 0.97, split: prod, note: "measured on the 71% auto-routed" }
  - { name: Latency p95, value: 62 ms, split: prod, note: "CPU, batch 1, 4 vCPU" }
limitations:
  - Confuses Billing and Refunds on tickets under 20 words
  - Accuracy drops to 0.81 on tickets forwarded from a partner mailbox because the signature dominates the text
  - Trained on the queue set of one inbox; a new queue needs a retrain, not a config change
ethics:
  - Input text may contain PII; the service logs only the queue and the score, never the text
```

## Where it works and where it does not

Per-queue numbers decide reuse more than the macro score. Three queues carry most of the errors, and all three exist because the human labels themselves disagree on the boundary. If your inbox has the same boundary, expect the same errors.

```table
title: Per-queue performance on the 2026-06 test split
columns: [Queue, Share of tickets, F1, Most common confusion]
rows:
  - [Billing, 18%, { v: 0.84, tone: warn }, Refunds]
  - [Refunds, 9%, { v: 0.82, tone: warn }, Billing]
  - [Login and access, 16%, { v: 0.95, tone: pos }, Account changes]
  - [Account changes, 8%, { v: 0.88, tone: warn }, Login and access]
  - [Shipping, 14%, { v: 0.94, tone: pos }, Order status]
  - [Order status, 11%, { v: 0.91, tone: pos }, Shipping]
  - [Product defect, 7%, { v: 0.92, tone: pos }, Returns]
  - [Returns, 6%, { v: 0.90, tone: pos }, Product defect]
  - [Partner and reseller, 3%, { v: 0.81, tone: neg }, Billing]
  - [Legal and privacy, 2%, { v: 0.93, tone: pos }, Account changes]
  - [Feedback, 4%, { v: 0.87, tone: warn }, Product defect]
  - [Other, 2%, { v: 0.76, tone: neg }, Feedback]
note: "F1 per queue; share is the fraction of test tickets with that label."
```

## How a ticket moves through the router

The confidence threshold is the only tunable that changes behaviour without a retrain. Lowering it raises coverage and lowers auto-route accuracy; the 0.85 setting was chosen so that auto-routed errors stay under the triage team's 3% re-route budget. Tickets the model cannot score at all go to triage with no suggestion, and that path fires on about 0.4% of traffic.

```flow
title: Intake path for one ticket
nodes:
  - { id: intake, col: 1, row: 1, kind: start, label: Ticket arrives }
  - { id: lang, col: 2, row: 1, kind: decision, label: "Language is EN or ES?" }
  - { id: score, col: 3, row: 1, kind: process, label: Score 12 queues }
  - { id: conf, col: 4, row: 1, kind: decision, label: "Confidence >= 0.85?" }
  - { id: auto, col: 5, row: 1, kind: end, label: Auto-route to queue }
  - { id: suggest, col: 5, row: 2, kind: end, label: "Triage with suggested queue" }
  - { id: triage, col: 2, row: 2, kind: end, label: "Triage with no suggestion" }
edges:
  - intake -> lang
  - lang -> score: "yes"
  - lang -x-> triage: "no"
  - score -> conf
  - conf -> auto: "yes"
  - conf --> suggest: "no"
```

## Reuse test

Run this list against your inbox before you commit. Two or more fails mean the model is not a fit as shipped. A retrain on your labels is the smaller project only if you already hold six months of human-assigned queue labels.

```checklist
title: Fit test for a new inbox
standard: Support Platform model reuse v2
groups:
  - label: Input
    items:
      - "[pending] 90% or more of tickets are in English or Spanish — measure on one week of traffic"
      - "[pending] Tickets carry text in the subject or first message, not only attachments — measure on one week of traffic"
      - "[pending] Fewer than 10% of tickets arrive via a forwarding mailbox with a long signature — see the partner-mailbox limitation"
  - label: Labels
    items:
      - "[pending] Your queue set maps one-to-one onto the 12 queues above — a queue with no counterpart forces a retrain"
      - "[pending] Your Billing and Refunds boundary matches the card's definition — else expect F1 near 0.83 on both"
  - label: Operations
    items:
      - "[pending] A triage team exists to receive the 29% of tickets below the threshold"
      - "[pending] Serving budget allows 62 ms p95 on 4 vCPU per replica, or a GPU"
      - "[pending] Ticket text must not be logged — the service already complies; confirm your log pipeline does too"
```