Production GenUI Playbook

Why this chapter matters

Most AI demos fail the moment they meet production constraints: auth, latency, cost, accessibility, audit, prompt injection, stale data, mobile networks, flaky tools, and skeptical users. A frontend architect's job is to make GenUI boring enough to operate.

This playbook turns the previous chapters into a release checklist.

Production readiness model

GenUI is ready for production only when it has controls in six areas:

Area	Required capability
Experience	Clear user intent, progress, correction, approval, and fallback
Protocol	Typed events, generated UI schema, cancellation, resume, terminal states
Security	Permission checks, prompt-injection defenses, component allowlists, audit
Quality	Evals, regression cases, groundedness tests, accessibility checks
Operations	Traces, metrics, alerts, cost budgets, kill switches
Governance	Versioned prompts, model/provider routing, review process, incident playbooks

The linked UX and research sources add one more operating rule: do not ship GenUI only because the generated surface is rich. Ship it because it improves a measurable outcome compared with the deterministic baseline, and because users can recover when generation is wrong.

MVP scope decision

Start narrow. A good first production GenUI feature usually has:

• one workflow
• one user role
• one data domain
• five to ten approved components
• a small number of tools
• clear human approval boundary
• measurable task outcome
• deterministic fallback

Avoid starting with "an assistant that can do everything." That creates unclear evals, unclear permissions, and unclear product value.

Release architecture

1. Define the workflow contract

Document:

• user intent
• allowed inputs
• required context
• tool capabilities
• generated UI components
• approval gates
• terminal outcomes
• failure states
• audit requirements
• success metrics

2. Build the event contract

Minimum event set:

Event	Purpose
workflow.started	Create visible task state
workflow.progress	Explain stage-level progress
retrieval.completed	Show context quality and sources
tool.started	Show external work
tool.completed	Update progress and evidence
component.proposed	Render generated UI after validation
approval.requested	Require user decision
workflow.failed	Preserve state and recover
workflow.completed	Commit outcome and next actions
workflow.cancelled	Stop UI and backend work

3. Build the component registry

Start with fewer components than you think you need. Each component should have:

• versioned schema
• prop validation
• design-system mapping
• accessibility contract
• analytics naming
• permission constraints
• loading and error states
• snapshot tests
• fallback rendering

4. Build evals before broad rollout

Evaluation must cover:

• expected happy paths
• ambiguous user requests
• insufficient context
• stale data
• unauthorized data
• prompt-injection attempts
• invalid generated UI
• tool failure
• cancellation
• mobile/slow-network behavior

Add UX-specific evals:

• deterministic baseline comparison
• messy prompt variation
• generated UI repair effort
• rejection reasons
• time to first useful state
• accessibility of streamed/generated states
• user trust in sourced vs unsourced generated content

Metrics

Product metrics

• task completion rate
• time to completed task
• time to first useful generated state
• user correction rate
• approval rate vs rejection rate
• repeated use rate
• escalation/support rate
• repair effort per generated artifact

AI quality metrics

• grounded answer rate
• citation coverage
• invalid schema rate
• tool success/failure rate
• hallucinated action rate
• eval pass rate by scenario
• generated UI intent-fit score

Frontend metrics

• time to first useful event
• time to first generated component
• generated component validation failures
• INP during streaming/rendering
• client error rate by component type
• cancellation rate

Operations metrics

• cost per successful task
• tokens per workflow
• model latency
• retrieval latency
• tool latency
• workflow timeout rate
• provider error rate

Security and abuse review

Review these before launch:

Risk	Control
Prompt injection in retrieved content	Treat retrieved text as untrusted, isolate instructions, test malicious sources
Generated UI injection	Component allowlist, schema validation, no arbitrary HTML/script
Unauthorized retrieval	Metadata filters before context assembly
Tool misuse	Least privilege, side-effect classes, approval gates
Data leakage in logs	Redaction and source-id logging
Overbroad agent permissions	Workflow-scoped capability exposure
Misleading generated facts	Citations, uncertainty states, groundedness evals

Accessibility review

Generated UI must pass the same bar as normal UI:

• keyboard navigation through streamed components
• focus placement when new approval panels appear
• live-region behavior for progress updates
• labels for generated controls
• no visual-only confidence indicators
• reduced-motion support for streaming or animated progress
• screen-reader access to citations, tables, and generated drafts

Cost controls

Cost is architecture:

• cancel backend work when the user cancels
• set max steps per workflow
• set token and tool budgets
• use smaller models for classification/routing where acceptable
• cache safe retrieval and deterministic tool results
• sample or batch expensive evals where appropriate
• track cost per successful task, not only total spend

Rollout plan

1. Internal dogfood with trace inspection.
2. Limited user cohort with manual review of failures.
3. Shadow evals against real user prompts.
4. Feature flag by tenant, role, and workflow.
5. Gradual rollout with cost and quality alerts.
6. Post-launch review of corrections, rejects, and support tickets.
7. Expand component registry only after the first workflow is stable.

Pre-launch readiness review

Use this as a go/no-go checklist.

Area	Required evidence
Product value	Deterministic baseline and expected GenUI improvement are documented
Workflow	State diagram includes pause, cancel, fail, resume, and terminal states
UI registry	Components have schemas, fallbacks, accessibility contracts, and tests
Retrieval	Source metadata, permission filters, citations, and freshness rules exist
Tools	Side-effect classes, approval gates, idempotency, and audit logs exist
Security	Prompt injection, generated UI injection, and data-leak tests exist
Evals	Happy path, messy prompt, malicious input, invalid UI, and tool failure cases run
Observability	Trace id connects frontend, workflow, model, retriever, and tools
Cost	Budget limits, cancellation, and cost per successful task are measured
Rollback	Feature flags can disable model, retriever, tool, and component independently

If a row has no evidence, the feature is not production-ready. The team can still run a prototype, but it should not be marketed as a reliable product capability.

Evals as release gates

Treat evals like integration tests for probabilistic systems.

Eval suite	Example cases
Intent	vague prompt, contradictory prompt, prompt with missing object
Schema	unknown component, invalid prop type, unsupported version, oversize payload
Retrieval	no source, wrong source, stale source, unauthorized source
Tool	timeout, duplicate retry, permission denied, partial success
UX	user rejects suggestion, edits draft, switches workflow, reopens thread
Safety	prompt injection, generated URL injection, hidden destructive action
Accessibility	generated dialog, generated table, live progress, approval panel

Release gates should be explicit:

• no critical security eval failures
• no unauthorized retrieval leaks
• invalid generated UI rate below threshold
• task completion above deterministic baseline for target workflow
• p75 time to first useful state within budget
• cost per successful task within budget
• rollback tested in staging

Runtime kill-switch matrix

One feature flag is not enough. Production GenUI needs layered disablement.

Switch	What it disables	User fallback
ai.workflow.enabled	Whole AI workflow	Deterministic form/search
ai.model.provider	Specific model/provider	Route to backup or disable AI
ai.retrieval.enabled	RAG context	Ask user to select sources manually
ai.tool.send_email	Risky tool	Keep draft-only mode
genui.component.approval_action.v2	One generated component	Render plain approval fallback
genui.streaming.enabled	Streaming patches	Use final response mode
ai.agent.delegation.enabled	A2A/multi-agent delegation	Single-agent or manual workflow

Kill switches should be documented in the incident playbook and owned by on-call engineers.

Runbook: invalid generated UI spike

1. Detect spike in generated_ui.validation_failed.
2. Break down by component type, version, model, prompt version, and client version.
3. If one component dominates, disable that component version.
4. If one prompt/model dominates, roll back prompt/model routing.
5. If old clients dominate, ship compatibility adapter or stop sending unsupported components.
6. Preserve sample payloads with sensitive fields redacted.
7. Add regression fixtures from the failed payloads.
8. Run evals before re-enabling.

The important behavior is graceful degradation. The user should see a fallback summary or deterministic workflow, not a broken page.

Runbook: hallucinated or unsupported action

1. Disable the affected tool/action capability.
2. Identify whether the model proposed an action, the protocol accepted it, or the backend executed it.
3. Check approval payload hash and audit logs.
4. Reconcile any external side effects.
5. Add eval case for the prompt and context that produced the action.
6. Tighten tool descriptions, prompt constraints, and server-side policy.
7. Review whether the UI made the risk visible enough.

In production, "the model made a mistake" is not a root cause. The root cause is missing boundary control.

Incident playbook

Incident	Immediate action
Data leak suspected	Disable workflow, preserve traces, rotate exposed secrets if needed, audit retrieval/tool logs
Invalid UI flood	Disable generated component type or schema version
Cost spike	Lower step limits, disable expensive tools, route to cheaper model, pause cohort
Tool side-effect bug	Disable tool capability, identify affected workflow ids, reconcile external systems
Low-quality answer regression	Roll back prompt/model/retriever version, run eval diff
Accessibility regression	Disable affected generated component and fall back to deterministic UI

Anti-patterns

• Shipping a general assistant without workflow boundaries.
• Treating evals as optional because manual testing looked good.
• Measuring tokens but not successful task cost.
• Letting prompts change without versioning.
• Adding generated components without accessibility and security contracts.
• Hiding AI failures behind generic "something went wrong" messages.

Verification checklist

• Workflow, event, generated UI, tool, and retrieval contracts are documented.
• The feature has deterministic fallback and kill switches.
• Evals cover quality, security, permissions, invalid UI, and tool failure.
• Observability correlates frontend events with agent traces and model calls.
• Cost, latency, and invalid schema alerts exist.
• Accessibility review includes streamed and generated states.
• Incident response identifies which flags disable which AI capabilities.

Exercises

1. Write a release checklist for a GenUI feature that drafts and sends customer emails.
2. Define a kill-switch matrix for model, retrieval, tool, and generated component failures.
3. Create an eval set for invalid generated UI payloads.
4. Design telemetry that connects user rejection to retrieval quality and generated UI type.

Source lens

This playbook synthesizes the official AG-UI, A2UI, MCP, A2A, LangGraph, CrewAI, Strands, Pinecone, Weaviate, AI SDK, LangSmith, assistant-ui, CopilotKit, NN/g, Google Research, and GenUI research-study material with the book's existing architecture guidance on performance, security, accessibility, observability, reliability, and governance.