GenUI Evaluation Governance

Why this chapter matters

GenUI systems fail differently from deterministic UI. The same release can pass unit tests and still generate an inaccessible component, cite the wrong source, call an unsafe tool, leak data through logs, or create a confident but wrong workflow. Evaluation governance is the operating model that keeps probabilistic behavior within product, security, accessibility, and reliability boundaries.

Do not treat evals as a research artifact. Treat them as release infrastructure.

Evaluation layers

Layer	Question	Example gate
Intent	Did the system understand the user's goal?	Correct workflow selected for fixture prompts.
Retrieval	Did it use allowed, fresh, relevant context?	No unauthorized source included.
Reasoning/output	Is the answer or generated UI grounded and useful?	Groundedness score above threshold.
Schema	Is generated UI valid and compatible?	Zero critical schema violations.
Tool use	Are actions allowed, idempotent, and approved?	No destructive tool without approval.
UX	Can users recover from errors and ambiguity?	Rejection and correction flows tested.
Accessibility	Is generated UI navigable and understandable?	Keyboard, focus, labels, live regions pass.
Security	Does the system resist malicious input/context?	Prompt injection fixtures do not change policy.
Cost/latency	Does the workflow meet operating budget?	p75 time and cost per successful task within budget.

Eval ownership

Every production GenUI workflow needs an eval owner, not only a model owner.

Owner	Responsibility
Product	Defines success, baseline, user harm, and launch threshold.
Design/UX	Reviews recovery, trust, consent, and generated UI usability.
Frontend	Owns component registry, schema validation, accessibility, and client performance.
Backend/platform	Owns tools, retrieval, tracing, rate limits, and side-effect controls.
Security/privacy	Owns injection, authorization, data leakage, and audit tests.
Support/operations	Feeds real failures back into fixtures and runbooks.

Fixture strategy

Create fixtures from four sources:

• designed scenarios before launch
• red-team prompts and malicious retrieved content
• anonymized production failures
• product changes that alter policy, data shape, or component behavior

Each fixture should include:

• user prompt or event sequence
• user role and permissions
• allowed data sources
• retrieved context sample
• expected workflow
• allowed components/tools
• expected refusal or approval behavior
• accessibility expectations
• cost/latency budget
• owner and review date

Release gates

A GenUI workflow should not launch unless it has explicit thresholds:

Gate	Example threshold
Critical security	0 unresolved critical failures
Unauthorized retrieval	0 leaks in fixture suite
Invalid UI	Below agreed rate, with safe fallback
Tool safety	0 unapproved destructive actions
Task success	Better than deterministic baseline for target cohort
Accessibility	No critical generated component accessibility failures
Latency	Time to first useful state within product budget
Cost	Cost per successful task within finance/product budget
Observability	Trace connects UI, model, retriever, tool, and outcome
Rollback	Kill switches tested for model, retriever, tool, and component

Online evaluation

Offline evals are necessary but incomplete. Production needs online signals:

• user correction rate
• rejection reason
• generated component validation failure
• citation click/use rate
• tool approval/denial rate
• workflow abandonment point
• support escalation
• repeated prompt reformulation
• time to first useful state
• accessibility error telemetry for generated surfaces
• cost per successful task

Connect these signals back to fixture creation. A production failure that does not become an eval will likely return.

Trace review

Traces should answer:

• what user intent was inferred
• what context was retrieved and filtered out
• what policy checks ran
• what model/prompt/tool versions were used
• what UI schema was generated
• what validation failed or passed
• what the user accepted, edited, rejected, or abandoned
• what cost and latency each stage added

Redact sensitive payloads. Store identifiers that allow investigation without copying private content into debugging tools.

Anti-patterns

• Testing only happy-path prompts.
• Measuring answer quality but not generated UI validity.
• Shipping generated components without accessibility fixtures.
• Letting production failures remain support anecdotes.
• Updating prompts without eval diffs.
• Treating "human approval" as a substitute for permission checks.
• Measuring total token spend instead of cost per successful task.
• Keeping eval thresholds vague until launch week.

Governance cadence

Cadence	Activity
Every PR touching prompts/components/tools	Run targeted eval subset and schema tests.
Weekly	Review new production failures and add fixtures.
Monthly	Rebaseline quality, cost, latency, and rejection reasons.
Before model/provider change	Run full eval suite and compare traces.
Before component registry expansion	Add accessibility, security, and fallback fixtures.
After incident	Add regression fixture and update kill-switch/runbook coverage.

Exercises

1. Build an eval matrix for a GenUI workflow that drafts a renewal email, retrieves account context, and asks for approval before sending.
2. Define launch thresholds for task success, invalid UI, tool safety, accessibility, latency, and cost.
3. Convert five support tickets into anonymized eval fixtures.
4. Design a trace schema that lets an engineer debug wrong generated UI without exposing private customer data.

Source lens

Use current AI framework and protocol docs for implementation specifics. Use this book's stable architecture material for the enduring controls: typed contracts, least privilege, accessibility, observability, incident response, cost budgets, and governance.