Rubric: AI Full-Stack and GenUI Architect
Purpose
Use this rubric to evaluate whether an engineer can build AI-assisted product systems that are useful, safe, observable, and maintainable. Demos are not enough. The system must have product boundaries, evals, fallbacks, and governance.
Capability rubric
| Capability | Level 3 | Level 4 | Level 5 |
|---|---|---|---|
| Product boundary | Separates deterministic UI from AI-assisted surfaces. | Designs fallback, escalation, and user-control behavior. | Creates product standards for AI interaction boundaries. |
| GenUI registry | Uses a limited component registry. | Defines schemas, validation, accessibility, versioning, and telemetry. | Governs generated UI across teams and clients. |
| Streaming protocol | Streams text and progress. | Handles component events, tool calls, errors, replacement semantics, and resumability. | Defines protocol contracts used by multiple agents or clients. |
| RAG architecture | Retrieves context with citations. | Designs source quality, chunking, freshness, context packing, and stale-source behavior. | Runs retrieval evaluation and content governance. |
| Tool and agent safety | Adds tool calls. | Classifies tool risk, requires approvals, audits actions, and supports rollback. | Establishes MCP/A2A boundary standards. |
| Evals | Uses manual testing. | Builds prompt fixtures, regression suites, red-team tests, and release gates. | Maintains eval-driven product quality over time. |
| Observability | Logs model calls. | Tracks cost, latency, retrieval quality, tool outcomes, corrections, and UI failures. | Connects AI telemetry to roadmap and governance. |
| Security and privacy | Avoids obvious leaks. | Designs tenant isolation, permission propagation, prompt injection controls, and data minimization. | Teaches secure AI architecture across teams. |
Required evidence
- Component registry contract.
- RAG/citation policy.
- Tool risk matrix.
- Evals report.
- Observability dashboard design.
- Security and privacy review.
- Model outage fallback plan.
Minimum passing standard
Readiness requires level 4 in GenUI registry, tool safety, evals, observability, and security/privacy. A GenUI architect should be able to prevent a model from becoming an implicit product owner.