UX Audit scannerAudit software
Replacing opaque AI scores with transparent, reproducible formulas across 5 audit types in a UX intelligence platform. Redesigned the scoring engine across IA Scorecard, Form UX Audit, Microcopy Audit, Cognitive Walkthrough, and Content Audit. Each score is now computed from structural data using weighted formulas with severity penalties - no AI inference in the scoring loop.
Full-Stack UX: Research + Design + Engineering
Responsibility
Owner. End to end.
- 01Company owner — strategy, positioning, pricing, and every conversation that becomes product.
- 02UX Research — interviews with designers and PMs to understand why the AI-only scorer wasn't trusted.
- 03Design — the full scoring engine, the report surface, the marketing site, and the rubric content.
- 04AI-assisted engineering — prompt engineering + deterministic formulas; the AI identifies, the formula scores.
Responsibility
Owner. End to end.
- 01Company owner — strategy, positioning, pricing, and every conversation that becomes product.
- 02UX Research — interviews with designers and PMs to understand why the AI-only scorer wasn't trusted.
- 03Design — the full scoring engine, the report surface, the marketing site, and the rubric content.
- 04AI-assisted engineering — prompt engineering + deterministic formulas; the AI identifies, the formula scores.
Why the AI scorer broke trust
The scoring trust problem.
Inconsistent scores across runs.
The same page scanned twice could yield different scores because the AI model picked numbers subjectively each time.
No explainability.
Users received a score like "72/100" with no breakdown of how it was calculated. They could not identify what to fix to improve the score.
No auditability.
Stakeholders could not verify scores. There was no formula to inspect, no weights to examine, no methodology to reference.
Score inflation.
Without severity penalties, a page with 3 critical accessibility failures could still score 85/100 because the AI averaged surface-level positives.
Why the AI scorer broke trust
The scoring trust problem.
Inconsistent scores across runs.
The same page scanned twice could yield different scores because the AI model picked numbers subjectively each time.
No explainability.
Users received a score like "72/100" with no breakdown of how it was calculated. They could not identify what to fix to improve the score.
No auditability.
Stakeholders could not verify scores. There was no formula to inspect, no weights to examine, no methodology to reference.
Score inflation.
Without severity penalties, a page with 3 critical accessibility failures could still score 85/100 because the AI averaged surface-level positives.
Competitive analysis
What do users actually trust?
| Tool | Score method | Auto justification |
|---|---|---|
| Google Lighthouse | Weighted average of audits, each pass/fail | Scores are deterministic — same input = same score |
| WAVE | Issue count by severity | No visual score — lets severity speak for itself |
| Axe / Deque | Pass rate with severity weighting | Critical issues can cap the maximum score |
| SiteImprove | 100-point weighted categories | Transparent weighted breakdown per category |
Determinism builds trust.
Users trust tools they can reproduce. If the same input yields different numbers, trust erodes immediately.
Severity caps prevent false confidence.
Axe's approach of capping scores when critical issues exist prevents dangerously misleading "green" scores.
Justifications reduce support load.
Tools that "show calculations" generate fewer "why this score?" support tickets.
Category breakdowns enable action.
An overall score of 72 is useless. "Labels: 40, Validation: 45, Accessibility: 65" tells you exactly what to fix.
Competitive analysis
What do users actually trust?
| Tool | Score method | Auto justification |
|---|---|---|
| Google Lighthouse | Weighted average of audits, each pass/fail | Scores are deterministic — same input = same score |
| WAVE | Issue count by severity | No visual score — lets severity speak for itself |
| Axe / Deque | Pass rate with severity weighting | Critical issues can cap the maximum score |
| SiteImprove | 100-point weighted categories | Transparent weighted breakdown per category |
Determinism builds trust.
Users trust tools they can reproduce. If the same input yields different numbers, trust erodes immediately.
Severity caps prevent false confidence.
Axe's approach of capping scores when critical issues exist prevents dangerously misleading "green" scores.
Justifications reduce support load.
Tools that "show calculations" generate fewer "why this score?" support tickets.
Category breakdowns enable action.
An overall score of 72 is useless. "Labels: 40, Validation: 45, Accessibility: 65" tells you exactly what to fix.
Two load-bearing choices
Trustworthy scores, without new AI calls.
Penalty-based formulas over ML models.
Severity caps.
A critical insight from Axe/Deque: if critical issues exist, the overall score must be capped regardless of how many other things pass. This prevents the "green score masks red problems" misleading case where 4/5 pass but the one that fails is load-bearing.
Two load-bearing choices
Trustworthy scores, without new AI calls.
Penalty-based formulas over ML models.
Severity caps.
A critical insight from Axe/Deque: if critical issues exist, the overall score must be capped regardless of how many other things pass. This prevents the "green score masks red problems" misleading case where 4/5 pass but the one that fails is load-bearing.
One engine, five audits
Dimensions with explicit weights.
Scoring dimensions.
Pattern across audit types.
One engine, five audits
Dimensions with explicit weights.
Scoring dimensions.
Pattern across audit types.
Same page, two scoring engines
From vibes to a defensible score.
- No breakdown.
- No explanation.
- No formula.
Same page, two scoring engines
From vibes to a defensible score.
- No breakdown.
- No explanation.
- No formula.
Live surfaces
In the wild.
Live surfaces
In the wild.