Automated vs. Manual Accessibility Testing: The 70/30 Framework
Automated tools catch ~57% of WCAG issues. Here's a systematic approach to cover the remaining 43% without burning QA budgets.
The Coverage Reality
The accessibility testing industry has a dirty secret: no automated tool catches more than 57% of WCAG violations. The commonly cited 'tools catch 30%' figure is outdated — modern scanners like RegLayer, axe-core, and WAVE have improved significantly. But ~43% of issues require human judgment.
What automation catches well: color contrast, missing alt text, missing form labels, duplicate IDs, heading order, ARIA attribute validation, link text, and language attributes. What it can't: meaningful alt text quality, logical reading order, keyboard operability edge cases, and cognitive load.
The 70/30 Framework
Our recommended approach: 70% automated (continuous, every deploy) + 30% manual (quarterly, focused). The automated layer catches regressions instantly. The manual layer catches the nuanced issues that need human perception.
The key insight: don't try to manually test everything. Focus manual testing on: user flows (can a screen reader user complete checkout?), dynamic content (do live regions announce correctly?), cognitive patterns (is the information architecture logical?), and custom widgets (do complex interactions work with assistive tech?).
- Automated (70%): Run on every PR, blocks merge if critical violations found
- Screen reader testing (15%): Monthly testing with NVDA + VoiceOver on key user flows
- Keyboard testing (10%): Quarterly audit of all interactive components
- Cognitive review (5%): Annual review of information architecture and content clarity
CI/CD Integration Pattern
The most effective pattern: run accessibility tests in your CI pipeline like any other test. Block PRs that introduce critical/serious violations. Warn on moderate violations. Track minor violations as tech debt.
# .github/workflows/a11y.yml
name: Accessibility Check
on: [pull_request]
jobs:
scan:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: npm ci && npm run build
- name: RegLayer Scan
run: |
npx reglayer-cli scan --url http://localhost:3000 \
--fail-on critical,serious \
--report-format json \
--output a11y-report.json
- name: Upload Report
if: always()
uses: actions/upload-artifact@v4
with:
name: a11y-report
path: a11y-report.json