VENNStart
Guide

How I run Playwright accessibility testing with axe-core

Automated a11y tests for a Next.js monorepo with axe-core and Playwright, including dark mode, a keyboard focus walk, legacy debt, and what axe can't see.

Lê Anh Tuấn4 min read

My side project (a Next.js monorepo with a separate API) has a spec that runs axe-core through Playwright on every main screen in light and dark. It also does a keyboard walk and runs DOM rules of my own. The repo uses @playwright/test 1.63.0 and @axe-core/playwright 4.13.0.

What axe can check automatically

axe handles what the DOM and computed styles can decide, like contrast or a missing accessible name. It can't tell whether alt text makes sense or where focus goes after a dialog closes. A green run doesn't mean the app is accessible. Use a screen reader for those checks, with the suite catching DOM and style bugs before the manual pass.

When you pass WCAG tags, axe only runs the rules tagged with them. region is tagged best-practice and never runs in a WCAG-only scan. target-size is tagged wcag22aa, and a tag list that stops at wcag21aa skips it. I got caught by that one.

One helper, readable failures

One helper gives every test the same AxeBuilder setup:

// e2e/lib/ui-audit.ts
import { AxeBuilder } from '@axe-core/playwright'
import type { Page } from '@playwright/test'

export const AXE_TAGS = ['wcag2a', 'wcag2aa', 'wcag21a', 'wcag21aa'] as const

export function runAxe(page: Page, selector?: string) {
  let builder = new AxeBuilder({ page }).withTags([...AXE_TAGS])
  if (selector) builder = builder.include(selector)
  return builder.analyze()
}

For a third-party widget you can't fix, exclude() skips that element, its whole subtree and every rule on them. When one rule reports false positives on one component, disableRules(['rule-id']) in that test is less blunt.

The gate only blocks on serious and critical, because with moderate mixed in the serious ones are hard to spot. I flatten results.violations to one line per element and pass that as the expect message:

import type { AxeResults } from 'axe-core'

export function formatAxe(results: AxeResults): string {
  const lines: string[] = []
  for (const v of results.violations) {
    if (v.impact !== 'serious' && v.impact !== 'critical') continue
    lines.push(`${v.id} (${v.impact}) x${v.nodes.length}: ${v.help}`)
    for (const node of v.nodes) {
      const target = node.target
        .map((part) => (Array.isArray(part) ? part.join(' ') : String(part)))
        .join(' >>> ')
      lines.push(`  - ${target}`)
      if (node.failureSummary) lines.push(`    ${node.failureSummary.replace(/\s+/g, ' ')}`)
    }
  }
  return lines.length ? `\n${lines.join('\n')}` : 'no violations'
}

The summary report groups by rule instead of by screen. One colour token change once cleared eleven violations on three screens that looked like separate bugs.

Login, dark mode, timing and mobile

Each test logs in through the form, which is slow. I'll move to the setup project the Playwright docs recommend, which logs in once and saves storageState (keep playwright/.auth out of git).

Dark mode needs its own pass, since contrast bugs are rarely symmetric. For a prefers-color-scheme theme, page.emulateMedia({ colorScheme: 'dark' }) is enough. This app reads the theme from a cookie on the server. Add it after the first goto and you audit the light DOM under a dark label. I assert <html> is in the right mode before running axe, because a wrong cookie still passes on the default theme. That kind of silent failure annoys me most.

An early scan audits the loading skeleton and comes back clean. I wait for main, then networkidle, then for Recharts to finish drawing. Any console error fails the test.

The gate runs at 1440 wide, but a separate audit of 22 routes at 1440 and 375 found scrollable-region-focusable only on mobile: six scroll areas on the system map page that the keyboard couldn't get to on a phone.

Dialogs have to be open to be scanned

axe only sees what's in the DOM. Dialogs, menus and selects rendered through a portal don't exist until they're opened, and the gate scans each screen as it loads, so it never sees them. The separate audit opened the dialogs before running axe. In the spec that means opening one, waiting, and scanning just that subtree:

test('edit dialog has no serious violations', async ({ page }) => {
  await page.goto('/settings')
  await page.getByRole('button', { name: 'Edit profile' }).click()

  const dialog = page.getByRole('dialog')
  await dialog.waitFor()

  const results = await runAxe(page, '[role="dialog"]')
  expect(results.violations.filter((v) => v.impact === 'serious' || v.impact === 'critical'),
    formatAxe(results)).toEqual([])

  await page.keyboard.press('Escape')
  await expect(dialog).toBeHidden()
  await expect(page.getByRole('button', { name: 'Edit profile' })).toBeFocused()
})

The last three lines check two things axe can't: whether Escape closes the dialog and whether focus returns to the trigger.

A keyboard walk catches what axe can't

The most useful part of the spec presses Tab through the page and checks that each focused element visibly looks focused. My first draft read outline-style at rest and flagged every control (54/54 on the queue screen, 81/81 on analytics), because focus rings only exist under :focus-visible. The current version records each element's resting style first, then presses Tab for real and compares:

for (let step = 0; step < maxTabs; step++) {
  await page.keyboard.press('Tab')
  const focused = await page.evaluate(() => {
    const el = document.activeElement as HTMLElement | null
    const i = el?.getAttribute('data-ui-audit-tab')
    if (!el || i == null) return null
    const s = getComputedStyle(el)
    return { i: Number(i), outline: `${s.outlineStyle} ${s.outlineWidth}`,
             shadow: s.boxShadow, border: s.borderColor, bg: s.backgroundColor }
  })
  if (!focused || seen.has(focused.i)) continue
  const rest = baseline[focused.i]
  const hasOutline = !focused.outline.startsWith('none') && parseFloat(focused.outline.split(' ')[1]) >= 1
  const changed = focused.shadow !== rest.shadow || focused.border !== rest.border || focused.bg !== rest.bg
  seen.set(focused.i, hasOutline || changed)
}

A walk that lands nowhere would pass too. The spec asserts it reached at least 80% of tabbable controls.

Walking the app with Tab also turned up radio groups whose focus-within:ring-ring was a dead class. The theme never declared --color-ring, so Tailwind generated nothing for it, and the ring-2 ring came out in currentColor, a different focus colour from the buttons next to it. A sr-only file input was still reachable by Tab, putting focus on an invisible 1x1 control.

Test the rules and isolate existing violations

On every run, each custom rule is tested against broken markup injected into a real page (an icon-only <button><svg/></button>, two <h1>s, white on white) and then against clean markup. The axe wiring scans #ui-audit-harness and expects button-name and color-contrast. A test that quietly stopped checking looks exactly like a clean app.

The first full run was red on every screen. Instead of the violation fingerprints the Playwright docs suggest, I went cruder, with a strict gate by default and an environment variable that downgrades some rule ids to warnings:

const known = (process.env.UI_AUDIT_KNOWN ?? '').split(',').map((s) => s.trim()).filter(Boolean)

const blocking = violations.filter((v) => !known.includes(v.rule))
const warnings = violations.filter((v) => known.includes(v.rule))

if (warnings.length) console.warn(`downgraded: ${known.join(', ')}`)
expect(blocking, formatViolations(blocking)).toEqual([])

The variable is empty by default. The weakness is that it works per rule, so skipping color-contrast also skips new contrast bugs. If the debt drags on, I'll switch to fingerprints.

The real bugs it found

Most were muted text with opacity, like calendar day numbers in text-muted/70 at 2.99:1 and the composer placeholder at 2.95:1. The month view had a comment saying not to use /70, and the week view used it anyway.

Tab triggers had aria-controls pointing at panels that were never in the DOM, on 16 nodes. That one isn't fixed yet. The onboarding pager dots were 16px wide with 4px gaps and failed target-size. Only the separate wcag22aa audit caught it, because the gate's tag list, which I typed myself, stops at wcag21aa.

Playwright needs Postgres, a seeded workspace and Chromium, so the GitHub workflow only runs fast gates and the e2e suite runs through a local verify script before each PR. That holds up while I'm the only one opening PRs. Next is running the a11y project in CI with a Postgres service container, adding wcag22aa to the shared tag list and adding a mobile project.

  • Playwright
  • Accessibility
  • axe-core
  • Testing
  • Next.js
  • E2E

I'm Tuấn (Venn), a web developer in Ho Chi Minh City. I mostly build Next.js sites and web apps, and I take freelance work.

Work with me

More posts

  1. 01Guide4 min read

    Deploying Next.js and NestJS to AWS EC2 with Docker Compose

    How I deploy a pnpm monorepo to one EC2 box. Images get built in GitHub Actions and Docker Compose pulls them from GHCR. The server has no code.

  2. 02Guide4 min read

    How I store OAuth tokens for social media APIs in NestJS

    OAuth for Facebook, TikTok and YouTube in a NestJS API. State and PKCE, encrypted tokens, refreshing before each publish, and when to ask for a reconnect.

  3. 03vennee.dev3 min read

    llms.txt on my Next.js site, and now Lighthouse grades it

    vennee.dev has a hand-written llms.txt and an llms-full.txt built from the blog. Notes on the spec, .txt routes in Next.js, and Lighthouse 13's new audit.