Setting Up axe-core in a GitHub Actions Pipeline

A GitHub Actions accessibility job runs @axe-core/playwright against your built pages on every pull request and blocks the merge if any WCAG violation is found. The single failure it prevents is the silent regression: an ARIA refactor or a colour-token change that breaks a sortable data grid and reaches production because no human happened to re-test that page. This guide gives the complete, annotated workflow — build, cache, serve, scan, gate, report, and matrix — as it belongs in the broader automated accessibility testing pipelines architecture.

Spec reference

There is no ARIA attribute to configure here; the “spec” is the contract between three moving parts.

Part Normative behaviour Default
@axe-core/playwright AxeBuilder Injects axe-core into the page under Playwright control and returns a results object with a violations array Runs all enabled rules on the whole document
.withTags([...]) Restricts the run to the named WCAG tag sets If omitted, runs every rule including best-practice
GitHub Actions job exit code The job fails when any step exits non-zero A test runner exits non-zero when an assertion fails

Default behaviour worth overriding: without .withTags, axe includes best-practice rules, which are advisory, not WCAG failures. Restrict tags to the levels your team commits to (wcag2a, wcag2aa, wcag21aa, wcag22aa) so an advisory finding cannot block a merge.

The success criteria this job can enforce in a real browser are 1.1.1 Non-text Content (Level A), 1.3.1 Info and Relationships (Level A), 1.4.3 Contrast Minimum (Level AA), 1.4.11 Non-text Contrast (Level AA), and 4.1.2 Name, Role, Value (Level A). It confirms the presence but not the firing of a 4.1.3 Status Messages (Level AA) live region.

When to use this vs. when NOT to

Use a GitHub Actions axe job when: you ship a web app with a build step, you want a required merge gate, and you can host the built output on a port inside the runner. This is the correct home for full-page, real-browser scans — contrast and computed roles that the jsdom-based unit layer cannot compute.

Do NOT use it as your only accessibility check. The job scans a static snapshot after each interaction you script. It will not press Tab for you, will not tell you the focus ring is invisible, and will not confirm a screen reader announced anything. Pair it with the manual protocol in the testing and auditing section. The misuse to name explicitly is the “green badge” fallacy: treating a passing axe job as a WCAG conformance claim. axe covers a minority of the criteria; the badge means “no machine-detectable violations”, nothing more.

Do NOT scan the dev server. Development output differs from production in minification, hydration timing, and warning overlays. Build, serve the static output, and scan that.

Annotated workflow

The JavaScript test first. It reads the route list, scans each one, waits for stability, and writes a machine-readable report per route.

// tests/a11y.spec.js — scans one route supplied by the CI matrix.
import { test, expect } from '@playwright/test';
import AxeBuilder from '@axe-core/playwright';
import { writeFileSync, mkdirSync } from 'node:fs';

const route = process.env.A11Y_ROUTE || '/';

test(`no axe violations on ${route}`, async ({ page }) => {
  await page.goto(route);
  // Wait for the grid to hydrate so we scan the real accessibility tree,
  // not a pre-hydration shell (avoids CI-only false negatives).
  await page.getByRole('grid').waitFor({ state: 'visible' });
  await page.waitForLoadState('networkidle');

  const results = await new AxeBuilder({ page })
    // SC 1.4.3 & 1.4.11 (contrast) and 4.1.2 (Name, Role, Value) run here.
    .withTags(['wcag2a', 'wcag2aa', 'wcag21aa', 'wcag22aa'])
    .exclude('.third-party-embed') // skip DOM we do not own
    .analyze();

  // Persist the report so a red build is debuggable from the artifact.
  mkdirSync('axe-results', { recursive: true });
  const slug = route.replace(/\W+/g, '_') || 'root';
  writeFileSync(`axe-results/${slug}.json`, JSON.stringify(results, null, 2));

  // Non-zero exit on any violation — this is what blocks the merge gate.
  expect(results.violations).toEqual([]);
});

Now the workflow. It builds once, caches the Playwright browser, serves the output, and shards the scan across a route matrix so ten routes do not run serially.

# .github/workflows/a11y.yml
name: accessibility
on:
  pull_request:
    branches: [main]

jobs:
  axe:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false        # scan every route even if one fails
      matrix:
        # Each route is a real pattern under test — e.g. the sortable grid.
        route: ['/', '/reports/grid', '/reports/dashboard']
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: npm

      - run: npm ci

      # Cache the Playwright browser download, keyed on its version, so the
      # ~100 MB Chromium download is skipped on cache hit.
      - name: Resolve Playwright version
        id: pw
        run: echo "v=$(node -p "require('@playwright/test/package.json').version")" >> "$GITHUB_OUTPUT"
      - uses: actions/cache@v4
        id: pw-cache
        with:
          path: ~/.cache/ms-playwright
          key: pw-${{ runner.os }}-${{ steps.pw.outputs.v }}

      - name: Install Chromium
        # On cache hit, still install OS deps but skip the browser binary.
        run: npx playwright install --with-deps chromium

      - run: npm run build

      # Serve the BUILT output (not the dev server) on a local port.
      - name: Serve built site
        run: npx serve -s dist -l 4173 &

      - name: Wait for server
        run: npx wait-on http://localhost:4173

      # Scan the matrix route. A violation exits non-zero and fails the job.
      - name: Run axe scan
        env:
          A11Y_ROUTE: ${{ matrix.route }}
        run: npx playwright test tests/a11y.spec.js

      # Upload the report even on failure so the violation is inspectable.
      - if: always()
        uses: actions/upload-artifact@v4
        with:
          name: axe-report-${{ strategy.job-index }}
          path: axe-results/

To make it block merges: open the repository’s branch protection rules for main and mark the accessibility / axe job as a required status check. A required job that exits non-zero prevents the pull request from merging.

CI event behaviour

How the job responds to each situation, and what the developer sees.

CI event / trigger Job behaviour Merge outcome Where to look
PR opened, no violations All matrix legs pass, artifacts uploaded Allowed Green check
A route has a violation That matrix leg exits non-zero; others still run (fail-fast: false) Blocked (required check red) axe-report-N artifact, that route’s JSON
Playwright browser cache miss Chromium downloads (~30 s), then scan proceeds Unaffected Cache step log
Grid not hydrated before scan waitFor throws a timeout; job fails clearly Blocked Playwright trace / error
Third-party embed fails a rule Excluded subtree is skipped; own DOM still checked Depends on own DOM .exclude() selector in spec

Integration context

This workflow is the browser layer of the automated accessibility testing pipelines cluster. Run it after the faster jest-axe unit layer described in writing jest-axe tests for data grid components — unit tests fail in seconds on static ARIA regressions, so most breakage never reaches this slower browser scan. When a route in the matrix is a sortable grid, script a sort interaction before analyze() so the scan asserts the aria-sort attribute contract in its post-interaction state, not just on load.

Gotchas

1. Caching the browser but forgetting the OS dependencies

actions/cache restores the browser binary, but Chromium also needs system libraries (libnss3, libatk, and others) that live outside ~/.cache/ms-playwright. If you skip npx playwright install --with-deps on a cache hit, the browser launch fails with a cryptic shared-library error. Always run install --with-deps even when the cache hits — it is fast when the binary is already present and installs the OS deps.

2. Scanning before hydration

CI runners are slower than a developer laptop. A scan that calls analyze() immediately after goto() can run against a server-rendered shell before React or Vue has hydrated the grid, so role="grid" and aria-sort are absent and axe reports false negatives (or the assertion passes on an empty page). Always wait for a concrete, interactive element — getByRole('grid').waitFor() — before scanning.

3. fail-fast hiding the full picture

The default fail-fast: true cancels every other matrix leg the moment one fails. On an accessibility scan that is the wrong trade: you want to know all the routes that regressed in one run, not fix one and rediscover the next tomorrow. Set fail-fast: false so every route reports.

FAQ

Should the workflow scan the dev server or the built output?

Scan the built production output. The dev server ships unminified code, source maps, and development-only warnings that never reach users, and it can differ from production in hydration timing and CSS. Build the site, serve the static output on a local port in the job, and point Playwright at that. You then test exactly what deploys.

Why does my Playwright axe job pass locally but fail in GitHub Actions?

The two most common causes are a browser-version mismatch and a timing difference. CI installs a fresh Playwright browser that may differ from your local one, changing computed contrast or role resolution. CI is also slower, so a scan can start before the grid has hydrated. Pin the Playwright version, install with npx playwright install --with-deps, and wait for a stable state before calling analyze().

How do I make the accessibility check block merges?

The job must exit non-zero on a violation, which it does when the test assertion fails. Then mark the workflow’s job as a required status check in the repository branch protection rules. A required check that exits non-zero prevents the pull request from merging until the violations are fixed.

← Back to Automated Accessibility Testing Pipelines