NVDA Smoke Tests with Guidepup

Unit tests and axe scans check the DOM; they cannot tell you what NVDA actually says. Guidepup drives real screen readers — NVDA on Windows and VoiceOver on macOS — from test code and returns spoken phrases. With Playwright, you can open a data grid, move through it with NVDA commands and assert that headers, values, sort state and announcements are spoken. These tests are slower and more fragile than unit tests, so a small set of smoke tests on critical paths is the right scope.

This guide belongs to screen reader smoke testing.

Spec reference

Guidepup — libraries @guidepup/guidepup and @guidepup/playwright for driving NVDA and VoiceOver.
NVDA — requires a Windows environment with audio-free operation configured by Guidepup’s setup.
SC 4.1.2, 1.3.1, 4.1.3 — the criteria whose real-world effect these tests observe.
Limits — speech output differs by NVDA version and settings; pin versions.

The test reads what NVDA says, not the DOM.

When to use — and when not to

Use NVDA smoke tests for a handful of critical paths: arriving at the main grid, navigating cells, sorting, filtering with a count announcement, and saving an edit.

Do not try to replace manual screen reader testing or assert exact transcripts of entire pages. The misapplication: a hundred NVDA tests with full-string assertions that break on every NVDA update and get disabled.

Annotated code example

// playwright.config.js — use Guidepup's NVDA fixture on a Windows runner
import { nvdaTest as test } from '@guidepup/playwright';
import { expect } from '@playwright/test';

test('grid announces headers, values and sort', async ({ page, nvda }) => {
  await page.goto('http://localhost:3000/invoices');
  await page.locator('h1').click();                                    // start reading near content
  await nvda.perform(nvda.keyboardCommands.moveToNextTable);           // T
  await nvda.perform(nvda.keyboardCommands.moveToNextColumn);          // Ctrl+Alt+Right
  await nvda.perform(nvda.keyboardCommands.moveToNextRow);             // Ctrl+Alt+Down
  const log = (await nvda.spokenPhraseLog()).join(' | ');
  expect(log).toContain('Amount');                                     // header announced
  expect(log).toMatch(/1,240\.00/);                                    // value announced

  await page.getByRole('button', { name: 'Amount' }).focus();
  await nvda.press('Enter');
  expect((await nvda.lastSpokenPhrase())).toMatch(/sorted ascending|ascending/i);
});

# GitHub Actions: Windows runner with Guidepup setup
jobs:
  nvda:
    runs-on: windows-latest
    steps:
      - uses: actions/checkout@v4
      - uses: guidepup/setup-action         # pin to a tagged release
      - run: npm ci && npx playwright install chromium
      - run: npx playwright test --project=nvda

Keyboard & AT behaviour

Smoke test	Key assertion
Arrive at grid	Table name and size spoken
Navigate cells	Column header on column change; value
Sort	“sorted ascending” or “ascending”
Filter	Count message spoken
Save an edit	“Saved” message spoken

Keep screen reader tests few.

Step-by-step implementation

Provision a runner. A Windows CI runner with Guidepup’s setup action, or a dedicated Windows VM.
Pin versions. Record NVDA, browser and Guidepup versions; update deliberately.
Write a few tests. One per critical path, each starting from a known focus position.
Assert key phrases. toContain or regex for headers, values, states and messages.
Handle flakiness. Add short waits after actions that trigger announcements, retry once, and quarantine tests that flake repeatedly until fixed.

Integration context

VoiceOver equivalents are in VoiceOver smoke tests with Guidepup, and Playwright-based approaches in writing a screen reader smoke test with Playwright. Speech logs are covered in capturing NVDA speech logs. Choosing which combinations to cover is in building a screen reader test matrix. Cheaper checks of the same messages are in testing live region announcements in unit tests.

Gotchas

Starting position. NVDA reads from wherever its cursor is; set focus to a known element before each sequence.

Timing. Announcements may arrive after commands complete; wait for expected phrases with a timeout rather than fixed sleeps where possible.

Settings. NVDA verbosity settings change output; Guidepup’s setup applies a known profile — do not alter it.

Edge cases

Browse versus focus mode. Commands differ; ensure the grid is in the mode you expect before asserting.
Localisation. NVDA speaks in the system language; run tests in one locale.
Headless browsers. Screen readers need a real, visible browser window.

Worked example

A finance product had component tests for its grid announcements, but users still reported silent sorts in NVDA. The team added five Guidepup smoke tests on a Windows runner. The sort test failed immediately: NVDA announced the new state only after focus moved. Adding a polite sort confirmation made the test pass and fixed the user-facing issue. The suite runs nightly and on release branches in under three minutes.

Common mistakes

Too many screen reader tests.
Exact transcript assertions.
Unpinned NVDA versions.
No known starting position.

Testing checklist

A Windows runner with Guidepup and NVDA is configured.
Versions are pinned and recorded.
Five to ten smoke tests cover critical paths.
Assertions use key phrases.
Flaky tests are retried once and quarantined if needed.

When a Guidepup test fails, reproduce it manually with the same NVDA version and the speech viewer open. Run the same commands in order and compare the speech viewer output with the test log. If they match, the product changed; if they differ, the test’s timing or starting position is wrong. Keeping this manual reproduction step quick is what keeps the automated suite trusted.

Keeping the suite healthy

Review the suite every quarter. Remove tests whose paths no longer exist, add one for any screen reader bug that reached users, and check run time stays within a few minutes. When NVDA releases a new version, run the suite against it on a branch before updating the pinned version, and fix assertions that depended on incidental wording. A small, current suite that people trust beats a large one that is routinely ignored.

Choosing the first five tests

Start with the paths where a silent failure would hurt users most: reaching and reading the main grid, sorting it, filtering it and hearing the count, saving an edit and hearing the confirmation, and one error path such as a failed save. Those five cover table semantics, state announcements and live regions — the three areas where DOM-level tests most often pass while real speech fails.

Key points

Guidepup lets tests observe real NVDA speech.
Keep tests few, on critical paths, with key-phrase assertions.
Pin versions and start from a known focus position.
Reproduce failures manually with the speech viewer.

Design system notes

Provide smoke tests for the design system’s grid and table components, so product teams inherit coverage and only add product-specific paths.

FAQ

Does Guidepup work on Linux?

NVDA requires Windows; VoiceOver requires macOS. Orca is not supported in the same way.

How long do tests take?

Tens of seconds each; keep the suite small.

Can tests run on every PR?

Usually on main and nightly; on PRs only for critical components.

What should I assert?

Key phrases: headers, values, states, messages.

Do I still need manual testing?

Yes; smoke tests catch regressions, not usability problems.

How do I debug failures?

Read the spoken phrase log and reproduce with NVDA’s speech viewer.

VoiceOver smoke tests with Guidepup — macOS
Capturing NVDA speech logs — logs
Building a screen reader test matrix — coverage

← Back to Screen Reader Smoke Testing