> ## Documentation Index
> Fetch the complete documentation index at: https://www.bugzy.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Test engineering

> AI-powered test code generation, debugging, auto-fixing, and maintenance — keeping your Playwright test suite healthy.

Bugzy's test-engineer subagent handles the full lifecycle of test code — generating new Playwright scripts, debugging failures, auto-fixing broken tests, and maintaining the suite as your application changes.

## Test generation

When creating new tests, the test engineer:

1. Reads test cases (from markdown files or an external TMS like Zephyr Scale or Testiny)
2. Generates Playwright TypeScript test scripts with proper assertions
3. Creates Page Object classes for reusable interactions
4. Adds appropriate waits, retries, and error handling
5. Tags tests with `@smoke`, `@regression`, or custom tags

Generated tests follow your project's existing patterns. If you have an established test suite, the agent reads your `CLAUDE.md` conventions and matches your style.

## API testing

Bugzy supports testing REST API endpoints using Playwright's built-in `APIRequestContext` — no additional framework needed. API tests use the `{ request }` fixture instead of `{ page }`, so no browser is launched (making them significantly faster).

API testing capabilities include:

* **Endpoint discovery** — during exploration, the agent looks for OpenAPI specs, route definitions, and API documentation
* **Test generation** — produces API test cases alongside UI tests, using `test_layer: api` metadata
* **Authentication** — supports Bearer tokens, API keys, and session-based auth patterns
* **Error validation** — tests for proper 4xx/5xx responses, error payloads, and rate limiting
* **Mixed tests** — combines API setup with UI verification in a single test (e.g., create data via API, verify in dashboard)

API tests use the same Playwright runner, Bugzy reporter, and `test-runs/` output structure as UI tests.

## Auto-fixing

When a test fails due to a test issue (not a product bug), the test engineer automatically:

<Steps>
  <Step title="Analyze the failure">
    Reads the error message, stack trace, and screenshots to identify the root cause.
  </Step>

  <Step title="Identify the fix">
    Determines whether the issue is a stale selector, a timing race, an outdated assertion, or a changed page structure.
  </Step>

  <Step title="Apply and validate">
    Modifies the test file, re-runs the test to confirm the fix works.
  </Step>

  <Step title="Retry if needed">
    If the first fix doesn't work, tries an alternative approach — up to 3 total attempts.
  </Step>

  <Step title="Commit the fix">
    Once validated, commits the fix to the repo so the suite stays green.
  </Step>
</Steps>

## Knowledge base

The test engineer builds and consults a `knowledge-base.md` file that captures:

* Application structure and navigation patterns
* Common selectors and interaction patterns
* Known flaky areas and workarounds
* Corrections from [disputed findings](/docs/capabilities/triage-failures#disputed-findings)

This knowledge base improves over time. Corrections from your team feed directly into future test generation and fixing decisions.

## Server-side verification (SSH)

Bugzy can verify server-side state during test execution by connecting to your servers via SSH. This catches backend issues that browser-only testing misses — incorrect log entries, failed database mutations, misconfigured services, or missing file uploads.

To enable SSH verification, add these environment variables to your project (Settings > Test Data > Environment Variables):

| Variable        | Value                                                    | Secret? |
| --------------- | -------------------------------------------------------- | ------- |
| `SSH_TEST_HOST` | Your server hostname or IP (e.g., `staging.example.com`) | No      |
| `SSH_TEST_USER` | SSH username (e.g., `deploy`)                            | No      |
| `SSH_TEST_KEY`  | Your SSH private key (full PEM block)                    | Yes     |
| `SSH_TEST_PORT` | SSH port (optional, default: 22)                         | No      |

<AccordionGroup>
  <Accordion title="Setting up SSH keys">
    1. **Create a dedicated read-only user** on the target server (required):
       ```bash theme={null}
       # On your server (as root or sudo)
       sudo useradd -m -s /bin/rbash bugzy-test
       sudo mkdir -p /home/bugzy-test/.ssh
       sudo chmod 700 /home/bugzy-test/.ssh
       ```
       Using `rbash` (restricted bash) as the shell prevents the user from running commands with `/` in the path, changing `PATH`, or using `cd` to navigate outside allowed directories.

    2. **Restrict the user to read-only commands** (required):
       ```bash theme={null}
       # Create a bin directory with only the commands Bugzy needs
       sudo mkdir -p /home/bugzy-test/bin
       sudo ln -s /usr/bin/cat /home/bugzy-test/bin/cat
       sudo ln -s /usr/bin/grep /home/bugzy-test/bin/grep
       sudo ln -s /usr/bin/tail /home/bugzy-test/bin/tail
       sudo ln -s /usr/bin/head /home/bugzy-test/bin/head
       sudo ln -s /usr/bin/ls /home/bugzy-test/bin/ls
       sudo ln -s /usr/bin/df /home/bugzy-test/bin/df
       sudo ln -s /usr/bin/free /home/bugzy-test/bin/free
       sudo ln -s /usr/bin/uptime /home/bugzy-test/bin/uptime
       sudo ln -s /usr/bin/curl /home/bugzy-test/bin/curl
       sudo ln -s /usr/bin/psql /home/bugzy-test/bin/psql 2>/dev/null  # if installed
       sudo ln -s /usr/bin/systemctl /home/bugzy-test/bin/systemctl 2>/dev/null

       # Set this as the only PATH for the user
       echo 'export PATH="$HOME/bin"' | sudo tee /home/bugzy-test/.bash_profile
       sudo chmod 644 /home/bugzy-test/.bash_profile
       sudo chown -R bugzy-test:bugzy-test /home/bugzy-test
       ```
       With `rbash` + a restricted `PATH`, the user can **only** run the commands you've symlinked. Commands like `rm`, `kill`, `dd`, or `chmod` are not available — the user literally cannot execute them.

    3. **Generate a dedicated key pair** for Bugzy testing:
       ```bash theme={null}
       # On your local machine
       ssh-keygen -t ed25519 -C "bugzy-testing" -f bugzy_test_key -N ""
       ```

    4. **Authorize the key on the server**:
       ```bash theme={null}
       # Copy the public key to the server
       cat bugzy_test_key.pub | ssh root@your-server \
         "sudo tee /home/bugzy-test/.ssh/authorized_keys && sudo chmod 600 /home/bugzy-test/.ssh/authorized_keys && sudo chown bugzy-test:bugzy-test /home/bugzy-test/.ssh/authorized_keys"
       ```

    5. **Paste the private key** (`bugzy_test_key`, not `.pub`) into the `SSH_TEST_KEY` environment variable in Bugzy. Include the full PEM block, from `-----BEGIN OPENSSH PRIVATE KEY-----` to `-----END OPENSSH PRIVATE KEY-----`.

    6. **Verify the restriction works**:
       ```bash theme={null}
       # These should work:
       ssh -i bugzy_test_key bugzy-test@your-server "cat /var/log/app.log"
       ssh -i bugzy_test_key bugzy-test@your-server "uptime"

       # These should fail:
       ssh -i bugzy_test_key bugzy-test@your-server "rm /tmp/test"       # command not found
       ssh -i bugzy_test_key bugzy-test@your-server "/bin/rm /tmp/test"  # rbash blocks /
       ```

    <Warning>
      **Do not skip steps 1-2.** The Bugzy agent runs commands on your server autonomously. A restricted user with `rbash` and a limited `PATH` ensures the agent can only run read-only commands — even if it makes an unexpected decision. Never use `root` or an unrestricted user for `SSH_TEST_USER`.
    </Warning>
  </Accordion>

  <Accordion title="What can the agent check?">
    * Application logs (`tail /var/log/app.log`)
    * Database state (`psql -c "SELECT * FROM orders ORDER BY id DESC LIMIT 1"`)
    * Service health (`systemctl status myapp`)
    * File uploads (`ls -la /uploads/`)
    * Internal API endpoints (`curl http://localhost:3000/api/health`)
    * System resources (`df -h`, `free -m`, `uptime`)
  </Accordion>

  <Accordion title="Firewalled servers">
    If your server is not reachable from the public internet, use [mcp-tunnel](/docs/integrations/mcp-tunnel) to establish a secure connection from the Bugzy execution environment to your internal network.
  </Accordion>
</AccordionGroup>

## File uploads in tests

Tests that involve file uploads (profile photos, CSV imports, PDF attachments) can use the **Test Data** file upload feature. Upload files through the dashboard's Test Data page and reference them via environment variables:

```typescript theme={null}
const filePath = process.env.PROFILE_PHOTO;
await page.locator('input[type="file"]').setInputFiles(filePath);
```

Files are stored securely in cloud storage and automatically downloaded to the test workspace before each run. Supported formats include images, PDFs, CSVs, spreadsheets, and text files (up to 10 MB each).

## Test maintenance

As your application evolves, existing tests may break due to UI changes, new authentication flows, or restructured pages. The test engineer handles this during every test run:

* Updates selectors when elements are moved or renamed
* Adjusts assertions when expected behavior changes intentionally
* Refactors Page Objects when page structure changes significantly

## OAuth test authentication

Applications that use OAuth login (Google, GitHub, Microsoft) can be tested without automating the provider's login page. Bugzy manages OAuth tokens automatically:

1. **Configure** your OAuth app credentials in the Test Data page (client ID, client secret, scopes)
2. **Authorize** with your test user account through a standard OAuth flow
3. **Run tests** — Bugzy refreshes tokens automatically and injects them into the browser

The agent inspects your application to determine how it stores auth state (localStorage, cookies, or sessionStorage) and injects the token accordingly. If a token expires or is revoked, the dashboard prompts you to re-authorize.

Supported providers: Google, GitHub, Microsoft. Multiple providers can be configured per project.

## Learn more

<CardGroup cols={2}>
  <Card title="Extend test coverage" icon="shield-plus" href="/docs/capabilities/extend-coverage">
    The full test generation workflow from plan to automation.
  </Card>

  <Card title="Triage failures" icon="bug" href="/docs/capabilities/triage-failures">
    How failures are classified and how auto-fix fits into triage.
  </Card>
</CardGroup>