Execution flow
Container properties
Containers are single-use. No state persists between executions — every run starts from a clean Git clone with fresh credentials.
The 15-step test execution pipeline
When you trigger a test run, Bugzy executes a 15-step pipeline:1
Run Tests Overview
Load task configuration and read
tests/CLAUDE.md for project-specific test instructions.2
Security Notice
Enforce security boundaries — the agent operates within defined permissions.
3
Parse Arguments
Extract test selection criteria: file pattern, tag (
@smoke), specific file path, or “all”.4
Read Test Strategy
Load
test-execution-strategy.md for context on test tiers and priorities.5
Clarification Protocol
If the request is ambiguous, confirm with the team before proceeding.
6
Identify Tests
Resolve the selector to specific test files and confirm the selection.
7
Run Tests
Execute selected Playwright tests inside the container with Chromium. Generate JSON reports.
8
Normalize Results
Convert raw test output into a standardized format.
9
Parse Results
Extract pass/fail status, error messages, screenshots, and traces from JSON reports.
10
Triage Failures
Classify each failure as a product bug (real application issue) or a test issue (broken selector, timing problem, flaky assertion). Uses the knowledge base for accuracy.
11
Fix Test Issues
The test-engineer subagent auto-fixes test issues — broken selectors, timing problems, stale references. Retries up to 3 times.
12
Log Product Bugs
File product bugs in your connected issue tracker (Jira, Azure DevOps, Asana, Linear) with screenshots and reproduction steps.
13
Handle Special Cases
Address edge cases — missing test files, invalid test cases, browser automation failures.
14
Update Knowledge Base
Record learnings in
knowledge-base.md for future triage accuracy.15
Notify Team
Post a summary to Slack, Microsoft Teams, or email with pass/fail counts, bug links, and fix details.
Where test artifacts live
All test artifacts are committed to your project’s Git repository:
Because everything lives in Git, you get full version history, code review via PRs, and the ability to run tests locally with standard Playwright commands.
