Agentic QA Engineer
VG Recruiting AgencyYerevan | Full time
Job description
This is a hands-on role for a Senior QA Automation Engineer who has already integrated agentic AI workflows into a real-world QA team.
You will design and build AI agents that work alongside QA engineers to transform Jira tickets into test cases, generate and maintain Playwright and Selenium tests, explore applications through Model Context Protocol (MCP), and triage failures in CI pipelines.
Working closely with the QA Manager, developers, and product managers across engineering teams, you will establish the patterns, safeguards, and metrics that determine where AI agents can operate independently, where human review is required, and how their impact on software quality is measured.
Job responsibilities
- Design, build, and operate AI agents covering the entire QA lifecycle, including requirements and ticket analysis, test case design, test generation, execution, and failure triage.
- Build multi-agent pipelines that transform Jira tickets into reviewed test cases and automated tests integrated into CI.
- Use Model Context Protocol (MCP) servers, such as Playwright MCP, to enable agent-driven browser interaction, exploratory testing, and self-healing locators.
- Develop and maintain the prompts, tools, context, and skills used by AI agents, applying version control practices similar to those used for regular code.
- Define human-in-the-loop checkpoints to ensure AI-generated outputs are reviewed when necessary and trusted when sufficiently validated.
- Extend existing Selenium and Playwright automation frameworks and contribute to the Playwright migration, leveraging AI agents to accelerate the process.
- Use AI agents to convert manual regression test cases into automated tests while continuously expanding test coverage without overloading the QA team.
- Write clean, maintainable, and reviewable test code in Java, Python, and TypeScript, following established design patterns such as the Page Object Model.
- Integrate AI-generated and AI-maintained tests into CI/CD pipelines, ensuring parallel execution and clear reporting.
- Develop evaluation datasets and metrics to assess agent performance, including test case accuracy, false positive rates, test flakiness, and time saved per release.
- Test MCP servers and AI agent tools, evaluating tool responses, data accuracy, authentication and permissions, error handling, and behavior when used by actual AI agents.
- Identify and reduce hallucinated assertions, fragile automatically generated tests, and low-value test coverage.
- Investigate failures and identify their root causes, distinguishing actual product defects from test instability, environment issues, and AI agent errors.
- Apply security-focused practices to AI agent tooling, including secure secrets management, restricted data access, and protection against prompt injection risks.
- Collaborate with the QA Manager, developers, and product managers to identify where AI agents can provide the greatest value within the weekly release cycle.
- Help QA engineers integrate AI agents into their daily workflows through collaborative sessions, documentation, and reusable templates.
- Communicate results, trade-offs, and lessons learned to engineering leadership in clear, accessible terms.
Required qualifications
- At least 8 years of experience in software quality assurance, including a minimum of 5 years in test automation.
- At least 1 year of hands-on experience integrating agentic AI into an actual QA team's workflows, beyond personal use of AI coding assistants.
- Experience building at least one AI agent or multi-agent workflow that generates, executes, or triages tests within a production QA process, with the ability to explain how its performance was measured.
- Hands-on experience with MCP servers, such as Playwright MCP or similar solutions, and AI coding agents such as Claude Code, Cursor, or GitHub Copilot.
- Strong programming skills in Java, Python, and TypeScript, with the ability to write clean, maintainable, and reviewable test code.
- Extensive hands-on experience building and maintaining automation frameworks using Playwright and Selenium WebDriver, as well as design patterns such as the Page Object Model.
- Strong manual and exploratory testing skills, with the ability to determine when human testing is more appropriate than automation.
- Solid experience in API testing and working knowledge of SQL for validating data behind user interfaces.
- Experience running automated tests reliably in CI/CD pipelines and working with Git.
- Sound judgment in determining when AI-generated outputs require human validation, with the ability to explain these
Contact information
Interested candidates are encouraged to apply here.
Please clearly mention that you have heard of this job opportunity on telegram channel VG Recruiting Agency.