This is a remote position.
Job Description:
We are seeking an experienced QA Lead to drive end-to-end quality assurance for an advanced Agentic AI platform. This role will be responsible for defining quality standards, developing automated evaluation frameworks, leading test strategy, and ensuring release readiness for AI-powered conversational solutions. The ideal candidate will have strong expertise in Python-based test automation, AI/LLM testing, conversational AI validation, and Retrieval-Augmented Generation (RAG) systems. Responsibilities include building evaluation harnesses, creating test plans, executing regression testing, validating data quality, conducting AI safety and prompt injection testing, managing defect triage, and leading a team of QA and Data Testers. The QA Lead will collaborate closely with engineering, product, architecture, and data teams to ensure reliable, scalable, and high-quality AI solutions are delivered while maintaining robust quality gates and release standards.
Requirements
QA Leadership
Python Test Automation
AI/LLM Testing
Conversational AI Testing
RAG (Retrieval-Augmented Generation) Testing
LLM Evaluation Frameworks
Golden Dataset & Regression Testing
Intent & Context Validation
Multi-Turn Dialogue Testing
Data Quality Testing
Prompt Injection & AI Safety Testing
Release Readiness & Quality Gates
Defect Management & Test Strategy
CI/CD Integrated Testing
Team Leadership & Mentoring