Senior Software Quality Engineer – AI

Résumé du poste

Develop and execute comprehensive quality strategies and scalable automation frameworks for AI-enabled software products. Evaluate AI responses for accuracy and reliability while integrating quality checks into CI/CD pipelines.

Détails du poste

We are seeking a Senior Software Quality Engineer – AI Focused to drive quality, reliability, and automation for AI-enabled software products. The role will collaborate with Development, DevOps, Product, and QE teams to design robust testing strategies and scalable automation frameworks. The engineer will test AI/ML and Generative AI features, APIs, microservices, integrations, and data pipelines while ensuring reliable and secure product delivery. The position is full-time and onsite in Toronto, ON. Key Requirements 5+ years of experience in Quality Engineering, QA, SDET, or Test Automation. 5+ years of programming experience with Java, Python, Groovy, or similar languages. Strong experience building and maintaining test automation frameworks. Hands-on experience with Selenium, Playwright, Cucumber/Gherkin, or RestAssured. Experience with API, database, integration, regression, and microservices testing. Experience with JUnit, TestNG, ReadyAPI, qTest, and JMeter. Hands-on experience testing AI/ML, Generative AI, conversational AI, or LLM-based applications. Knowledge of AI quality concepts such as prompt evaluation, accuracy, relevance, grounding, hallucination detection, toxicity, bias, and response consistency. Experience with CI/CD, Agile, BDD, TDD, ATDD, and risk-based testing. Strong analytical, problem-solving, communication, and documentation skills. Key Responsibilities Develop and execute comprehensive quality strategies, test plans, test cases, and test data. Design, develop, and maintain scalable test automation frameworks. Create automated tests for AI/ML and Generative AI features. Evaluate AI responses for accuracy, relevance, safety, hallucination, reliability, and latency. Test APIs, microservices, databases, integrations, data pipelines, and third-party services. Create AI evaluation datasets, expected behaviors, quality thresholds, and regression coverage. Integrate automated testing and AI quality checks into CI/CD pipelines. Identify, document, prioritize, and track defects through resolution. Analyze test coverage, quality metrics, AI evaluation results, and production feedback. Collaborate with Development, DevOps, Product, and QE teams to improve product quality and release readiness.

Ce que vous ferez

Develop and execute comprehensive quality strategies and scalable automation frameworks for AI-enabled software products. Evaluate AI responses for accuracy and reliability while integrating quality checks into CI/CD pipelines.

Exigences

Requires over 5 years of experience in Quality Engineering and programming with Java or Python. Must have hands-on experience testing AI/ML and Generative AI applications along with proficiency in automation tools like Selenium or Playwright.

Compétences indiquées

• CI/CD · Souhaitée
• Agile · Souhaitée
• Java · Souhaitée
• Python · Souhaitée

Autres compétences pertinentes

Relevées dans la description du poste. Confirmez les exigences importantes ci-dessus.

• Quality Engineering
• Test Automation
• Java
• Python
• Groovy
• Selenium
• Playwright
• Cucumber
• RestAssured
• API Testing
• Microservices Testing
• AI/ML Testing
• Generative AI
• LLM Evaluation
• CI/CD
• Agile

Domaines d’emploi

• Software
• Technology
• Data & Analytics
• Engineering