Senior AQA Engineer (with AI experience)

N-iX is looking for a Senior QA Automation Engineer to lead the validation and verification strategies for EDGE Group’s AI transformation. You will be responsible for defining "what good looks like" for non-deterministic AI systems, ensuring that Large Language Models (LLMs) and predictive engines meet the strict reliability standards of the defense sector. Qualifications 5+ years in QA Automation Experience testing data-driven systems, AI, or ML models Strong analytical skills for defining test criteria in non-deterministic systems Ability to collaborate with engineers, data scientists, and product teams Experience in regulated industries (defense, aerospace) will be a plus English level - at least Upper-Intermediate Technical Requirements Strong experience in Python and test automation frameworks (Pytest, Selenium/Playwright, API testing) Experience with performance testing tools (JMeter, Locust, K6) Familiarity with LLM evaluation tools (e.g., DeepEval, TruLens) Knowledge of SQL and data validation tools Experience with CI/CD pipelines (GitLab) and version control (Git) Exposure to test reporting tools (Allure, TestRail) Key Responsibilities 1. AI & LLM Testing Build automated tests to evaluate AI outputs for accuracy, consistency, and reliability Validate Retrieval-Augmented Generation (RAG) systems using metrics like relevance and correctness Create regression tests to detect changes in AI behavior (prompt drift) 2. System & Integration Testing Test integrations between AI systems and enterprise platforms (e.g., SAP) Perform API testing, including security and access control validation Design performance tests for latency, scalability, and system throughput 3. Quality & Governance Align test cases with system requirements and user needs Support release readiness through structured testing reviews Track and manage defects, ensuring traceability to data, models, or requirements We offer*: Flexible working format - remote, office-based or flexible A competitive salary and good compensation package Personalized career growth Professional development tools (mentorship program, tech talks and trainings, centers of excellence, and more) Active tech communities with regular knowledge sharing Education reimbursement Memorable anniversary presents Corporate events and team buildings Other location-specific benefits *not applicable for freelancers