Software Engineer in Test – AI Voice & Clinical Systems

Software Engineer in Test – AI Voice & Clinical Systems Working Hours: Full overlap with MT Location: Remote / LATAM About the Team Our client is building next-generation AI-powered clinical tools used by dental professionals across North America. Our products combine voice recognition, AI-assisted workflows, and clinical automation to improve the speed and accuracy of dental charting and documentation. Current product areas include: AI Voice Scribe — real-time clinical transcription and note generation Voice Perio — AI-assisted periodontal charting through voice commands Voice Charting — hands-free clinical data entry integrated into dental workflows This is a fast-moving engineering team working at the intersection of AI, voice technology, and healthcare software. About the Role We are looking for a highly technical Software Engineer in Test (SDET) to lead quality strategy for our AI voice and clinical products. This is not a traditional QA role focused only on manual testing or UI automation. We are looking for someone who can think deeply about how to validate non-deterministic AI systems, build scalable automation frameworks, and define quality standards for products powered by speech recognition, LLMs, and NLP workflows. You will be the first dedicated QA/SDET hire on the Innovation team and will play a critical role in shaping how quality is approached across both our internal engineering team and external development partners. The ideal candidate combines strong automation engineering fundamentals with hands-on experience testing AI-powered systems such as conversational AI, transcription platforms, voice assistants, LLM features, or NLP workflows. Responsibilities AI & Voice System Testing Design and execute testing strategies for AI-powered voice and clinical workflows Validate speech-to-text accuracy, NLP/entity extraction, and AI-generated outputs Build approaches for evaluating non-deterministic systems, including hallucination detection, regression tracking, and model output validation Create and maintain benchmark datasets and evaluation methodologies for AI quality measurement Test real-world voice workflows involving accents, noisy environments, and clinical terminology Automation Engineering Design, build, and maintain scalable automation frameworks and test infrastructure Develop automated functional, integration, regression, API, and performance tests Integrate automated testing into CI/CD pipelines and release workflows Improve test reliability, maintainability, and execution speed across environments Quality Strategy & Ownership Define quality standards, risk-based testing strategies, and release criteria Establish metrics and monitoring approaches for AI quality and regression detection Partner closely with engineering and product teams to identify risks and improve release confidence Help shape quality processes across both internal and partner engineering teams Collaboration Work closely with engineers, product managers, designers, and stakeholders throughout the development lifecycle Participate actively in sprint planning, refinement, reviews, and retrospectives Clearly communicate testing strategies, quality risks, and recommendations in written and verbal form Required Qualifications 4+ years of experience as an SDET, Software Engineer in Test, or QA Automation Engineer Strong hands-on automation experience using frameworks such as Playwright, Cypress, Selenium, pytest, or similar Experience building or extending automation frameworks independently Strong programming skills in Python, JavaScript/TypeScript, Java, or C# Experience testing REST APIs and distributed systems Strong understanding of CI/CD pipelines and automated quality gates Excellent analytical and problem-solving skills Strong written and verbal communication in English Preferred Qualifications Experience testing AI/ML-powered products or workflows Experience with conversational AI, transcription systems, voice assistants, chatbots, or NLP-based products Familiarity with LLM evaluation, AI output validation, regression detection, or prompt/workflow testing Experience testing cloud-based SaaS products and distributed systems Exposure to performance testing tools such as k6, JMeter, or Locust Experience working in healthcare or regulated software environments is a plus Familiarity with dental software or healthcare terminology is a plus What We’re Looking For Successful candidates typically demonstrate: Strong automation engineering depth Ability to reason about probabilistic/non-deterministic systems A thoughtful approach to AI quality validation Strong ownership mentality and initiative Clear communication and documentation skills Comfort operating in ambiguous, fast-moving environments Locations LATAM Remote status Fully Remote About Tight Line Tight Line represents the perfect synergy of expertise and ambition. Born from the merger of Rapid River (est. 2013) and Cactus Code (est. 2016). Our mission is simple yet bold: to attract the brightest minds in software development and empower them with meaningful, cutting-edge projects for clients across the globe. https://tightlinesoftware.com/ Founded in 2024 Co-workers 160+