AI Operations Analyst

Job Description This is a remote position. AI Operations Analyst Reporting to Manager, Production Support & Service Reliability, monitors AI-driven workflows in production, reviews output-quality incidents, supports fallback and human-review decisions, and partners on trust-sensitive escalations. This role helps ensure AI-enabled workflows remain reliable, supportable, and safe in live finance-oriented environments. Key Responsibilities AI Workflow Monitoring · Monitor AI-driven workflows in production for abnormal behavior, trust issues, degraded outputs, and recurring exception patterns. · Review signals related to output quality, fallback use, low-confidence patterns, and workflow anomalies. · Escalate issues that could affect live user trust, business quality, or finance-sensitive outcomes. Incident Investigation · Support investigation of output-quality incidents, orchestration failures, prompt or workflow regressions, and tool-call problems in production. · Help distinguish true software defects from model behavior issues, data-quality problems, or process misunderstandings. · Partner closely with Engineering, QA, and business stakeholders on judgment-heavy incidents. Operational Controls · Support human-review patterns, fallback routes, and safe-operating responses for AI-enabled features. · Help define and apply production review practices for higher-risk workflows. · Document evidence and issue patterns in ways that support repeatability and better release decisions. Continuous Improvement · Identify recurring AI operations issues and recommend improvements to supportability, monitoring, and quality feedback loops. · Feed production learnings back into QA, Product, and Engineering. · Help improve operational trust in AI-enabled solutions over time. Requirements Required Qualifications · 4+ years of experience in production support, AI operations, business process operations, analytics, QA, or a similar role. · Strong analytical skills and comfort evaluating nuanced workflow or output issues. · Ability to work across engineering and business teams in ambiguous situations. · Experience with AI-enabled workflows, operational analytics, business process validation, or support for workflow-heavy systems. · Strong written communication and documentation discipline. · Bachelor's degree preferred. Key Traits for Success · Thoughtful, precise, and calm under pressure. · Strong at spotting subtle trust or quality issues others may miss. · Comfortable balancing evidence, judgment, and business impact. · Focused on safe operations and clear escalation. Benefits Base salary plus performance-based bonus. Actual compensation packages are determined by evaluating a wide array of factors unique to each candidate, including but not limited to skill set, years and depth of experience, education, certifications, cost of labor, and internal equity.