LLM Red Team Intern (Evaluation Systems)

Internship | Remote | LLM Evaluation | Reports to CTO or Safety Lead About Elloe Elloe is the immune system for AI. We don’t train models — we protect their outputs. We trace every hallucination, enforce every policy boundary, and create an audit trail for every critical LLM interaction. Our modules (TruthChecker™, AutoRAG™, Autopsy™) are embedded in hospitals, banks, and regulatory sandboxes. Our job is to make sure these systems are safe before anything hits production. This role will help us break, stress-test, and harden the models used by governments and enterprises alike. About the Role You’ll red team real-world LLM deployments, design eval harnesses, and help scale Elloe’s output-level safety layer. This isn’t just prompt tuning — it’s forensic risk mapping. You’ll work directly with product and safety leads to uncover failure patterns and codify guardrails for GenAI systems under real-world scrutiny. What You’ll Own 1. Red Teaming & Risk Testing Create prompts to trigger hallucinations, policy violations, or failure scenarios Stress test Elloe-protected deployments using open and proprietary models Document behavioral exploits across use cases (healthcare, compliance, gov) 2. Evaluation Design Build truthsets and scoring rubrics tied to factuality, policy, or ethical standards Benchmark Elloe’s modules across model types (Claude, GPT-4, Gemini, open models) Collaborate with product to refine and expand our eval harnesses 3. Safety Intelligence Identify blind spots in current detection logic Recommend scoring methods or red flag thresholds for deployment Support internal model comparison reports or customer safety audits Who You Are ML/AI researcher or engineer (undergrad, grad, or early career) Experience working with LLMs, eval sets, and prompt design Strong attention to detail, grounded in safety and adversarial thinking Bonus: exposure to safety benchmarks like TruthfulQA, MMLU, or red teaming tools Why This Matters This is real-world alignment, not research theater. You’ll be helping define how AI gets deployed responsibly — with traceability, transparency, and real-time protection. You’ll leave this role with: Exposure to high-stakes LLM safety deployments Published frameworks or scoring methods used by enterprises Mentorship from technical founders operating at the bleeding edge of AI safety Logistics & Application Start Date: Rolling Duration: 12–16 weeks Compensation: Research stipend Location: Remote-first; flexible for global candidates To Apply: Share a jailbreak or eval idea you’d love to run against GPT-4.