Position: LLM – AI Quality Analyst (Personalization) – Vietnamese

Type: Short-Term Contract

Location: Remote

Commitment: 30–40 hours/week, 4-hour overlap with PST

Engagement Length: 2 months

Start Date: Immediate

Role Responsibilities

Design multi-turn conversational prompts (1–5 turns) using personal context
Evaluate personalized AI responses for grounding, integration, and helpfulness
Assess correct usage of personal data and identify flawed inferences or hallucinations
Review integration quality to ensure personalization feels natural and not over-narrated
Conduct side-by-side (SxS) evaluation and ranking of model responses
Identify personalization errors, reasoning gaps, and grounding issues
Write clear, structured rationales referencing specific conversation turns
Extract and verify “Debug Info” to confirm correct data source utilization
Maintain strict data hygiene by deleting evaluation conversations

Requirements

Vietnamese proficiency (reading and writing)
Strong experience in data annotation, AI quality evaluation, content moderation, or related roles
Strong analytical skills for evaluating nuanced and ambiguous AI outputs
Experience with creative prompt engineering and multi-turn conversations
Understanding of personalization concepts and AI response evaluation
High attention to detail for SxS comparisons
Excellent written communication and feedback documentation skills
BS/BA degree or equivalent experience in a relevant field
Willingness to use a primary personal Google account with enabled personal data sources
Self-motivated and able to work independently in a remote setup
Desktop/laptop with reliable internet connection

Application Process

AI Model Evaluator (Vietnamese) | $11/hr - Remote

About this role