Position: Computer Science - Domain Experts (Pay Per Task)

Type: Short-term Contract

Compensation: $250 per task upon approval

Location: Remote

Commitment: Up to 40 hours/week with 4 hours overlap with PST

Duration: 3 Months

Start Date: Immediate

Role Responsibilities

Design advanced computer science problems that test AI model performance across conceptual and reasoning-heavy topics, including systems design, algorithms, cybersecurity, and other advanced CS areas
Develop clear, step-by-step reference solutions with rigorous logic, well-structured explanations, and defensible reasoning
Evaluate AI-generated responses for correctness, reasoning quality, completeness, clarity, and alignment with the intended problem requirements
Identify flaws, gaps, edge cases, and near-miss reasoning patterns in AI outputs
Collaborate with researchers to refine benchmarks across undergraduate, graduate, and PhD-level computer science topics
Provide structured feedback to improve problem quality, benchmark reliability, and model evaluation standards
Apply computer science concepts creatively to challenging reasoning and problem-solving tasks

Requirements

Candidates with a Ph.D., completed or currently pursuing, in Computer Science or closely related fields are encouraged to apply
Strong expertise in advanced computer science reasoning and problem-solving across domains such as:
Algorithms, Systems Design, Cybersecurity, Databases, Networks, Theory of Computation, Operating Systems, and related CS fields
Ability to design challenging conceptual problems that require deep understanding rather than straightforward recall or coding
Strong analytical, critical thinking, and problem-solving skills
Excellent written communication skills, with the ability to explain complex technical ideas clearly and logically
Strong attention to detail in evaluating reasoning, assumptions, definitions, edge cases, and solution validity
Ability to provide structured, high-quality feedback on AI-generated responses
No prior AI experience required
Python and Data Analysis knowledge is desirable but not mandatory
LinkedIn profile must be mentioned in the resume
Candidates will go through a background check after onboarding. Before verification they will not receive any work

Application Process

Computer Science Specialist | Remote

About this role