Generalist - Labeling

1. Role OverviewMercor is collaborating with a leading AI lab on a short-term project focused on improving preference ranking models for conversational AI systems. We’re seeking detail-oriented generalists—ideally with prior experience in data labeling or content evaluation—to assess and rank model outputs across a variety of domains. This opportunity is well-suited for professionals comfortable with nuanced judgment tasks and working independently in a remote setup.2. Key ResponsibilitiesEvaluate and compare AI-generated responses based on quality, coherence, and helpfulnessAssign preference rankings to pairs or sets of model outputsFollow detailed labeling guidelines and adjust based on evolving criteriaProvide brief written explanations for ranking decisions when requiredFlag edge cases or inconsistencies in task design or model output3. Ideal QualificationsPrior experience in data labeling, content moderation, or preference ranking tasksExcellent critical thinking and reading comprehension skillsComfort working with evolving guidelines and ambiguityStrong attention to detail and consistency across repetitive tasksAvailability for regular part-time work on a weekly basis4. More About the OpportunityRemote and asynchronous — set your own hoursExpected commitment: 10–20 hours/weekFlexible workload depending on your availability and performance5. Compensation & Contract Terms$25–35/hour depending on experience and locationPayments issued weekly via Stripe ConnectThis is a freelance engagement; you’ll be classified as an independent contractor6. Application ProcessSubmit your resume to get startedhttps://work.mercor.com/jobs/list_AAABmZxe1Lvr8tEByeFKTr9q?referralCode=3694f086-4466-4dd1-a1e6-78cca8cd64a7&utm_source=referral&utm_medium=share&utm_campaign=job_referralComplete a short form to highlight your relevant experienceYou may be asked to complete a brief assessment to evaluate task fitExpect a response within 3–5 business days7. About MercorMercor is a talent marketplace that connects top experts with leading AI labs and research organizationsOur investors include Benchmark, General Catalyst, Adam D’Angelo, Larry Summers, and Jack DorseyThousands of professionals across law, engineering, research, and creative fields collaborate on contract-based projects shaping the future of AIWe consider all qualified applicants without regard to legally protected characteristics and provide reasonable accommodations upon request.