Domain Contributor - Evaluation Specialist
In short AI
Remote evaluation work testing and improving AI models, as a contractor paid per task across history, law, finance, journalism and tech. About 15 hours per week.
Description
- Role
- Design domain-specific tasks and questions to probe AI model limits and reasoning flaws. Verify AI-generated answers against primary sources and document the shortcomings found, following the task guidelines and evaluation rubrics.
- Requirements
- Proven hands-on experience with AI model training, annotation, RLHF or evaluation
- Deep expertise in a core domain (history, law, finance, journalism or technology)
- Advanced research skills and verification of claims against authoritative sources
- Excellent written and verbal communication
- Ability to precisely follow detailed instructions and evaluation frameworks
- Preferred
- adversarial task design, graduate-level research, Fortune 100 experience, professional fluency in a non-English language
- Conditions
- Contract: independent contractor, paid per accepted task
- Salary: not stated
- Hours: about 15 hours per week, minimum number of tasks per week required
- Location: remote
- Benefits
- none stated
Role Title: Domain Contributor - Evaluation Specialist
Role Type: Contractor (~15 hrs/week)
Location: Remote
micro1 is engaging Domain Contributor - Evaluation Specialists to support an internal project evaluating and improving advanced AI models. In this role, you'll apply your expertise to help train next-generation AI systems. Your work will shape how models learn, reason, and perform through high-quality, real-world input. Flexible, pay-per-task work across history, law, finance, journalism, and tech, for experts with prior AI training experience.
Scope of Work
- Design nuanced, domain-specific tasks that probe AI model limits and surface reasoning flaws.
- Author questions and scenarios demanding deep subject knowledge, beyond general understanding.
- Assess AI-generated outputs, validating accuracy against authoritative, primary sources in your area of expertise.
- Document clear rationales for model shortcomings, specifying missed reasoning, source errors, or nuances.
- Apply detailed task guidelines and evaluation rubrics consistently to each submission.
- Submit completed deliverables independently on a rolling basis, meeting project requirements for written and verbal clarity.
Preferred Qualifications
- Proven hands-on experience with AI model training, annotation, RLHF, or evaluation.
- Deep expertise in a core domain: history, archival or library science, law (paralegal, legal academic, compliance), finance (equity research, PE/VC, forensic accounting), journalism, or technology/industry history.
- Advanced research skills with a track record of verifying complex claims against authoritative sources.
- Exceptional written and verbal communication skills to articulate findings and feedback.
- Demonstrated ability to precisely follow detailed instructions and evaluation frameworks.
- Background in adversarial or “stomp the model” task design is advantageous.
- Graduate-level research, experience at a Fortune 100 company, or professional fluency in a non-English language a plus.
Compensation Structure
Compensation is output-based; experts are paid per task that meets the project specifications. The time required to complete work may vary depending on the expert’s experience and workflow. Minimum submission requirements apply. Experts must submit a minimum of tasks per week.
Start Timeline & Availability
We typically fill roles within 48 hours and are looking for experts ready to jump in right away. If selected, we expect you to start your first tasks within 24–48 hours of completing onboarding.
Originally posted on Himalayas
How to apply
Tasks start 24–48 hours after onboarding; roles are typically filled within 48 hours. AI
Prepare your application
A cover letter and a short pitch for this job, written from your profile, and what to learn next.
Sign in and create your profile from your CV. It is private: we share it with no one.
Something wrong or unlawful in this ad? Report it
Similar jobs
- Independent Consultant, Sales, Marketing & Business Strategy (Domain Expert)micro1
- Sales and Business Development Expertmicro1
- Software Engineer - Open Source Contributionsmicro1
- Independent Consultant, Finance, Data & Engineering (Domain Expert)micro1
- Senior Property Damage and Restoration Expertmicro1
- Senior Claims Investigation and Evidence Review Expertmicro1
Training for this job
- ISTQB® Testing with Generative AIGalileuonline · daytime · 14 h · 1320 € + IVA · starts 09/12/2026Asked for in this ad: AI / machine learning
- Utilizar Ferramentas da Ia na OrganizaçãoServiço de Formação Profissional de Santarémonline · daytime · 25 h · free · starts 02/12/2026Asked for in this ad: AI / machine learning
- a Inteligência Artificial no QuotidianoServiço de Formação Profissional de Viana do Casteloonline · evenings · 8 h · free · starts 03/12/2026Asked for in this ad: AI / machine learning
Languages AI
Published on (1)
- Himalayas seen 16 Nov
First seen on 16 Nov, 23:33