colours
radaremprego

English-Speaking AI Data Evaluation Specialist | Large Language Models

Workster Jobs recruitment agency
Work modeon-site
Hoursrotating shifts · nights · weekendsconfirmed in the text
not stated in the adcontract · salary
posted 2026-08-24 · advertised for 90 days ·

Where

Lisboa (Lisboa)

improve the location

More jobs: Systems analysts in Lisboa (1 195)

Sign in to see the travel time

Median rent ⓘ
T1870 €T21 310 €

In short AI

Evaluating and annotating responses generated by large language models (LLMs) for a leading global customer experience company, on-site in Lisbon. Rotating 24/7 schedule with two rotational days off. Includes specialized initial training.

Description

a long ad (3 044 characters): the AI summarised it; the full text is in the next tab.

Role
Review, evaluate and annotate AI-generated responses according to detailed annotation and quality guidelines, assessing quality, relevance, factual accuracy, safety and policy compliance. Identify inconsistencies, edge cases and potential model issues, escalating when necessary, and contribute to the training and improvement of AI systems through detailed evaluations and feedback.
Requirements
  • Native-level English fluency required
  • English at B2 level or above
  • Strong understanding of generative AI, LLMs and AI evaluation methodologies
  • Ability to interpret and consistently apply complex written guidelines
  • Strong analytical thinking, attention to detail and reading comprehension
  • Comfortable working with structured text, datasets and AI-generated content
  • Confidentiality and discretion when handling information
Preferred
  • experience in AI data annotation, LLM evaluation, RLHF, SFT, quality assurance, content labeling or Trust & Safety
  • experience in similar analytical roles
Conditions
Contract: not stated
Salary: not stated
Hours: rotating 24/7, Monday to Sunday, with 2 rotational days off
Location: Lisbon, Portugal (on-site)
Benefits
  • Private health insurance
  • Meal allowance
  • Specialized onboarding and training in AI evaluation
  • Professional development and ongoing learning
  • International and multicultural working environment

Workster is partnering with a leading global customer experience company to recruit English-speaking AI Data Evaluation Specialists for an innovative artificial intelligence project based in Lisbon, Portugal.

If you’re passionate about AI, language technology, and data quality, this is a unique opportunity to contribute to the development of next-generation Large Language Models (LLMs). You’ll work with cutting-edge AI systems by reviewing, evaluating, and improving AI-generated responses, helping create safer, more accurate, and more reliable conversational experiences for millions of users worldwide.

Your Role
  • Review and evaluate AI-generated responses using detailed annotation and quality guidelines

  • Label and annotate language model outputs with accuracy and consistency

  • Assess AI-generated content for quality, relevance, factual accuracy, safety, and policy compliance

  • Apply complex annotation guidelines across a wide variety of scenarios and content types

  • Identify inconsistencies, edge cases, and potential model issues, escalating when appropriate

  • Analyze structured text, datasets, and AI-generated content to support model improvement

  • Deliver high-quality work while consistently meeting productivity, quality, and accuracy targets

  • Maintain strict confidentiality and comply with all project standards, security requirements, and internal procedures

  • Contribute directly to the ongoing training and improvement of advanced AI systems through detailed evaluations and feedback

  • 24/7 rotating schedule (Monday to Sunday) with 2 rotational days off

Your Qualifications
  • Native-level fluency in English

  • Good command of English (B2 level or above)

  • Strong understanding of Generative AI, Large Language Models (LLMs), and AI evaluation methodologies

  • Previous experience in AI data annotation, LLM evaluation, Reinforcement Learning from Human Feedback (RLHF), Supervised Fine-Tuning (SFT), Quality Assurance, Content Labeling, Trust & Safety, or similar analytical roles is highly desirable

  • Excellent analytical thinking and attention to detail

  • Ability to interpret and consistently apply complex written guidelines and policies

  • Strong reading comprehension and written communication skills

  • Comfortable working with structured text, datasets, and AI-generated content

  • Ability to work efficiently in a fast-paced, quality-driven environment while meeting performance targets

  • Professional, objective, and capable of handling confidential information with discretion

The Offer
  • Office-based position in Lisbon, Portugal

  • Comprehensive onboarding and specialized training in AI evaluation processes

  • Opportunity to work on one of the world’s most advanced artificial intelligence projects

  • International and multicultural working environment

  • Career development opportunities within the rapidly growing AI industry

  • Private health insurance

  • Meal allowance

  • Ongoing learning and professional development

  • Modern office facilities with a collaborative team culture

  • Unique opportunity to shape the future of generative AI and language technologies

How to apply

Apply on the original ad ↗

Prepare your application

A cover letter and a short pitch for this job, written from your profile, and what to learn next.

Sign in and create your profile from your CV. It is private: we share it with no one.

Something wrong or unlawful in this ad? Report it

It stays visible while we review it. We decide within 7 days (anything clearly unlawful, such as a scam, sooner) and tell you the outcome and why.

Similar jobs

Salary compared ⓘ

This jobMedian?2 800 €
2 100 €3 429 €

219 jobs · Systems analysts · Lisboa district

see the best paid →

Training for this job

Published on (1)

First seen on 31 Oct, 13:44 · last seen on 21 Nov, 21:17