colours
radaremprego

Remote Python Senior Software Engineer, AI Training

Hourspart-timeconfirmed in the text
Contractfreelance
not stated in the adwork mode · salary
posted 2026-11-01 ·

In short AI

Evaluate interactions with AI coding agents (OpenAI Codex, Claude Code, Cursor) for reasoning quality, explanations and engineering judgment. Remote, 10-20 hours per week, on projects lasting from a few weeks to several months.

Description

a long ad (3 080 characters): the AI summarised it; the full text is in the next tab.

Role
Assess, end to end, AI-generated coding interactions in real scenarios: whether the response makes sense, whether the preamble and reasoning are useful, whether it reflects strong engineering judgment. No production code is written. Give written, opinionated feedback and define what a high-quality response looks like.
Requirements
  • Senior, staff or principal-level software engineering experience (or equivalent)
  • Strong background in TypeScript/JavaScript or Python
  • Hands-on experience with at least one of: OpenAI Codex, Claude Code or Cursor
  • Written and spoken English at B2 or above (required, with recorded videos)
  • Evaluate code without executing it or reviewing every line
Preferred
  • experience in prompt design or evaluation, mentoring senior engineers or defining engineering standards
Conditions
Contract: independent, ongoing, projects from two weeks to a few months
Salary: 100-200 USD per hour, depending on experience and location
Hours: 10-20 hours per week, flexible
Location: remote, worldwide (eligibility may vary by country)
Benefits
not stated

Contract | Remote, worldwide | $100-$200/hour depending on experience and location | 10-20 hrs/week

Please note that eligibility may vary depending on your location, as some regions may not be supported by the client.

Check out this Loom video for more details:

We're looking for highly experienced software engineer (SR+) to help evaluate the quality of interactions with modern coding agents such as OpenAI Codex and Claude Code.

This is not a traditional engineering role. You won't be writing production code. You'll be evaluating something harder: whether the model thinks like a great engineer.

What this role actually is

You will assess how AI coding agents behave in real-world scenarios, focusing on:

• Whether the response makes sense

  • Whether the preamble and reasoning are useful
  • Whether the output reflects strong engineering judgment
  • Whether the interaction feels right to an experienced developer

This role is about engineering taste. Syntax correctness is the easy part.

What you'll be doing

  • Evaluate AI-generated coding interactions end to end
  • Judge whether outputs are useful, correct at a high level, and aligned with how a strong engineer would think
  • Assess the quality of explanations and reasoning, not just the code
  • Distinguish between levels of response quality (what makes something a 2 vs a 4)
  • Give clear, opinionated feedback in writing on what worked, what didn't, and what felt off or misleading
  • Help define what great looks like when working with tools like Cursor, Codex and Claude Code

What we mean by "taste"

We're looking for engineers who can answer questions like:

  • Does this feel like something a strong engineer would actually say?
  • Is this explanation helpful, or just technically correct?
  • Is the model guiding the user well, or just dumping output?
  • Would this interaction build or erode trust?

You should be comfortable making subjective but rigorous judgments, and explaining them clearly.

Who you are

  • Senior, Staff or Principal-level engineer (or equivalent experience)
  • Strong background in TypeScript/JavaScript or Python
  • Hands-on experience with at least one of OpenAI Codex, Claude Code or Cursor
  • Deep familiarity with modern AI-assisted dev workflows
  • Able to evaluate code without executing it or reviewing every line
  • Strong written and spoken English (B2 or above). You'll be writing detailed feedback and recording short video explanations, so this is a hard requirement
  • Comfortable giving direct, opinionated feedback
  • High bar for what good engineering looks like

Nice to have

  • Prior exposure to prompt design or evaluation workflows
  • Experience mentoring senior engineers or defining engineering standards

Engagement details

  • Rate: $100-$200/hour depending on experience and location

• Hours: 10-20 hours/week, flexible

  • Duration: ongoing. Projects run from about two weeks to a few months each, and we offer new ones to evaluators who do well
  • Start: as soon as you clear the take-home and a project has an open seat
  • Process: one take-home evaluation exercise with a recorded Loom walkthrough. No interview

Originally posted on Himalayas

How to apply

Process: one take-home evaluation exercise with a recorded Loom walkthrough; no interview. Start as soon as the take-home is cleared and a project has an open seat. AI

Apply on the original ad ↗

Prepare your application

A cover letter and a short pitch for this job, written from your profile, and what to learn next.

Sign in and create your profile from your CV. It is private: we share it with no one.

Something wrong or unlawful in this ad? Report it

It stays visible while we review it. We decide within 7 days (anything clearly unlawful, such as a scam, sooner) and tell you the outcome and why.

Similar jobs

Training for this job

Languages AI

Published on (1)

First seen on 16 Nov, 23:06