About The Role
As an AI Trainer, you will improve large language models by producing high-quality training signals through data labeling, RLHF preference rankings, and rubric-based evaluations. You will support end-to-end LLM training pipelines with consistent, well-documented decisions that improve instruction following, reasoning quality, and content safety outcomes.
What You Will Do
• Perform data labeling for NLP tasks (intent classification, summarization checks, factuality review, named entity recognition)
• Execute RLHF work: pairwise ranking, rubric-based scoring, and short rationales
• Conduct prompt evaluation and response quality scoring using detailed rubrics
• Run QA evaluation, track error categories, and contribute to guideline updates
• Support content safety labeling for policy-sensitive topics
• Collaborate with engineering partners on edge cases, sampling strategies, and quality thresholds
Core Workstreams
• LLM evaluation (instruction following, reasoning quality, refusal behavior)
• Training data quality audits (spot checks, adjudication, gold set creation)
• Annotation operations (calibration sessions, reviewer alignment)
• Occasional multimodal evaluation or computer vision annotation when required
Qualifications
• Mid-senior experience in data operations, evaluation, QA, or applied AI workflows
• Strong written communication and ability to produce consistent decision rationales
• Attention to detail with rubric-based scoring and annotation guidelines compliance
• Familiarity with NLP concepts, LLM behavior, and training data quality practices
Employment Details
Remote, full-time role with Rex.zone.
Compensation:
$30–$50 per hour (USD), depending on scope and assignment.
How To Apply
Apply on Rex.zone via the AI Trainer jobs in Brazil page.