Remote Data Annotator

Remote Data Annotator (Full-Time, Remote) Rex.zone helps candidates discover and apply to remote data annotator roles supporting modern AI/ML training pipelines. In this role, you will create labeled datasets and evaluation signals that improve training data quality and large language model performance across NLP, computer vision, and content safety workflows. What You Will Do Apply detailed annotation guidelines to label and review text, images, audio, and multi-turn conversations Perform LLM evaluation and prompt evaluation (helpfulness, harmlessness, factuality, reasoning quality) Complete RLHF-style preference labeling, rubric scoring, and critique writing to generate high-signal feedback Run QA evaluation using sampling plans, inter-annotator agreement checks, and corrective actions Document edge cases, ambiguity, and rationale to improve rubrics, taxonomies, and consistency Flag dataset issues (duplicates, leakage, label noise) and follow confidentiality/data-handling standards Required Qualifications Mid-to-senior experience in data annotation, data labeling, or QA evaluation Strong guideline compliance, consistency, and written reasoning for critique-based tasks Comfort with structured labeling tools, taxonomies, and metrics-driven remote work Familiarity with NLP/LLM evaluation or content moderation taxonomies Preferred RLHF workflows, preference ranking, and rubric-based scoring experience Computer vision annotation exposure (bounding boxes, polygons, segmentation QA) Understanding of training data quality concepts (bias, representativeness, label noise) Compensation Base hourly rate: $30–$50/hr (USD), depending on project needs and qualifications.

Create labeled datasets and evaluation signals to improve AI/ML training data quality and LLM performance. Perform RLHF-style preference labeling, prompt evaluation, and QA checks across text, image, and audio workflows.

Requires mid-to-senior experience in data annotation or QA evaluation with strong guideline compliance and written reasoning skills. Familiarity with NLP/LLM evaluation or content moderation taxonomies is required.

Relevées dans la description du poste. Confirmez les exigences importantes ci-dessus.

• Data Annotation
• Data Labeling
• QA Evaluation
• LLM Evaluation
• Prompt Evaluation
• RLHF
• NLP
• Computer Vision
• Content Moderation
• Rubric Scoring
• Preference Labeling
• Critique Writing

• Data & Analytics
• Technology
• Software
• Science & Research

Expérience minimale

2+ ans

Postuler avant le

29 août 2026

Langue de l’offre

anglais

Heures de travail

40 heures par semaine

Exigences de lieu

Country, Montreal, Quebec, Canada

Niveau d’expérience

Mid-Senior level