Senior Data Labeling Jobs in Ottawa (Remote)

Overview Rex.zone is hiring for Senior Data Labeling roles supporting AI/ML training workflows end to end, including data labeling, RLHF, prompt evaluation, QA evaluation, and training data quality audits for large language model evaluation and computer vision annotation. This is a full-time, remote opportunity aligned with Ottawa talent. What You Will Do Execute senior-level data labeling across text, image, and multimodal tasks with consistent decision-making and high accuracy Perform RLHF-style preference labeling/ranking using rubric-based guidelines Run prompt evaluation and QA evaluation to identify failure modes and policy gaps Audit training data quality via sampling, error categorization, and root-cause analysis Ensure annotation guidelines compliance; propose clarifications and edge-case examples Support NER/span labeling and content safety labeling as needed Resolve disagreements, calibrate annotators, and stabilize inter-annotator agreement Document decisions and acceptance criteria for downstream model training teams Core Workflows Gold set creation, consensus labeling, disagreement analysis Quality systems: spot checks, batch acceptance, error tracking, corrective actions Evaluation datasets for LLMs (rubric scoring, adversarial/safety probes) Computer vision annotation (bounding boxes, polygons, segmentation) where applicable Who You Are Experienced in data labeling/annotation with strong edge-case handling and consistency Comfortable with RLHF, prompt evaluation, and QA evaluation concepts Able to mentor peers and communicate clearly when guidelines conflict with real data

Execute senior-level data labeling and RLHF preference ranking across text, image, and multimodal tasks. Perform quality audits, prompt evaluations, and calibrate annotators to ensure high-quality training data for LLMs.

Requires experience in data labeling and annotation with a strong ability to handle edge cases and maintain consistency. Candidates must be comfortable with RLHF and QA evaluation concepts and be able to mentor peers.

Relevées dans la description du poste. Confirmez les exigences importantes ci-dessus.

• Data Labeling
• RLHF
• Prompt Evaluation
• QA Evaluation
• Training Data Quality Audits
• Computer Vision Annotation
• NER
• Span Labeling
• Content Safety Labeling
• Inter-annotator Agreement
• Gold Set Creation
• Consensus Labeling
• Bounding Boxes
• Polygons
• Segmentation
• Root-cause Analysis

• Data & Analytics
• Technology
• Software
• Science & Research
• Engineering

Expérience minimale

2+ ans

Postuler avant le

29 août 2026

Langue de l’offre

anglais

Heures de travail

40 heures par semaine

Exigences de lieu

Country, Ottawa, Ontario, Canada

Niveau d’expérience

Mid-Senior level