Training Specialist

About HumanSignal

HumanSignal works with teams building AI to create and deliver high-quality real-world data. The company supports customers from dataset creation and annotation through final delivery, helping them ship stronger AI products more quickly.

The team builds datasets from the ground up, sources and manages subject-matter experts who assess model outputs, and runs the work through its own platform, Label Studio, an open-source data labeling and evaluation tool used by more than 1 million practitioners worldwide.

The company focuses on complex data operations such as real-world collection, multimodal workflows, and multi-step processes. Its enterprise platform helps advanced ML and AI teams run their own data operations, while its services team expands capacity when internal resources are limited.

If you want to help shape how the next generation of AI products is built, this opportunity may be a strong fit.

Role Overview

This position sits at the center of HumanSignal’s data delivery process. The team handles demanding, high-stakes projects end to end, including scoping, protocol design, and final delivery across on-site and distributed expert workforces spanning 50+ knowledge domains, 30+ languages, and 75+ countries.

The work includes RLHF, evaluations, red-teaming, and custom multimodal data creation, supported by Label Studio Enterprise and guided by strict quality standards, ethical sourcing, and data security practices. The Training Specialist helps ensure contributors understand the task, the quality bar, and the right way to work from the beginning.

This is a fast-moving role that requires ownership, adaptability, and strong follow-through. You will support complex programs, work through changing client requirements, balance quality with deadlines, and create the structure needed to keep training effective at scale.

What You'll Do

• Create project-specific training assets quickly, often within 24 to 48 hours, for new AI data labeling initiatives.
• Develop clear onboarding and enablement materials such as written guides, annotated examples, scoring rubrics, and assessments that turn technical instructions into contributor-friendly direction.
• Review learner performance data, recurring errors, and quality signals to find training gaps and design targeted fixes.
• Coach operations teams, including SPLs and Ops Associates, on instructional design, feedback delivery, and quality review methods.
• Build repeatable templates, frameworks, and self-service resources that keep training consistent across programs and customers.
• Measure training impact using quality metrics, completion rates, and satisfaction feedback, then refine content and methods quickly.
• Partner with Product and Engineering to identify tooling improvements that make contributor learning easier and more scalable.

Required Qualifications

To succeed in this role, you should bring at least 3 years of experience in instructional design, training, or technical content development. You should also be able to turn complex ideas into simple, accessible guidance for people who are not subject-matter experts.

The role calls for someone who can produce strong training materials on short deadlines, communicate clearly in writing, think in a structured way, and use data and performance metrics to improve training outcomes.

Preferred Qualifications

Experience in AI or ML, including familiarity with supervised fine-tuning, RLHF, or model evaluation, is a plus. A STEM background or technical training in fields such as computer science or data science is also preferred.

Additional value will come from experience with LMS platforms or annotation tools such as Scale, Labelbox, or Prodigy, as well as backgrounds in customer success enablement, sales enablement, or technical consulting.

Compensation

The role is listed with a salary range of USD 60,000 to USD 125,000.

Additional Information

Level: Individual Contributor

Location: San Francisco, CA