Global Data & Annotation Operations

Challenge

ML models are only as good as their training data. Jumio needed high-quality, consistently labeled data across identity documents from 200+ countries, covering data extraction, fraud detection, and biometric verification. This required a massive, well-coordinated annotation operation.

Solution

Building the Operation

Scaled annotation operations from a small team to 350+ experts distributed across multiple geographies. Designed workflows, quality assurance processes, and tooling to maintain consistency at scale.

Quality Assurance

Implemented multi-tier review systems with inter-annotator agreement metrics, automatic quality checks, and continuous feedback loops between ML engineers and annotation teams.

Cross-Functional Coordination

Managed the interface between ML research (what data is needed), annotation operations (how to produce it), and product teams (what to prioritize). Ensured annotation guidelines evolved alongside model capabilities.

Tooling & Process

Built internal tooling for annotation task management, quality dashboards, and performance tracking. Standardized processes to reduce onboarding time for new annotators and maintain output quality.

Results

• Managed 350+ annotation experts globally
• Supported 3 ML verticals: Data Extraction, Fraud Detection, Biometrics
• Created training datasets covering 200+ document types across countries
• Maintained production-grade quality standards at scale

Key Insight

Data operations at scale is as much a leadership and process challenge as a technical one. The difference between good ML and great ML often comes down to data quality—and data quality comes down to how well you build, train, and manage annotation teams.