As high-quality public web data approaches exhaustion, synthetic data generation combined with Direct Preference Optimization (DPO) has emerged as the definitive pipeline for training domain-expert enterprise models. However, uncurated synthetic data quickly triggers Model Collapse. In this guide, we outline our production pipeline for rejection sampling, multi-judge filtering, and memory-efficient alignment using Unsloth.
01 The Rejection Sampling Quality Filter
High-utility synthetic data relies on aggressive downstream pruning:
- Deterministic Test Execution: For code and reasoning traces, samples must execute inside a containerized sandbox with zero failing assertions;
- Dual-Judge Cross-Validation: Candidate responses are evaluated across orthogonal frontier judges, accepting only samples that achieve unanimous approval.