Poolside's synthetic data pipeline for code pre-training

Poolside generates synthetic code data by pairing templates with supplementary context and tuning difficulty. The pipeline spreads generations across an axis of phrasing, ensuring tasks are neither trivial nor too hard for the model to learn.

Featured · Marah Abdin, Robert McHardy