AnalysisAI Models
Jun 22, 2:08 AM
Sakana Fugu: AI agent improves GPT training recipe
An AI agent ran 123 experiments over ~14 hours on a single H100 GPU, achieving a best bits-per-byte (BPB) of 0.9774, outperforming frontier models like Model C (0.9781) and Model A (0.9822). The agent autonomously iterated on training code, adjusting batch size, depth, learning rates, and optimizers to improve a small GPT's validation loss. Fugu-Ultra's best single run reached 0.