MathFormer: Tiny model tests if symbolic math is pattern matching or reasoning

View on Reddit

The 4M parameter seq2seq model reaches ~98.6% accuracy on expanding factorized symbolic expressions with no prior math knowledge. The project explores whether LLMs learn reasoning or just pattern matching in math.