Wells Fargo pre-trained a transformer on billions of transactions

Freddy Lecue describes pre-training a transformer from scratch on billions of customer transactions, predicting the next transaction instead of the next word. The single backbone is then fine-tuned for fraud detection, credit line decisions and anti-money laundering.

People · Freddy Lecue